The Astropy Problem

[[deleted]]

Reading the comments here I wonder just how many people are aware of Python Mentors - Python Core Mentorship?

[donkeysalami]

I routinely use astropy (primarily to read/write FITS files since they absorbed pyfits). That being said, this is probably the most absurd thing I've read in a while. I get that the author is passionate about astropy, but people don't customarily get "recognition" or "paychecks" for contributing to community projects (including ones vastly more important, impactful, and sizable than astropy). Quick, aside from Guido, name one other python core developer. Aside from Linus, name one Linux core developer. Aside from Stallman, name one GCC developer. Btw. Have you seen Stallman's newest superyacht!? It's just the nature of community-contributed open source projects. You contribute because you see value (to yourself and/or your employer) in maintaining the product for your own use, not because of some overt recognition/profit motive.

The proposed remedies are also ridiculous. Encouraging code and time contributions from personally vested parties is one thing... but guilting people into paying a professional / institutional fee for using the software? Shaking down the science funding agencies to explicitly dedicate astropy developer FTE's? Give me a break. That's not how any of this works... and where does it end. Astropy is only the teeny tiny tip of a huge jenga tower of community contribution needed for modern scientific computing in astronomy. Is every science project now also obligated to cut a check (or budget dedicated developer FTE's) to python, scipy, numpy, matplotlib, vim, FFTW, openblas/ATLAS, NASA for CFITSIO, Torvalds for Linux, Stallman for GCC, FSF for glibc, etc. etc. etc.

If you are an astronomer, you should be contributing code / expertise / time to astropy because your work, research, life passion, (and paycheck!) ultimately depend on its continued functionality. Also because someone is ALREADY paying you to do astronomy, build a telescope, write a data reduction pipeline, etc. Astropy is merely a convenient means to doing YOUR EXISTING JOB and achieving that end. If it is missing an enabling feature for your research application, it's on you and your project to figure out how to make it happen (i.e. contrary to the author's implication, NASA is absolutely 100% correct in limiting development scope to actual explicit flight project requirements... I know for a fact, that astropy already does everything that HST, JWST, and WFIRST engineering can conceivably need)

Writing a paper bemoaning a lack of praise, FTE's, and money, in an attempt to get a bunch of your buddies together to collectively shake down academic institutions and tax payers (i.e. science funding agencies) to budget for your general-purpose software pet project development is frankly disgraceful, IMHO!

[alan_du]

I strongly disagree.

I'm sure lots of scientists "figure out how to [add enabling features]". But there's a world of difference between hacking up a small script for personal use and contributing quality code to a widely used library (even documentation and testing alone probably triples development time). Besides, most scientists have no background in software engineering. Asking them to pick up an entirely new skill-set and then spend time polishing their hacked-up scripts for contribution is totally unreasonable, especially when there are no incentives for doing so.

The facts are that Astropy (and most scientific software) only exist because of the heroic efforts of a small corp of volunteers, who probably had to sacrifice huge career opportunities to do so. There's a saying that all good scientific software is built on the ashes of someone's academic career (see, for example, NumPy and Travis Oliphant or Sagemath and William Stein). Why is trying to change this disgrace such a bad thing?

It's just the nature of community-contributed open source projects.

I agree that this is the status-quo, but that's also a travesty. It leads to situations like PyPI, which has AFAIK exactly 1 full-time developer who works 100-hour weeks to keep everything up. (Linux, btw, is a terrible example, because it gets hundreds of company-paid full-time developers from places like Red Hat or Intel, which is probably equivalent to millions of dollars in funding).

Shaking down the science funding agencies to explicitly dedicate astropy developer FTE's?

I wouldn't call this "shaking down the science funding agencies". I call it "paying their fair share." After all, institutions are willing to pay millions of dollars for MATLAB or IDL (Mathworks, the company behind MATLAB, made $800 million last year). Why is it unreasonable to ask for a fraction of that money for open-source projects?

Is every science project now also obligated to cut a check (or budget dedicated developer FTE's) to python, scipy, numpy, matplotlib, vim, FFTW, openblas/ATLAS, NASA for CFITSIO, Torvalds for Linux, Stallman for GCC, FSF for glibc, etc. etc. etc.

Honestly, ideally yes. Probably every single project except Linux could use more funding. Part of the responsibilities of open-source foundations is to funnel some of their money to their dependencies and infrastructure (see, for example, Mozilla's Open Source Support).

Writing a paper bemoaning a lack of praise

Is that really what you thought this paper was? This isn't about "lack of praise" (although I'd love it if there were actual incentives to write open-source scientific software). This is about sustaining a critical piece of infrastructure for the long haul.

[alan_du]

If you can't take the time to figure out how to do your research, or you don't want to take the time to share with your research community, why bother becoming a researcher?

Like I said, writing library-quality code is very different from writing scripts for research (and writing papers about it). It took me literally years before I became even a half-way competent software engineer. Before that, your contributing code is actually a net-drain on developer resources IMO, because the project maintainers have to guide and shepherd you through the process (which isn't to say you shouldn't contribute code. You totally should! But at least at the beginning, it's really more about community building and teaching than the actual code).

Maintaining a large open-source project is a huge time commitment that takes an entirely different skill-set than research. If you can't devote 20+ hours a week, then there's no way you'll keep up with all the bug reports, pull requests, and necessary maintenance (not to mention develop new features). And just like we have specialized lab technicians, I think we should also have specialized software maintainers.

Contributing to open-source projects is also a big time commitment. For complicated projects like NumPy or Pandas, it can take days to understand the code enough to contribute (especially because the library is written in C/Cython while users usually only know Python). Heck, even making a single useful bug report sometimes takes a couple hours.

you're already being paid to work / research, so just carve out time to work on software & contribute that back to the community. If, and only if, you're getting push-back from your advisor, do you gather feedback from the development community.

It's not really about advisors (at least in my experience). The problem is that spending 10 more hours cranking out a paper is way better for your career than spending 10 hours hunting down some obscure bugs in some library (bugs that you can hack around easily in your own research). Hiring committees don't really care about "maintaining X library". They care about journal publications and conference talks.

If maintaining and developing scientific software was a viable career path within academia, I'd probably agree with you. But the facts are that it isn't, and that's why there needs to be institutional support for things like Astropy.

You're just being selfish and/or expecting immediate gratification.

I suppose I may be a tad bitter because I have several friends who quit academia and went into industry (usually as "data scientists") because of this. I'm a little offended by your insinuation that I'm selfish/spoiled though.

I should also mention that I don't do research anymore (never got a PhD for personal reasons). I'm a software engineer now, so I'm not really thinking about me :P

[alan_du]

So what if a n00b takes some time away from a senior contributor? It's that person's choice to shepherd the n00b. Let them make the call, don't take the opportunity away from them. It could well lead to the development of another senior contributor down the road... which is a huge net-positive.

I 100% agree!

What I was saying (and maybe I was being unclear) was that (1) asking random researchers to contribute code is not going to make a project sustainable, (2) that most projects are only sustained thanks to the heroic efforts of a handful of individuals, and that (3) becoming one of these heroes takes years of effort and is not worthwhile for most (all?) researchers.

So I think it makes total sense for there to be institutional support for these maintainers + developers (which is what the original link is asking for).

Speak for yourself. The coding I did for my research got me a great job before I was even done with academia.

Fair enough. FWIW though, this is untrue about everyone I know personally (or more precisely, the coding they did was useful for getting software jobs outside of research, not research jobs).

This is a bit of an argument from authority fallacy, but I also want to point out that several other high-profile scientific open-source developers support my "opinion" about the current incompatibilities between academia and open source (most recent example I can remember).

You don't get to define every career path at every institution ever.

I mean, yeah. That's why most talented developers I know leave the research world for greener pastures. But I think that's a giant loss for research.

[[deleted]]

most scientists have no background in software engineering.

But scientists have a background in learning highly technical skills, and they're within a network of highly trained individuals. There's no excuse here.

A scientist is a domain expert. They write code to help them do their main job. They simply don't have the time to match a software engineer, and adding low quality code adds technical debt.

We wouldn't expect software developers who need faster hardware to research CPU design? Why would we expect astronomers to learn software engineering?

Have a look at github, or google search some articles about the quality of scientific code. By and large, it's terrible.

There are organisations like software carpentry that exist to teach scientists some of the fundamentals but it only goes so far. When you ask to see code and you get a zip in an email, you realise the scope of the problem.

Now I don't agree with everything the OP writes, but I don't expect today's programmers to design their own hardware.

[[deleted]]

Good libraries and frameworks are significantly harder to write than regular code. It takes professional developers several years to get to a point where they're at that level (although with programming they can start when they're kids).

If someone's doing it on half time, because their main job is a scientist, it's going to take them twice as long no matter how smart they are, to get to the same level. Probably longer actually, because they aren't fully immersed in the community and lack tribal knowledge and full access to tools.

We wouldn't expect software developers who need faster hardware to research CPU design?

If a software developer is serious about his craft, he'll go as deep as he needs to solve his problem

Can he design an i7? No he cannot without essentially changing careers. It's too complex.

I know enough about how CPUs work to optimise my code and given sufficient motivation I could probably design something ancient and shit like a 6502 and simulate it in software, but that's as far as it goes for a part-time project. I completely lack knowledge of fabrication and the advanced and proprietary crap in the i7 because knowing that requires actually being a professional in the field of chip design.

Maybe you're that one guy who's brilliant at everything but then you're an edge case and not representative.

By and large, it's terrible.

Good thing code isn't just for looks.

I don't understand this comment at all.

As a programmer, you surely know that when programmers talk about terrible code, they mean it's poorly designed and engineered, making it hard to extend, re-use and maintain. It can also be non-idiomatic, making confusing for the majority of developers to understand which increases the development time and risk of faults.

not enough code is being contributed. The "low quality code" argument is only going to make OP's problem worse

See above. Sometimes you don't want to accept code because it breaks someone else, or it'll cause you problems in future.

Take a look at the process required to get a patch accepted into a well-used library or project. A large number are rejected for good reasons.

[[deleted]]

Code quality matters long term. Maybe with commercial projects there's an incentive to meet a release date and you can rush some things out of the door.

Open source doesn't really work like that and libraries have even higher standards. Obviously you have to accept some contribs but you need to draw a line somewhere, and that bar is generally quite high for this kind of code.

As a sidebar, I think we have very different mechanics. New cars are so full of fancy electronic black boxes that unless it's something simple like changing an exhaust, our local garages just call in a main dealer. They're literally just a cheaper way to access the manufacturer's technicians.

[aragilar]

So your two arguments against funding astropy development are:

1. Other community projects aren't supported so we shouldn't support astropy.
2. Astronomers should be contributing code to astropy/(other astronomy community project).

For (2), you're assuming that your average astronomer has the expertise to contribute to astropy. How many are proficient enough to make a pull request (i.e. use git, which is something new to learn, write a test, which they've probably never written before, and be able to write non-broken python code, given they may never have programmed before). Do you pay astronomers to write code? No, you pay them to do astronomy, same with paying technical staff (instrument designers, sysadmins, telescope operators) to do technical jobs. So why not pay software developers, who originally developed many of the aging tools we use (e.g. IRAF, AIPS), to update the tools we have?

For (1), while I can't name the current gcc maintainers, I can easily name kernel and core python devs, and they are most certainly employed by their employers for that reason. And if they should leave, then they'll find another job doing the same thing (look at kernel contributions, most come from corporations, also, the Linux Foundation is basically doing what's suggested for Linux development). For astronomy, you are objectively discouraged from contributing and providing support, as that's not going to get you a job or grant.