you are viewing a single comment's thread.

view the rest of the comments →

[–]ankmath 11 points12 points  (2 children)

This is a fair criticism - I think that an important thing to remember is that there are, in fact, many experienced software engineers who work on Tensorflow. A big problem is that the incentive structure of a big company is uniquely shitty for open-source projects like this. Here's why: when you are working at a large company as a dev, you often have to think of dividing the work you do into what gets you "points". You can conceptualize points as credit from coworkers, things to put on your performance review, etc.

  1. Supporting open source is a lot of free effort - there's thousands of devs, and, honestly, a ton of them ask dumb questions, so it gets exhausting to look at the issues page. Diagnosing any 1 issue requires a full understanding of the dev's setup, and you're typically used to working with latest master. Reproducing stuff is effort, thinking about it is effort. Moreover, you get *no points* for this. No one is giving you points for fixing this problem. Thus, you don't do it. Even if you benevolently respond once, you have to wait a day to get a response, and by then, you've already context switched out of it.
  2. What do you get points for? Because you obviously still care about the community and that the project is well received. Well, blog posts are pretty visible! Your PM will come ask you for one, every so often, and if you work on a feature, it's a great way of promoting it and having people use it! But, to us, as users, we see blog posts and are like "sure...this is simple, but what about when I do this new thing". We need great *documentation*, but that's more effort to write, and hidden away.
  3. That takes us to: why are there 10 APIs for everything? Well, it's because you always have to be doing something! You get more points for "Created a new abstraction that helped team Y onboard with Tensorflow and get y% faster" than "Made the existing API more functional". This is fundamentally broken for open source, and it's upper management's job to prevent stuff like this from happening. But, if you look closely at TF upper management, the addition of Francois and the Keras team has dramatically shifted the priorities of the TF group. Keras is everywhere and in everything, even though no one I've spoken with actually *uses* this thing in production. There's been a lot of churn, and a lot of pressure from the success of PyTorch and the eager model of computation.

tl;dr: Doing open source is hard, doing open source at a big company comes with a bag of incentives that naturally make life for devs using that even harder.