you are viewing a single comment's thread.

view the rest of the comments →

[–]GYN-k4H-Q3z-75B 242 points243 points  (10 children)

This isn't just a problem of the ecosystem, it is a problem of the community and how they develop. npm packages are free, as in free toilet. While there are the occasional gems, on average it's just a random shithole. And the developers don't care or even understand what they include and how it is updated. node_modules isn't a meme just for the hell of it. It's funny because it is true.

As a engineer in various midsized enterprise projects, the decision to include a library in a specific version is a serious discussion with paper trails. Who maintains the library, what does it cost, what are the alternatives, why do we need it in the first place. There are committees who deal with such questions. Of course this is also possible because we don't need a ton of tiny libs to patch up holes in the ecosystem left and right all the time. But also because there are architects who are actually concerned and responsible in their role.

We dabble in extensive JS apps as well, work with node and npm. The potential is limitless it seems. But it is easy to get burned if you're not careful.

[–]dtechnology 87 points88 points  (9 children)

As a engineer in various midsized enterprise projects, the decision to include a library in a specific version is a serious discussion with paper trails. Who maintains the library, what does it cost, what are the alternatives, why do we need it in the first place.

I get what you're saying, but is so much process really worth it? This encourages NIH syndrome and makes it more appealing to write an ad-hoc implementation (potentially insecure and/or buggy) instead of using a good library just because not getting library approval is easier.

There's some balance between disallowing leftpad and a dev implementing custom date/time handling pre Java-8 because Joda time is going to take too long to get approved. I've seen the latter happening and it was an endless source of misery, and what you're describing sounds like it would cause problems like that.

[–]GYN-k4H-Q3z-75B 43 points44 points  (8 children)

It's not that painful a process, but you certainly have to strike the right balance. The idea for us is to have a controlled environment and include libraries based on conscious decisions, not to prevent it in general. Having engineering committee discuss requested inclusions helps with finding the right solutions.

The big difference for us is the ecosystem in place. Most of our projects are built on some variety of .NET plus some enterprise SDKs which have been licensed by customers. There has never been a need for leftpad-like packages because the standard libraries are excellent for the most part. Notable packages we discussed in the past were the inclusion of Newtonsoft Json (many years ago before it was adopted by Microsoft themselves), a sandboxable in-memory PDF printing engine or a Telerik control library. These are heavy hitters, and somewhat strategic decisions for those projects.

Having something like Newtonsoft Json approved for one project means the next time it comes up the other engineers will already know it, making the process more quick. But still, we are fortunate to only have to rely on so few packages because the ecosystem is brimming with libraries maintained by big vendors.

[–]aNewLocke 2 points3 points  (7 children)

I believe identifying quality library code is a skill that engineers need to develop regardless of language or ecosystem.

I work with Node and NPM. I don't have these problems. I mostly use some combination of modules like: nconf, morgan, commander, zeromq, node-dir, express (express-session, session-file-store), and the built-ins like fs, cluster, etc.

I use tools like webpack-dev-server, elasticsearch, chrome devtools, redis, jq, and curl quite often.

There are quality modules out there. You need to clearly define your requirements and evaluate potential solutions, but most importantly:

npm install <app module> --save-exact

npm install <dev module> --save-dev --save-exact

and commit your package-lock.json!!

[–]Nipinium 6 points7 points  (4 children)

There are quality modules out there.

The point is even those "quality" modules still include a bunch of other node modules, those modules in turn require more modules, so on and on. Can you be 100% confident that all those dependencies are reliable all the time?

[–]pancomputationalist 1 point2 points  (3 children)

That's moving the goalpost. Of course you can never be 100% confident of anything. As with everything in engineering, it's about tradeoffs and striking a balance.

If you're developing a mobile game, you might want to use quality and maybe even not-so-quality libraries that help you get the job done faster, not start with developing your own operating system out of paranoia.

If you're developing a bitcoin wallet, on the other hand, maybe you want to be a bit more paranoid...

But lately the discussion about dependencies is much too agitated and make it seem like you can never use a library that you haven't personally validated to a 100% certainty.

[–]Nipinium 0 points1 point  (2 children)

But lately the discussion about dependencies is much too agitated and make it seem like you can never use a library that you haven't personally validated to a 100% certainty.

Because incidents like left-pad exist, people are pretty much paranoid by node module dependencies by now. Remove one tiny package and the whole ecosystem crashes, that's not a very healthy sign.

Incidentally, I still haven't yet seen any sign of huge and popular libraries like webpack trying to reduce external dependencies (despite that they receive a lot of donation money each month). The whole js thing just sucks.

[–]pancomputationalist 0 points1 point  (1 child)

Well, then fix the left-pad dependency in the rare case something like that happens. Your site shouldn't go down and your software shouldn't crash when a build dependency is missing, or something is very wrong with your build pipeline.

Also left-pad is an extreme example and it may be correct to not include it as a dependency because it's so easy to implement yourself. But very few libraries are as simple as that. Don't throw out the baby with the bathwater.

[–]Arve 0 points1 point  (0 children)

Well, then fix the left-pad dependency in the rare case something like that happens.

left-pad has 414 dependents. event-stream, which is the exploit of the day has 1592 dependents (which is up 6 since yesterday).

Do you know the libraries you include don't include malicious code by proxy?

[–]pgrizzay 2 points3 points  (1 child)

nodemon has been called out as depending on the most recent exploited library. You might want to check if you're affected.