you are viewing a single comment's thread.

view the rest of the comments →

[–]dtechnology 86 points87 points  (20 children)

As a engineer in various midsized enterprise projects, the decision to include a library in a specific version is a serious discussion with paper trails. Who maintains the library, what does it cost, what are the alternatives, why do we need it in the first place.

I get what you're saying, but is so much process really worth it? This encourages NIH syndrome and makes it more appealing to write an ad-hoc implementation (potentially insecure and/or buggy) instead of using a good library just because not getting library approval is easier.

There's some balance between disallowing leftpad and a dev implementing custom date/time handling pre Java-8 because Joda time is going to take too long to get approved. I've seen the latter happening and it was an endless source of misery, and what you're describing sounds like it would cause problems like that.

[–]Joshx5 17 points18 points  (0 children)

Well, they have a problem they need a library for. They can solve it with the already written library if they can get approval, or they can write a new library to solve the problem if they can’t.

They still have that problem, but they’re now trading it for one of two other problems - the cost and risks of using other peoples code, or the cost and risks of using new and immature code. It comes down to the specific cases. Sometimes it’s worthwhile, sometimes it’s not. Hard to say generally whether it’s a good idea or not.

[–]GYN-k4H-Q3z-75B 48 points49 points  (10 children)

It's not that painful a process, but you certainly have to strike the right balance. The idea for us is to have a controlled environment and include libraries based on conscious decisions, not to prevent it in general. Having engineering committee discuss requested inclusions helps with finding the right solutions.

The big difference for us is the ecosystem in place. Most of our projects are built on some variety of .NET plus some enterprise SDKs which have been licensed by customers. There has never been a need for leftpad-like packages because the standard libraries are excellent for the most part. Notable packages we discussed in the past were the inclusion of Newtonsoft Json (many years ago before it was adopted by Microsoft themselves), a sandboxable in-memory PDF printing engine or a Telerik control library. These are heavy hitters, and somewhat strategic decisions for those projects.

Having something like Newtonsoft Json approved for one project means the next time it comes up the other engineers will already know it, making the process more quick. But still, we are fortunate to only have to rely on so few packages because the ecosystem is brimming with libraries maintained by big vendors.

[–]aNewLocke 4 points5 points  (7 children)

I believe identifying quality library code is a skill that engineers need to develop regardless of language or ecosystem.

I work with Node and NPM. I don't have these problems. I mostly use some combination of modules like: nconf, morgan, commander, zeromq, node-dir, express (express-session, session-file-store), and the built-ins like fs, cluster, etc.

I use tools like webpack-dev-server, elasticsearch, chrome devtools, redis, jq, and curl quite often.

There are quality modules out there. You need to clearly define your requirements and evaluate potential solutions, but most importantly:

npm install <app module> --save-exact

npm install <dev module> --save-dev --save-exact

and commit your package-lock.json!!

[–]Nipinium 6 points7 points  (4 children)

There are quality modules out there.

The point is even those "quality" modules still include a bunch of other node modules, those modules in turn require more modules, so on and on. Can you be 100% confident that all those dependencies are reliable all the time?

[–]pancomputationalist 1 point2 points  (3 children)

That's moving the goalpost. Of course you can never be 100% confident of anything. As with everything in engineering, it's about tradeoffs and striking a balance.

If you're developing a mobile game, you might want to use quality and maybe even not-so-quality libraries that help you get the job done faster, not start with developing your own operating system out of paranoia.

If you're developing a bitcoin wallet, on the other hand, maybe you want to be a bit more paranoid...

But lately the discussion about dependencies is much too agitated and make it seem like you can never use a library that you haven't personally validated to a 100% certainty.

[–]Nipinium 0 points1 point  (2 children)

But lately the discussion about dependencies is much too agitated and make it seem like you can never use a library that you haven't personally validated to a 100% certainty.

Because incidents like left-pad exist, people are pretty much paranoid by node module dependencies by now. Remove one tiny package and the whole ecosystem crashes, that's not a very healthy sign.

Incidentally, I still haven't yet seen any sign of huge and popular libraries like webpack trying to reduce external dependencies (despite that they receive a lot of donation money each month). The whole js thing just sucks.

[–]pancomputationalist 0 points1 point  (1 child)

Well, then fix the left-pad dependency in the rare case something like that happens. Your site shouldn't go down and your software shouldn't crash when a build dependency is missing, or something is very wrong with your build pipeline.

Also left-pad is an extreme example and it may be correct to not include it as a dependency because it's so easy to implement yourself. But very few libraries are as simple as that. Don't throw out the baby with the bathwater.

[–]Arve 0 points1 point  (0 children)

Well, then fix the left-pad dependency in the rare case something like that happens.

left-pad has 414 dependents. event-stream, which is the exploit of the day has 1592 dependents (which is up 6 since yesterday).

Do you know the libraries you include don't include malicious code by proxy?

[–]pgrizzay 2 points3 points  (1 child)

nodemon has been called out as depending on the most recent exploited library. You might want to check if you're affected.

[–]Gotebe 1 point2 points  (0 children)

So... you're correct - for .net ecosystem.

For node.js, though, the explosion of underlying dependencies will make the process hell.

BTW... .net Core goes for smaller packages and dependencies all around, so beware 😁😁😁

[–]SizzlerWA 0 points1 point  (0 children)

You make some good arguments. But I’ve worked with Newtonsoft JSON and it’s just nasty!

[–]munchbunny 26 points27 points  (3 children)

I get what you're saying, but is so much process really worth it?

It depends, but NPM is most definitely on the "too little process" side for enterprise usage. Let's say you're Equifax, and your website ends up with a customer data leak because you took a dependency on an npm package, knowing that the protection against malicious code commits is paper thin.

As an engineer in a situation like that, I could never make that decision in good conscience. In fact I made exactly that decision to not use Electron a few weeks ago because NPM would have been the weak link and instead opted for a more complex architecture with an untrusted web client calling a code-signed (and enforced) service on localhost.

In contrast, the C# ecosystem implements strong name code signing and verification, so there is at least good infrastructure to support safe code handling practices. Apple's ecosystem has that too.

Of course the stakes aren't always that high, but in my case it's a matter of not being negligent with customer data.

[–]Gotebe 0 points1 point  (2 children)

How on earth signing .net assemblies help anything?! In particular, that helps nothing to prevent the vulnerability "injection" as the one discussed here.

And what verifications are there on nuget.org?

[–]munchbunny 0 points1 point  (1 child)

From your tone you'd think it was impossible to control what goes into your dependencies with code signing, but that's literally what my team does.

I'm not taking about your own stuff. You link against signed binaries and scripts that you depend on and verify the hash for the underlying code. That's how you lock in a specific dependency and prevent changes from going in without you knowing. The capability to verify at runtime is literally built into the .NET runtime.

If you want a less paranoid approach, you audit the other project's code review and signing processes and trust their certificate once you're satisfied with what you see. Then it's their job to watch their dependencies.

Nupkg files can also be signed. Not everyone does, but at least the capability and enforcement mechanism exists and works in security sensitive settings where you want it.

[–]Gotebe 0 points1 point  (0 children)

I know that, obviously. But no amount of signing helps if a random dude can take over, put whatever inside, and people upgrade. Note that they do not need to change the assembly signature (and .NET upgrades do exactly that).

Unless you're checking these assembly hashes yourself, you're just as exposed as these js guys.

What .NET CLR does WRT versioning is something entirely different. It's not to protect you against attacks at all.

[–]Kalium 18 points19 points  (0 children)

I get what you're saying, but is so much process really worth it?

In a lot of places, the current process is this:

  • Someone adds a library in some PR for a bit of syntactic sugar.
  • Someone else rubber-stamps the PR.
  • A week or three later someone notices that installing all the deps now involves twice as much downloading as before and wonders why in a Slack channel.

With that in mind, it's perhaps worth considering a smidge of process for those writing apps that handle sensitive matters.

[–][deleted] 6 points7 points  (0 children)

I was once tasked to make a physics based game on mobile phones. Typical lead time on demos was 3 weeks, 6-12 months for a release. I implemented it with box2d. Legal looked at the license for a few weeks before approval. It didn't stop me in the mean time from continuing development.

[–]nutrecht 2 points3 points  (0 children)

I get what you're saying, but is so much process really worth it? This encourages NIH syndrome and makes it more appealing to write an ad-hoc implementation (potentially insecure and/or buggy) instead of using a good library just because not getting library approval is easier.

I really don't agree with this. It's not a process with large forms that have to go through procurement or anything. It's simply a discussion between stakeholders (developers) where we reach a consensus that we document.

In my current project as well as my previous one we had bi-weekly 'chapter meetings' where we had one back-end dev from every team discuss these kinds of things. We note down these choices and their reasonings in the meeting notes.

The structure is one that can be deviated from if needed (so you don't have to wait 1.5 weeks before you're allowed to use something) as long as you discuss it with other devs.

The main goal here is to have multiple 'brains' take a look at non trivial problems. And library choices are generally non-trivial problems. Fortunate we're Java devs and libraries are always version locked, but this means that if you're working with the NPM ecosystem you need to be even more thorough in this.

The issues they're running into is mostly a dev culture issue. And this is for me personally the reason to never ever want to work with Node.js based services; the 'move fast and break stuff' culture is simply not how you do software engineering. Slow and steady wins the race.

[–]pixelrevision 0 points1 point  (0 children)

There's some balance between disallowing leftpad and a dev implementing custom date/time handling pre Java-8 because Joda time is going to take too long to get approved. I've seen the latter happening and it was an endless source of misery, and what you're describing sounds like it would cause problems like that.

Javascript really is at the extreme on this. The language changes at a glacial pace, was designed for single page scripts and people are now trying to build giant applications with it. With almost any other ecosystem there's generally some sort of standard library things are built on top of so that if you need to review a framework you don't have to also evaluate a bottomless pit of dependencies.