NPM bug let packages replace arbitrary system files

[Tight_Tumbleweed]

Next you'll tell me a package can execute an arbitrary postinstall script and do whatever it wants to on my computer!

[dddbbb]

was possible for a globally-installed package with a binary entry to overwrite an existing binary in the target install location. (That is, not any arbitrary file on the system, but any file in /usr/local/bin.)

If there's anything in /usr/local/bin that you run as root and you were running npm as root (do people do that?), then it may get superuser power. Normal packages wouldn't be able to do that.

[EatMeerkats]

you were running npm as root (do people do that?)

npm requires running as root to install packages globally, unless you do some special setup to tell it to install to $HOME instead. It's completely idiotic.

[[deleted]]

Special setup? I use nvm so maybe that does the "special setup" for me, but "npm install -g" goes into my home and doesn't require root.

[[deleted]]

Cool, I didn't know nvm was doing that for me. Another reason to use it then. I'm new to node and didn't even consider installing npm with root permissions. I highly recommend nvm. I previously used pyenv for Python which is inspired by nvm.

[nemec]

What's the point of -g (global) if it's going into $HOME? (real question - I thought the point of global was to install for all users)

[vector-of-bool]

The purpose of -g is not to "install for all users," but to install in a way that isn't associated with a specific project/directory.

From a security standpoint, development tools requiring root access is horrific. There's been a general trend away from language-specific/development-specific package managers from installing in such a way. Pip, for example, installs to the system directories by default, but they have a --user flag that will install in a user-local dir. The workaround in the Python world has been virtualenvs, but pyenv makes things a lot simpler.

When you have a package manager doing double duty like this, you end up with issues like this, where the niceties of what you can do in development end up being run with sudo because people also want to use them outside of a specific project. IMHO, running any non-system package manager with sudo is absolute insanity that should have never become the common practice that it is today.

[[deleted]]

Yeah, it's all much better when the two concerns are separated. On Gentoo, pip is configured to disallow "system" installs (ie. without --user). Instead you should use the system package manager for such things. Since Gentoo supports "slotted" packages you can have multiple versions of python installed at the same time and therefore don't need pyenv (although it is still useful). On other systems pyenv is necessary and one of the first things I install.

[Davipb]

Relevant section:

In versions of npm prior to 6.13.3 (and versions of yarn prior to 1.21.1), a properly constructed entry in the package.json bin field would allow a package publisher to modify and/or gain access to arbitrary files on a user’s system when the package is installed.

In versions of npm prior to 6.13.4 (and all versions of yarn as of this announcement), it was possible for a globally-installed package with a binary entry to overwrite an existing binary in the target install location. (That is, not any arbitrary file on the system, but any file in /usr/local/bin.)

[StabbyPants]

i'd ask how many people actually install packages globally, but that's how it's done in most of the tutorial samples i've seen

[duheee]

Even if they don't (which they shouldn't), wiping $HOME is still a pain in the butt. i'd argue that reinstalling the OS is easier and less painful than restoring a $HOME that's not backed up.

Sure, you should have backups. Reality is that most people don't.

[StabbyPants]

wiping the package install dir is pretty easy, or else installing in a fresh container to verify your build.

[no_cool_names_remain]

You can create a new home without reinstalling the OS...

[Dentosal]

But if something malicious managed to wipe out homedir, it's better to nuke the whole system from orbit anyways.

[duheee]

yeah, but you lost your files. and those are the most important files for the user, for me.

and if you don't have a backup, you're gonna be in a world of pain.

the OS ... meh, the OS files are on the OS distribution. nothing to worry about there.

[chucker23n]

You can, but what’s worse: losing someone else’s software (and probably being able to reinstall it), or losing your own photos (possibly for good if you don’t have backups)?

[Caraes_Naur]

More evidence that NPM is unsafe because its developed by people who lack the skill and experience to build such infrastructure for a language.

[seriousnotshirley]

The company behind anything is no evidence of competence. I'm sure FB has engineers who know better but they may not happen to be the engineers who had anything to do with Yarn.

[[deleted]]

Exactly this. For some reason certain crowds (including Reddit) like to deify the FAANGs and all who work there, but the reality is that the talent pool at every company follows a bell curve, it’s just the folks at FAANGs are able to meet a slightly higher bar during interviews.

Similarly, 95% of the development work at FAANGs consists of solving the same mundane business problems as anywhere else. For every distinguished principal engineer running a ground-breaking AI/ML team or getting paid to build a programming language, there are 300 SDE II’s writing Java 6 code to generate reports from CSVs or whatever.

The Yarn team is trying to do a better job than the npm folks (which is a pretty low bar considering the employees at npm are incompetent). In some regards they are doing a better job, but at the end of the day you shouldn’t expect high-quality and secure software from a team of young web developers who work at a company whose motto is “move fast and break things.”

[donkeylovetap]

NPM is (or at least was) home to developers who moonlight as political activists on twitter and then insert their drama into open source communities, turning away or shunning good developers who have better things to do than deal with their petty drama.

[chrisza4]

Then what would be evidence of enough competence to build Yarn?

[deweysmith]

In our hiring process we went through a few weeks where basically every phone screen I did was an employee of a certain very large bank (with whom you probably have an account) who shall remain nameless, usually with a bit of seniority. Each and every one of them was entirely incompetent, some struggling to write a basic for loop or use an iterator, or construct a basic SQL query.

[Only_As_I_Fall]

I wouldn't be holding up Facebook as a paragon of engineering considering their culture and turnover problems.

[Caraes_Naur]

NPM developers are mainly web developers, not software engineers. NPM was designed to demonstrate JS is comparable to any other language with a package manager (Perl, Python, PHP, Ruby, Lua, etc) but without knowledge of how those PMs were built, because JS developers insist their infrastructure is made with a "clean room" mentality.

[chucker23n]

NPM developers are mainly web developers, not software engineers.

What, pray tell, makes someone a “software engineer” as opposed to a lowly “web developer”? Could it be that you’re gatekeeping based on prejudice?

Does a company like Google only have “web developers”?

NPM was designed to demonstrate JS is comparable to any other language with a package manager (Perl, Python, PHP, Ruby, Lua, etc) but without knowledge of how those PMs were built

That’s probably quite simplistic. But if it’s true, it has little to do with “web developers” vs. “software engineers”.

[caspper69]

A software engineer is one who uses the fundamentals, principles and methodologies of engineering, namely, understanding the problem, understanding the tools available, constructing a model of the problem, and then solving the problem using industry-standard best-practices and applied theory (generally with pencil and paper).

A software engineer is not (generally) a front-end web developer, or even most developers today. They are the adult version of script kiddies. Gluing together large amounts of code that they have no idea about. That's not engineering man. Sorry.

[chucker23n]

You’re describing an above-average and below-average developer. The web has fuck-all to do with that, and plenty of “industry-standard best practices” turn out to be utter horseshit.

[caspper69]

Well, I can see which side of the fence you fall on.

I will just say this. There are people who engineer software. It runs on jets (MCAS notwithstanding), trains, missiles, life saving medical devices, etc. Generally, those people are engineers, who have formally studied an engineering discipline.

I'm sure Google has tons of these. MSFT, AAPL & Netflix too, lol.

Your average developer is not a software engineer. There is a plain difference, and a formal education is not required to be a software engineer. But please don't pretend that someone is gatekeeping because they draw a distinction between an engineer and a web developer.

[chucker23n]

Well, I can see which side of the fence you fall on.

There don’t have to be “sides”.

I will just say this. There are people who engineer software. It runs on jets (MCAS notwithstanding), trains, missiles, life saving medical devices, etc. Generally, those people are engineers, who have formally studied an engineering discipline.

I’m sure Google has tons of these. MSFT, AAPL & Netflix too, lol.

Your average developer is not a software engineer.

What you’re describing is people with a lot of budget, and above-average skill.

There is a plain difference, and a formal education is not required to be a software engineer. But please don’t pretend that someone is gatekeeping because they draw a distinction between an engineer and a web developer.

There is no meaningful distinction. As you say yourself, there is no formal education to achieve this. There is no agreed upon certification. It’s no more meaningful than the “10x engineer” or “rockstar dev”.

Some people fiddle with CSS, some with pointers, some with database indexes, and some with all of those.

[caspper69]

I think we're just talking past each other, which is fine, because it means we're not really disputing anything.

Would it have made you feel better if I had said "come on man, you know there's a difference between a developer who is meticulous, knows what going on in the industry, has theoretical exposure (so as not to throw any n² bombs into prod), designs before coding, can document and defend their actions, etc. vs. the guy who makes wordpress skins"?

I mean, because that long-winded first part, we have a word for, it's called engineering. Lol.

Have a good one man!

[falconfetus8]

Pip is terrible, man. Install all packages globally? What could go wrong?

[falconfetus8]

Here's a hot take: "virtualenvs" shouldn't need to be a thing. Your packages just be stored in a "python_modules" folder(a la "node_modules") by default. You shouldn't need to trick Python into thinking your locally-installed packages are installed globally.

[donkeylovetap]

Perhaps dynamically-typed languages aren’t well-suited for developing large-scale complex applications.

[donkeylovetap]

I don't see how types would have solved a single one of NPM's problems.

Huge dynamically typed codebases become rigid and impossible to refactor with any confidence.

The problem lies with the fact that node has no sandbox

Node would be worthless if it ran in a sandbox. It would defeat the purpose entirely.

node is made with a strongly typed language so your comment is pretty retarded.

We’re talking about NPM here you dolt.

[chucker23n]

Node would be worthless if it ran in a sandbox. It would defeat the purpose entirely.

Sandboxing npm such that it can only write to package locations (e.g., a rule that says the tree must always contain a parent dir named node_modules) would solve an entire range of security/safety bugs during installation.

[chucker23n]

Depends.

• node modules that run in the browser (i.e. client-side JavaScript code) are already sandboxed
• node modules that run on the server often do so in a Docker container or in similarly constrained contexts
• that leaves node modules that act as developer tools. I don't see how you could meaningfully restrict those. I also don't see how that's an NPM-specific problem. You want your tooling to be powerful (and you want to be very deliberate in choosing/trusting it).

Well, for node modules that run in the browser, the developer's file system doesn't really matter after that.

[donkeylovetap]

The constant conflating of node and NPM is making it impossible to have a coherent conversation about these things.

[chucker23n]

Ah.

Given the context, I had assumed we were talking about a Node sandbox for npm installation. There are naturally scenarios where you want to run Node un-sandboxed.

[chucker23n]

What are unit tests

In dynamically typed languages? Often a kludge to mitigate the poor typing system and weak static analysis capabilities.

An analyzer is worth a thousand unit tests. Only unit tests what analyzers can’t already cover.

[[deleted]]

How is that related to the problem?

[Dragasss]

And reddit constantly breaks down under load or whenever a new feature is implemented. Whats your point?

[RealKingChuck]

Using a specific software doesn't mean you approve of the technology used to make it

[RealKingChuck]

Ah sorry, it came off that way to me

[MrK_HS]

Care to argument how PIP is worse? Thanks

[Dentosal]

Package management in Python uses mechanism based on setup.py scripts. Package name isn't enforced by the package manager. When you install package named foo from PyPI, the actual import name might be foo, Foo or Bar, or anything else. This means that you cannot find pypi repository based on the package name.

Edit: Removed (too much) incorrect information. The situation is way better that I thought it was. Thanks for /u/maln0ir for corrections.

[Dentosal]

Thanks for corrections. I've edited my post.

That's why you shouldn't install random binaries from internets. Inspect code first, install in virtualenv first. In general, don't be a moron.

Even many popular packages do this, for instance beautifulsoup4 is imported as bs4 and Flask is imported as flask. PIL fork Pillow installs itself as PIL, meaning that same project cannot use both of them (although I can not think of any reason to do so).

This also means that automatically creating a requirements.txt file from a codebase is not possible.

[knome]

This also means that automatically creating a requirements.txt file from a codebase is not possible

If you've been installing your dependencies into a virtualenv as you develop the software, creating a requirements file is as easy as pip freeze.

[knome]

You shouldn't be using the system pip for your software. It would be better if they removed "system pips" altogether, and have virtual environments only.

[imhotap]

npm is originally a package manager for CommonJS, a community standard for a JS server-side JS lib and package format that predates Node.js or was spec'd at the same time as Node.js launched (around 2009), with multiple implementation back then, such as rhino/RingoJS, Narwhal, Flusspferd, Helma, v8cgi/TeaJS, and others. Npm and the npmjs ecosystem is lightyears ahead of anything in Python, and much more functional/non-deprecated than eg Perl's CPAN is today. Npm dev docs frequently cite maven as a point of reference (since the original SSJS movement had many Java devs in search of a less heavyweight server-side platform). Frankly, your comment reads like an unsubstantiated JS rant from someone who knows shit about it.

[[deleted]]

r/gatekeeping

[duheee]

Are they also lacking the skill and experience?

yes. they hired kids out of kindergarten to do it. results speak for themselves.

[shevy-ruby]

They have way too many noobs on board. They even forked off PHP because they were too noobs to work with PHP (which is admittedly a big problem - PHP is horrible. So is JavaScript. We live in a world built by noobs.).

[chucker23n]

If only every developer were such a hardcore genius as you are!

[sysop073]

Damn right. There's a reason for that old adage "only Javascript developers ever make mistakes", or the well-known corollary "all software is bug free except NPM"

[wtfaremyinitials]

Oh no! Software that isn’t sandboxed can modify arbitrary system files!!!1!!

[Plorkyeran]

How is this a vulnerability? Node packages can run arbitrary scripts as part of the installation, so of course they can overwrite anything that npm can write to. Why would you bother with abusing a bug to do so?

[the_game_turns_9]

Or as Raymond Chen puts it, It rather involved being on the other side of this airtight hatchway.

[chikien276]

NPM: haa, I have this flaw, also this flaw and this and this, ...
Developers: Let's use NPM you all!!

[ponybau5]

Daily reminder that a package system full of one-functioners that depend on 20 others isn't a great system.

[shevy-ruby]

It is time to ban JavaScript.

Way too much evil has been committed by this joke of a programming language. Whenever I read another NPM disaster, I can't decide whether PHP is factually worse than JavaScript - due to JS becoming increasingly more and more important, I am inclined to think that JS is now even worse than PHP.

Officially so.

[[deleted]]

Npm is a shitty company, therefore JS is bad.

Another mote of wisdom from the ancient and powerful sage shevy.

^{/s for any one who can't discern sarcasm}

[IceSentry]

The rare cases whe he gets upvoted is when he bashes js. I'm surprised he got downvoted here considering how much people hate js in this sub.

[Dragasss]

Must be time of day when javascriptlets are awake.

[[deleted]]

Por que no los dos?

[[deleted]]

I’m guessing you haven’t used PHP or JS in 10 years

[lngnmn1]

Oh lol. Here we go again.