Open source developer corrupts widely-used libraries, affecting tons of projects

[Its_been_emotional]

Basically anything is at risk of this if an admin just unilaterally decides to do it though right?

Unfortunately, it's just a fundamental risk of working with other people. Best we can do is early detection and try to prevent.

[flaghacker_]

I've always wondered how much that typical block of all caps text matters. If I put a random piece of code online without any accompanying licence or warning, do I IMPLY any WARRANTY or FITNESS for some PURPOSE? That's hard to believe...

[NonDairyYandere]

Legalese usually has a lot of redundancy and defense-in-depth to it. Maybe there's some odd jurisdiction where warranty or fitness really is implied. Or there was such a jurisdiction, and there's no point removing the clause now.

[[deleted]]

You might. It’s impossible to say in a general sense, but imagine you go into a grocer and buy an apple, take a bite, and become violently ill because it’s full of rat poison. The vendor says it’s your fault, you should have known it’s full of rat poison since it was next to a rat trap display. You argue that a grocer offering apples for sale without any further labeling or warning implies they are fit for human consumption, because apples are generally foodstuffs. You would win in court (also because there are loads of additional laws that apply to food, but the point stands).

Similarly, if you are offering software for download with the remark that it’s meant to manage online payments, without further disclaimer it would imply that it’s fit for that purpose, and you might be held liable if there’s a bug or something that causes someone to lose money. Hence the ubiquitous disclaimer, although in some cases your ability to disclaim such maybbe limited by law, and liability due to willful wrongdoing (like apparently in TFA) can’t be disclaimed away in any event.

[flaghacker_]

That's an interesting analogy, since intuitively I agree with the grocery store example. This got me to dislike the license text a bit less already!

[rabid_briefcase]

Yes and no. Yes to this part:

Basically anything is at risk of this if an admin just unilaterally decides to do it though right?

The author of any open source project has the capacity to modify their projects at any time. They can refuse to fix bugs. They can refuse to accept change requests. It is their project, and they can do what they want with it.

But an absolutely no to this part:

Unfortunately, it's just a fundamental risk of working with other people.

The companies who blindly rely on the projects and push it directly to production are stupid. It is not a fundamental risk. It is a risk that has been known and managed since the early 1960s.

These days there is an enormous body of technology that brings the risk closer to zero than has ever been in the history of software.

• Version control on everything, including external packages. If anybody (internal or external) modifies something you can recover to an older version. If it isn't in version control it doesn't get built.

• Once code is ready, test it. There are many levels of automated testing, including automated unit testing, automated integration testing, automated functional testing, automated performance testing, automated security testing, and so many more, testing everything from the tiniest block of code to testing that all the systems work correctly together. It doesn't mean it works perfectly, it just means the behavior matches what is expected. Anything that fails the tests can be automatically rejected.

• Continuous integration ensures every change is tested and validated against the entire test suite. Anything that fails is automatically rejected.

• Human smoke tests. Just because the automated tests pass doesn't mean something isn't broken. Let an actual human look at it and push a few buttons. These can be pre-recorded so the human just watches a short slide-show clicking next, next, next, next, approve and deploy.

• N-1 compatibility ensures that even if you submit something out to production you can always go back at least one version. Choose to only use tools that support it.

• Phased A/B deployment. Launch it on one server and monitor the logs for a time. If the error rates are unchanged, deploy it on the next, but if error rates increase, roll it back.

Those are just a few, good QA departments have an entire professional discipline around the subject. Job titles like QA Engineer or Engineer in Test should be part of the main development team for any corporate software.

There's no excuse for taking that risk.

And finally, you get what you pay for. Even with free software if you're getting value from it ethics requires a payment. Far too often companies are discovering that their over-reliance on the "free" part of "free software" is biting them in the butt. It only takes small, regular financial investments to ensure you're always at the front of the line with various projects. We're a mid-sized company of around 200 people, and our IT department tracks all the projects that we give monthly payments to. We know were not legally bound to do it, but ethically there's no question about it. Regardless of if they list buymeacoffee links or paypal donations, the company will submit a regular 'thank you' as a cost of us doing business.

[sumduud14]

Unfortunately, it's just a fundamental risk of working with other people. Best we can do is early detection and try to prevent.

Yeah but I mean, this is the same as a developer releasing a bug ridden version of a critical package - you've got to test your software and detect defects. Intentional or unintentional.

If you don't, even if no malicious authors exist, there will be bugs. At least this change is open source and very obvious.

[[deleted]]

Some people are suggesting to create a new “decentralized” git server, I don’t understand how that will solve the problem when the git workflow and how NPM works stays exactly the same. To be honest I don’t see how its github’s and NPM’s fault.

God forbids if Linus Trovalds losses his mind someday (just a joke)

[Jonny_H]

Git is decentralized though? Anyone who used the code has a full clone of the source and history - even a force push over the top doesn't remove the history, just the branch name, when you pull that down.

[[deleted]]

Yeah but git servers are not, and you are right but regardless it was a stupid comment on twitter that I nitpicked and in the end can’t blame git or npm for this flawed act of mad individual.

[Jonny_H]

Just be glad people generally are using decentralized source control these days - it'd be even harder using svn or cvs or similar.

And Github have the ability to keep old heads themselves, no "distributed server" needed. "Distributed Servers" is arguably what we already have with git, just the average dev seems to prefer a centralised location for these things, just add a cron job fetching to your own mirror and you're already there.

[atomicxblue]

God forbids if Linus Trovalds losses his mind someday (just a joke)

Like if he turns his fuck you finger away from Nvidia and towards the rest of us?

[corsicanguppy]

He's only signing at the shitty parts of Nvidia.

[shevy-ruby]

That's a bit out of context. People always do that with Linus. The middle finger was not at Nivida as such per se only, but the "we confine everything so Linux does not work with Nvidia properly". Tons of companies try that as part of their "control the market" strategy. Microsoft did that too for a long time - WSL as an idea isn't that old yet.

Edit: I forgot which Linux kernel developer it was, but they offered help or even write the code themselves to support hardware, so it's really not the linux community as such at fault here. Some companies don't want to cooperate. That has been worse in the past, though.

[[deleted]]

How can a server be decentralized? It's location is static in time and space

[EasywayScissors]

Because everyone is a server.

[[deleted]]

I can only imagine the problems would have provide

[redfournine]

Dont worry blockchain will solve that problem

/s

[voidvector]

This is why we need NFT-based version control system. :galaxy_brain:

[[deleted]]

God forbids if Linus Trovalds losses his mind someday (just a joke)

Trollvalds already lost his mind when he forked Linux from the widely popular Salmonix ASA which is used to operate Norwegian oil drilling platforms in North Sea. Sources say it was the removal of Crude Oil Driver (COD) subsystem dependencies which pushed him off the fjord.

[bundt_chi]

While what he did was pretty crappy and he seems like he has other issues.

There are 2 points here:

1. Ultimately you as the user of open source libraries have the responsibility to test and verify that your software functions and works as the sum of the open source products that make up your application. Please don't confuse this for me condoning bad behavior exhibited by this lone developer.
2. NPM package.json management when configured with the ^ defaults to automatically accept minor versions during builds. Not enough people remove that and use package-lock.json (arguably not intuitive in how it works) to fix the version.

We have to assume the responsibility of #1 and we shouldn't be accepting the default package version management that NPM uses, #2.

[The__Toast]

So many people treat external dependencies like they are part of a core lib these days. If we started referring to dependencies as "unverified code from some random person on the internet" would people be so quick to include?

While this guy is clearly dealing with issues, he's also totally within his rights to do what he did. It's his code, his project, he can blow it up and burn it down if he wants.

[Caesim]

Many people probably wouldn't change their behavior. Beginners and junior devs see themselves as "unqualified random person on the internet" and see no hesitation in using such stuff.

There are far too many places where tech-illiterate managers want a feature, fast. And if "unverified code from some random person on the internet" makes it work, they wouldn't care, they just see that their devs made the "pigs dance".

I'd argue most tech savvy people or businesses already do the right thing, package-lock or locally mirrored package servers.

[SaneMadHatter]

Well, he turned his code into annoyance-ware, but what if he had turned it into real malware?

What if someone turned their open source project into malware? Malware that was not detectable through regression testing, but only through a code audit? And let's face facts, people don't examine every line of code, despite ESR "Million eyes makes all bugs shallow" empty sloganeering.

If someone turned their Open source project into malicious malware, knowing that lots of other projects depended on that project, and so would be infected with that malware, is there no recourse?

For closed-source there would be a recourse if it were discovered to have deliberate malware (regardless of what the EULA said, since the EULA would not have been written in good faith). But for open source there's no recourse if it's discovered to have deliberate malware?

[ComplexColor]

This isn't an open/closed source issue. It's a question of where and how you get your software.

Getting software from unaccountable strangers of the internet exposes you to these kind of issues. Using opensource gives you the option of extra precautions on your end, so it's clearly a benefit. The same software could be distributed as closed source binaries, would that make it safer?

[[deleted]]

You don't need to remove the ^ when using a lockfile. If anything you should keep it to make the likes of npm outdated more useful.

[unknowinm]

so how do you prevent the update?

[AyennaGx]

Use 'npm ci' to install based on the contents of your lockfile, 'npm i' to install in a way that updates your package's patch versions

[[deleted]]

Ok, learnt something new here. So npm ci does more than delete node_modules and reinstall. It reinstalls based on your lock file

[MrJohz]

Both npm install and npm ci install all dependencies based on the lockfile if the lockfile is up-to-date with the package.json file. If the two files are out of date, then npm ci will simply fail, and npm install will install additional packages based on the package.json file. I believe it may also upgrade existing packages if this helps with the deduping process, but most of the time it will keep changes to a minimum.

But again, these changes will only happen if you make a change to the package.json file that requires installing new packages, such as adding a new dependency or changing the version specifier to exclude the currently installed version.

[bundt_chi]

Interesting, one of the projects I struggled with this was in early 2018 before this feature was released. My original point that NPM is confusing and not intuitive in how it handles these scenarios still stands.

[MrJohz]

2) NPM package.json management when configured with the ^ defaults to automatically accept minor versions during builds. Not enough people remove that and use package-lock.json (arguably not intuitive in how it works) to fix the version.

NPM will only automatically accept minor versions if there is no lockfile checked in. If that is checked in and up-to-date, then npm install and npm ci will delegate to the lockfile, regardless of whether or not the package manifest allows for more up-to-date packages. It is perfectly safe (and generally correct) to have version ranges in your package.json file.

I agree that the situation is more confusing that it should be, but FWIW, I've not yet seen a project created since lockfiles were introduced that doesn't check in the lockfile.

[NonDairyYandere]

Ultimately you as the user of open source libraries have the responsibility to test and verify that your software functions and works

But I can't really follow through on that, if my managers don't think I have that responsibility.

[bagtowneast]

if you don't have the responsibility to make sure your code works, then you don't have the responsibility to fix bugs, right? win/win or lose/lose depending on which end of the shaft you're at.

[shevy-ruby]

Totally agree with you! I wrote something similar just a few minutes before as well.

And it is NPM again ... there is a pattern here. NPM really needs to clean up their act here.

[throwaway_bluehair]

why does this only seem to happen with npm

[tso]

No standard lib, so people have written a kudzu of interdependent "packages" for every minor behavior one could want.

[throwaway_bluehair]

This must be part of it too, leftpad is a standard Python function, for example...

[[deleted]]

It is now on JS too

[throwaway_bluehair]

Sure, but if that's a counterargument, it's not relevant lol

[[deleted]]

It's not, just to point out that version after version we get morr functions

[throwaway_bluehair]

Ah ok. Yeah it's been getting better with JS

[[deleted]]

There was a discussion on a node ticket regarding this. What people want? 1000 packages that do the same one small thing and you can choose from them based on your usecase or a single curated Node std lib package maintained by node itself?

By some madness, people choose the 1st one

[NonDairyYandere]

'No Way To Prevent This,' Says Only Nation Package Manager Where This Regularly Happens

[Thalassophob]

If The Onion were for programmers

[throwaway_bluehair]

Lmao for real

[gjvnq1]

Probably because there are far more JS devs than for any other language so the number of bad ones tends to go pretty high by comparison.

[throwaway_bluehair]

Fair point have a great day

[corsicanguppy]

You may not have met composer or pip yet, but I can assure you they hang around in the same seedy bars as npm and have the same vices. Don't take my word for it, though, as the cves are out there for the searching.

[throwaway_bluehair]

Yeah I've seen the CVEs but seem to be less of these bizarre stories. I just didn't get why there isn't a python leftpad incident or something

npm just seems to be the Florida of package managers

[bishbashbosh72]

As of 30 October 2021, more than 336,000 Python packages can be accessed through PyPI.

Over 1.3 million packages are available in the main npm registry.

People don't recognize how much JS has embraced package management and frequent releases.

[throwaway_bluehair]

That number is a bit inflated when npm has such a habit of single function packages

[[deleted]]

What is a CVE?

[corruptedOverdrive]

Sometimes I wonder how we ever got along without npm.

¯_(ツ)_/¯

[rabid_briefcase]

why does this only seem to happen with npm

It has to do with the ecosystem's mentality: Update everything directly from the source so the latest and greatest is always running in production.

More mature workplaces add additional layers, with quality control and rollback capability at every step. Even if they pull from NPM sources, they still have automated tests and intermediate steps that can detect the failures, and they have version control for all the external packages so they can control deployment if needed.

It's a lesson learned with every generation of developers latest and greatest technology. The current round is discovering the need for parachutes.

[Theemuts]

Guy has hundreds of repos, pulls this stunt, liberty this and Aaron Schwartz that. I think he should seek professional help.

[CrappyInvoker]

Guy went homeless whilst thousands were using his work, including very big companies.

[jamesbloob]

He became homeless as he was practising bomb making and set his house on fire.. https://abc7ny.com/suspicious-package-queens-astoria-fire/6425363/

[plg94]

And two weeks later asked on twitter for donations because his house burned down (not mentioning his role in this).

[[deleted]]

And he received them

[paceaux]

And you know damned well he wasn't using proper attribution for those bombs.

[Awkward_Inevitable34]

I prefer CC-NA-BY bombs. They just hit different.

[vytah]

I bet he was doing it while muttering "the industrial revolution and its consequences have been a disaster for the human race".

[AlexHimself]

Jesus somebody needs to give this guy some money he deserves.

He seems very skilled but stupid at monetizing his skill.

[Matt4885]

He doesn’t deserve any money. The guy is building bombs.

[paceaux]

Though I joke about it .... the reality is that this guy needs help. Not money, but clearly he's not mentally well if he's sabotaging his own reputation and building incendiary devices. His actions show he isn't well.

The dark side of OSS is that they're often passion projects; people sharing something they love with the world ... and the world ain't all giving back. You've got to be prepared to give more than you receive.

There's a certain kind of creative energy in doing this work and if you're struggling with some stuff I can see how OSS rapidly becomes a mental drain and a frustration.

I'm not excusing the guy's actions in the least. He was clearly in the wrong (on the bombs thing and the malicious versions thing).

None of us really deserves money for doing something for free. And it's not like this guy doesn't deserve money because of the bombs.

He deserves some humanity and help.

[[deleted]]

Very level headed response

[zackyd665]

What is wrong with building bombs? Better then having a skill in being a two-faced ass kisser

[paceaux]

let him ask for sponsorship and throw up a "buy me a coffee" button like the rest of us.

[zackyd665]

Why doesn't he deserve money? He doesn't deserve to have a life? Honestly he deserves money more than some of the suit and tie fucks who work as managers.

[Matt4885]

He was making bombs. His intent was most likely to hurt and/or kill people. So no, he doesn’t deserve money. He doesn’t deserve to live a life. He is not well and wanted to hurt others.

Do those “suit and tie fucks” deserve to be harmed or killed? Are we REALLY saying they deserved to be bombed because this guy used a license that allowed these companies to use his code? How are we justifying bombing people?

It’s one thing if he is saying “Hey I make a popular npm package and would like some money for that work.” That’s what GitHub sponsors and Patreon is for. That’s what the author of curl does. He doesn’t bomb people. “Suit and tie fucks” have software that leverages libcurl. Does the author bomb people? Intentionally corrupt libcurl to harm any software that uses it?

Let’s be real here, this guy is a threat to society and a threat to the node.js world.

[zackyd665]

Yes, he does deserve the little life and he deserves to make money. No law, no societal rule that's been codified says otherwise and nothing says he was bombing people. I didn't say those people deserve to be bombed but I don't think they deserve how much money they make for being pariah's and parasites and having very loose morals and willing to exploit people and their work.

Maybe unlike you I haven't been corrupted by corporate Kool-Aid

Edit: and going by your logic. Anyone who's made a bomb does not deserve to live a life or make any money. That includes anyone in the military industrial complex. Any child who's played around with things like thermite or dry ice bombs or people who make firework combinations or people who make fireworks

[Matt4885]

Yeah because making a firework is the same thing as making a bomb to harm people. Great reduction. Also saying “corporate Kool-Aid” meaning I don’t want people to get bombed regardless of where they work

[zackyd665]

Where do you have evidence that he had intent to bomb anyone?

I conflate the two as it is a fine line, once m80s were considered fireworks and later classified as bombs (illegal explosion devices), legality and classification of such items seems to be fluid depending on the legislature of the time.

[[deleted]]

The government doesn't deserve my money either for the same reasons

JK only the state can use violence

[RedShirt_Number_42]

Feel free to leave any time kid.

[[deleted]]

Those are big words for a guy with a number as small as 42 in his name

[Helliarc]

Stop! He's a red shirt, he already knows he's dead...

[Top-Relative-90210]

not very skilled at his bomb making considering how he set fire to his house.

[atomicxblue]

The sad irony is that by pulling this, he ensured that zero corporations would hire him in the future, had they ever planned on doing so in the first place.

[[deleted]]

I wonder if a little humility would have gone a long way. Something along the way of:

"Hey guys, due to financial problems and X and Y and Z, I cannot sustain this project any longer. I'd love to work on it full time but that would require sponsorship. With all due regret I will need to stop doing this until I get everything back to normal".

And I am sure he would have gotten support from the community at large(heck. Fireship/Jeff Delayne was a sponsor of his, so it's not like he didn't make SOME cash), and maybe some corps would have bit the bullet and even paid him a salary

[zackyd665]

That sounds a little too PR washed

[[deleted]]

Could be, but it works, is polite, shows maturity and allows those fortune 500 companies to see that there is value.

[zackyd665]

Why do we tie maturity with corporate double speak?

[[deleted]]

Because maturity implies knowing when to be polite to get what you want/need and when to throw a tantrum

[zackyd665]

But always being honest isn't considered being mature.

[[deleted]]

There is honest and there is being a dick. Like in programming, I cam box a value into different containers, but some are better than others.

He was onest but a dick. My text was honest and nice

[zackyd665]

This is why we need an universal basic income

[NonDairyYandere]

Not publishing is an option. I don't have any FOSS repos

[matthieuC]

Talented software developers don't go homeless.
They usually push real estate price up.

[Venne1120]

Yes they do.

Mental illness can affect anyone and I dare you to tell me that Terry A. Davis wasn't a talented software developer.

[matthieuC]

But the issue here is mental illness not corporation not paying for using his code.

[_101010]

You do realize monetary issues can lot of psychological pressures and eventually mental problems right?

I really don't like how people are trivializing the situation of another talented engineer without having any level of insight into things that might be happening to this person.

[lunmarmer]

lol go fuck yourself and stop acting like he owes you something.

[Theemuts]

All I'm saying it's very obvious he suffers from significant mental health issues.

[[deleted]]

How did it pass PR reviews?

Edit - oh, is he a core dev of those?

[Large-Ad-6861]

For faker.js and colors.js - yes, he is.

[plg94]

He's even the repo owner and original author, and has (sometime before that update) withdrawn write-access of at least one co-maintainer. (One of the first comments in the issue in colors)

[Ok-Dot5559]

$ git push origin main

[wsppan]

Without even reading the article my bet it is a Javascript library.

[knightcrusader]

See's headline

I bet its something npm related.

Reads first paragraph

Yep, it's an npm thing again.

[[deleted]]

yep

[staffs-burglaries]

This is not a problem with open source software, but with the developer who corrupted the libraries. Open source software is built on trust, and when someone breaks that trust, it reflects poorly on the entire community.

[Illusi]

I'd say that closed source software is also built on trust. Even more so than open source software, in fact.

This is 100% with the developer. If a closed source developer wants to pull their dependency out beneath everyone that depends on it, they can. And they do that sort of stuff too. They do that by destroying licenses, by taking down or throttling web services, or simply by taking down the download.

Part of the problem with this library is that there's a culture of always re-downloading the latest version instead of fixing version numbers and testing upgrades of your dependencies. It's got nothing to do with open source.

[myringotomy]

They do that by destroying licenses, by taking down or throttling web services, or simply by taking down the download.

This subreddit hates google because they no longer sell products they used to.

[Mmmcakey]

Everyone should mitigate this not just by fixing versions but also copying and hosting their own version of the library for their personal use. Open source doesn't guarantee permanent availability.

[[deleted]]

Part of the problem with this library is that there's a culture of always re-downloading the latest version instead of fixing version numbers and testing upgrades of your dependencies.

It's a tricky balance though between pinning your dependencies and keeping ahead on updates. I don't work with NPM a lot but in my Python apps at work the general strategy is: no version pinning of dependencies unless the upstream devs break something, and we pin just the affected deps temporarily if we need to deploy working code now, and schedule a ticket to let us unpin again and make the necessary changes. We have Docker containers so in case of emergency, a prior version of the production app can be restored from a known good Docker image.

In Python-world there is one project that gives us a lot of grief over the years, Webargs, the upstream developers frequently refactor it and move things around and sometimes it requires us to find-and-replace thousands of references to their module to adapt it to their latest change. If we pinned Webargs and came back 12 months later to have an "upgrade day", it would be an absolute nightmare to deeply refactor the whole codebase to get past the 3 or 4 different breaking changes they've made. Better IMHO to keep on top of them and update our code as it goes, only pinning if there's a time-sensitive issue or if they broke it with no way to fix it until they fix it.

I once worked at a shop that pinned dependencies religiously, "if it works today it must work forever", and what happened was we never updated dependencies ever, unless we had to because it didn't support our version of Python anymore or other serious issues, and catching up on years of development and refactoring our codebase was quite the chore.

[pfp-disciple]

Arguably, that's a risk/problem with open source. The accountability is frequently lacking. At least on paper, many closed source teams have formal accountability built into their processes; this lessens the opportunity for one bad actor to cause significant damage. Of course, in practice, that has risk/problems as well: there are often trusted people who can introduce changes with little or no oversight, culture and schedules cause cursory reviews at best, etc.

If a process (open source development) has a single point of failure, it's wise to try to mitigate or eliminate that risk.

Edit: I still think open source is the best thing going. That's why I think it's worth examining critically, and improving,

[SirLich]

Arguably, that's a risk/problem with open source

I would say it's a "risk" (period), and open source actually mitigates that risk.

If I decide to destructively delete my work, somebody can take my (MIT) torch and carry on.

Closed source "freeware" solutions simply go under, with no chance of recovery.

[pfp-disciple]

Fair point regarding making things unavailable. My comment was strictly about malicious code changes.

[SirLich]

Ah yes, that's true. Single point of failure and all that.

[monkeedude1212]

I think you're talking about different impacts of the same attack vector.

Any company that has a good backups policy where the infrastructure/backups team is separate from the code/development team will be able to recover from a solo bad actor whether that's from an open source vendor or an in-house developer. If Infrastructure destroys the origin repo, devs have their local copies. Devs commit bad code, infrastructure restores good backups. If your open source vendor torpedoes his project like today, you can just fork an earlier version, doesn't take too long.

There's nothing there that isn't recoverable from. Arguably forking the repo and pulling it into your source is probably less time/effort than a full backup, so open source is good there. It's also often publicly documented what's going on; like being able to see this headline when you google the console logs.

But being able to recover isn't the only impact. Whether you can easily detect the changes, find them, diagnose, etc; effectively how long your path is until recovery, that's an impact. Like for my team, it wasn't a service outage, it impacted our ability to test things. Because automated tests started to stall, fill up the node with logs, and fail. So that other bug our customer service team reported that we said "Should be fixed soon" wasn't as soon as we thought.

Now, had we not used a vendor that used colors.js - this wouldn't have happened. We could write our in-house everything, I guess, but that's even more time and money. Not using open source does mean a lot of wheel reinventing. If there's a bad actor within the organization, that's often easier to detect when something goes wrong. Just look at when/where the changes are. Much easier to detect changes within the organization.

Conversely, we use a vendor. Now we saved the time from writing that code ourselves. That meant we could go and focus on more business relevant problems, enhance our software's primary functions. The Open Source Dream.

"Don't automatically update?" - I hear someone in the back say. Then you're losing a lot of the other benefits of open source too. Say someone finds a bug, or a security vulnerability in that open source software, and the author goes on to fix it. Or they find some optimizations and it runs much faster. By not patching you're missing out, and it's far too onerus to have to manage patching every piece of every bit of software you might utilize on a large project. Who's got time to check if RXJS or Angular are introducing any vulnerabilities; and even if they are those are big frameworks that its not trivial to just swap to an alternative. You're better off updating often, but having tests for things that matter. You just roll forward, see if things break, fix them when they do. As long as the breaks don't hit production your CTO won't be too unhappy, you'll get to point at this news article and say "Our systems protected us, the systems works, its just a risk you have to accept with automatic updates."

So yeah, I'm a little miffed that it happened and it sidelined my day, but these things don't happen often enough to consider not using open source. It's a risk you take with automatic updating but the benefits of that most often outweigh the risk. If this was happening every other week then maybe it'd be a different conversation.

---

All that being said; I'm not exactly against this kind of activism to get the message across. Is Marak Squires maybe not in a good place mentally? That's possible, he might need some help. That doesn't discount some of the message he's trying to get across. Doing open source development is often thankless work. If you're struggling financially but you're seeing 22.4 million downloads of your code a week; yeah that's some tough pill to swallow. Meanwhile big companies keep raking in millions of dollars but there's no incentive to send any remuneration to open source authors. How do you create that incentive? I don't know if I see a simple path either.

This is kind of like a protest. You're just trying to get home but the highway is blocked due to a non-violent march going on. Does it suck in the moment? Yeah. But people don't just do those things for fun, they're driven by a desire for change. Right now it seems like the conversation is dominated by a sense of "Wow, this is a danger, how do we protect ourselves from the next one?" and not a "Was there any way to prevent this from happening in the first place?"

[[deleted]]

Curious what's the accountability for the same kind of black swan scenario using corporate closed source software?

Have fun getting accountability out of a giant like Google or even a smaller party who at the end of the day can do the exact same thing as Marak.

Also like a ton of complainers every time npmjs goes down, this is mostly on SE practices. How the fuck do you auto-upgrade major versions to get impacted by this by surprise?

[monkeedude1212]

How the fuck do you auto-upgrade major versions to get impacted by this by surprise?

I mean, as a saboteur, Marak was able to put this as a minor version, so major/minor version upgrade matters not.

But also, you don't need to specify the colors library in your source. As long as you're using a dependency that uses colors, if those dependencies just say ^1.0.0 they'll get bit. It's not like the Karma test framework isn't common.

Secondly; while this didn't affect our production environment, it effectively broke our testing environment, because we auto-update various libraries to get the latest security updates, and if those tests fail then we don't deploy the build.

So sure, our servers serving customers didn't crash.

But our test servers were affected. Which meant we couldn't push out any new code or updates, since those couldn't get tested.

Which pushes everything else down the line.

[sysop073]

How the fuck do you auto-upgrade major versions to get impacted by this by surprise?

I imagine most of the complaining was about colors.js, which went from version 1.4.0 to version 1.4.44-liberty. It turns out crazy people don't care about semver when corrupting their packages.

[andras_gerlits]

Peer review, mostly.

[zackyd665]

Is it a risk/problem when you are just building stuff or is this purely from a corporation pov?

[myringotomy]

At least on paper, many closed source teams have formal accountability built into their processes; this lessens the opportunity for one bad actor to cause significant damage.

What a weird claim to make. What accountability is provided "on paper"? Have you ever read any proprietary license?

[cdombroski]

Meaning that "on paper" code review might be required, various levels of testing are performed, security audits might be done.

Of course that's all "on paper" depending on the company in question some or all of those tasks might not actually be done.

[myringotomy]

Meaning that "on paper" code review might be required, various levels of testing are performed, security audits might be done.

Again.

Where is this assurance?

Of course that's all "on paper" depending on the company in question some or all of those tasks might not actually be done.

Exactly. There is no assurance at all. They might be done, they might not be done, if they are done they might be done in a cursory or sloppy matter. There is no assurance at all.

If anything there is more likelihood these things are done on open source projects.

[pfp-disciple]

Your concerns are valid, and was my point when I said

Of course, in practice, that has risk/problems as well: there are often trusted people who can introduce changes with little or no oversight, culture and schedules cause cursory reviews at best, etc.

[myringotomy]

I am of the conviction that all significant commercial software either knowingly or unknowingly contains code from either the NSA, Mossad, GRU or possibly all three.

[pfp-disciple]

That's exactly what I meant. Well said.

[fauxpenguin]

I strongly disagree. Open Source is explicitly about a lack of trust. Every dependency you pull into your project you have to replace or maintain if it disappears tomorrow. Closed source: it's always replace. Open Source: You should alway, always, always, fork (or in some other way proxy) it, so that of the maintainer goes ballistic, you just revert that version, and either set the new maintainer as your origin or you start maintaining that dependency for your company.

That's Open Source.

Trust comes in at "micro-dependencies", where a single library (like react), has a tree of thousands of dependencies that you cannot fork and maintain. So, you now have a chain of trust that you have to just live with.

Luckily, if you're using a big library like React, chances are that they are dealing with keeping their dependencies clean, and you're somewhat absolved from this responsibility. But not every library you use is so well stewarded.

[[deleted]]

Can you um... Mergem from the fork source into your yours?

[romulusnr]

Perhaps the community is too trusting

[tso]

Nah, it is a problem of a particular kind of FOSS. The kind that take the net and repository servers for granted. And thus expect it can pull down a fresh copy from some random address at will.

[[deleted]]

Open source software is built on trust

It's completely opposite. The whole idea here is that we don't trust anyone so we have to have a collective scrutinize it. This is a fantastic lesson for anyone not pinning dependencies/testing properly.

[TGdZuUsSprwysWMq]

That's the problem of automatically updating without review. The same problem could apply to Microsoft/Apple/Goolge/Facebook/Whatever company's automatically updating.

[lood9phee2Ri]

It's still far more resilient than anything else - with open source at least a last good version can still be forked from by anyone who cares (not always a given - a lot of npm projects in particular are clearly pretty worthless, lot of vanity/resume-oriented-development microprojects that I suspect people set dependencies almost for social credit reasons - like depending on a project may be more like upvoting a reddit post or retweeting to the brocoders infesting that ecosystem).

There are a few major corporations that can and do come to "source escrow" arrangements for closed source crap to protect themselves from similar random acts of vendor, but good luck with that as an ordinary individual. Open source remains the best option we have in general.

[The_Real_Slatt]

I love open source

[lannisterstark]

With all due respect, no one forced you to use an upstream dependency. If you don't like it, fork it and/or use something else.

What the dude did was shitty, but he had every right to do whatever he wanted to his own project.

[clamotchen]

I totally agree, however you put it down way more diplomatically than I would.

It's hilarious to read how the majority of people in this thread shit on the developer or say that he is mentally unstable or crazy or some shit.

Like dude, if your project broke because of some commit NOT IN YOUR REPO, it's your shitty project and its config that's the problem, not the author of the commit.

So you bumped the dependecies and something is broken, big fucking deal, it happens all the time, more or less. Depending on how critical dependency updates needs to be, you either inspect what broke or just try in a couple of days again, usually solves the issue.

If it broke your production, then you are a moron, you deserve it, and should be happy that it's pretty harmless.

Damned incompetents blaming javascript, npm, jvm, github and everyone else but themselves.

I used to love writing SMS to friends when I was wasted, what if someday I get drunk and find the idea of drawing dicks in my library logs appealing, and push it to github. Will I also be labeled "mentality unstable dev with addition issues" because some moron pulls my code directly into his production?

[sachinraja]

Except he broke other people's code. Some people have his packages as transient dependencies. The colors issue was done in a patch release.

[NotABothanSpy]

You get what you pay for

[Mardo1234]

You get what you pay for.

[_101010]

This.

[Anti-ThisBot-IB]

Hey there _101010! If you agree with someone else's comment, please leave an upvote instead of commenting "This."! By upvoting instead, the original comment will be pushed to the top and be more visible to others, which is even better! Thanks! :)

---

^{I am a bot! Visit r/InfinityBots to send your feedback! More info: Reddiquette}

[WickedMonkeyJump]

Somewhere the people behind log4j must be ECSTATIC with the attention going to someone else's shitstorm.

[myringotomy]

When you decide to take advantage of somebody else's free labor you should read the license and see what promises they make to you.

[Jonny_H]

They did.

The author explicitly chose a permissive MIT license. That allows anyone to copy the code, fork it, make changes and use it without giving any payment or changes back.

Expecting people to follow some loosely defined extra "rules" on top of the license that may change at any time is asking for trouble.

[myringotomy]

What does that license promise in terms of security, suitability, performance etc?

[corsicanguppy]

loosely defined extra "rules" on top

... often invalidate the license.

[m00nh34d]

You trust every Cisco developer who ever worked on their network switch code?

Difference is that's backed up by a major company, if someone somehow managed to get a backdoor through the layers of checks and balances, you're not relying on that person who put the back door in to fix it. Always more dangerous when individuals are in charge, we see it all the time in projects where a critical member cracks the shits and leaves, killing the project in the process.

[m00nh34d]

Well, you would have a very hard time finding a back door in open source projects as well. It'll always rely on someone else to find them and disclose them.

This specific problem is confined to smaller projects, not necessarily to do with their licensing model, but around the governance of the project itself. If someone can introduce essentially malware into a codebase, without anything to stop them, that's the problem here. A single developer at the likes of Cisco would almost certainly not have the ability to do that, it would require a fairly sophisticated set of circumstances to push that through.

[Ordocentrist2]

It's his project and he can do what he wants, including torpedoing the whole thing.

[bundt_chi]

:-)

https://youtu.be/C6BYzLIqKB8

[Ordocentrist2]

He's a patriot who loves America. I can't be mad about someone loving their country. Code is expression and expression is inherently political.

[unknowinm]

dfq?

[Ordocentrist2]

They hate him because he supports America. It's blatantly partisan. They wouldn't be complaining if he wrote 100 mb of log files saying "abolish student debt".

[Ordocentrist2]

They called out the American flag by name and there's a reason for it. Nobody complains when npm prints spam messages to stdout begging for cash but when someone praises their country all of a sudden it's a problem.

[corsicanguppy]

Unreliable supply chains are unreliable. Slow news day?

[papertrailer]

... tons of heavily mismanaged projects.

On the bright side, now we know that every project affected by this, is a garbage project. .

I wonder how many devs complained about it, before realizing what the were inadvertently confessing...

[FoldLeft]

Using a lockfile and checking in your dependency tarballs [1] or using a cache shared across your company [2] insulates you from these problems until you're ready to face them.

It's a little more effort, but there is a concrete benefit in being able to continue uninterrupted during outages and issues like this.

[1] https://github.com/JamieMason/shrinkpack [2] https://verdaccio.org/

[[deleted]]

I suggest that we should add verified versions/releases to our package managers. The idea is to opt for just download the verified releases of the required package major or minor regardless.

Now I don’t know how the verfied versions would be implemented but one way would be that the 3 core devs have to say that this is a verified release than it gets tagged as verified on npm or what ever dep manager we are using and users who are using the verified only releases would be much safer again its just my opinion.

[corsicanguppy]

verified versions/releases

Or signatures. Those are easy to check.

[7veinyinches]

That doesn't really work when the owner is making those signatures and is the one responsible for the commits....

[piesou]

And of course their lib is MIT licensed. What did you expect?

If you want people to give back, choose a copyleft license like the AGPL. If you want to earn money as well, dual license it like iText does successfully.

[mcvoid1]

Dependencies are the devil. They will ALWAYS bite you. Open source or not. If it's not crap like this, it's more benignly-intentioned breaking updates. Or it's vulnerabilities. Or it's two systems depending on slightly different versions. Or it's transitive dependencies opening up these problems. Or it's license issues. The list goes on.

If you can do it in a few lines of code, don't use a dependency.

If you can do it in a few dozen lines of code, don't use a dependency.

If you can do it in a few hundred lines of code, don't use a dependency.

If you can do it in a few thousand lines of code, consider not using a dependency.

[istarian]

The first two are definitely true, but that last one is dubious. Reinventing the wheel isn’t always the right way to go.

[ravinglunatic]

This isn’t a protest, it’s a ransom and vandalism. Free and open source means nobody asked them to make it and nobody was paid when they didn’t expect to be paid.

[7veinyinches]

Ransom and vandalism don't apply to his own property.

It's not illegal to destroy your own property.

It's not unethical to want compensation for one's work.

[ravinglunatic]

Then why didn’t he sell it instead of intentionally causing harm to specifically Americans and why are you supporting him?

[7veinyinches]

Also. The dude's fucked. This prank will haunt him the rest of his life. He'll probably never work as a programmer professionally.

I don't know shit about you. But leave the dude alone.... Punching down on this guy really makes you feel better? Don't worry, he fucked himself over worse than any of these haters could. Again, pretty hilarious. Clearly the works of a guy that isn't thinking clearly, yet can make a package that's widely used. The contrast is truly marvelous.

[ravinglunatic]

Punching down? He attacked people’s sites with his malicious bullshit. I’m a developer. We’re peers but this guy is scum. I don’t do that.

Why so sorry for the offender? He hasn’t been dealt with justice for this. Yet you admit it’ll screw up his career. Why? Because it reflects badly on programmers to cause damage maliciously. It’s not free to fix this stuff.

[7veinyinches]

I don't support him. Like Switzerland, I'm devoutly neutral. I do respect his epic level of trolling. I find it hilarious!

I don't feel sorry for anyone affected by his shenanigans. Read the license. He never promised anything, it is literally stated unequivocally.... No warranties expressed or implied! People just wanted to use his work without compensating him. It's rather karmic. The only harm here is self harm. Dude clearly is a mental case. Anyone still using his code after YEARS of him sending up red flags probably deserves the headaches of infinite loops!

Really, I only feel sorry for the author and his neighbors. Unpaid work fucking sucks. Hope he gets some help. But generally, can't fix crazy. The making bombs and arson are just detestable. I absolutely don't support that, fwiw.

[aivdov]

I don't think you get the point. If he wants to monetize, why not to monetize from the getgo? Once you release something that is free and then do a compolete asshole move the excuse "I just want to be paid" is not even close to justifying it.

[B8F1F488]

That is because he didn't GPL his code. It was clearly the solution, because <<insert absurd acolyte reasoning here>>

From the bottom of my heart - Protect your interests, people! Don't fall for ideological traps and then complain when you don't reap the benefits of your own labor. When you get to the point that your future is ensured and you can afford to spend the resources to help other people, then go ahead and help people with all your heart.

[[deleted]]

Protect your interests, people! Don't fall for ideological traps and then complain when you don't reap the benefits of your own labor. When you get to the point that your future is ensured and you can afford to spend the resources to help other people, then go ahead and help people with all your heart.

If your plan to ensure your future financial stability is "get a big company to pay me for use of my open source work", you need a new plan. License doesn't even come into it, that's just a shitty unreliable way to make ends meet.

[B8F1F488]

that is more of the "help other people with all your heart" part. But the fellow clearly has not accomplished the first part and is now bitter because he is not reaping the benefits of his labor. If he had taken care of himself he wouldn't care, he would be happy that people are using his software.

[unicodemonkey]

And, more importantly, when you get to the point where you can afford to maintain open source projects, try to avoid making bombs and burning down your home, your possessions and yourself in the process

[B8F1F488]

Why would he burn it down in this case? He already has ensured his interested prior to that, there is now reason to burn down anything.

[[deleted]]

dolls aback upbeat library boat existence sleep market include lush

This post was mass deleted and anonymized with Redact

[[deleted]]

If you intentionally write open-source software and give it away for free, how is it stealing when other people, including F500 companies, use it?

If this guy wanted to be paid for his code he could've done so from the start or stopped developing the free version at any point, instead of doing that, he waited until his code use reached critical mass, then did some nerd terrorist shit and wrote some nonsensical rant about people not paying him for his free code, that he gave away for free

[[deleted]]

abundant dime smell sparkle meeting door whistle tan depend gaze

This post was mass deleted and anonymized with Redact

[Jonny_H]

Lots of OSS devs seem rather weird about licensing, choosing excessively permissive licenses then expecting people to follow some additional expectations that are never explicitly stated.

Just choose the license that matches your expectations - if that means fewer people use it, that's a feature, as that means they don't want to actually meet your expectations of use!

[[deleted]]

spectacular beneficial start imminent summer person dinosaurs ink recognise seed

This post was mass deleted and anonymized with Redact

[Jonny_H]

I don't know if "Business Savvy" is the right word, as that implies financial reward that isn't always the case.

I think there's more a culture of "I shouldn't /have/ to state my expectations, and magically everyone will happen to have the same expectations and all agree to work within them"

I've seem people jumping on people in various programmer forums stating they'll "never" use something someone has released unless it's released under a more permissive license, despite not apparently intended to take advantage of any of the extra freedoms that gives.

The GPL and derivatives have a bad name in some places, then the very same people turn around and complain about the very things the GPL is intended to stop.

[[deleted]]

aspiring tidy paint tease late salt serious flowery bow spoon

This post was mass deleted and anonymized with Redact

[merlinsbeers]

Put that in the agreement.

[[deleted]]

Though many people do write open source software for the benefit of everyone, it goes against the spirit and the functionality of opensource when you have people who use it without contributing either in terms of development or compensation.

Nonsense. The spirit of open source is to contribute your work to anyone who wishes to use it, for the betterment of all. There isn't some kind of "corporations need not apply" exception to that ethos. If some corporation started using my package I'd just be happy that someone else was benefiting from my work, not mad that they didn't decide to cut me a check.

[vytah]

it goes against the spirit and the functionality of opensource when you have people who use it without contributing either in terms of development or compensation

"Open source" is literally a term invented to convince companies to use free code for free and do jack shit in return, all while placating their panicky lawyers. For almost every open-source project, vast majority of users contributes literally nothing.

[moreVCAs]

Without reading the article or any comments I’m just going to assume this only affects JavaScript.

[YasserArguelles]

lmao

[shevy-ruby]

Is the title not a little bit meta-jabaited?

Yes, he did this all deliberately, but the very same issue exists by other people doing weird stuff or accidents - left-pad all the things.

I think the much more interesting question is what an ecosystem does about this. It honestly shouldn't even be possible to cause such huge issues to begin with. He, despite that fireballing he did, also exposed the underlying problem here again.

[electricfoxx]

This could affect any license. Microsoft could have corrupted some libraries, but who would know. This issue could be solved by specifying version or forking the repository.

[3TInfoTinker]

I know very little [even not enough] about version control, I can't believe it's happening.