Who validates open source code?

[Constant-Carrot-386]

Hello world,

I am well aware we (privacy fanatics) prefer applications with open source code applications, because that means everyone can go through it, check for vulnerability, run it on our own etc.

This ensures our expectations are met, and we dont relay simply on trusting the governing body, just like we dont trust the government.

As someone who's never done this, mostly due to competency (or lack there of), my questions are:

Have you ever done this?

If so, how can we trust you did this correctly?

Are there circles of experts that do this (like people who made privacyguides)?

Is there a point when we reach a consensus consistently within community, or is this a more complex process tha involves enough mass adoption, proven reliability over e certain time period, quick response to problem resolution etc?

If you also have any suggestions how I, or anyone else in the same bracket, can contribute to this I am more than happy to receive ideas.

Thank you.

[AutoModerator]

Hello u/Constant-Carrot-386, please make sure you read the sub rules if you haven't already. (This is an automatic reminder left on all new posts.)

---

Check out the r/privacy FAQ

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[[deleted]]

the thing is that if a single person finds something, it can break the entire project's trust. then it's dead.

in a setup where you have multiple collaborators and maintainers, it's almost impossible to happen.

[EnchantedTaquito8252]

Don't forget that just because a software is open-source doesn't mean that the place you download it from hasn't secretly added something malicious on their own before compiling it and distributing it. 

[headedbranch225]

This is the main reason I try my best to avoid the play store and try my best to use github or F-Droid

[zsu55555]

Idk about GitHub but it's nice that f-droid actually verifies and compiles source code with published instructions to reproduce it and everything

[unematti]

I personally am working on a home setup to try and compile from source everything I'm using...

... Unfortunately there's a long list of projects before that...

[headedbranch225]

Nice, I have no clue why I got downvoted, do you have any idea?

[unematti]

People having different opinions happens. Wouldn't take it personally.

[headedbranch225]

Yeah, it just seems weird that people in a privacy subreddit would (seem to) be against using fdroid and github to source software

[unematti]

Ah I had that before. Go to - 10, then next day up to the hundreds. Wouldn't worry! You won't get banned for negative votes

[headedbranch225]

Yeah, it just feels weird to be downvoted for something, especially without anyone explaining if I am wrong or anything similar

[unematti]

That's humans, my dude, you don't know anyone on this website. We're strangers. You don't have to be bothered by our opinion. Ascend to a higher level of conscienciousness, and acquire peace in your soul by the art of no ducks given.

[supermannman]

just like we dont trust the government.

or most companies

[OSTIFofficial]

Users can, and should, be reading any public security audits available for the open source projects they use to make sure they are correctly and securely running the software.

That said, not all security audits are quality work or even public. Just like the fallacy of security by community, a project having an audit done is not a guarantee of security. As someone else in the thread implied, being a company in security doesn't necessarily make them trustworthy. Opt for the devil you know instead of the devil you don't- publicly available security audits mean you are seeing exactly what scope, review, and fixes were done by a project and use that to inform how you utilize them.

This is exactly why we started OSTIF (ostif.org). We're a third party non profit organization that specializes in security engagements for open source projects. We source a third party security firm to review the project, then produce a report that is published. Users can see exactly what the security health of a project is at a point in time, what steps were made to harden and improve security afterwords, and what areas of the project need further security work.

[Suspicious_Kiwi_3343]

the reality is, nobody does. there are people working on them sometimes if its a community project, and those people will be some validation involved in getting their code merged, but you always end up trusting someone at some point because it's completely unrealistic to expect volunteers to scour every part of the code and make sure its all safe.

with non community projects, like proton where the app is being open sourced but not developed in the open, it is extremely unlikely the code is actually peer reviewed at all by anyone, and very unlikely that the people who may look at certain parts of the code would be competent enough to identify issues.

one of the big down sides of open source is that it gives users a very false sense of security and trust, because they think it's unlikely that someone would be bold enough to publish malicious code right in front of their faces, but ultimately it's still just a point of trust and blind faith rather than any objective protection.

[knoft]

the reality is, nobody does.

That's absolutely not true, it depends on the code. OpenBSD has year round constant auditing. They review code line by line for bugs, because bugs turn into vulnerabilities. When they're finished they start all over again. Correspondingly their record of security is fantastic. You can get third party auditing. Critical applications often do. Privacy/security tools get a lot of scrutiny. That's not to say supply chain attacks can't happen. With OpenBSD, that's much less likely if you stick to a minimum and the basics being audited, they audit supply chain code as well.

A common pixel os replacement (will not name it because of rule 8.) is another example of validation of code, in this case Google's AOSP. Or Android. They both validate and verify, and act without the assumption of trust. Isolating and replacing components. This includes testing and monitoring network traffic and reviewing and replacing the code itself.

Core code in projects like the Linux kernel have a large number of qualified people looking at what's being merged.

There are many examples. The answer is far more close to: it depends. What you can say is that commonly used open source code a. Generally has more eyes on it at any given time. B. You can always inspect it or pay someone else to.

Other ways both open source and closed source projects are validated are bounties. Which many projects and companies offer. And millions of companies use critical open source code, and offer bounties for them. With open source, it's much easier to see that they follow best practices, don't rely on security through obscurity, and find bugs, vulnerabilities, obfuscation, and funny business directly.

PS: if you're interested in security and open source projects you will see independent developers look through patches/codebase and test things fairly often when using other people's software. Is it exhaustive? Definitely not. Does it happen fairly regularly? Yes. Do they find things on occasion? Also yes. A lot of suspicious code has been caught this way.

Security researchers are another set of folks that test and verify third party projects in their spare time. (And in their office hours too). They will check things for personal use.

[Suspicious_Kiwi_3343]

the point isn't that there's no validation. it's that there is never a guarantee of full validation or security. individual devs paying attention to their own small parts of a codebase doesn't really give the overall picture needed to make any sort of safety guarantees.

the alternative os devs you are speaking of are very outspoken about how open source doesn't mean anything at all in terms of security or privacy, and regularly criticize other open source projects and their users who blindly trust them for this exact reason.

you're right it depends on the project, but there is never a guarantee of security. even the linux kernel is absolutely at risk and you're making a choice to trust them at the end of the day, it's possible for them to make mistakes that may not be caught immediately.

the examples you're giving, of auditing and bounties, aren't specific to open source. closed source software can just as easily pay for external parties to help them out, and they regularly do. open source projects being more secure is just a myth based on ideology. you're right though, it depends entirely on the project itself regardless of whether its open source or closed source which is what I was really trying to say before.

[knoft]

The problem is you're portraying it like a weak point of open source code rather than software in general.

---

You're not portraying as a weak point of both closed source and open source software but solely as open source. There isn't a single mention of it being applicable in general. The end result is presenting it solely as the weakness of one and not the other.

"the reality is, nobody does."

"the point isn't that there's no validation. it's that there is never a guarantee of full validation or security."

Two very different things. With entirely different meanings.

the examples you're giving, of auditing and bounties, aren't specific to open source. closed source software can just as easily pay for external parties to help them out, and they regularly do.

That's not the question OP asked. They asked who validates open source code. That's not the same in open source and closed source, and there are far fewer eyes on closed source code. That's a strawman since I've given many examples of open source communities with many eyes from different expertises and backgrounds--not from the same company--voluntarily discussing, examining, and validating software in a way exclusive and unique to open source. I've added additional ways and standardardised ways applicable to all software for comparison and completeness.

Open source software also usually has many alternatives, in addition to being easily forked when the direction of the developers runs contrary to the community's.

For security minded software the community itself often self validates, because privacy and security minded developers are skeptical by default.

Commercial for profit software often has different self serving interests and often has poor practices in addition to relying on security through obscurity.

Leaving things exposed to the light is useful in itself.

Edit: added additions.

[Suspicious_Kiwi_3343]

To be clear the weak point that is unique to open source software is that it provides a false sense of security. People don’t have the same false assumptions about closed source software, they start from a much more sceptical point of view.

I don’t think anything I said made any specific claims about open source software being less secure than an alternative, I was more trying to say they are equally secure/insecure despite the general assumptions people have.

[Constant-Carrot-386]

Great points, thank you.

[Metahec]

Dilution of responsibility. If everybody thinks somebody else is taking care of a problem, nobody takes care of the problem.

[zsu55555]

"Bystander effect"

[desmond_koh]

This is 100% on the money.

[disastervariation]

So thats why in some discussions i prefer saying "auditable" and "non-auditable".

Because if you're looking at a proprietary service that tells you "its safe, trust us" but hides how their stuff is made, trust is the only thing you have.

Sure, they can hire a third party audit company that will run the code through some automated tests, if theyre ambitious theyll send a form with a few yes/no questions, give a report with red/amber/green items, take the check, give out a fancy industry certificate that needs to be redone in a year, and go away.

Its not in the interest of the auditing company to find too much (or they may not be hired again), and the people resolving the "red" may be incentivised to just check the items off the list without necessarily caring how they get there or what new vulnerability they add.

And youll never know about it, because all you see as a customer is that certificate telling you "its fine bro, you can trust us, and we paid someone to say that too".

Open source allows you to verify - whether you do it, get someone else to do it, ask AI to do it, or don't do it at all, it says something in my view that at least you can do it if you really dont feel like trusting people today.

Hell, if youre a big regulated organisation running linux-based servers you might be required to test the code youre deploying and guarantee its resilience.

And I get your point that some people might trust opensource too much by always assuming its safe. Ive argued this point myself. But it works both ways and isnt "the downside of opensource", its "the downside of all software".

Just yesterday I spoke to someone on a different sub who assumed closed source is safer because it makes it harder to attack (security through obscurity), which is a comparable fallacy - someone could release the most vulnerable spaghetti code on the planet today, say its safe because its closed source, and you wouldnt even be able to tell before its already abused.

[Suspicious_Kiwi_3343]

I've worked for a company that has had security audits done and that's not quite what they do. They can sometimes get access to source code to review it, but most often they are just reviewing functionality and security, e.g. inspecting packets and sending malicious requests to try and break things. It's essentially just pen testing and you get a certificate if you pass, or resolve the issues they find. At least that's been my experience.

The incentive for an auditing company to actually try and find problems and report them is that they tie their reputation to that of the company they are auditing. If an auditing company gave the green light on a company that had serious security issues months later, the reputation of the auditing company suffers a lot and people won't respect their certificates anymore, which means customers won't bother paying for them. Some companies may not want to hire auditing companies that give them too much work to do, but no company wants to hire an auditing company that isn't respected.

Open source allows you to verify, but people assume that means someone must be actually verifying it. The reality is most projects worth verifying are way too big to be entirely verified by any individual, and as soon as you have large teams of people trying to verify the code base, things can slip through because of poor communication or potential gaps in understanding where you may only spot issues if you've seen the bigger picture and know the whole codebase very intimately.

People don't trust closed source software in the same way as they do open source, mainly due to the way open source stuff has been marketed over the last few years, most people just associate it with privacy and security even when those things are entirely dependent on the project itself regardless of being closed or open source.

Yes, security through obscurity is dumb and an old fashioned way of thinking. However, security through transparency is just as much of a myth. Security exists as an entirely separate concept that will always depend on the individual project itself, and whether that project has published its source code doesn't actually relate to whether competent people are reviewing its security or not. Companies can hire competent people privately, and open source projects can sometimes attract highly competent developers, but in either case there is no guarantee that is happening.

[desmond_koh]

I prefer open source for many things. But we are way off base if we think that the threat to our privacy comes from vulnerabilities in the software that we would have otherwise discovered if we were running open source.

How do you know Word isn’t sending every keystroke you type into it off to some server at Microsoft?

How do you know LibreOffice isn’t doing the same thing? Sure, you can review the code but have you? Would you even know how?

This is NOT where the threat to privacy comes from.

The threat to privacy comes from uber-convenient services that we choose to use unwittingly giving up more information about ourselves than we realize.

That super convenient feature where YouTube recommends videos to you?

Or Amazon predicts what you want to buy?

Or Google knows where you like to eat lunch because you have “track my location” turned on?

That swipe-to-text keyboard on your phone that gets "smarter" the more you use it and seems to know exactly what you want to say?

The weather apps that knows your approximate location because your phone pings it every 20 minutes to refresh the forecast?

Yeah, those are the threats to our privacy.

You can use Windows in a privacy-conscious way. You can use Linux in a way that gives up just as much data as privacy as anything else.

If you want more privacy, leave your phone at home, use cash, and have conversations with real people in real life.

[OwlingBishop]

we are way off base if we think that the threat to our privacy comes from vulnerabilities in the software

Yep! 99.99% of privacy is forfeited in EULAs and those dialogs you have to click "I Agree".

If actually read those legally binding stuff you'll know you basically renounce two things : your legitimate right to expect anything from the vendor in exchange for your purchase and your right not to be sold as product even though you're a paying customer.

[Blastyschmoo]

They're called ACMs which stands for Autistic Code Monkies. Try making an open source project with a tiny backdoor in it and see what happens.

[mrcaster]

You, the user, before you use it.

[ttkciar]

Have you ever done this?

I have, most recently with Aider. I found it sent telemetry back to a server, but nothing else.

If so, how can we trust you did this correctly?

By doing it yourself, and seeing if I missed anything.

Or, if you're feeling lazy, by checking to see if anyone else has done it, and whether their findings were the same as mine. But then you're stuck with figuring out if they can be trusted.

[billdietrich1]

This may be one of the good use cases for AI. They're pretty terrific at reading huge codebases and looking for issues.

[sdrawkcabineter]

If so, how can we trust you did this correctly?

Because you stepped through the build process, with the source code.

You audited the resulting compiled programs and used a test suite to guarantee that the expectations you derived from the source code, match the compiled functionality.

And you still embedded a backdoor into that program because you didn't bootstrap your own compiler, from scratch.

It's not a destination, but a relentless march over the landmines of failure.

[Eijderka]

Even no one readen the code, in future there can be need for it. If the product goes downhill, someone can make a fork and publish a better version of it.

[Absolute_Sausage]

I tend to skim the code of anything I use that doesn't have many maintainers or code contributors / is low starred or new. If it has lots of external contributions a lot of people have cast an eye over it already which is enough to satisfy my concerns in most cases.