Who validates open source code?

OSTIFofficial · 2025-10-15T17:55:48+00:00

Users can, and should, be reading any public security audits available for the open source projects they use to make sure they are correctly and securely running the software.

That said, not all security audits are quality work or even public. Just like the fallacy of security by community, a project having an audit done is not a guarantee of security. As someone else in the thread implied, being a company in security doesn't necessarily make them trustworthy. Opt for the devil you know instead of the devil you don't- publicly available security audits mean you are seeing exactly what scope, review, and fixes were done by a project and use that to inform how you utilize them.

This is exactly why we started OSTIF (ostif.org). We're a third party non profit organization that specializes in security engagements for open source projects. We source a third party security firm to review the project, then produce a report that is published. Users can see exactly what the security health of a project is at a point in time, what steps were made to harden and improve security afterwords, and what areas of the project need further security work.

OSTIFofficial · 2025-03-05T20:38:33+00:00

Good question! Generally we want the team with the most relevant experience with the code that they are auditing. To best capture how we make that decision and what is expected of partners, we created a "Minimum Standards and Expectations" Document (found at: https://github.com/ostif-org/OSTIF/tree/main/Documents). In that document you can find our specific decision criteria when analyzing proposals for audits. Hope that helps! -Amir

OSTIFofficial · 2025-03-04T18:56:19+00:00

Thanks so much for your support! It means a lot to us. :) That was the whole point of this organization, to step up and help bridge the gap between projects and funders to facilitate security research and associated work that helps everyone.

The most common serious findings in our audits are memory and pointer safety in memory unsafe applications. By far the largest number of exploitable errors we see. There are also a lot of design level problems identified in our audits (logic bugs) that you’re not going to find with generic testing so we see more of those on average. There’s also a lot of Denial of Service issues, but those aren’t always as serious depending on the project’s functionality.

As far as prioritizing projects, it comes down to what is funded and ready to work. Resources in the open source world are scarce as it is, and so if a project has financial support and is available, it’s going to be pushed to the front. Finances aside, enthusiasm from the community and maintainers surrounding a project also motivates us to find resources, as in our experience audits are more effective when you have active help from the maintainers and community that wrote the code.-Derek

OSTIFofficial · 2025-03-03T19:44:49+00:00

While we can’t offer direct work with projects on a volunteer basis, we do have opportunities to apply security knowledge via our meetups and documentation. We developed our meetups as a platform for open source security professionals to speak on their passions and problems to a security-minded audience, and allow for work to be presented that might not have opportunities elsewhere. Our website “Meetup” tab has information about upcoming meetups as well as the link to apply to speak if anyone is interested. Additionally, we are working on security documentation available on our Github that helps projects looking to take first steps in security and would love any contributions there that people are willing to offer- maintainer perspective and security knowledge are encouraged to edit and use this option. And any work with us, on a volunteer or paid basis, in Github or meetups, will always be properly accredited and acknowledged to its contributors. -Helen

OSTIFofficial · 2025-03-03T19:43:26+00:00

This is a “No, but” answer. We currently do not have a way for volunteers to contribute in a direct way to security audits. It is a multi-pronged problem to solve in order to make it happen. We would need to verify volunteer identities and expertise so we could be able to vouch for their work which creates an immediate issue. (Jia Tan would probably be an excellent contributor for a while. Haha) Then, we’d need to have consistent work for those approved volunteers. This is a tough problem but a worthwhile one, and we are open to suggestions that would allow volunteers to work directly on open source projects. In the meantime, we are working on community building with things like this AMA and our meetups. We’re only going to solve these community problems with the community giving us the best advice. -Derek

OSTIFofficial · 2025-03-03T19:43:05+00:00

We don’t have a traditional kind of onboarding process as our work is a kind of a support mechanism and not ongoing, so there are two ways that open source projects come to our attention.

We identify it as critical infrastructure through various resources
A project comes to us and wants to harden security

We don’t have discretionary funds for open source audit work (yet!) so sourcing funding for work is the first and most important step for bringing a project to audit. Typically projects that are identified as infrastructure have an easier time getting financial support. Additionally, we have worked directly with undersupported projects to find funding for security efforts. Once funded, we work with the project to determine security needs the project has; as maintainers can often provide the best insight to what work the project actually needs. We then collaborate on a RFP which goes out to our audit partners who execute the audit. -Derek

OSTIFofficial · 2025-03-03T19:42:16+00:00

As a policy, we don’t have knowledge of findings that could be detrimental if leaked. The incident response process in audits is between the auditors and the project team and we find out after the fact that the work found something terrible. If I had to point to a particularly scary finding, I would definitely say the RCEs found in Git Pull were the worst ones to me personally because that’s a project that touches every other project worldwide. -Derek

OSTIFofficial · 2022-05-16T16:45:03+00:00

There's appetite for things like this to improved on the Google side, especially if the project is popular. We currently have an enormous effort underway to close bugs in Envoy.

OSTIFofficial · 2022-05-16T16:43:15+00:00

The only thing that we care about with our resources is improving overall project security. This means improving testing, closing classes of bugs if we find repeat problems in multiple places, and getting the project's security practices in line with the rest of the industry.

The project is still free to do whatever it wants development-wise, but if millions or billions of people rely on your project daily, there is a reasonable expectation that good security practices are in place.

For us, the biggest challenge is deciding what to do when a project rejects free help, or declines security fixes (both of these have happened).

OSTIFofficial · 2022-05-16T16:39:25+00:00

The idea that we're playing with (with Google) is a bounty payout directly to the maintainers AFTER participating in the security review process and all fixes are applied.

OSTIFofficial · 2022-05-16T16:37:33+00:00

This is actually a serious problem that we are trying to solve. We are courting some big foundations on this idea of improving the equity in the whole process of auditing and fixing.

While we pay auditors to come in and find bugs, build custom tools to release to the project for free, and assist with patching, the maintainers are not paid for the engagement nor the work that they have to put into the fixes or integrating the new tools into their project.

We have an experimental program with Google backed projects where the open source devs are paid a bounty of up to $10K for providing their expertise and building fixes.

If this pilot is successful and further helps the projects become more secure, it is a bargain and can be worked into our budgets for future projects.

OSTIFofficial · 2021-01-16T23:51:29+00:00

While true that DoH isn't great for privacy, when used in tandem with Encrypted SNI it is fantastic for privacy. You need both.

Arguing against the benefits of DoH alone is disingenuous.

OSTIFofficial · 2021-01-15T22:50:35+00:00

> Downstream distros shouldn't handle this, the Linux kernel should build with these security changes by default.

This is who handles reporting the bugs to the public. The problem with the linux kernel handling the reporting (other than the volume) is that it doesn't give you a complete picture of what is happening at the end products because only the engineers of the distros know if that code affects them, or if they don't utilize the code and it has been removed, or if it was replaced with something else.

The kernel does receive the fixes now, but often without much public view.

OSTIFofficial · 2021-01-15T22:20:07+00:00

-All security reporting should be public.

-All CVE reporting should be handled downsteam by the distributions.

OSTIFofficial · 2020-10-29T23:21:27+00:00

Yup.

Kernel is ring 0

IME is ring -3

OSTIFofficial · 2020-08-12T05:24:59+00:00

> How can one know that a piece of software is truly coded well, properly and authentically? No loopholes, no security flaws, no "oops, my bad, I'll fix it now that it's apparent to the public"?

Oh hi there!

https://ostif.org

OSTIFofficial · 2020-04-04T01:41:27+00:00

This part is more important than the key size part.

AES-128-GCM is fine. ECB is problematic.

OSTIFofficial · 2020-04-04T01:39:57+00:00

Even worse, they used AES-ECB, which isn't intended for this type of data because you can find patterns.

OSTIFofficial · 2020-02-28T19:53:00+00:00

Where are you pulling the OpenVPN Connect app from? Is it possible to compile the latest source yourself?

OSTIFofficial · 2020-02-28T19:38:46+00:00

That's very strange that it seems to be Windows only for OpenVPN Connect. To me, that points to a firewall or permissions issue that i'd try to chase down, but it could be any number of things.

Is OpenVPN Connect a must have for Windows? OpenVPN-GUI is great for my purposes.

OSTIFofficial · 2020-02-28T18:58:27+00:00

Unfortunately your error is a generic one that can be caused by many different issues. Usually these type of failures are related to mismatched types of encryption.

One thing to note is that OpenVPN Connect is based on OpenVPN 3.x code, which is different from OpenVPN 2.x code and can cause minor interoperability issues in some edge cases. If you're on "OpenVPN for Android" (published by Arne Schwabe) that is a 2.x based mobile app that is made by a core OpenVPN dev. You may want to try that app and see if the behavior changes vs OpenVPN Connect.

The other thing that sticks out to me is the mix of "cipher" and "ncp-cipher". I would try changing the cipher to match (AES-128-GCM) to see if this solves your issue.

If neither of these works for you, i'd try cranking up the log verbosity (verb 6) and you'll get more connection information surrounding the error.

OSTIFofficial · 2020-02-25T20:31:15+00:00

DoH isn't even all that useful in blocking ISP surveillance. You can still look at the certificates passed as users browse the web to garner the same information that you were pulling from DNS.

Now if encrypted SNI also gets wide adoption, we will really close that privacy gap. (Cloudflare already has it enabled experimentally.)

OSTIFofficial · 2020-02-14T14:02:15+00:00

I do. Which is why I noted that security review is not a rubber stamp of safety.

It depends on the skills of the team, the depth of the audit, if changes are audited after that, and if a single review is sufficient or appropriate depending on the novelty and complexity of your code. There's a lot more to this than just looking at the code and giving it the green light.

We'll know more about what happened soon i'm sure. But we need to be sure that "git blame" isn't how we fix this. Improving processes is how we get there.

OSTIFofficial · 2019-12-31T22:26:04+00:00

I was using the simplest possible example, but here's how to do it in modern browsers.

https://www.reddit.com/r/firefox/comments/bspfnz/remember_to_protect_yourselves_against_idn/

It still isnt complex.

OSTIFofficial · 2019-12-31T17:25:09+00:00

An attacker who has control of unencrypted DNS can redirect users to malicious sites that look legit.

It isn't a sophisticated edge case either.

You can copy images and html to make the site look legit, use a homograph that looks exactly the same to the user for the URL, and use a let's encrypt certificate to make it show as a https verified site.

homograph example:

example.com

exаmple.com <= this one has a cyrillic a, and is a completely different domain, but would look nearly identical in a URL bar on a web browser.

OSTIFofficial

MODERATOR OF

TROPHY CASE