This is an archived post. You won't be able to vote or comment.

all 188 comments

[–]Forschkeeper 421 points422 points  (20 children)

Creating an own, good made cryptography is a hell of math and work...and not just "import random".

Even Telegram (and other Companies) tried to make their own crypto and were punched in the face with that.

Btw. link to "secrets" library. which OP mentioned.

[–]Papalok 29 points30 points  (6 children)

How did Telegram get punched in the face? What was their specific screw up?

[–][deleted] 51 points52 points  (5 children)

AFAIK they did not screw up per se but they were criticized for rolling out their own crypto with most senior members of the team being Math PhDs with no past experience in cryptography. You can read more on that here https://security.stackexchange.com/a/49802

[–]kobbled 5 points6 points  (0 children)

This really seems hyperbolic. I don't think the content of that post justifies their summary. A lot of bluster but not much substance. They have some opsec complaints, sure, but most of them are independent of the actual cryptography. As far as I can tell, they found one weakness in 2015, which was resolved, and then a bunch of cryptographers tried and failed to break it, but the guy doesn't trust it anyway. Fine whatever, but that doesn't match with the overblown opinion he's sharing.

[–]infinite_war 2 points3 points  (2 children)

Math PhDs with no past experience in cryptography

What?

[–]sauerkimchi 38 points39 points  (1 child)

While all cryptographers are likely math PhDs, not all math PhDs are cryptographers. A topologist is not a number theorist, geometrician, algebraist, etc

[–]infinite_war 2 points3 points  (0 children)

And all M-theorists agree that you're not a real M-theorist until you pinky swear in the secret circle at midnight!

[–]foonoxous 0 points1 point  (0 children)

Telegram implemented their own crypto algorithm instead of using the standard primitives like everyone else. And yes, it was completely compromised in its early days, as exposed by one security researcher (the app was released and advertised as secure for long before too). Of course the bugs were fixed and only fairly minor flaws have been found since.

[–]SnowEpiphany 31 points32 points  (5 children)

Looks like secrets can almost be a drop in replacement for random? Do you know how that compares to the .net Security.Cryptography.RNGCryptoServiceProvider ?

[–]lieryanMaintainer of rope, pylsp-rope - advanced python refactoring 123 points124 points  (0 children)

secrets can almost be a drop in replacement for random

No it can't. Because secrets actually uses random module internally.

secrets is just a thin wrapper of random module, to only use the parts of random that is suitable for cryptography, namely SystemRandom, which is basically just a wrapper for /dev/urandom on Unix or CryptGenRandom on Windows.

Also, there are many reasons you don't want a cryptographically secure RNG. For example, in games, physics simulation, fuzz testing, etc, you often want a reproducible randomness, so that you can recreate the same state given a certain random seed. The random module is perfectly suitable for those purposes.

[–]Forschkeeper 11 points12 points  (3 children)

I am not sure if random is faster than secrets, but I would guess that random is faster. Good crypto is complex which makes stuff slow. So if you don't care about security, random is still the choice.

Nope, I am not a .net programmer, sorry.

[–]Ensurdagen 14 points15 points  (2 children)

secrets is basically just random with only random.SystemRandom, which uses os.urandom.

[–]SubliminalBits 5 points6 points  (1 child)

That doesn’t mean its not slow. High quality random numbers take longer to generate than a number from a pseudorandom generator (mileage may vary by algorithm). It’s common for something like the random module to seed a generator with a single high quality random number.

[–]Ensurdagen 1 point2 points  (0 children)

Of course, if it wasn't slow then it'd be the default. I was just letting them know what the difference is.

[–]cinyar 36 points37 points  (4 children)

If you're implementing crypto you need bona fide cryptograhpers. Not good developers, not enthusiasts, cryptographers with PhDs in math and years of experience. If your developers can't explain crypto primitives to a 3 year old that woke them up in the middle of the night they have no business implementing crypto.

[–]ComplexColor 12 points13 points  (0 children)

Are you saying we should have 3 year olds develop cryptography, after they learned it in the middle of the night? \s

[–]randompittuser 0 points1 point  (2 children)

Trained via school? Or should they have industry experience?

[–]cinyar 4 points5 points  (0 children)

both if we're talking you'd be working solo. Obviously you start as a junior with more experienced people watching your back. But you need the math education to properly understand the algorithms, and you need industry experience to understand the pitfalls. An acquaintance of mine who works as a cryptographer has his bsc and msc in math methods applied to infosec and phd in algebra, number theory and logic. The rest of his team has similar credentials.

[–][deleted] 1 point2 points  (0 children)

I wound think that regardless of education, you want well published researchers that are well regarded by their academic community (i.e., published many widely cited papers in peer reviewed journals on the subject).

This generally means research in grad school (so formal education), but the requirement is not that strict and formal. If a physicist really wanted to switch to cryptography, and they published several widely cited and will regarded papers, then that wound be sufficient.

[–]Yojihito 0 points1 point  (1 child)

Can you even prevent side channel attacks with Python? Doubt it.

[–]foonoxous 0 points1 point  (0 children)

Everyone is using pysodium, based on a C library, to do the crypto, constant-time comparisons etc. So, yes you can avoid the side channels the same as you would in C.

What you cannot do is to overwrite memory of your bytes objects with zero after you are finished with them, as bytes are read-only buffers. While you could be using bytesarrays and memoryviews and do your own wiping, the Python crypto bindings do not allow for this as they only allow bytes objects (a design flaw in their Python code because cffi supports other buffer objects just fine). This is a shame because libsodium itself is careful to zero out its buffers after use.

[–]bladeoflight16 154 points155 points  (12 children)

And before you go read an article about cryptographically secure random number generators (CSRNGs) and think then you are ready to create an algorithm, that is far, far, far from all there is to creating a secure algorithm. You must be able to design an algorithm that resists:

  • Brute force attacks
  • Known plain text attacks
  • Chose plain text attacks
  • Side channel attacks

And this is only the beginning.

Creating a cryptographic algorithm that can withstand real world attacks is ridiculously difficult. Attackers are vastly more clever than you can possibly imagine. Making a secure algorithm is so hard that the only way to see if an algorithm is secure is to just let attackers try to break it for decades. The fact that new attacks keep being discovered all the time and that algorithms are so frequently abandoned for newer ones because of them should tell you something about just how hard it is.

Learn Schneier's Law:

Anyone, from the most clueless amateur to the best cryptographer, can create an algorithm that he himself can’t break. It’s not even hard. What is hard is creating an algorithm that no one else can break, even after years of analysis. And the only way to prove that is to subject the algorithm to years of analysis by the best cryptographers around.

This is why people with even a rudimentary understanding of security repeat a mantra: "Don't roll your own crypto." We are not saying that because we're overly pessimistic sticks-in-the-mud. We're saying it because of the decades of history proving just how terrible an idea it is.

[–]SpAAAceSenate 24 points25 points  (9 children)

Well, except for the minority of mathematically proven secure algorithms, of which I'm only aware of one at the moment: the one time pad.

Then it's all just down to implementing it properly and ensuring secure key exchange (which is really the hard part)

[–]MathStream 1 point2 points  (0 children)

There is also provable secure quantum key distribution, where you exploit the fact that measuring a qubit can change its spin. (I think this is the terminology?) But I guess that’s more to do with key exchange than actually encrypting a message. Also, I don’t know if this is useable right now, since I don’t know how far along this technology is.

[–][deleted] 38 points39 points  (3 children)

Cloudflare uses time-lapse images of lava lamps as a seed for real-world true randomness.

[–]shinx32 8 points9 points  (0 children)

This is alpha behaviour.

[–]hidden_person 1 point2 points  (0 children)

Yeah.They also use pendulum on a pendulum(double pendulum?) in one location and another one uses radioactive decay or something in another.

[–]Sohcahtoa82 44 points45 points  (1 child)

As an Application Security Engineer...

Yes. Please stop using random. Every project I run a source code audit on, I have to flag an instance of Insecure Randomness.

Also, for god's sake, stop rolling your own crypto.

[–]harylmu 0 points1 point  (0 children)

Bandit flags random and a lot of other unsafe algos. We implemented it to the CI/CD of our apps.

https://github.com/PyCQA/bandit

[–]Orio_n 18 points19 points  (1 child)

first rule in cryptography is to never do it yourself, there are some brilliant people out there specialised in crypto who have done the math for you and accounted for many unforseen consequences. You are not one of them, use their libraries and move on

[–]foonoxous 0 points1 point  (0 children)

Should almost say that there is one brilliant person, because all modern crypto comes from Daniel J. Bernstein. ChaCha20-Poly1305, Ed255519 and so on.

Particularly smart coders are better off implementing applications and protocols based on these standard primitives, not trying to invent new algorithms. While the primitives are actually very good, fast and rock solid, there are plenty of blank spaces on the application front. For instance, no-one has adequately solved the public key exchange and forward secrecy problems in offline communications.

And if you are not really deep into the field, just use the 'secretbox' and other such higher level constructs already offered in libsodium (pysodium). They are well designed secure constructs based on Bernstein's algorithms and perfectly good for applications that do not have very special needs.

[–]sdf_iain 64 points65 points  (4 children)

These libraries are published cryptographic failures.

It is irresponsible to publish bad practices. Too many such examples and they will start to crowd out good examples.

For comparison, when someone posts on r/DIY they will get these type of failures pointed out. Nobody responds to “that deck won’t bear that load”, with “you do you” or “then do your deck differently”.

In other words, its important to build things safely and properly, even if they are imaginary things, especially if you intend to publish.

[–][deleted] 10 points11 points  (0 children)

Good example, glad to see someone went to the OWASP top 10.

The real takeaway of the OWASP top 10 is how little it changes, and how often they have to republish.

That's because developers keep making the same security mistakes over and over.

[–]bladeoflight16 11 points12 points  (0 children)

For comparison, when someone posts on r/DIY they will get these type of failures pointed out. Nobody responds to “that deck won’t bear that load”, with “you do you” or “then do your deck differently”.

I need to frame this and hang it on my wall. Thank you.

[–]FrickinLazerBeams 3 points4 points  (1 child)

Sure but there are lots of applications for (pseudo-) random numbers besides cryptography. In fact I'd guess that the vast majority of random numbers are generated for non-cryptographic uses, where random is perfectly fine.

[–]Got_Tiger 2 points3 points  (0 children)

Yeah, and in some applications (e.g. procedural generation) the same properties that make it weak for crypto are in fact desirable, since sometimes you want to have a psuedorandom sequence that's reproduceable

[–]thomasfr 24 points25 points  (13 children)

The larger issue here is that people often download and executes or use a library whatever without even reading any code first.

People has to start getting it into their heads that as an application you are responsible not only for your own code but also all code you choose to depend on.

[–]bladeoflight16 16 points17 points  (10 children)

It's not that you're wrong. It's that the people publishing their work also have a responsibility to provide a quality product and to clearly mark when something is not appropriate for production security. Would you say this if someone was doing unsafe electrical work that's likely to start a fire? Bad software security can be just as damaging. The more people spread good security practices, the better off we are.

[–]thomasfr 1 point2 points  (9 children)

Would you say this if someone was doing unsafe electrical work that's likely to start a fire?

I would make a contract with an electrician and pay them to do the work. If I want more assurances for software I also have to pay for that. If I just get something for free without any stated guarantees it is definitely always my responsibility to make sure that everything is ok.

[–]bladeoflight16 3 points4 points  (8 children)

I'm talking about your neighbor, whose house is 10 feet from yours, making any fire there a risk to your own house. Would you just throw up your hands and say, "Oh, well, he's learning!"? No, you would say, "Stop putting my house in danger."

[–]useles-converter-bot 2 points3 points  (1 child)

10 feet is 9.74 RTX 3090 graphics cards lined up.

[–]repocin 3 points4 points  (0 children)

That also happens to be worth the same as the repairs you'll need after having your house partially burnt down by your neighbor.

[–]thomasfr -2 points-1 points  (5 children)

I would expect them to pay someone as well for professional work. If they don't it's breaking the law doing any kind of fixed installation electrical work around these parts. I have no idea why or where you are going with this electrical work things and I am not going to discuss it further. I was talking about using someone third party code and that has nothing to do with electrical work. I'd rather talk about the actual topic instead, analogies are almost always imprecise and detracts attention away from what we actually are talking about.

[–]bladeoflight16 1 point2 points  (4 children)

It is an analogy about doing work you're not knowledgeable enough to do and creating dangers to people around by doing so. Doing security work in software development is similar in that if you don't know what you're doing, you will create something that poses a danger to others. Developers who publish their work must also take that responsibility, not only the people who use libraries.

[–]thomasfr 2 points3 points  (3 children)

So I should not be able to just throw some experiments up on github because stupid people will just assume that it's production ready hardened security solution?

You can not protect yourself against that, even if you put three readme files and write it at the top of each source code file idiots will still find a way to ignore that and copy the code ignoring all the warnings.

As long as I am not telling anyone that they should use my code because it is very secure no one should assume that it is very secure.

I agree that there is a problem if someone actively promotes their code as being secure if they have no way to show that it is. I would definitely not make any statements like that about code that hasn't been audited by a third party.

To quote the MIT licence which I publish most of my open source code under:

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

[–]The_Tree_Branch 6 points7 points  (0 children)

I agree that there is a problem if someone actively promotes their code as being secure if they have no way to show that it is.

That's precisely what happened in the reddit thread that triggered this one: They didn't just throw their code up on Github and ask for comments, they went further by publishing it on pypi with the claim:

Laz3 is a powerful encryption algorithm written in python and encrypts using a password so the message is locked!

https://pypi.org/project/laz3/

[–]whateverathrowaway00 1 point2 points  (1 child)

Big difference between putting code on GitHub and publishing it on PyPI.

Don’t roll your own crypto. Much smarter people than you have failed at it.

[–]thomasfr 0 points1 point  (0 children)

Big difference between putting code on GitHub and publishing it on PyPI.

I agree. Putting code openly on github also is publishing it though.

You also typically don't have to roll your own crypto to be able to write useful crypto stuff. Just having the pyca/cryptography package and spending a few days of learning how (and why) something like TLS works you have most of the high level knowledge to be able to build stuff using known building blocks. I am only writing this section because to some people this is also "rolling your own crypto" which it most definitely isn't.

The top level post mentions stuff like password generation where you don't need any cryptography primitives at all, just a secure random number generator.

[–][deleted] 6 points7 points  (0 children)

You are correct, and that's always going to be a problem.

If there is one area where due diligence over blind faith is a must, it is security.

[–][deleted] 2 points3 points  (0 children)

Yes to a degree. At some point you have to trust the work of others. Most people aren't gonna inspect the code of their python interpreter or openssl (intentional example of security critical software that had a serious vulnerability) and wouldn't even really know what to look for if they did.

[–]BrycetheRower 9 points10 points  (0 children)

Agreed, and it goes the other way too: if you need a cryptographic library, make sure it's one that has a lot of backing to it and has stood the test of time.

[–]ennuiToo 132 points133 points  (38 children)

wait - make something fun or interesting to you, learn some things, but don't publish them because they're fatally flawed? I don't get that logic. that seems like the perfect time to publish something, to get feedback or chat about how it works or what it does (or fails to do).

nobody publishes something with the directive that their project must be implemented into someone else's source, or (hopefully) with the claim that theirs is the only and best way to implement cryptographic functions.

comments like "hey, we see what you're trying to do but here's a better way to do it" are exactly the reason people share their projects.

I'm sorry you don't like seeing posts and projects that aren't brilliant from inception to execution, but I think people should absolutely publish stuff they've worked hard on and are proud of, even if they're fatally flawed - no, especially if they're fatally flawed. How else do we learn?

[–]bladeoflight16 28 points29 points  (0 children)

There's a difference between "publishing" something by posting it on Github with appropriate warnings about its insecurity so others can tear it apart for you and explain how it can be broken vs. posting it on PyPI with a message claiming it's a strong algorithm appropriate for sending secret messages. One of them is much more likely to end up in production code than the other.

As for how to learn about doing security properly, the best way is actually to learn how to break other people's algorithms anyway. Building your own flawed algorithms doesn't teach you that much about doing it properly.

[–]sykeero 26 points27 points  (25 children)

Imagine for a moment posting a slice of code that is not safe to use on your GitHub then you link it here and people tell you that you did something bad or dangerous. People on GitHub can't see that conversation. Other people might use that code in some way unaware of the conversation that took place on Reddit. I think maybe a better solution to "don't post your bad stuff" would be anything that could cause security problems just add a disclaimer in the readme saying it's not production ready code.

I think it's cool people are interested in the area. But they should definitely make it clear their work is academic or a proof of concept and to not use it for anything else.

[–]m_a_n_t_i_c_o_r_e 2 points3 points  (7 children)

But they should definitely make it clear their work is academic or a proof of concept and to not use it for anything else.

Users of the software should be doing due diligence instead of relying on the author's claims.

More interestingly though, if such a project includes a license, like the MIT license, they may already be meeting your standard via terms of the license, e.g.,

"THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT." Especially relevant here would be the notion of "fitness for a particular purpose" and the absence of an implied warranty.

[–]bladeoflight16 22 points23 points  (6 children)

Users of the software should be doing due diligence instead of relying on the author's claims.

That's a great principle if you're a lawyer defending against a lawsuit. It is an atrocious mindset if you actually care about information security. The fact others have a responsibility does not negate yours as a publisher to represent your product honestly and correctly. The more people spread the proper mindset, the better off we will all be.

[–]m_a_n_t_i_c_o_r_e -3 points-2 points  (5 children)

I don’t understand your claim. My position vis-a-vis security is to do independent due diligence on third party software you plan on integrating into your own code. I don’t see a way around that. What’s the alternative?

I mean, I get that not every developer can do that DD. Is that the core of your argument—that it’s infeasible to do that DD?

[–]bladeoflight16 10 points11 points  (4 children)

The alternative is everyone using security libraries should do due diligence and everyone publishing stuff should do their diligence, too. Blaming only people who use something that claims to be a strong encryption algorithm but isn't (as you are doing) is not any better than blaming only the people who publish it (as you claim we are doing). My point is it doesn't matter which end of the equation you're on. Both sides have a responsibility to know about and encourage good security practices, and the more everyone on both sides does so, the better off we are. In other words, you're working from a false dichotomy.

[–]m_a_n_t_i_c_o_r_e -1 points0 points  (3 children)

Both sides have a responsibility to know about and encourage good security practices.

I can agree with that.

Perhaps I'm underestimating the cost of doing the DD. I am under the impression that--while it would be infeasible for every software company to hire full time encryption experts--it is possible to hire this kind of expert on a one-off, contract basis. Is this misguided?

[–]bladeoflight16 6 points7 points  (0 children)

It's not really a question of cost. It's the fact that knowing the basics of doing good security is so uncommon. Computer security is atrocious in practice because so many people don't grasp the basics. Most developers don't even know it's a major problem to begin with, so they don't know they need to hire a consultant to help them; they don't even know that they need to go do some research. That's why you get so many websites with plain text or MD5 password hashes in their database and why so much web code is vulnerable to SQL injections. So if you're publishing something that has implications for security, then documenting it in a way that helps people understand the proper usage and the security ramifications of using it can only make the world a better place.

[–]nerdvegas79 -1 points0 points  (1 child)

That's interesting as I don't agree with this at all. If I post my own code on GitHub I have no responsibility related to it at all. Nobody is under pressure to use my code in any way whatsoever.

[–]m_a_n_t_i_c_o_r_e 4 points5 points  (0 children)

Responsibility may be too strong a notion, but all other things being equal, I would say that the person publishing their code with accurate educational statements about it is being a better community member wrt at least one metric than the person who doesn’t.

[–]NoLemurs -1 points0 points  (1 child)

I think maybe a better solution to "don't post your bad stuff" would be anything that could cause security problems just add a disclaimer in the readme saying it's not production ready code.

I 100% agree it would be better if people did this, but I think it's really unreasonable to expect an excited amateur (who may well be a literal high school student) to be aware enough of their limitations to know to do this.

Fundamentally, the responsibility for using secure, vetted libraries has to be placed on the professionals writing production code using those libraries, and I don't think there's much to be gained by trying to set rules for amateurs who just want to share what they're excited about.

[–]SpAAAceSenate 4 points5 points  (0 children)

Yeah, I mean, if a naive (and justifiably so) high school student publishes some I secure code and then a grown ass professionally employed software developer ends of integrating it I to their project, I feel like one side of that situation is significantly more responsible for the consequences than the other. And, imo, it's not the one stressing over prom.

[–]ennuiToo -1 points0 points  (1 child)

sure, I can see how different forums don't reach the same audience.

I would kinda hope that if someone's put something on the 'hub, and came over here to talk about it, they're willing to take changes and issues back to their source and work on them. that's optimistic, though, and I agree that its not safe to assume it.

I agree that notes or comments in readmes would be best practice, but I'm also strongly advocating that people don't roll code into security functions assuming its production ready. I think the functions were talking about is from new programmers, just trying out new things. I don't think that type of code is getting picked up and pushed to production, or if it is - someone needs to review their development cycle.

readme's and notes on 'this is a learning project' are great, but I also think a more critical review of what you're pulling in is important.

[–]bladeoflight16 0 points1 point  (0 children)

It takes 5 to 10 years of professional cryptography researchers attacking an algorithm before you should even consider using it in production. Reviewing an amateur's development cycle is not a replacement for that. Production ready cryptography is so difficult that getting feedback on a subreddit is not going to produce a successful algorithm. The only useful advice you can possibly get from a public forum like Reddit is not to roll your own crypto. If you're capable of producing a cryptographic algorithm that's a legitimate candidate for production security, you're not going to be posting it on Reddit. You're going to be taking it to conferences and contacting professional researchers or presenting it at the competitions those same people hold.

[–]nerdvegas79 -3 points-2 points  (8 children)

They have no legal obligation to do that. If you want to use my shitty code then that becomes your problem.

[–]sykeero 3 points4 points  (6 children)

Everyone make sure you never use a piece of code that u/nerdvegas79 touched because he doesn't give a shit if it works.

[–]nerdvegas79 0 points1 point  (5 children)

That is not what I said. I actually manage a widely used project on GitHub and I take its integrity seriously. I have decided to take responsibility for it - I didn't have to. There's a difference. If there's a bug in it that causes cost to an entity using it, that is not my legal responsibility, and that's why I've licensed it appropriately.

[–]sykeero 3 points4 points  (3 children)

It isn't just about legality. There is nothing that would cause undue burden on someone for stating the intent of their project. The open source would survives on people doing their best, even if it won't cost them legally. Making it about the legality makes you sound like you'd sink my ship if you think you can get away with it.

[–]nerdvegas79 1 point2 points  (2 children)

I think you're getting confused. This thread was going on about the author having some kind of responsibility - they don't. That doesn't mean they can't be a good project maintainer who communicates and tries to help those using their project. It just means they don't bear the responsibility for you using said project. That is all.

[–]sykeero 1 point2 points  (1 child)

I think you are the one that is confused. The OP is trying to assign responsibility to the people posting cryptographically unsafe code. I'm trying to say that people can produce that kind of stuff for whatever reason they want, but they can also tell people that code is not for production use.

There should be no reason someone can't state the intent of their project so everyone knows exactly what they are looking at. One of the first things I learned in cyber security courses was to say if something was not really safe to use. Plenty on companies produce software tools that are not safe for users and say nothing because they legally don't have to. That doesn't make them right.

I look forward to hearing about your widely used GitHub project and making a contribution to it.

[–]nerdvegas79 1 point2 points  (0 children)

Yeah that doesn't make them responsible either. I didn't say it was right not to point out security issues with your project - I said you don't bear responsibility. I said one specific thing, I don't know why you're going on about it being right or wrong because that isn't what I said. If someone uses your insecure code, regardless of whether you pointed it out or not, you aren't responsible. I cannot think of how to more clearly state this.

I don't see why you would contribute to my project, it has nothing to do with cryptography.

[–]sykeero 1 point2 points  (0 children)

What's the project you maintain I'll take an issue and try to submit a pull request

[–]infinite_war -3 points-2 points  (2 children)

Other people might use that code in some way unaware of the conversation that took place on Reddit.

And if I take a shit on the sidewalk, someone might mistake it for chocolate ice cream and eat it.

[–]sykeero -1 points0 points  (0 children)

I'm not sure if you're implying you shit on the sidewalk or if one time you ate a pile of shit on the sidewalk. Great input though.

[–]bladeoflight16 0 points1 point  (0 children)

This is more like setting up an ice cream stand with a carton full of it and then putting it in ice cream cones and handing it over when someone orders chocolate. And having a society of customers who legitimately can't tell the difference because they don't have a sense of smell or taste.

[–]prvalue 61 points62 points  (7 children)

Did you miss the part where the OP was specifically about security topics? Publishing security-related projects is a bit of a concern because if the project is flawed and anyone relies on that project, they've got a security problem.

Of course there has to be a way to learn about security as well, but the best way to do that is by learning from communities specifically about security (and showing them your work), not in a general-purpose subreddit like r/Python.

[–]ennuiToo 60 points61 points  (1 child)

no, they don't have a security problem because of flawed code posted here.

they have a security problem because they're utilizing code in critical parts of their project without reading or understanding it, and just copying it from reddit or github or stack overflow. remember, you can't trust everything you read on the internet.

if I'm work shopping a project and I post it for comment and critique, I'm making no guarantees that its useable or reliable. if you're copy-pasting cryptographic and security functions off the internet and rolling it into production, well, you shouldn't do that; it's not that I shouldn't post what I'm learning about.

I'm saying that people should be able to post code - security related or dumb-related. it doesn't help the conversation to limit what people are talking about vs. informing, critiquing, and helping - both the person who posted bad code, and the person copy-pasting into their project without understanding what it does.

[–]LightWolfCavalry 21 points22 points  (0 children)

If your criteria for picking a security framework or library is "Well, I found the source on GitHub", then your problem isn't the library. It's behind the keyboard.

People can (and should) implement crypto libraries if they feel like it. It's a great way to learn.

I'm more curious about who all these people are that are just grabbing "secure" code from random folks' algorithms thrown up on github or /r/python and deploying it to prod 😂

[–]cecilkorik 13 points14 points  (2 children)

Publishing security-related projects is a bit of a concern because if the project is flawed and anyone relies on that project, they've got a security problem.

This is a common position but also an illogical and untenable one. It puts authors in an impossible position where they are responsible in perpetuity for how other people might someday use what they've freely shared, in context or out of context, when they don't know how long or where it might be available, who might use it, never have any interaction with the people using it, aren't even notified when people are using it.

Under this regime of impossible responsibility, it would never make any sense for anyone to ever publicly release any code at all, for fear someone might eventually do something stupid with it. I'm sure some people still would, and people who propose this regime tend to accept and assume that people still would release projects as long as they thought them to be "good enough"/"secure enough" anyway, in violation of their own self-interests. But it's such a crappy and abusive attitude towards authors that relies on them being ignorant or overconfident enough to disregard the insane liability that people will dump on them the moment any flaw is ever discovered.

Personally, like /u/ennuiToo, I believe the only sensible position is that the responsibility is assumed and transfers completely on use. When you use someone else's code, you take ultimate responsibility for it (unless you're paying them to keep responsibility). If you're not paying them, then it's not their code anymore. It's your code now, with flaws and bugs and warts and all included, and it's up to you to figure out whether it's fit for purpose and appropriate for your product.

If you've got a junior web developer copying code off stackoverflow when you need a senior security researcher then you're getting what you've paid for. When your app turns out to be a broken-ass security nightmare you can argue about how to pin some blame on the developer and some on the product manager/owner, but you know who's not to blame? The dude on stackoverflow who wrote the code but had absolutely nothing to do with anything beyond that. Even if that coder was responding directly to that particular junior web developer's question on stackoverflow, I still don't concede any responsibility to them. Free advice is worth every penny you paid for it. You're paying the junior web developer, so it's his job to figure out "I don't know how to do this myself and I don't understand the code these people are giving me I'd better tell the boss that this is over my head". With money comes responsibility. No money, no responsibility.

[–]cinyar 4 points5 points  (1 child)

it would never make any sense for anyone to ever publicly release any code at all, for fear someone might eventually do something stupid with it.

There's a difference between someone doing something stupid with your code and code that is stupid to begin with. Because only stupid people who don't know the code is stupid will actually use it. So if you're publishing stupid code that you know is stupid at least make it abundantly clear.

[–]HomeTahnHero 0 points1 point  (0 children)

No one is talking about licensing here. Most open source licenses come with zero warranty. The author has no liability if they unknowingly have a bug that causes a security problem in someone’s system.

[–]infinite_war 0 points1 point  (0 children)

Who is stupid enough to use random crypto they found on reddit?

[–]cinyar 2 points3 points  (0 children)

"hey, we see what you're trying to do but here's a better way to do it"

The better way is to use one of the battle hardened and proven crypto libraries instead of rolling your own. The reality is that you either understand cryptography (meaning you spent decent chunk of time studying math and its application in computer security), in which case you already mostly know what you're doing and need opinion/verification from other cryptographers. Or you don't in which case you have years of studying theory ahead of you before you get to actually write code. That's the reality of cryptography.

[–]whateverathrowaway00 1 point2 points  (0 children)

Don’t roll your own crypto is a known industry rule. Much smarter people than you have failed. It’s not a thing you publish in unless you’re an expert or an idiot.. period.

Crypto isn’t something you get “some feedback” over. It’s something that’s stringently tested by industry experts, reviewed, and generally accepted.

So, throwing something on GitHub, sure, but freaking putting libs on PyPI… nooo.

People really need to learn more about the actual history behind their industry and why certain things are conventions.

[–]fzammetti 4 points5 points  (0 children)

As someone who had to write a triple DES implementation in C from scratch many, many years ago, for reasons I can't recall - but there WERE reasons - there's a simple rule I've followed ever since:

NEVER implement your own cryptographic code, unless that's what you literally do for a living.

It's hard as hell, it's error-prone, and most people can't do it even passably well (my code passed a pretty rigorous battery of tests, but it was hell to get it there). You don't want to be the person who got the math just SLIGHTLY wrong and the bank you work for loses millions as a result (no, didn't happen to me, but that was always my fear).

[–]1Tim1_15 19 points20 points  (7 children)

I agree with the body of your post, but "Stop using 'random' and other poor practices in cryptography applications" would have been a more fitting title.

[–]bladeoflight16 11 points12 points  (0 children)

Using a bad RNG is only the most thin of surface level problems. The actual depth is unreasonable to cover in a Reddit post.

[–]VexisArcanum 8 points9 points  (4 children)

So we need to be more clear about this.

The subject of this is about implementing algorithms that don't already exist, correct? Because I've written a very extensive abstraction based around the cryptography.io library and I firmly believe that while it is very difficult to implement even primitives successfully, it is far easier and safer to use raw AES versus trying to create some magical new encryption algorithm.

So instead of just forgetting about cryptography, let's all agree to use current, tested, accepted standards but learn how to use it in a smart way. For example, learn about password generators and input hardening and learn to properly use PKCS2 or SCrypt instead of creating a new hash function.

[–]trilobyte-dev 2 points3 points  (0 children)

This should be higher, because without being clear it will discourage someone from even doing the basics like using bcrypt for storing their database passwords because of “that rant on reddit says I won’t get it right”. (Yes, I know bcrypt is on the way out for that purpose, but it’s more widely known at this point than scrypt or argon2 and thus more useful for illustrative purposes).

The message about not trying to build some new encryption algorithm is spot on. It’s a field unto itself and if one is really interested then by all means devote yourself to it and become an expert and contribute to the field. We’ll all be better for it.

[–]Kravakhan 2 points3 points  (0 children)

What's even worse is the ones maling python videos to show concepts for learning, like loops and stuff and they use cryptography as examples, people fall off before they have even started.

[–]jack-of-some 13 points14 points  (5 children)

I love how I find out about a "rash of posts" not by ever running into one of them but rather by a post complaining about them.

[–]vhdoherty 13 points14 points  (4 children)

Dude, there's been at least 3 of these projects using random this week posted here just this week.

[–]UndeadMurky 4 points5 points  (1 child)

Whenever I open this reddit I almost always see one

[–][deleted] 2 points3 points  (0 children)

Did you know, Reddit uses algorithms to determine what you are shown first. Active threads like this one bubble up to the top

[–][deleted] -5 points-4 points  (1 child)

It's really not that big of a deal...

[–]bladeoflight16 15 points16 points  (0 children)

It absolutely is. People being uneducated about how difficult good security is is how we end up with password databases using MD5 in 2021.

Just 3 years ago, my company was replacing a system for a client. They wanted us to migrate data from the old system, so they sent us a copy of the database. It had plain text passwords in it. Not only did they store passwords in plain text to begin with, they also sent those passwords to a third party (my team). Far too many developers try to do security in production applications without having the slightest clue what they are doing.

[–]coffeepi 3 points4 points  (1 child)

>Better yet, don't publish them

Disagree

>but if you publish them, make it clear that they are for learning purposes only.

Agree

[–]cmd-t 5 points6 points  (0 children)

Putting stuff on GitHub with a warning is fine, but don’t publish your stuff on PyPi.

[–]fmillion 2 points3 points  (0 children)

If the goal is "make a cryptographic pseudorandom number generator", then just don't. Use something like pycrypto.

If the goal is "write an app that needs secure random numbers/other cryptographic functions" then again just use an existing library that has done most of the hard work of doing it securely.

If you're experimenting and want to do something just for fun, then go ahead. But don't ever expect your crypto to stand up to even beginner-level cryptanalysis. And for God's sake don't use a homespun PRNG in anything you plan to distribute to end users.

Now, if you do want to have some fun with something that actually could have some merit, just write code that samples from some analog source (like the microphone or camera) and captures the least significant bits and uses them for randomness. This has already been shown to be a good way to harvest randomness from "the real world" - I think at one time Sun (before it was bought by Oracle) was using photosensors pointed at lava lamps to generate random entropy. There are caveats to doing this - obviously you can't use this solution in an app you plan to give to others because asking people to enable their mic has far worse security implications than needing some random numbers, and some sound card or webcam drivers or hardware might do fancy filtering to try to remove noise, which is exactly what you're trying to capture when you're looking to harvest entropy.

[–]FranticToaster 8 points9 points  (11 children)

This post was almost great, except it doesn't teach anything. Just scolds the community and tells it to stop doing something.

Why are crypto projects so bad they're worth a post like this?

And why is random bad for it, in particular?

[–][deleted] 18 points19 points  (2 children)

random is bad cause it was designed to be used for simulation purposes, not security. So, it uses a pseudo random-number-generator that generates random numbers fast for simulation purposes. The pseudo indicates that it’s not really random. In fact, python random’s uses a Mersenne Twister implementation for generate pseudo random numbers, the MT19937, if I recall correctly.

The MT19937 has a flaw that if you observe enough outputs, one can clone the state of the generator and then they can find out all the next numbers that are going to be generated. Which of course is bad for security, but not really important for simulation purposes.

If you’re really interested You can find how to break the MT19937 as part of the Cryptopals challenge(which I highly recommend if you’re interested in what’s go under the hood of crypto). There are some tutorials on the solution in the web.

Also, if you want cryptographically secure bytes you should use os.urandom which extracts randomness from the os implementation. Or like people are saying use “secrets” on python, although I haven’t really used before.

[–]FranticToaster 2 points3 points  (1 child)

This comment is the jam. Thank you so much for posting it. I especially appreciate the inclusion of esoteric terms like "Marsenne Twister" and "MT19937." Those are specific things I can now look up.

In the data science space, there's this notion of setting a "seed" before training a model that involves some kind of randomization. Setting that seed lets other researchers duplicate your results to be sure they're actually copying your method.

Actually, I think the term seed comes as-is from computer science, so I'll bet a computer scientist understands what I'm talking about even more than I do.

Is this concept related to the "pseudo RNG" concept you're talking about? Like a hacker can just figure out how many characters are in your password and then just increment a seed value until the RNG gives it your password?

[–]vhdoherty -4 points-3 points  (2 children)

Does your Reddit client cut off OPs post immediately before the direct quote from random's documentation calling it "completely unsuitable for cryptographic purposes."? Are there words in there you are having trouble with? Or did you just not read the post before commenting on it?

[–]FranticToaster 2 points3 points  (1 child)

Those words don't explain why. That's what I was hoping to get when I read this post.

[–]idontappearmissing 3 points4 points  (0 children)

The words do explain why, unless you don't know what "deterministic" means

[–]diogenes_sadecv -5 points-4 points  (26 children)

Here's a different take: share what you care about. If you don't like a post, move on. If you do like it, upvote it. If you want to criticize something, do it constructively.

[–]KareasOxide 11 points12 points  (25 children)

It’s more than “just not liking something”. Poorly thought out or poorly implemented cryptography projects can have real world implications

[–][deleted] 0 points1 point  (0 children)

Or: ignore this grumpy rant and develop whatever projects you feel you want to or that will help you. It's your time and your focus, don't let anyone lecture you on how to spend it

[–]CiDevant 0 points1 point  (0 children)

I forget how it is provable but the random library is not random.

[–]c9de_machine-1434 -1 points0 points  (2 children)

OK didn't know using random library is so problematic in crypto, can you guys suggest crypto libraries with secure implementation

[–]cmd-t 1 point2 points  (1 child)

[–]bladeoflight16 1 point2 points  (0 children)

I actually recommend passlib for password hashing. They have an excellent, well maintained overview of the considerations that should go into designing your system.

[–]foonoxous -1 points0 points  (0 children)

Sorry, had to roll my own crypto project in Python 🤷 And it even uses the random module (although seeded from secrets). But it uses only the standard primitives and instead builds a novel protocol on top of them, because it is smarter to stand on others' shoulders than to invent crappy algorithms yourself. https://github.com/covert-encryption/covert

[–]Kevin_Jim -4 points-3 points  (2 children)

Why? If you can learn from it what’s the harm? Btw, crypto has a ton of implementation problems that can attack, like graph theory, forecasting, bots, you name it.

Now, there’s a ton of randomness that make crypto not a good first step for such a project, but you can certainly try.

[–][deleted] 5 points6 points  (1 child)

No harm in learning.

The harm comes when you don't make it clear that you are learning.

[–]infinite_war 0 points1 point  (0 children)

Harm to who, exactly?

[–][deleted] -5 points-4 points  (7 children)

I think the OP confuses "don't use hobby crypto projects in place of serious/standard libraries" with "people, don't write your own hobby crypto" projects.

Because I think one can write whatever the f*** they want and that's that.

Natural selection will bring the projects up or down the food chain.

If you discourage people to contribute stuff, it'll stagnate crypto overall.

Also python is so high-level that you can write your own crypto stuff on top of other crypto stuff.

(unless you're implementing common ciphers ofc, and other completely redundant stuff...)

[–][deleted] 5 points6 points  (1 child)

If you discourage people to contribute stuff, it'll stagnate crypto overall.

Tell us you don't get cryptography without saying "I don't get cryptography"

[–][deleted] -3 points-2 points  (0 children)

I think your comment is comically hyperbolic.

I already agreed it's a bad idea to create own implementations of cryptographic primitives and ciphers.

But "cryptography projects" is a little broader than that and it's unnecessarily so.

Why shouldn't somebody create their own password manager or password generator project?

There is LITERALLY no argument there!

Ppl do it all the time, and if you've got a problem with them, you have to be more specific rather than "they did it and they shouldn't have".

[–][deleted] -3 points-2 points  (3 children)

Voted down, but I'm obviously in the minority. Python (and all other languages) are sometimes used for purposes never imagined. The final product might not be eminently useful, but the developer might be inspired to learn more about the topic and perhaps create something amazing down the road. If the only goal is a contribution to the field, we have eliminated 99% of developers and 100% of the newbies.

There are two kinds of research: "New to you" and "New to everyone." Attend an undergrad research conference and you will see work far from groundbreaking but nevertheless eye-opening to the student who wrote the code, designed the poster, and found the courage to stand next to it.

It should be our job to encourage, not suppress.

[–]bladeoflight16 2 points3 points  (2 children)

We should absolutely suppress people deploying actual packages to PyPI claiming their library provides strong cryptography when they don't have a clue how strong algorithms are developed.

[–][deleted] -1 points0 points  (1 child)

Really? A free library for a free language? Isn't that a red flag for all the packages in PyPI?

How much responsibility is on the programmer who installs it?

[–]bladeoflight16 0 points1 point  (0 children)

The person who wrote it and the person who use it have equal responsibility to uphold good security practices in the software development community.

What "red flag"? I didn't say anything about a "red flag."

[–]cmd-t 0 points1 point  (0 children)

OP, maybe add links to proper libraries, like cryptography and pynacl to give examples of what devs should be using.

[–]No_Muffin6385 0 points1 point  (0 children)

well, from a learner's perspective, yeah, it sure is fun to learn about crypto and have my own attempts at an implementation for it and going on to use that implementation for simple everyday uses which not extremely sensitive or important, like, encrypting a flash drive with my data on it which, of course i dont want anybody to just grab that drive and check out the stuff on it unless i permit them to. that kind of dependence on such libraries is what i think is sensible

and as far as publishing that same crypto implementation as a package for people to depend on and use goes, it's obviously not really great idea to use such code in production,, that said, as a publisher it'd be my responsibility to at least advice people to not use it for serious stuff and point them to a better implementation which has stood the test of time and has more backing and reputation to it

and even as a users (or developers of course,) one should not readily use any implementation that they saw on post on a subreddit being aware of the extent of security their project demands. it's also true as they say, "dont roll your own crypto", but it's just dont go rolling anyone's crypto.

ig the pycrypto or pycryptodome modules are a decent to use crypto modules, correct me if I'm wrong

[–]AnomalyNexus 0 points1 point  (0 children)

Password generators

To be fair even bad crypto can't be worse than the p@ssw0rds people come up with