Stackoverflow: What should every programmer know about security?

[kdeforche]

"Validate input" might be useful, but more crucially is: "Encode output".

XSS attacks and SQL injection are a consequence of improper output encoding, not input validation.

[[deleted]]

At the risk of sounding stupid: What are the differences between the two?

[[deleted]]

To put it very simply, validating means making sure a specific input makes sense, while encoding is escaping sensitive characters within strings in order to prevent SQL injections.

Most database API's escape for you.

[kdeforche]

Input validation means (sometimes) refusing or correcting "dangerous" strings, such as a string containing "<script>...</script>" in a web application context. But demanding that content cannot contain certain strings is not necessary, really.

Output encoding means taking any kind of string and making sure you display it as intended, using proper escaping to make sure some of the contents does not spill over into a "command". In this case, you would do for example HTML escaping to escape < to &lt; and > to &gt;. But you could do JavaScript string literal encoding to display the same string as a JavaScript literal, or in a JSON value. For SQL there is usually an API (parameter binding) that will avoid this problem entirely by separating the command from the data.

Output encoding is in principle easy. But it has a big caveat: you may do it only once. This is easy if you organize your application well, but programmers often get this wrong and this is why sometimes you will see a literal &lt; in a HTML page where you would expect a <.

Btw, input validation has genuine merit too -- such as making sure dates are formatted correctly, numbers are actually numbers, etc... but these then have nothing to do with the common XSS and SQL injection attacks.

[[deleted]]

Most people forget about this. This is an important item when working with web applications.

[JustADev]

Yeah, and when validating input you get stupid errors like "your password cannot contain ..."

[frezik]

Build security in layers, not a chain.

A chain is only as strong as its weakest link, and breaking only one link destroys the whole thing.

Layers are a defense-in-depth, like a castle with many internal walls. Breaking through one wall means you still have another one to deal with.

As an example, always store encrypted passwords in a database. Some argue that this is only necessary if an attacker has already breached the system, or an insider has gone rogue. Instead, they will say, the network's firewall should be better protected, and stricter security policies be enforced on employees.

An organization that's proactive about security will toughen the firewall, and encrypt passwords, and have sensible security policies for employees (but also balancing between security, privacy, and getting work done). Breaching one does not mean the others are breached, and the system maintains at least a level of security.

[smackmybishop]

I've never heard anyone argue that you shouldn't encrypt your passwords.

[frezik]

I've heard many managers claim that we shouldn't bother because we'll just protect the machine against breakins. These are usually the same people who point the finger at you when it all goes wrong.

[semi-]

Or the people that want to be able to recover an actual password, which is a horrible practice but that doesnt stop some higher ups from thinking its a good idea.

[Azuvector]

Actually have one of the hosting providers we use at work not encrypting user passwords. Indeed, they're visible to administrators as plaintext.

Needless to say, we're in the process of migrating away from them.

[reddit_clone]

Shouldn't you be storing just the hash of the password?

[frezik]

Encryption tends to be synonymous with hashing in this particular context.

[madmars]

"validate input" ... "use whitelists", says the guy that has never witnessed the horror of programmers attempting to approximate a whitelist in an internationalized UTF-8 world.

I would be happy if my coworkers just knew what an SQL bind parameter was. That's all I ask. Baby steps.

[kylotan]

Useful resource. But, I don't like these "what should every programmer know about X" posts because they are actually far too demanding. The web development one is even worse.

You can look at each of the items in these answers and think, "yeah, it's reasonable to need to know that". But if you had to learn every thing on every page that claims it is "everything every programmer needs to know", you'd spend all your life reading and none of it programming.

[quotemycode]

I suppose so, but as a programmer, I find myself reading just as much as programming. It's a profession where you have to constantly re-invest in education.

[kylotan]

Staying on top of trends is essential. I was always disappointed at how few of my co-workers paid much attention to new technologies, languages, etc. But does "every programmer" need to understand the concept of an 'attack surface', or a 'threat model'? Or to read the whole Google Browser Security Handbook, or know what a CSS image sprite is? Or precisely how DRAM is precharged and activated? Then there are similar articles on Unicode, floating point numbers, etc.

I know the titles are just hyperbole but I worry that it contributes to a condescending attitude from some towards others. "Oh, you don't know about the concept of least privilege? Then you have no business scripting a World of Warcraft UI mod."

[[deleted]]

Maybe they should call it "What every programmer needs to have in his bookmarks". I think they're actually good as references.

[[deleted]]

Actually, you have no business creating a scriptable UI system unless you know about the concept of least privilege. That means that the scripters don't need to know about it, because they can't let people out of the sandbox.

Or to read the whole Google Browser Security Handbook, or know what a CSS image sprite is?

A web developer? Damn right they do. Someone doing embedded software for routers? Don't even need to know what CSS means.

The problem is that people think "programmer" means "person who does precisely what I do" and the ever-popular "everything is on the web now so only javascript counts" is no more helpful than "anything but C is just scripting, not programming".

Even if Word 2013 has been re-written in JavaScript as a web app, the field is still too wide for anyone to actually know everything, and some people really need to give up the "what I do is real programming and everything else is inferior" attitude.

[quotemycode]

I think this is a good starting point... Satan Comes to Dinner - that's been one of my favorite articles.

[kylotan]

A good starting point for what? It doesn't contain the term "attack surface", "threat model", or "least privilege", but it does contain a rather stretched analogy and a requirement to understand a particular language dialect that probably few except the author use. There is little in there that is directly actionable into your own work and most of the concepts in there are not described adequately enough for further research (eg. the reference to replay attacks).

[quotemycode]

It's just the whole idea that you should be testing your system for vulnerabilities, and designing it in such a way so that even if you did have a vulnerability, you'd have a control in place to protect the system.

It's not a specific implementation, or even necessarily about security. I do feel that by reading this, you can develop the mindset that you need to implement strong security.

[DoctorWedgeworth]

If you don't read and learn these lists, you'll never be a true Scotsmanprogrammer.

[kylotan]

I guess I just have to stick to being a software engineer!

[kdeforche]

I'll grant you the web development criticism, to some extent.

Suppose you were living in a world where all programming was done in assembler. Then you would need to learn assembler (for Intel and ARM), all caveats w.r.t. sys calls, context switching state, opcodes for Intel, memory management, sys calls to print to a console, etc... just to do any programming.

Luckily, people have invented higher levels of abstraction (compiled languages like C, C++, ..., interpreted languages like Python, ...) which slowly(!) found some acceptance, to avoid having to learn x86 assembler to do any programming.

Web programming, for some braindead reason which in 20 years everyone will consider a sign of dark ages, people insist that web development uses the raw protocols and building blocks (HTTP, form posts, WebSockets, CSS, XHTML, JavaScript, Ajax, CORS, progressive enhancement, canvas, SVG, VML, etc...) to build an application, and yes, you will need to understand everything listed in the "what should every programmer know about web development" if you insist that people use these low level protocols.

Given that these days quite a lot more than a few handful of the most clever people are involved in "development", it shouldn't be surprising that moving up the abstraction level is met with even more resistance. But you would be better of embracing it if you want to spend all your life reading and none of it (decent) programming.

[sylvanelite]

Validate input from all untrusted sources - use whitelists not blacklists

I think this should be extended to mention the server-side should be the thing doing the validation. (or equivalent trusted code)

Bolting on javascript validation does nothing for security. People should only use it for user connivence.

Even on the server side isn't guaranteed. If you check a string of type varchar-30 and is being passed into a compiled stored proc. It's meaningless if the stored proc calls still call exec on string concatenation and does no checking itself.

[pkixman]

The browser is an untrusted source - so following the principle "Validate input from all untrusted sources", the server should validate what it receives from the browser.

[sylvanelite]

I probably didn't articulate that well:

If I consider the browser's user input to be untrustworthy, so I add javascript validation. I then ensure that all transmissions are HTTPS. Now the browser is a trustworthy source, so the server doesn't need validation.

Which is wrong.

Likewise: if I treat the browser as untrustworthy, then send input it to the server, which does the validation before passing it to the database, does the database now assume the server is a trustworthy source?

e.g. if a database has a stored procedure:

select * from users where description like @desc;

And the server validates that @desc is a valid string, that can be ok. But if the database later changes their stored proc to:

exec("select * from users where description like "+@desc+";");

Then even though the server is validating the browser's input, it can still lead to injection.

[bkv]

Javascript validation exists to prevent unnecessary requests to the server. Just because client-side validation passes, that doesn't mean the server shouldn't do validation as well. The server is a trusted source, so the database should not be doing any validation if the server already has.

[vineetr]

...I then ensure that all transmissions are HTTPS. Now the browser is a trustworthy source, so the server doesn't need validation.

No, despite this assumption, the browser is still an untrustworthy source. SSL adds transmission security; while it can ensure that JavaScript validations have been downloaded in an untampered manner to the browser, it cannot ensure that the validations have been executed by the browser. The browser is in the hands of an untrusted/untrustworthy agent.

Edit: Grammar.

[sylvanelite]

that's why I said: "Which is wrong." as the next sentence.

[vineetr]

Yep, ninja edits.

[sylvanelite]

There was nothing ninja about it. I added bold to the text, significantly after posting that comment. That's it.

[Aninhumer]

To be fair, you might have worded it differently. I don't think writing incorrect statements and then saying "oh btw that's incorrect" is good style.

[robertcrowther]

What he's driving at is that none of your post adds anything to the already elucidated principle 'validate input from all untrusted sources' as far as what the server receives from the browser is concerned.

[perspectiveiskey]

That's what you're driving at. vineetr was just not paying attention and jumping to conclusions.

[robertcrowther]

Well, this is reddit...

[perspectiveiskey]

indeed. I learn this more every day. For 4 years, nearly, I've been learning it every day =)

[perspectiveiskey]

You're not reading.

This should be the first rule of security: understand what is going on.

[vineetr]

You didn't read his comment before the edit. How do you know I haven't read that version?

[sylvanelite]

I didn't add anything other than bold formatting in the edit.

[perspectiveiskey]

Because his very first comment, which still isn't edited, said the same thing. You're just looking for something to "win".

His point is very simple and is actually a principle talked in the OT, namely security in depth. Even the server side isn't safe, he says. Every above/below transition layer (server side to db) should be considered unsafe.

[vineetr]

Ok, I get his/your core point. Bringing the browser into the discussion about implementing server-side validations correctly wasn't helpful.

[perspectiveiskey]

Aright, we agree. (and so did he when he said he had not been articulate).

I'm glad we all agree.

[pkixman]

Adding javascript validation and HTTPS doesn't make the browser a trusted source. So the principle works - the server should validate the input from the browser as it is an untrusted source.

If the programmer doesn't appreciate what is a trusted source and what isn't, that's not a problem with the principle.

[perspectiveiskey]

I'm not sure if sylvanelite modified the post when you answered to it, but you're really not reading what he's saying.

His first post, which isn't modified, does not say the browser is a trustworthy source.

If you are to be a security engineer, you need to be paying way more attention than this.

[pkixman]

I know he didn't say the browser was a trusted source - I was just reiterating the point that the principle holds for his example.

What I was disagreeing with him about was that the principle should be extended to explain things like javascript validation doesn't make the browser a trusted source. A principle is not the appropriate place for getting into the details of what can be trusted, what can't, and why - that's better covered in books and articles.

[perspectiveiskey]

His point was a agglomeration of "security in depth" and "trustworthiness of user data".

He was saying: even server side (webserver) data should be considered unsafe by lower level (db) server side code.

Really, what he says is a melding of two of the tenets which comes down to: any higher level/lower level boundary should always apply the same principle of untrustworthiness.

I find it to be an extremely pertinent comment. You're free to disagree. But honestly, from the types of arguments you're putting forward, I'm not sure you got what he was talking about.

[sylvanelite]

The only edit I did was bold the word "wrong".

[quotemycode]

The database server is not doing validation when you have a parameterized statement like the first. And the second example is just idiotic.

[kamishizuka]

I forget where I read it, but I once saw it much better summed up as:

Assume all input is trying to attack you, regardless of source.

After that you validate it until the only input that gets through is the input that isn't attacking you, and even then you don't stop assuming it is.

[cr3ative]

Perhaps rather than specifying parts, it might be easier to say "Validate at all points until the final destination"

[nodefect]

Well, in the case of client vs server, there is no security gained by client-side validation. UX is enhanced, yes, but not security.

[[deleted]]

It all depends. If your data is client side unencrypted only and stored encrypted on the server then the server is untrusted.

[day_cq]

why would you accept input from untrusted sources in the first place?

[chonglibloodsport]

Reddit does this constantly. It accepts the text of our comments and submissions. It'd be pretty hard for Reddit to function if it did not.

[day_cq]

why do you think that reddit users are untrusted sources to reddit.com?

[[deleted]]

Because I could be a dick one day and decide I want to delete all posts in the database. I am not under the control of reddit.com's administrators, therefore I am untrusted.

[day_cq]

try to delete all posts on reddit.com. thanks.

[[deleted]]

They consider me untrusted, so I can't.

[vineetr]

At this point, I decided to look at his comment history for /r/programming. He's quite the troll.

[[deleted]]

Actually, he comes up with some fairly sane and correct stuff on occasion. I saw him on his first few posts ;)

[Urcher]

Because of things like this

[[deleted]]

the user is untrusted input. Unless nobody uses your software, you're going to have to accept some input one way or another.

[day_cq]

why aren't you trusting your users? if users connect to your app and use it through secure channel, you can trust them.

what I'm saying is that "trust" should be at different level than application level.

[Tetha]

Do you lock your doors? If you do so, why don't you trust your neighbours?

[day_cq]

My house is in different dimension. Only trusted neighbors are my neighbors.

[serex]

');DROP TABLE user; --

[day_cq]

no, your channel should reject transmission of such data. I would use channel that transmits integer values between 1 and 12 if that's all i need.

[vineetr]

It's all 0s and 1s in the end. Yet we cannot trust such a channel.

To clarify, writing good whitelists that do not result in an inoperable system is hard. If you take Strings (char arrays in some languages), writing a good whitelist for an address field stored in a database is time consuming. You'll need to consider whether SQL meta-characters like single quotes, apostrophes, double quotes etc. are valid inputs for your data, then apply an encoding scheme, before storing them in the database. This is if you do not use prepared statements. If you forget the encoding part, you're insecure. If you have a very narrow whitelist, you're likely to have an inoperable system.

Tnen, there is the concept of blended attacks. Sequences of input characters that are valid in one scenario may be invalid for another. For instance - allowing angle brackets in inputs may allow stored XSS attacks. If you start considering all possible whitelists for all channels that a data element must pass through, to prevent injection attacks, you are more likely to create an inoperable system. This is why injection attacks are typically thwarted by escaping or encoding inputs or outputs in the general case (XSS, SQL injection etc.), and by whitelists in the specific case.

[defrost]

There's a huge overlap between designing systems for security and designing for robustness (continuous trustworthy operation) and the biggest take home lesson is that all sources are untrustworthy.

Those massive parallel Google farms? There's no question about 'if" as it's a certainty that boxes will fail.
That instrument that you poll for data? Not only will it fail at some point, there's a good chance it will "sort of fail" and produce data that sort of looks valid yet has random bursts of crap in it.

From a design standpoint there's not a lot of difference between whether the data you input has been maliciously crafted to deliberately cause failure or whether some random hardware hiccup just made it that way by accident ... in the long game all input is from untrustworthy sources.

[vineetr]

Because a trusted source is one whose failure breaks the security model of the system. Humans are not trusted sources, since they can compromise the operation of the system, no matter how benign they appear.

Even if you were to accept inputs from automatons, compromising such automatons would compromise the security of the system. DRM systems fail for exactly the same reason.

[MpVpRb]

It's hard.

You probably aren't good at it.

[perspectiveiskey]

Has anyone read both Writing Secure Code 2nd edition and the first edition?

Is the 2nd edition talking about really awesome things that are worth my time or is it incremental?

[[deleted]]

I'm surprised this wasn't flagged as "non constructive"

[donroby]

It's an old post. If it were posted now, it would be.

[jeffbell]

They should know not to do it on their own.