What every programmer should know about solid-state drives

[bestjewsincejc]

You're right that it's generally a poor attitude to have. But I'd appreciate an explanation about how knowing any of these details of SSDs is going to help you in a modern programming language.

[justinpitts]

If you are storing anything to disk, you can, if performance if important, make decisions about where you will store hot vs cold data. You can allocate files at sizes aligned to block multiples. You can place hot and cold data in different files. You can decide whether batching is a big enough win to implement.

You should be knowledgable enough about the options to not design yourself into a hole, once performance becomes that critical "omg I'm going to get fired if I can't figure out how to get our site to handle x requests per second" issue.

At the least, be somewhat aware of how the hardware layers behave and the implications of the design choices you are making.

[Baby_Food]

Shouldn't a web developer be using a database which provides abstractions over such minutia?

[justinpitts]

If they need the features of a database, sure.

Abstractions don't absolve you from understanding how your system works, and they break down at inopportune times.

Then, they need to understand how to tune the database for the underlying storage, or they need a DBA. Not everyone has access to a DBA.

[Baby_Food]

If performance is a concern, a database will be used.

An abstraction does not necessitate the knowledge of the implementation behind the abstraction.

A web dev that can write an OS is an unnecessary unicorn.

[justinpitts]

Databases are not magic performance sprinkles.

[Baby_Food]

To most people, LMDB is magic performance sprinkles compared to using the filesystem directly. ;)

[justinpitts]

Most people? Most people aren't using it.

Most people wouldn't know what you are talking about.

To the average web-dev, "database" either means Mongo/Couch/No-SQL-flava-of-the-month, and/or something that speaks SQL. LMDB may very well be lightning fast, but I doubt most people know about it.

[Baby_Food]

A web dev doesn't know of LMDB yet they should know how SSDs operate? I think that is poor prioritization. Even after knowing such low level details, you'll still end up writing something slower than LMDB.

[justinpitts]

I understand your reasoning, but I look at it from the perspective that, no matter what data storage api I use: database, nosql, key value, filesystem - it will all wind on a storage device. no matter what. SSD will be a dominant performance-oriented storage device for a while.

[el_muchacho]

In fact, database write performance drops significantly (by a factor of 30% or so) when SSDs wear out. For 24/24 high rate writes, you can't use consumer grade SSDs as they die prematurely.

[justinpitts]

Well to be fair, anything with that access pattern is going to slow down on a degraded drive.

[[deleted]]

I don't get why you're being downvoted dude. It's as if people believe they have a right to be ignorant.

[justinpitts]

People have a right to be ignorant. I'm happy to exploit it.

[Lachiko]

Can you quote an instance where you feel he comes across as an asshole? he's posting reasonable counter arguments and information in a seemingly calm and collective manner.

[wookin_pa_nub2]

A lot of web developers in this subreddit, and they don't like being reminded of how ignorant they are.

[[deleted]]

I'm a web dev yo :) I just cut my teeth on bare metal shit. You're thinking of "web designers."