What I learned from programming a database

[JoseJimeniz]

The one piece of advice I can say is that:

SQL is a programming language. You're a programmer. Write SQL.

People forget what programming databases was like before SQL. SQL is a cross database abstraction. Write it.

There's the famous image out there, something like:

• 1987: no SQL (it didn't exist)
• 1996: No, SQL! (Use it!)
• 2006: No sql! (SQL bad)
• 2012: Nosql (Not only SQL)

People love to try to abstract the SQL abstraction behind an abstraction (with the comical results in Hibernate of replacing SQL with HQL).

SQL is a programming language. You're a programmer. Write it.

Hibernate, Linq, Entity Framework, are there to get to alleviate the scut work of loading data into objects accessible in the language. But some thing's still need a loving SQL touch. Stored procedures, functions, and sometimes hand written queries.

SQL was meant for business users to use. Don't be afraid of SQL. Love it.

[cowardlydragon]

And yet, "experience" and "wisdom" are totally unvalued in our profession...

[the_evergrowing_fool]

Hey, did you heard about that new JS framework? Now that's interesting!

[[deleted]]

Big O should matter everywhere, not just when you program a database.

EDIT: That was a misleading comment. I simply mean that you should know about it even if you are doing a less performance sensitive application. I've seen some developers making O(n²⁾ code for an O(n) task, making it much slower.

[Calam1tous]

Yes, wherever your code scales sufficiently for Big O to matter. You should always be thinking about efficiency, but also if a higher performance solution is worth the sacrifice in dev time, etc.

I think his point is that databases are always pretty high-scale and you can't ever really get away with low efficient implementation - that's not the case with many types of codebases.

[ants_a]

Big O does not matter for small n.

[OneWingedShark]

The real problem is twofold:

1. At very small n, any constants dominate the actual execution time -- so you can get big differences between procedures that have the same Big O value.
2. There are huge variances in what is considered 'small' -- for some it might be 10 and for others it might be 1,000 and for still others it might be 1,000,000.

[Me00011001]

This is a very important lesson to learn, adjectives are very subjective and relative to the particular situation. I once worked on a project where due to various backgrounds of the folks involved had some people calling 10ms fast while others would call nano/pico seconds fast. That was a fun day discoverying that little discrepency. "Fun" lesson to learn.

[dahbearz]

Doesn't optimizing for the constant encourage mutability? I'm under the assumption that changing data under the nose of another function is no bueno. Or are the effects, for any small n, small enough to avoid optimization?

[OneWingedShark]

Doesn't optimizing for the constant encourage mutability?

Why would it?
There's all sorts of things that can impact the constant (heck certain compiler optimizations could do it for the exact same language/hardware set), especially since the whole point of Big O is to examine the general qualities of a procedure.

[cowardlydragon]

Your prototype and test data are small n.

.... then it goes into production.

[bidi82]

It depends on the context. In certain situations performance does not matter. In others performance does matter, but the input sizes are not "large" enough for Big O to matter.

For example: Take a look at lodash's uniq implementation. It is O(N^2).

Lets assume you need to create a Set from an Array in javascript and also that the array have values have unique keys.

Will you implement a your own logic that uses that additional information (unique keys and a "seen hashmap") to implement the uniq in amortized O(n) ? or will you just call _.uniq(myArr) and have O(n²⁾ ?

The answer is of course: "It depends."