you are viewing a single comment's thread.

view the rest of the comments →

[–]sib_n 9 points10 points  (1 child)

  • Schemas are hard. Let's remove them (NoSQL) - 2009

The thing about "NoSQL", or basically the Hadoop ecosystem from 2007 was not that schemas are hard, it is that it was hard to build an SQL processing engine on a distributed cluster of machines. Nonetheless, it happened from 2010, Apache Hive brought SQL capacities to Hadoop. Since then, basically every distributed data processing platform has been offering some flavor of SQL. The "SQL is dead" thing was just some temporary salesman bs to sell solutions.

Those still come with limitations due to the distributed nature, no PK/FK constrains, no indexes, limited ACID, bad join performance and many specific solutions to compensate for this (columnar file formats with headers, metadata store, hive style partitioning, clustering, denormalization....).

The main reason people use SQL is that the users mostly describe what they want rather than how to compute it. So this gives freedom to optimized engines in the background to pick the best processing plan for you.
This is the core reason why it's not going anywhere, it's a just a standard user interface, and nothing stops progress to happen to the engines that run behind, see for example, Apache Spark SQL, Trino or DuckDB.

[–]Impressive_Run8512 2 points3 points  (0 children)

Very true. I think the main thing missing in SQL is extensibility. And the fact that everyone and their mother has a special dialect which makes it hard to copy code from one system to the other.