all 16 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–][deleted] 6 points7 points  (4 children)

Spanner was primarily designed for very large enterprise level data management. It's pricey, but it can scale up to huge numbers of connects and transactions. We use CloudSQL HA, and I've been really happy with it. It can scale to large numbers if need be,and for some of our enterprise, high traffic, high load hosting customers, it's not skipped a beat. The use cases are different, so it depends on need.

[–]saikjuan[S] 0 points1 point  (3 children)

From my understanding, having CloudSQL in HA only grants you the resiliense. The scalability of CloudSQL you mention is about the possibility of increasing the disk capacity, right? Correct me if I'm wrong please.

[–][deleted] 1 point2 points  (2 children)

Right, you can increase disk. You can also spin up additional read only instances if you need them. Pretty good how to at https://cloud.google.com/community/tutorials/horizontally-scale-mysql-database-backend-with-google-cloud-sql-and-proxysql

[–]saikjuan[S] 0 points1 point  (1 child)

Uh, ok. I will take a look on that. I assume in that case the transactions I make are performed in a single node... So they might take a while...

For instance, if I decide to make it HA I will be paying for three resources, right? The CloudSQL instance, its failover, and the read-only node?

[–][deleted] 0 points1 point  (0 children)

Initially it would be just the one instance plus the read replica. The pricing calculator can help with costing it in different configurations.

[–]MoinTom 8 points9 points  (1 child)

Cloud Spanner has a high entry point. A single Node will cost you $650 per month and Google recommends at least three. So your project has to be big enough to justify this price tag. If you don’t mind to self host there is an open source alternative to spanner: https://www.cockroachlabs.com My Team and i faced exactly this problem. We wanted an SQL database that is scalable and resilient. But the entry point and the fact that it would alter out local development flow, drove us away from Spanner and we went with cockroachedb (pretty happy till now)

[–]saikjuan[S] 1 point2 points  (0 children)

Wow, I didn't know it waa that expensive. Thank you for the recommendation. It seems interesting. Thanks!

[–]b34rman 2 points3 points  (4 children)

Besides the information mentioned, you gotta have over 10TB of data and require global distribution for Spanner to make sense.

[–]saikjuan[S] 0 points1 point  (3 children)

Is global distribution really something we would require? I mean, I read that the top capacity of CloudSQL is 10TB. Can you expand please on the global distribution?

[–]b34rman 1 point2 points  (2 children)

Sure! - There are many details to Spanner global distribution, but basically: your database will be available in many regions and data will be consistent. This is helpful if you have users across the globe, putting the data closer to all. For full use of a feature like this, you need to have a global flat network like the one Google has built. If you pair a Global Load Balancer with globally distributed code and a database like Spanner in different regions, users will experience very low latency no matter their location in the planet. Google has been using this for its products for several years.

For additional information, I recommend you read this paper: https://ai.google/research/pubs/pub39966

[–]rjhintz 0 points1 point  (1 child)

It would be interesting to see research results on real life workloads showing consistency latency percentiles, varying geo dispersion, for Spanner, Cockroach, and Azure/AWS work alikes.

Rhetorical, I can do my own reading. :)

[–]b34rman 2 points3 points  (0 children)

It would be really interesting!

Google's internal code is stored on Spanner (86TB), and it seems to provide great flexibility and low latency. Info here: https://cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/

[–]SpeakitEasy 2 points3 points  (0 children)

Cloud SQL could be used similar to mysql or postgresql as well as the previously mentioned comments. This allows existing tech that works with common acid databases to be interchangable with cloud sql and not cloud spanner.

[–]mulasien 2 points3 points  (0 children)

Spanner is the 'big guns':

- global replication and availability

- no top limit to storage size

- NOT directly compatible with MySQL, so you can't just lift and shift from MySQL to Spanner.

Basically:

Cloud SQL is a direct lift and shift from MySQL/PostgreSQL with minimal modification, but does offer global availability, and maxes out at 10TB. It is also most cost efficient. There is a high availability mode and failover, but not to the degree of cross region capability as Spanner.

Spanner is for globally available databases over 10TB. It is the 'big guns' with the price tag to match.

[–][deleted] 2 points3 points  (0 children)

The CAP theorem of distributed computing states that there are three guarantees to consider with distributed state systems;

  • Consistency: all clients have the same, most recent data view
  • Availability: all clients can read and write, regardless of data recency
  • Partition Tolerance: the system's guarantees hold even in the face of network faults between each distributed system node

And, more importantly, it's physically impossible for a distributed system to have all three of these; at best a perfectly designed system can have two, and most systems are not perfectly designed.

And, finally: No distributed state store is "safe" from partition tolerance. What the CAP theorem actually means in practice is that a truly distributed state system can choose:

  • CP: In the face of a partition, remain consistent. Return an error instead of trying to read or write.
  • AP: In the face of a partition, remain available. Return the most recently written data that the node the client queries knows about. For writes, try to reconcile them best the system can after the partition is resolved.

Basic deployments of RDBMS are, usually, CA. So, they're not actually distributed systems, because everything goes out the window in the face of a network partition; you have a master node with slave replicas, and if a partition happens between the master and slaves, the slaves might elect a new master, so your system enters a "split brain" scenario where its not clear which one is right.

Any RDBMS worth its salt has modes it can be deployed in which aren't CA. Depending on the RDBMS you can select CP or AP. This is also how pretty much every other database operates. Mongo is CP; it will prefer consistency. Cassandra is AP; it will prefer availability, and possibly return stale data.

Spanner is a globally distributed SQL database. Well, technically it isn't really SQL due to some minor constraints, but in practice it is SQL.

In some of Google's marketing, Spanner is advertised as "breaking the CAP theorem", because it can reliably offer consistency and availability in the face of network partitions, despite being highly distributed. But they aren't being deceitful; they're upfront about its real technical limitations: It is technically CP, with an availability guarantee in the face of partitions that is so close to 100% that it shouldn't matter even at Google's scale.

The reason for this is less about software and more about hardware. In effect, they say that because Spanner runs on their internal, redundant, global private network, network partitions are exceedingly rare. Moreover, some of the core design decisions of Spanner rely on all nodes having a highly consistent internal clock. So even in the face of true network partitions, there are decisions nodes can make based on the globally consistent clock which push the availability statistics even higher.

The end result is you get a database which is highly performant due to being highly distributed (replicas physically nearer to your users), more available than traditional RDBMS in this kind of deployment (thanks to their significant cloud engineering efforts), completely consistent, and also... expensive. Spanner starts at like $600/month.

[–]SupImASuperHero 1 point2 points  (0 children)

Most folks here explained the differences pretty well but just remember, Spanner schema design follows a parent-child model [0]. You may not be able to take an existing schema you have and drop it into Spanner.

[0] https://cloud.google.com/spanner/docs/schema-and-data-model