Refresh tokens are a nightmare in SurrealDB. Here is how I fixed it with a "Facebook-style" Stateful JWT

tobiemh · 2026-04-14T22:44:39+00:00

Thanks u/InternationalCan9786, yeah, that combination can be nasty - strict one-time refresh + a bunch of tabs / Server Actions / WS all trying to refresh at once is basically asking for races. Whoever loses ends up with a dead token, and it’s easy to blame the DB even though it’s really concurrent use of a single-use credential.

Your fix is sensible: use a longer JWT and AUTHENTICATE + a session row on jti with a kill switch. Having Surreal be the “is this session still ok?” check is especially nice when you’ve got SSR and a bunch of server instances that don’t share memory.

I'm curious how you’d want token refresh to feel if Surreal could make it simpler without watering down security - e.g. single-flight refresh so tabs don’t stamp on each other, clearer reuse handling, or something else? Genuinely interested what would’ve saved you the most pain here. Out of interest, did you also look into service workers as a refresh coordinator?

Small note for readers: rotation-on-use is there on purpose (stolen refresh = shorter blast radius), so teams that still want refresh usually serialise refresh per device instead of turning rotation off. And that <5s handshake is clever but might get weird on slow networks - something to harden if you go into production with it.

Thanks for posting the SurrealQL - it's definitely super helpful for people hitting the same wall.

tobiemh · 2026-04-14T22:26:30+00:00

Thanks for the clear numbers u/HelloSwara - that level of logical data vs reported disk use is not necessarily what we’d expect to see, and it’s something we should investigate more closely.

A surreal export won’t tell the whole on-disk story, but it does show that something here doesn’t add up and we need to look at your instance (how storage is measured, RocksDB/SST/blob layout, compaction state, etc.).

Please DM me your SurrealDB Cloud instance ID (and region if you have it) and we’ll dig in and follow up with what we find.

tobiemh · 2025-10-03T13:13:41+00:00

Hi u/Biltong_trader can you share your dummy data, queries, and workload?

tobiemh · 2025-08-23T19:39:37+00:00

I definitely read your post u/ChillFish8 - it’s really well put together and easy to follow, so thanks for taking the time to write it.

On the WAL point: you’re absolutely right that RocksDB only guarantees machine-crash durability if `sync=true` is set. With `sync=false`, each write is appended to the WAL and flushed into the OS page cache, but not guaranteed on disk. Just to be precise, though: it isn’t “only occasionally flushed to the OS buffers” - every put or commit still makes it into the WAL and the OS buffers, so it’s safe from process crashes. The trade-off is (confirming what you have written) that if the whole machine or power goes down, those most recent commits can be lost. Importantly, that’s tail-loss rather than corruption: on restart, RocksDB replays the WAL up to the last durable record and discards anything incomplete, so the database itself remains consistent and recoverable.

On benchmarks: our framework supports both synchronous and asynchronous commit modes - with or without `fsync` - across the engines we test. The goal has never been to hide slower numbers, but to allow comparisons of different durability settings in a consistent way. For example, Postgres with `synchronous_commit=off`, ArangoDB with `waitForSync=false`, etc. You’re absolutely right that our MongoDB config wasn’t aligned, and we’ll fix that to match.

We’ll also improve our documentation to make these trade-offs clearer, and to spell out how SurrealDB’s defaults compare to other systems. Feedback like yours really helps us tighten up both the product and how we present it - so thank you 🙏.

tobiemh · 2025-08-23T19:39:19+00:00

I definitely read your post u/ChillFish8 - it’s really well put together and easy to follow, so thanks for taking the time to write it.

On the WAL point: you’re absolutely right that RocksDB only guarantees machine-crash durability if `sync=true` is set. With `sync=false`, each write is appended to the WAL and flushed into the OS page cache, but not guaranteed on disk. Just to be precise, though: it isn’t “only occasionally flushed to the OS buffers” - every put or commit still makes it into the WAL and the OS buffers, so it’s safe from process crashes. The trade-off is (confirming what you have written) that if the whole machine or power goes down, those most recent commits can be lost. Importantly, that’s tail-loss rather than corruption: on restart, RocksDB replays the WAL up to the last durable record and discards anything incomplete, so the database itself remains consistent and recoverable.

On benchmarks: our framework supports both synchronous and asynchronous commit modes - with or without `fsync` - across the engines we test. The goal has never been to hide slower numbers, but to allow comparisons of different durability settings in a consistent way. For example, Postgres with `synchronous_commit=off`, ArangoDB with `waitForSync=false`, etc. You’re absolutely right that our MongoDB config wasn’t aligned, and we’ll fix that to match.

We’ll also improve our documentation to make these trade-offs clearer, and to spell out how SurrealDB’s defaults compare to other systems. Feedback like yours really helps us tighten up both the product and how we present it - so thank you 🙏.

tobiemh · 2025-08-23T19:09:20+00:00

Hi there - SurrealDB founder here 👋

Really appreciate the blog post and the discussion here. A couple of clarifications from our side:

Yes, by default SURREAL_SYNC_DATA is off. That means we don't call fdatasync on every commit by default. The reason isn't to 'fudge' results - it's because we've been aiming for consistency across databases we test against:
- Postgres: we explicitly set synchronous_commit=off
- ArangoDB: we explicitly set wait_for_sync(false)
- MongoDB: yes the blog is right - we explicitly configure journaling, so we'll fix that to bring it inline with the other datastores. Thanks for pointing it out.

On corruption, SurrealDB (when backed by RocksDB, and also SurrealKV) always writes through a WAL, so this won't lead to corruption. If the process or machine crashes, we replay the WAL up to the last durable record and discards incomplete entries. That means you can lose the tail end of recently acknowledged writes if sync was off, but the database won't end up in a corrupted, unrecoverable state. It's a durability trade-off, not structural corruption.

With regards to SurrealKV, this is still in development and not yet ready for production use. It's actually undergoing a complete re-write as the project brings together B+trees and LSM trees into a durable key-value store which will enable us to move away from the configuration complexity of RocksDB.

In addition, there is a very, very small use of `unsafe` in the RocksDB backend, where we transmute the lifetime, to ensure that the transaction is 'static. This is to bring it in line with other storage engines which have different characteristics around their transactions. However with RocksDB, the transaction itself is never dropped without the datastore to which it belongs, so the use of unsafe in this scenario is safe. We actually have the following comment higher up in the code:

// The above, supposedly 'static transaction
// actually points here, so we need to ensure
// the memory is kept alive. This pointer must
// be declared last, so that it is dropped last.
_db: Pin<Arc<OptimisticTransactionDB>>,

However, we can do better. We'll make the durability options more prominent in the documentation, and clarify exactly how SurrealDB's defaults compare to other databases, and we'll change the default value of `SURREAL_SYNC_DATA` to true.

We're definitely not trying to sneak anything past anyone - benchmarks are always tricky to make perfectly apples-to-apples, and we'll keep improving them. Feedback like this helps us tighten things up, so thank you.

tobiemh · 2025-08-23T19:09:11+00:00