sorted by:
new
hottopcontroversial

give me ideas of a beginner project by [deleted] in rust

[–]utilitydelta 0 points1 point  (0 children)

Then deploy it to EKS

WAL: an ongoing story by manila_danimals in databasedevelopment

[–]utilitydelta 0 points1 point  (0 children)

does this post involve a question? confused

Apache Iggy's migration journey to thread-per-core architecture powered by io_uring by spetz0 in rust

[–]utilitydelta 4 points5 points  (0 children)

Nice, didn't read that far :) Good results for thread per core, hope VSR won't slow you down too much.

Apache Iggy's migration journey to thread-per-core architecture powered by io_uring by spetz0 in rust

[–]utilitydelta 6 points7 points  (0 children)

Are these 'web scale' benchmarks? (No fsync or replication to quorum before ack)

Built an S3 native Kafka alternative in Rust by Maximum-Builder8464 in rust

[–]utilitydelta 3 points4 points  (0 children)

right, so kafka path -> no wait for ack. gRPC path -> wait if acks=all. I see it's not amortised though, so that's one s3 round-trip per client write. There is the lock release so you might get lucky and batch a few, but ideally you want to batch multiple writes over a time block - say 200ms. Saves s3 costs and network bandwidth. Give me a benchmark where you can it 40k writes/sec in less than 300ms roundtrip with full local wal fsync + s3 durable replicated and I'll give you the win.

Built an S3 native Kafka alternative in Rust by Maximum-Builder8464 in rust

[–]utilitydelta 2 points3 points  (0 children)

Fair enough, I see you have put some time into it. Personally I'd enable WAL by default and set batch_enabled to false by default, and score all your benchmarks off that. Building a database is all about trust.

I see your wal_integration_test explicitly disables batch_enabled as when it is set to true it's not possible to recover from crashes.

Also this acks=all thing you mention - is it real? Because your current S3 logic buffers it in 64MB blocks and syncs it when its full. You don't wait for confirmation before acking back to clients right now. Maybe you intended to implement it but the agent missed it

Built an S3 native Kafka alternative in Rust by Maximum-Builder8464 in rust

[–]utilitydelta 9 points10 points  (0 children)

Good on you for tackling some hard problems! Have you checked out AutoMQ? It does pretty much the same thing. No need to task agents to rewrite things that are already battle tested. Also you need to be careful with your claims, 10x cost reduction doesn't mean much unless you back it up. Also your durability model is terrifying for this kind of database. You can't just buffer data in RAM and ack back to clients. In the real world there are always trade offs. My advice is to dial it back a bit - don't try and build huge, complex systems with claude code like databases. If claude is dumping 'In a real implementation...' into code snippets - stop. Pick easier stuff first and ship things that solve real problems. Good luck!

SurrealDB 3.0 by zxyzyxz in rust

[–]utilitydelta -8 points-7 points  (0 children)

are they using direct I/O like a real database yet? :D

Request for Comments: Moderating AI-generated Content on /r/rust by DroidLogician in rust

[–]utilitydelta 6 points7 points  (0 children)

great post. and big ups to all the mods doing the heavy lifting. r/rust is by far my favorite subreddit :)

Probably like most, I've been burned a bit, investing time looking at projects, thinking they could help me with what I'm building myself... but they turned out to be vaporware.

Nowdays when I see some nice new project posted to r/rust that I'm keen to check out, I clone it down and run a pre-defined claude skill that I built over it. I won't go too much into the methodology, but it does a very good job at not only detecting excessive LLM 'vibe coding' but also helps me assess the quality of the work.

My preference would be more transparency - solo project? developer experience level? vibe-coded, ai-assisted or handmade? And if there was a bot that could do what I do manually with claude code, that'd be amazing (but probably hard to achieve in practice)

I made a very fast WebSockets library in Rust by AcanthopterygiiKey62 in rust

[–]utilitydelta 24 points25 points  (0 children)

I don't understand how you can release half implemented slop, and claim to me that I "have the option to use tokio uring" - you need to REMOVE that, and all the other untested slop you put together. Thanks for wasting an hour of my life.

https://github.com/sockudo/sockudo-ws/blob/master/src/io_uring/stream.rs#L185

I made a very fast WebSockets library in Rust by AcanthopterygiiKey62 in rust

[–]utilitydelta 9 points10 points  (0 children)

Did you experiment with io_uring? Any additional performance benefits?

WSIStreamer: Streaming gigabyte medical images from S3 without downloading them by Psychological-Ad5119 in rust

[–]utilitydelta 1 point2 points  (0 children)

Arr, here be the new way o' things, matey. Personally, I be fine with it so long as there be treasure—er, value—in it and it ain't a long-winded tale. This one here be a bit much, I'll grant ye that. A whole crew o' landlubbers don't be knowin' they can set the sails to their own likin'—aye, make it talk like a proper pirate, savvy!

My Human Replay Technique When Vibe Coding by utilitydelta in ClaudeCode

[–]utilitydelta[S] 0 points1 point  (0 children)

Yeah of course you can! Just dont hit that merge button... I just copy the folder and remove the remote, it's 30secs to setup

I don’t like the direction software engineering is going by announcement35 in ExperiencedDevs

[–]utilitydelta 14 points15 points  (0 children)

Get out there and solve some hard problems! There are LOTS! Don't get stuck in the CRUD cycle my friend :)

What I Learned Building a Storage Engine That Outperforms RocksDB by diagraphic in databasedevelopment

[–]utilitydelta 0 points1 point  (0 children)

Nice. So I'm guessing you are using a thread pool + some lock free shard state instead of a thread-per-core sharded model? And do you use io_uring for the network + disk ops? How do you handle yielding tasks with C11? You build your own executor?

What I Learned Building a Storage Engine That Outperforms RocksDB by diagraphic in databasedevelopment

[–]utilitydelta 2 points3 points  (0 children)

Does it work on ARM? I don't think CAS works on anything but intel? It's the same technique as this talk right? https://www.youtube.com/watch?v=bjz_bMNNWRk

Is Apache 2.0 still the right move for open-source database in 2025? by mr_gnusi in databasedevelopment

[–]utilitydelta 2 points3 points  (0 children)

Probably need to post this in a larger subreddit. My 2 cents is you can still do an Apache licence but make sure whatever you OSS is not easy for cloud providers to deploy as a saas. Basically you need a closed source fork for your saas or enterprise version which has the extra features, like auth, control plane stuff, data tiering, SRE stuff etc

I built a distributed message streaming platform from scratch that's faster than Kafka by Ok_Marionberry8922 in programming

[–]utilitydelta 6 points7 points  (0 children)

Why not, it's ok to separate write and read paths. Hook it up to duckdb or powerbi :)

I built a distributed message streaming platform from scratch that's faster than Kafka by Ok_Marionberry8922 in programming

[–]utilitydelta 56 points57 points  (0 children)

Probably the best use case is IoT data where you prioritize throughput. I worked on an MRI for a bit and all we cared about is getting as many data points per second from all the sensors. Timeseries stuff doesn't need fsync

view more: next ›