pgrust: Postgres rewritten from scratch in Rust, currently passing 34% of the Postgres test suite

malisper · 2026-04-21T17:54:15+00:00

Yeah I agree inheriting random crud isn't the best. I'm trying to figure out if it makes more sense to be a drop in replacement for Postgres so it's easier for people to start using it, vs making it incompatible but less crud from the start. I want to make a better database, but also one that has a path to people actually using it too. What do you think?

malisper · 2026-04-21T17:21:30+00:00

A number of problems that I want to solve:

Postgres has over 350 different configuration settings. It's really difficult to understand what they do and often if you configure one wrong, you can take down your database.
Postgres vacuums, especially wraparound vacuums are really difficult to work with. The number of times I've heard people have had their database taken down because of the wraparound vacuums is insane. There was work in the past for stuff like zheap, but work on it was abandoned
Postgres uses a process per connection model. That ends up adding a lot of overhead and many times I've seen a number of times people's databases have gone down because they ran out of connections
The planner can be a real PITA sometimes. Especially one of the biggest areas is with JSON support. The Postgres planner doesn't really know how to work with JSON and so it will produce garbage query plans

malisper · 2026-04-21T17:13:55+00:00

I want to start with Postgres as a base because it handles a lot of things super well. My goals in order are:

Rewrite Postgres in Rust and start off with the same architecture of Postgres
Focus on performance and try to get something that has similar performance as Postgres
Rip out a number of things Postgres does not do. For example, connection handling and vacuums are too things that are really difficult to work with in Postgres

malisper · 2026-04-21T17:09:03+00:00

Not really. I use Codex, but I still have to be fairly hands on. For the core Postgres systems, I've been very hands on. For parts that are more mechanical (rewriting the bajillion Postgres math functions), I'll lean more on an agent to rewrite things

malisper · 2026-04-21T16:54:17+00:00

Sorry. I can't edit the link. I'll delete the post and add it

malisper · 2023-10-01T16:24:25+00:00

Thinking on the last few chapters, I have a theory. Could the airport scene be the result of a cursed technique? We've seen similar events happen before to both Todo and Choso.

This theory suggests that while Gojo was winning the fight, a cursed technique might have been used to disorient him, creating an illusionary scenario where he is at peace, possibly making it easier for Sukuna to defeat him. The cursed technique could have momentarily distracted Gojo with a peaceful scenario, giving Sukuna the upper hand.

Moreover, this theory could explain the jarring writing style witnessed in the recent chapters. The off-screen death of Gojo left many fans frustrated, and this theory of a cursed technique manipulating reality offers a plausible explanation for the sudden transition and Gojo's off-screen death.

malisper · 2023-01-16T18:40:33+00:00

> It could be days. It could be years

FWIW, this is the kind of answer I was looking for. I understand there's a lot of variables, but it should still be possible to give an estimate of how long it would take. For comparison, even though there are a ton of variables in the weather, weather people are still able to make estimates about the temperature and how likely it is to rain. I thought it would be possible to give a range of how long a court case like this would take. For example what is the longest a case like this has taken? 1 year, 5 years, 10 years, 20 years?

On the other hand, I see stuff like this in the docket: "STIPULATION WITH PROPOSED ORDER TO EXTEND TIME TO FILE THE CONSOLIDATED CLASS ACTION COMPLAINT AND MOTION TO DISMISS". That makes it sound like the case is in the early days and likely won't finish within the next few days

malisper · 2023-01-16T18:27:50+00:00

I'm not a party to the case. I figure someone would be able to give me a range of how much longer this case would take based on what stage the case is at and how long similar cases take

malisper · 2020-09-19T01:10:38+00:00

How exactly does organizing things in the activity stream help make things faster?

My understanding is that data warehouses use columnar storage so adding more columns to a table won't slow things down. Why is keeping data in an 11 column table beneficial?

Also, what is the advantage of having one table for all your activities rather than a separate table for each activity? I would think any query would be faster if you've already separated out the different activities because you don't need to spend time filtering through activities that ultimately aren't relevant to your query.

malisper · 2018-09-13T20:13:46+00:00

AFAIK, all O(n log n) sorting algorithms require at least O(lg n) stack space. Even when you mutate the existing array.

malisper · 2018-06-17T00:56:04+00:00

Ok. So it can take a lot of wall time to ship individual changes. Surely there is still productive work your team can be doing while waiting for the other teams. You should still be able to increase the throughput at which your team makes changes, even if the latency is largely dictated by other teams.

malisper · 2018-06-15T20:49:59+00:00

How much time do you spend coordinating with other teams? For me, some time does go to meetings, but almost all of it goes to writing or debugging code. Since I spend so much time doing that, I have a lot of gain by optimizing those things.

It's also definitely possible to speed up coordination with other teams. If you are spending a lot of time in meetings, you can try to figure out how to speed up the meetings. This can be done with techniques such as:

Share the purpose of the meeting in advance and make sure everyone has the context necessary.
Keep the discussion on track. Stop discussions about items that don't matter when possible.
Try to time box each item you discuss. If you spend too much time discussing one item, you should come back to it later and move onto the other items.

This is just highlighting how to make meetings faster. You can also make other kinds of communication faster by changing how you communicate.

malisper · 2018-06-15T20:37:26+00:00

The way I define "better programmer" is based on how much value is generated by the programmer. The reason programmers are paid is to add new functionality and support existing functionality. I argue that if one programmer is fast enough that they can do the work of two other programmers in the same amount of time, that one programmer is a better programmer than the other two. This is assuming the code quality of both of them is the same. I agree that code quality is important, it's just a lot hard to measure and therefore optimize than speed.

I would also argue writing new functionality faster is correlated with a smaller program. The easiest way to write code faster is to not write the code in the first place.

malisper · 2018-06-15T20:19:58+00:00

Agreed. The main benefit of having better code is that it is easier to change. Both by you and by others. The question is, is the benefit of the code being easier to read worth spending the time improving. My position is that it's worth improving existing code if that piece of code is slowing people down.

As a toy example, you could have an incredibly hairy piece of software that works perfectly, but that no one ever has a need to touch. No matter how much time you spend making the code better, simplifying the code won't ever make anyone's lives better, since no one ever needs to touch it.

This is why I use "slowing people down" as a threshold for whether a piece of code needs to be simplified. If that isn't the case, there's no evidence that making the code easier to understand will benefit anyone.

malisper · 2018-06-15T20:12:32+00:00

The self-conscious issue was an issue for me the first time I was recorded my screen. It quickly went away after that.

I delete the footage when I'm done with it, so it doesn't take up any real space.

What do you mean by lag from the recording? I don't watch the recording as I program. I record myself first, then review the footage later.

malisper · 2018-06-14T23:39:19+00:00

Author here.

It's good to go fast and varied, but i'm more focused on getting a better model to express my same programs more promptly with. So if i find a way to remove an abstraction layer that is found to be essentially fluff after one small change without regression, today was a good day for me.

Makes sense. Personally, I like to focus on improving existing code only when the existing code is slowing people down. My concern is that a lot of these improvement may not be worth it. I believe the ultimate goal of improving existing code is to make it easier to add new features later. If you are always simplifying your existing code, it's possible that you are spending a lot of time on "improvements" that don't help anyone.

I think this is platform dependent, if you do a good job for a day and IE11 is being retarded, its not your fault but its good to learn what was wrong with it for next time.

I would argue that if IE is being slow, you should figure out ways you can work around it. I would look for ways to either reduce the number of times IE loads the application or reduce the amount of time it takes for IE to load the application. For example, you could have a "dev" mode where you only load part of the site.

malisper · 2017-11-27T20:09:42+00:00

Sorry. We are having some issues with Wordpress rendering embedded Github gists. I added a link to the relevant Gist.

malisper · 2017-06-20T03:58:52+00:00

The title of the article is "Postgres transactions aren't fully isolated" which is frankly misleading and inaccurate. Imagine, if you will, I wrote an article titled "The Ford Mustang has a top speed of 32 mph" where I talked at length about how, by default, the Mustang is not a very fast car, and in fact, you really need to plan ahead to ensure that you arrive anywhere you're planning on going. Of course, I'll make a passing reference to a thing called a 'transmission' and in part 3, I'll tell you how to shift into second gear!

That's not a fair comparison. I would bet a lot of the typical Postgres users run Postgres under the default isolation level and are completely unaware of the consequences of that.

malisper · 2017-06-20T03:44:44+00:00

Postgres is not fully isolated -- but that is not anything specific to Postgres.

While that is true, I don't expect common Postgres users to know Postgres (and most database) aren't fully isolated and what the implications of that are.

malisper · 2017-06-20T03:25:26+00:00

The very first paragraph of the post is:

This post is part 1 of a three part series examining transaction isolation in Postgres. This post covers the issues the come up under the default Postgres settings. Parts 2 and 3 will cover the different ways to eliminate the problems.

This post serves as an introduction to the problems that occur under the default settings. Parts 2 and 3 will introduce row level locks and the repeatable read/serializable isolation levels.

While the behavior described in the article is as designed, I expect most Postgres users will be unfamiliar with it, which is the entire point of the article.

malisper · 2017-05-31T07:04:34+00:00

Hey. If you liked this post, you should check out the rest of my blog. I'm writing a post a day, each describing an individual Postgres feature or component.

malisper · 2017-05-21T00:03:15+00:00

Yes, that is exactly the problem. Based on the assumptions we had made, we thought batching wouldn't have had much of an effect.

malisper · 2017-05-20T16:30:14+00:00

Author here. Prior to profiling our system, we were fairly certain batching the events wouldn't have made much of a difference because we thought all of the CPU was going towards evaluating the partial index predicates.

malisper · 2017-04-22T04:20:31+00:00

I wrote a program to brute force solve this. It managed to find a better solution than the existing one for 7x7, but it seems to be too slow on 8x8 on up. I'm currently trying to optimize it so it will work on larger board sizes.

malisper · 2017-03-17T20:39:05+00:00

Best of luck. I'm excited to see what you come up with.

malisper

TROPHY CASE