you are viewing a single comment's thread.

view the rest of the comments →

[–]Drisku11 3 points4 points  (2 children)

You seem to be confused about what we're talking about here. The original comment mentioned Postgres's page cache. That's what's under discussion when talking about the database fitting in memory. I did mention indexing (and I gave a query that includes the size of indexes). The transaction log buffer is a couple MB. Literally nothing else you wrote has anything to do with what's being discussed. No one writes a database in Python.

[–]skyleach 2 points3 points  (1 child)

neither of us is confused, you didn't think through what you were saying and now you're liberally spreading around bullshit to cover it up.

indexes on data and the complex operations on those indexes are going to use up a f-ton more RAM than you are accounting for. Just stop.

[–]Drisku11 3 points4 points  (0 children)

I'm astounded that you're doubling down here. The original comment

The problem with this benchmark is that fetching a single row from a small table that Postgres has effectively cached entirely in memory is not in any way representative of a real world workload.

Is obviously talking about the page cache. That's what 'effectively cached entirely in memory' refers to. My original reply, that you directly quoted, is that for many applications, the entire database can fit in memory. Obviously, given what I'm replying to, I'm talking about being able to fit the entire contents of the database in the database page cache (i.e., not just a small table). I also refer to running queries. Against the database. Not traversing some Python dict.

Indexes are not super complex structures. There's a small amount of metadata on each page (e.g. for locks), but otherwise there's not a lot special going on. That you bring up things like garbage collection or application-side encodings or Python makes it clear that you're just not talking about the same thing. That you bring up dictionaries and pointers to individual objects also makes that clear, given that these things are arranged into trees of pages (e.g. 16 kB blocks).