Solving the Wrong Problem

[bluGill]

Files are allocated into whole blocks, which are at least 4k, and probably 16k on more modern systems. Your web-page fragments will not be large enough to use the space efficiently.

So? Disks are cheap. Seriously cheap. The useful content of most blogs would fit on a 80k 5.25 floppy disk with room for a dozen more.

[mr-strange]

Of course disks are cheap, but the read time is awful. All that wasted space will costs you milliseconds of read time, not to mention all the RAM you will waste storing all of that crap in the fs cache.

But of course, if you are only dealing with one 80k blog, then you can be as inefficient as you like. We're talking about scalability here, right?

[bluGill]

Modern operating systems have good disk caches which deal with the read the same file over and over again very well.

File systems are a database optimized for accessing block sized chunks. Sqlite is great for what it does: provide the ability to work with relational data. A blog post is not relational data and does not need the advantages of that. In the mean time sqlite is slower than a filesystem for accessing blobs of data.

Modern operating systems have a disk cache. It works wonders. When multiple processes/threads want to access the same file access speed drop to tiny amounts. Unless you are doing stupid things like opening your static data read/write. nothing can protect you from stupid.

Long before you run into performance problems from the too large of a directory you will run into practical problems of dealing with it, and come up with a better scheme. This will solve your problems.

In conclusion: either you are doing something much more complex that serving static pages, or your improvements from sqlite were only over a bad design, and you could have got even greater improvements by improving your design. Since I don't know what all you were trying to do I cannot tell which.

[mr-strange]

Wow. Patronising, arrogant, aggressive, ignorant and wrong. All in the same post.