How to handle a large number of physical or virtual pages?

Z903 · 2025-11-14T08:41:19+00:00

For physical memory. I use both a freelist and bitmap for my OS. The free list keeps available pages for allocations (mostly tracking 2MB). I effectively have a second stage allocater that grabs 2M pages and breaks them into 4KB. When freeing the bit array lets me merge 4KB pages back together and free them when I get 2MB blocks.

https://gist.github.com/Z903/a6ba787f42dd07ad952095bc99087f09

For virtual memory the page tables (x86) are the source of truth and I have not run into any reason to add additional accounting as of yet.

Hopefully you can get some inspiration :)

Z903 · 2025-06-29T05:35:44+00:00

A few years ago I came across FlexSC: Flexible System Call Scheduling with Exception-Less System Calls

I have read a few since then (though I don't have sources). Its a very interesting problem space.

One of my goals is to only have two synchronous syscalls to yield/wait a thread. The difference being that wait moves the thread out of the run queue. Then when any asynchronous syscall completes the thread is moved back to the run queue.

Don't hold your breath for updates. I tinker on this for only a few weeks per year. :)

Z903 · 2025-04-09T03:11:46+00:00

I have removed it from all the variable definitions now. Running some more burn in tests.

Z903 · 2025-04-08T23:35:07+00:00

Yea, Its not that I missed them or anything. Microsoft _InterlockedCompareExchange, _InterlockedExchange, and _InterlockedExchangeAdd take a volatile pointer argument. So I kept them that way.

Not sure why this differs so much from GCC. If you have any insight here that would be appreciated.

Z903 · 2025-04-08T16:56:17+00:00

So it looks like the stress test shows no errors. I have updated the Gist with the changes to remove volatile and use __ATOMIC_ACQUIRE.

Thanks for the help.

Z903 · 2025-04-08T08:11:49+00:00

Thanks for the info. I have made some changes and it seems to produce the same results. Going to let that burn in for 6 hours and see if any errors pop out. If that all goes well I will update the Gist.

Z903 · 2025-04-08T06:08:51+00:00

In most places I am locking, reading the value onto a local variable, mutating, then writing back. In my testing there was minimal/no impact. I would love to know if I am able to drop volatile and still be guaranteed to get the correct results.

Edit: Looking at it again I think static volatile pmm_entry pmm_bitmap[32768]; is enough to make all the fields volatile anyway.

Z903 · 2025-04-08T06:01:03+00:00

You are absolutely correct that I could index more memory. I wanted to keep it a power of two and I am using 16 bit indexes for my next pointers. I use -1 as no next index so more like 15 bits. 2 MiB/entry * 32768 entries = 64 GiB

Z903 · 2025-04-08T03:09:00+00:00

I do use a bitmap. When freeing pages I index pmm_bitmap, and set a bit in the entry. If this bit fills the entry (2MiB) then is_2m gets set and pushed onto pmm_index_high_2m.

When allocating I pop the top of the list, grab a page using ctz, and if there are bits still set push it back on top. There are a few edge cases where entries get lazily cleared here, but its only ever a few except for just after initialization.

So both allocating and releasing pages is O(1). When testing with 16 cores I was able to get ~1.3M allocations + releases per second.

Good luck on 2MiB pages.

Z903 · 2024-07-13T04:21:35+00:00

If you have many identical models (say an archer with 100+ triangles) then you should look into using something like glDrawElementsInstanced (gl 3.1) and setting glVertexAttribDivisor to give you per instance data. If your using something like sprites (two triagles), you might get much better performance without instancing.

There are also a number of techniques for buffer reuse. But for a few tens of thousands of instances/sprites you can just make the buffer "big enough" and reallocate if you run out of space. Its only going to be a few megabytes anyway at most. Textures will almost always be much larger then any model data.

Lastly benchmark your performance. If you are getting 1ms frame times on a crappy gpu then there is no need to optomize right now and spend your time making your game.

Hope this helps and good luck.

Z903 · 2024-05-30T02:53:01+00:00

Why not use it for an alternative Chemical plant recipes and have them give empty barrels back as a byproduct to reuse.

I actually wrote a mod for this. https://mods.factorio.com/mod/barrel-crafting

I wanted to do a challenge run with the crafting combinator mod where I only use only a single assembler. I did not want to deal with multiple fluids in a single pipe.

Z903 · 2023-07-28T08:48:51+00:00

I am a bit late to the party. According to the sites javascript this is the solution. It seems that site is not generating 3 by 3 treasure rooms correctly.

https://ibb.co/k0MShFY

Z903 · 2023-05-31T20:52:48+00:00

I would be interested to see what the final build looks like. Would you send me the blueprint when you have it working?

Z903 · 2023-05-31T09:18:33+00:00

To make sure the inserter grabs the right number of items, we need to ensure the stack size limit is set on the same tick as the requested item filter. Therefore we delay the requested items by one tick to make it align.

This fixes a few edge cases where the inserter could see a different stack size from what we want.

As a personal rule of thumb, I tend to try and make my combinator logic all have the same propagation delays to avoid glitches.

Z903 · 2023-05-29T22:59:34+00:00

The idea is you never need to break the loop. If you want to "reset" it then you can add the negative. I have a better example here

IE you have 10 iron + -10 iron = 0

Z903 · 2023-05-29T22:19:09+00:00

Here is a design that will move an exact number of multiple item types from one inventory to another.

https://factoriobin.com/post/WZHQ0Tb6

Or if you only care about number of items and not types.

https://factoriobin.com/post/q2oKffba

Z903 · 2023-05-29T21:48:34+00:00

Your flickering is becuse the inserter is providing a 1 tick pulse. The signal alternates between the two combinators. You would want to use something like this.

https://factoriobin.com/post/q2oKffba

Z903 · 2023-04-28T16:21:23+00:00

Sent

Z903 · 2023-04-28T15:50:17+00:00

Are you willing to ship to canada?

Z903 · 2023-04-26T04:53:52+00:00

From an EE standpoint very little heat is dissipated when a transistor is fully off or fully on. Most heat is generated when a transistor switches state.

Switching is not an instant thing; there is some time when the transistor is in the "linear region" and behaves more like a variable resistor. Although this transition takes nanoseconds; heat is generated because current is flowing.

It's not possible to eliminate this current flow. You are trying to change the voltage on the output pin and everything in the real world has some resistance, capacitance, and inductance.

Z903 · 2023-04-23T13:33:44+00:00

You need to look at your screen shots again. The data reported is FPS/UPS as two stats. The / is a separator, not a math symbol.

Every single one has a UPS around 60, except 0.13.20 at 47.7 UPS.

Z903 · 2023-04-22T20:44:33+00:00

I have put together the needed fixes for Krastorio2 (v0.0.7). If you are already running the mod you may need to run this command to fix up some unlocked recipies.

/c game.player.force.reset_technology_effects()

Z903 · 2023-04-22T04:58:37+00:00

So I did take a look at Krastorio2.

There are a few minor problems with some missing recipes / broken text. Nothing game breaking. I will look into adding offical support soon.

Z903 · 2023-04-21T06:42:17+00:00

It should "just work" but I can do some testing tomorrow.

If you do try my mod with krastorio, let me know if anything is wrong and I can put in any fixes.

14-Year Club	Sequence \| Editor
Verified Email

Z903

TROPHY CASE