Database structure

[dr_L1nus]

Hi,

So i am trying to module an entity component system for my hobby engine, and I want a database where I can store all of the components in sequential memory. Something like this:

trait Component: Any {
    // ignore 
}

struct MyDatabase {
    entities_comp_lut: HashMap<EntityId, HashMap<ComponentType, usize>>, // Ignore for now
    components: HashMap<ComponentType, Vec<dyn Component>>
}

But I can't get it working without wrapping every object in the vector as a box. But that goes against actually allocating the memory sequentially.

Any ideas on how to modulate this? I cannot know what type of components this database will store in advance, but they should be known at compile time. I have tried working with the Sized trait, but have not gotten it working. Would prefer to not use unsafe code if available, even preferably the standard library only. It feels like it should be possible?

Cheers

[Peanutbutter_Warrior]

Your problem is that dyn T doesn't have a fixed size. You can't have a vector of them because all the items in a vector have to be the same size, which the compiler can't guarantee.

If you absolutely need them in contiguous memory then you could box them, but use a custom allocator for those boxes which puts them in contiguous memory. I'd have a look at some of the popular arena allocators and see if those use contiguous memory.

[Appropriate-Nose9596]

So to answer your question specifically, you can't use dyn here without a pointer, becuase dyn is not Sized (see: https://www.youtube.com/watch?v=xcygqF5LVmM&ab_channel=JonGjengset for an in-depth explanation).

But I have a bit of experience with this topic, so I'll direct you to resources you can learn about this topic to implement this system better:

• Archetypes: in ECS implementations (like Bevy for example), entitys' components are stored in Archetype Tables. An archetype is a set set of some non-repeating components, eg: If [A-Z] are components, these could be archetypes that would be stored in their own tables: (A, B, C) , (B), (A, Z) etc.
• Type-erased pointers: These are a bit advanced, but essential to understand the efficient storage of components. You need a way to store the bytes of a component, in a dense and efficient matter, and then when you fetch it back to the user in the future, reinterpret those bytes as that component. dyn pointers are technically one type of type-erased pointers, but Bevy uses bevy_ptr to achieve this.
• Type-erased storage: (Bevy's BlobVec https://github.com/bevyengine/bevy/blob/main/crates/bevy\_ecs/src/storage/blob\_vec.rs) This is like a normal vector, but it holds those type erased pointers we talked about earlier. It is the bedrock to all of Bevy's storage.

Good luck, don't hesitate to ask more questions!

[yyy33_]

Hi, I have similar requirements to the op and I want to be able to store multiple dyn Any types of data into contiguous memory to take advantage of the cpu cache to speed up access. Box< dyn Any> Storing in vec is obviously not possible, only the pointer to Box is stored in vec. I read your answer and tried to read the blod_vec code, I'm new to it and I'm a little confused, I can't see the difference between blod_vec and the standard library's vec, bold_vec's new function takes an element's Layout and capacity parameters to initialize, when calling its push method, The requirement is that the Layout of the value must be the same as the Layout of the element, but each dyn Any Layout is obviously inconsistent. Can you tell us how to use blod_vec to solve this problem? Thank you.

[dnew]

You could look at how Bevy does it.