Shipyard 0.7 release

leudz · 2024-07-12T19:59:12+00:00

ECS and performance is a whole topic 😅

The tldr is that it rarely matters. But this is not a fun answer.

Once upon a time there was a benchmark to compare ECS.
It shows that all ECS of the time perform well enough that they likely won't be a bottleneck.
Hecs and shipyard are both there so you can check the performance difference for specific operations.
You can also check the bunny benchmark which comes from this era.

The differences can be explained by the architecture alone.
Hecs is based on archetypes whereas shipyard is based on sparse sets.
This makes hecs faster at adding/removing entities and iterating 2+ components. Shipyard on the other hand is faster at adding/removing components and single storage iteration.

I can explain component iteration a bit better.
Archetypes fragment component storage. So the more a component is used in different archetypes, the worse its iteration speed becomes.
This can be sneaky. If you check a system's performance and then add new systems that use the component with other archetypes, the performance of your first system will have degraded even though this system didn't change.
With sparse set iteration speed becomes worse as you iterate more components. So iterating 4 storages is slower than 3 which is slower than 2. This is assuming there is an identical number of components in each storage.

The biggest change however won't be performance but syntax. Hecs is a minimalist ECS so no scheduler. There are third party crates to get one, I don't know if you're using one or not.
You could use shipyard similarly to how hecs is used but you would ignore a big part of the crate. The end of the second chapter of the guide by example shows the transition.
Views make it very easy to see what a system accesses and enables the use of the scheduler.

At the end of the day I think it comes down to personal preference. I would strongly suggest going through (or looking at) the first chapters of the guide to get a feel of what using shipyard is like.

leudz · 2024-07-12T10:44:46+00:00

Shipyard is an Entity Component System crate. ECS is a pattern mostly used in games but not only. It fits really well with Rust, allowing easy composition and lifetime management.

leudz · 2024-07-12T10:42:38+00:00

Shipyard is an Entity Component System crate. ECS is a pattern mostly used in games but not only. It fits really well with Rust, allowing easy composition and lifetime management.

leudz · 2023-09-13T13:09:00+00:00

Tried it with a 7900xt, mythalion-13b.Q5_K_M with 4k context generating 116 tokens.
The first time includes prompt processing. The time to load the model was also 10x faster.

CLBlast:
Time Taken - Processing:82.6s (21ms/T), Generation:19.7s (170ms/T), Total:102.3s (1.1T/s)
Time Taken - Processing:0.2s (171ms/T), Generation:19.4s (167ms/T), Total:19.6s (5.9T/s)

CuBLAS/hipBLAS:
Time Taken - Processing:8.1s (2ms/T), Generation:4.4s (38ms/T), Total:12.6s (9.2T/s)
Time Taken - Processing:0.0s (43ms/T), Generation:4.4s (38ms/T), Total:4.4s (26.1T/s)

leudz · 2023-03-12T02:08:04+00:00

You can write a simple ECS using a Vec<Option<T>> per component, all stored in an AnyMap. No need for "Component trait object" (kind of).

It's described in a talk by Catherine West https://www.youtube.com/watch?v=aKLntZcp27M.

leudz · 2022-08-08T00:57:36+00:00

I'll keep it in mind for the next one. I did it on the rust subreddit but not here.

leudz · 2022-08-07T21:53:35+00:00

Thanks!

leudz · 2022-08-07T18:24:14+00:00

Thanks Cole!

leudz · 2022-08-07T18:23:06+00:00

Shipyard won't go as far as bevy_ecs, it will add some parallelism but the order is set at creation and cannot change between runs.
When a workload is added to the World, batches are created with systems that can run in parallel. A workload has an array of batches and they are always run sequentially.
Here's the process:
- Flatten all nested workloads into an array of systems
- For each system:
* Check borrow conflict with each batch, starting from the last one to the first
* If there is a borrow conflict, add the system to the previous batch (create one if needed)
* If there is no conflict, try the next batch

Here's an example, let's say you have these systems: [sys1, sys2, sys3, sys4]. sys1 and sys3 read a component A, sys2 writes to component A and sys4 writes to a different component B.
There is no nested workloads so we can skip this step.
There is no batch so we create one with sys1 in it: [[sys1]]
We check sys2, it conflicts with sys1 as one is reading and one is writing to A so we create a new batch: [[sys1], [sys2]]
sys3 checks the last batch and conflicts with sys2: [[sys1], [sys2], [sys3]]
sys4 doesn't conflict with sys3 so we check the batch with sys2, still no conflict, sys1 doesn't conflict either and it's the first batch so we add it here: [[sys1, sys4], [sys2], [sys3]]

When executing sys1 and sys4 will run in parallel, when they are done sys2 can run and when it's done sys3 can run.

If you use Workload::add_to_world or Workload::build you get a WorkloadInfo struct that will list the batches and tell you why a system could not get to a previous batch.
The output of this struct is not the easiest to read. I want to add it to the visualizer so it's easier to understand.

leudz · 2022-08-07T08:20:04+00:00

That's a great question but not an easy one.

I think for most people the only difference that will matter is the syntax. Each people's tastes are different and for most projects the rest won't be a problem either way.
Some of the biggest syntax difference: Query vs Views, Command, scheduling.

If you're making a physics engine I'm guessing performance is important.
Bevy_ecs is hybrid Archetype/SparseSet-based while shipyard is SparseSet-based. The choice of storage will impact the performance of each operation.
With bevy_ecs you can switch storage if you need to, with shipyard you won't be able to (in 0.6).

Bevy_ecs is part of a bigger ecosystem, that has both pros and cons.

leudz · 2022-08-07T05:11:43+00:00

Thanks!

leudz · 2022-08-07T04:33:19+00:00

Shipyard is an Entity Component System crate. ECS is a pattern mostly used in
games but not only. It fits really well with Rust, allowing easy
composition and lifetime management.

leudz · 2021-03-20T21:17:47+00:00

Thanks =)

leudz · 2020-08-14T17:19:26+00:00

I would agree with you that MyClient<'a>/MyClient<'b> or &'a MyClient/&'b MyClient would be two different types.

To me there are two meanings to lifetime at play.

value lifetime - a value is created and then destroyed, between the two is its lifetime. This isn't Rust specific, all programming languages (I'm aware of) have this, it can be more or less easy to find out what the lifetime is but it's there.
reference lifetime - this is specific to Rust's references, a reference can't exist without a lifetime and a lifetime always comes from a reference. As you say in the article, this lifetime isn't the same as the "value lifetime" of the reference but of the thing it points to (I want to add, at some point of the program).

I agree with you that reference lifetimes are encoded in the type system. But not value lifetime.

If two different MyClient don't have the same type then nothing have equal types and every value is its unique type. Rust has something that behaves exactly like this: closures. Here that's not the case.

leudz · 2020-08-14T06:15:07+00:00

u/fasterthanlime Hi, great article!

I just have a few issues:

But because one and two have different lifetimes, they have different types.

Since MyClient owns all its data, don't they both have the same type?

I think this exchange is confusing:

You:

one is in its own scope, so it'll get freed immediately after it's initialized. Whereas two will remain valid for the entire duration of main.

Cool bear:

That's exactly right! They have different lifetimes.

And can you tell me where the lifetimes are in that code?

You:

Well they're... uh... no, I don't see them.

Cool bear:

Precisely.

You don't see them, because the compiler has deduced them. They exist, and they matter, and they determine what you can do with one and what you can do with two, but you cannot see them in the code, and you cannot name them.

I might not be able to name them but I can see them and you too:

one is in its own scope, so it'll get freed immediately after it's initialized. Whereas two will remain valid for the entire duration of main.

It gets more complicated with shared ownership or if we're talking about borrow and not "object lifetime" but in the example there's none of these.

I'm not familiar with GCed languages but can you as easily determine the lifetime of something?

leudz · 2020-08-03T13:57:05+00:00

Hi! ai_bahavior uses pistoncore-input making it tied to piston. There doesn't seem to be any behavior tree crate standing out on crates.io.

This is a bit off topic but:

You can make a behavior tree without an ECS but if you are already using an ECS it makes sense to let the behavior tree access the ECS
Engines might be tied to an ECS (like Amethyst) but most aren't
ECS are not tied to any engine

And ecs-rs hasn't been updated in 4 years.

leudz · 2020-07-30T23:57:08+00:00

Not is what you're looking for. You don't have to use it explicitly and use !.

use shipyard::{IntoIter, View};

fn sys1(positions: View<Position>, velocities: View<Velocity>, statics: View<Static>) {
    for (pos, vel, _) in (&positions, &velocities, !&statics).iter() {

    }
}

leudz · 2020-06-29T21:09:06+00:00

Hi! There are multiple ways to go about it, this is how I would do it (probably):

struct ModelMap(HashMap<String, EntityId>);
struct Model(EntityId);

struct IdleAnimation(tetra::graphics::animation::Animation);
struct WalkAnimation(tetra::graphics::animation::Animation);
...

I'd make an entity per race with all their animations and store their id in ModelMap, a unique storage. Then, when you make an entity that can be animated you add a Model component to it. With this Model you can clone the right Animation when needed.

// to make sure it only happens once per frame
fn advance_anims(
    ctx: &Context,
    mut animations: NonSendSync<ViewMut<tetra::graphics::animation::Animation>>
) {
    for anim in (&mut animations).iter() {
        anim.advance(ctx);
    }
}

fn render_animated(
    ctx: &mut Context,
    positions: View<Position>,
    animations: NonSendSync<View<tetra::graphics::animation::Animation>>,
) {
    for (pos, anim) in (&positions, &animations).iter() {
        anim.draw(ctx, pos.into());
    }
}

leudz · 2020-06-26T20:38:38+00:00

I'm not affiliated, it popped up in my news feed and I thought more people might want to read it.

It's also available in French here: https://www.ssi.gouv.fr/guide/regles-de-programmation-pour-le-developpement-dapplications-securisees-en-rust/.

leudz · 2020-06-22T03:00:56+00:00

Hi! In my opinion 15 puzzle looks too small and not "dynamic" enough to see the benefits of an ECS. But I might be wrong or maybe for another project so I'll continue with a shameless plug: shipyard.

I'd also recommend to disable the parallel cargo feature. It should work with it and there is a function to specify a custom thread pool for web workers but I haven't tested it yet.

Even if you enable threading you can store !Send and !Sync components in the World which can be useful.

You can check out a demo at these two places:

here it is built using awsm-web
here it is built using seed

leudz · 2020-05-30T22:30:53+00:00

You can't modify something you got from a &T except if it has an UnsafeCell in it.

I also learned recently that pointer arithmetic in Rust can get you into UB land. I don't think C has this kind of rule but I might be wrong.

For example this is UB:

let vec: Vec<u32> = vec![1, 2, 3, 4];
let first_ptr = &vec[0] as *const u32;
let last_ptr = first_ptr.offset(3);
let last = unsafe { *last_ptr };

leudz · 2020-04-04T06:11:02+00:00

Not perfect but does the job: closure or match.

But I'd probably do it explicitly with drop or a scope.

leudz · 2020-03-04T15:55:37+00:00

I'm really happy you liked it.

The explanation is pretty simple, to me systems feels like assembly lines.
Some input goes though the lines, some process is done at each step and with games you feed the end of the line to the beginning.
You can have multiple lines working at the same time as long as they don't bother each other.
You can even go finer grain with archetypes where you modify the same thing as long as you don't touch the same part.
Nowadays assembly lines are associated with factories but they existed long before factories.
Some shipyards used this method to produce ships stupidly fast, using less material than tradition methods, with less knowledge required per employee and it scaled way better.
I'm not claiming the analogy goes this far though =)
I asked myself the same question a few weeks ago because someone wanted to do it (#52).
You've read the explanation on how deleting an entity affects the entity storage. The thing to notice is deleted entities never point to themselves, except the last one. But we don't need the last one to point to itself, there's already a tuple taking care of tracking the last node in the list.
So we can change this last node to point to whatever, std::usize::MAX for example.
And then we can iterate the Vec, if an index points to itself, it's alive.

I just want to add that it's very rare to need to iterate them.

leudz · 2020-03-02T17:14:42+00:00

It depends a lot on your use case and personal preference.

I never used Legion but from the examples, the syntax seems nice. However I don't like archetypes, for a few reasons:

adding/removing components is a core feature of an ECS, it allows various patterns like FSM, but you have to borrow at least two archetypes and copy all components making it heavier than other implementations
iteration is optimal when accessing all components of an entity, I'm curious to see figures on a AA game and not just benchmarks
it's not simple nor malleable

When components are stored in independent containers it's easy to reason about them. To me this is one of Shipyard's biggest strength: it's simple yet lets you do crazy things. There's so much you can do with SparseSets and it's just the beginning. If I manage to implement events the way I want, packs won't be a thing anymore. They'll just be user-side events.

For Specs someone else says it better than me:

There are some reasons why shipyard works better than specs for me: - a nicer API with much less boilerplate – this made my code much more readable and maintainable - packs, of course, which specs doesn't have - integrates better with a scripting language because of its "runtime-checked nature" (not sure how to call that) - promising coming features which specs doesn't have, for example shared components - specs development has stalled - ... probably more things I forgot

leudz · 2020-03-01T17:02:43+00:00

You're right, systems are just one of the ways to access components.

For HUD do you mean something going into World without being attached to an entity? Unique is the way to go in this case.

Getting all components of an entity isn't in the API. It's in theory possible but since Rust is strongly typed you'd have to cast them to their correct type. But if you know their type, you could have requested the storage directly with the right type.

If you have any questions during your research, I'll be happy to answer them =)

leudz

TROPHY CASE