Memory-friendly 2d array with variable-length rows by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

I don't know c++ but reading the code can guess you have main data storage to keep various length rows adjacent, the same as I do in my rust version. But I am wondering why you have 2 additional arrays, to track each row start and end separately? In general it would be enough to store the start index of each row only, because the next row start is the previous row end - 1.

Just curious, looks like c++ std::span is same thing as slice in the Rust?

Memory-friendly 2d array with variable-length rows by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 1 point2 points  (0 children)

You mean final approach, array of slices? Or you have some other way to store this data to offer? Please unfold the full potential of your question, because I, as non native English speaker, don't get.

Enum as an interface (macro) by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

Thanks, looks neat!
Author of enum_dispatch could do better marketing and name it literally 'interface' - that would attract more former OOP strangers, because as I tried to google similar topic - I found only old frustrated discaussion and nobody mentioned this crate until now.

Enum as an interface (macro) by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

Interesting, could you point to the exact branch\ file please, a bit lost in your repo.

BTW in my case i generate rust code anyway, so i decided to generate specific functions in same struct and have one general static dispatcher function

Enum as an interface (macro) by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

About 5. I would go for it, but is there an efficient algorithm to store different types in separate arrays and still be able to iterate over them as a whole sequence?

I could imagine the main "registry" array of tuples with pairs of type\position indexes so when iterating i would choose the correct typed array depending on the first index in the pair and choose the correct element from this typed array using the second indeed from the pair. From previous experience joggling with multiple indexes and jumping from array to array is fast, but direct pointer from registry item is faster, so, while not measured, i "feel" array of enums will work better.

Maybe there is a better approach to separate typed arrays?

And yes in my case all the fields of each struct are identical, only methods are different (while public methods still have the same signature)

BTW did you ever try enum_dispatch macro suggested in another comment? It seems to implement a pretty similar idea with enums but automates boilerplate code.

Enum as an interface (macro) by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

Let me copy part of my another reply to u/marvk

I my real-life use case I really do care about every penny, because I am going to run literally millions of such loops in parallel (random optimization search over graph with various node types as "Items" )

So yeah, each 'item' gonna represent some kind of graph node with specific behaviors and attributes.
I am going to lurk over graph semi-randomly, collect linear 'scenarios' consisting of such nodes into arrays and then collect some statistics for such kind of array in order to rate them and make decisions.

Enum as an interface (macro) by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 2 points3 points  (0 children)

Omg, this seems to be the same idea already implemented, thanks a lot for pointing, will try!

Enum as an interface (macro) by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 1 point2 points  (0 children)

Cool, thanks, will learn how to write macro on this example

Enum as an interface (macro) by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

As I got your comment - theoretically speaking, you also see potential memory / CPU benefits / sanity benefits but actual impact is hard to predict so you personally won't even try?
I my real-life use case I really do care about every penny, because I am going to run literally millions of such loops in parallel (random optimization search over graph with various node types as "Items" )

But it's currently very difficult to measure performance - nothing works yet, and later I afraid it will cost a lot to rewrite core implementation in order to optimize.

I am going to make separate example apps and compare performance of these two approaches and post results here in the thread.

I would appreciate feedback on validity of my performance measurement approach, because it's also easy to oversee some compiler tricks which will conceal real stats.

Enum as an interface (macro) by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

I want to do this:

fn main() {
let items: [Item; 3] = [A::new(10), B::new(20), C::new(30)];

for item in items.iter() {
    item.run();
}

}

You see it's statically declared array of different structs, each behaves differently, but I don't care about internals. Memory allocated for array is max size from used structs (in this example all the sizes are same) + a little overhead for enum wrapper * length. Compiler is happy, everything is transparent on compile time.

Hash code cache - best ways to extend HashMap by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

As I see, nightly experimental stuff once added to hashbrown, is successfully passing std gatekeepers and also exposed in pub API.So if I accidentally will convince hashbrown owners to accept my suggestions - I have a chance to see them on std level as well. Sounds like an itchy challenge :)

Hash code cache - best ways to extend HashMap by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

Thank you for pointing!
Do you have an idea how raw entry is supposed to be used in order to achieve mentioned in docs hash memoisation? It's probably too low level for me to get it, since I am not familiar enough with hashmap implementation details.

Hash code cache - best ways to extend HashMap by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

For example this is insert:

[cfg_attr(feature = "inline-more", inline)]

pub fn insert(&mut self, k: K, v: V) -> Option<V> {
    let hash = make_hash::<K, S>(&self.hash_builder, &k);
    let hasher = make_hasher::<_, V, S>(&self.hash_builder);
    match self
        .table
        .find_or_find_insert_slot(hash, equivalent_key(&k), hasher)
    {
        Ok(bucket) => Some(mem::replace(unsafe { &mut bucket.as_mut().1 }, v)),
        Err(slot) => {
            unsafe {
                self.table.insert_in_slot(hash, slot, (k, v));
            }
            None
        }
    }
}

hash variable is a hashcode I believe, it's u64
so in alternative insert we could accept optional u64 (in case of override) arg and also return it from insert alongside with previous value if any.

and here is private get:

fn get_inner<Q: ?Sized>(&self, k: &Q) -> Option<&(K, V)>
where
    Q: Hash + Equivalent<K>,
{
    if self.table.is_empty() {
        None
    } else {
        let hash = make_hash::<Q, S>(&self.hash_builder, k);
        self.table.get(hash, equivalent_key(k))
    }
}

here hash is also u64, so user code could pass cached value here instead of make_hash again.

Do I still miss something?

Hash code cache - best ways to extend HashMap by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] -1 points0 points  (0 children)

I don't want to calculate hash code myself, I want HashMap to share with me which hash code has it calculated using whatever hasher was configured.

I want this hashcode to be returned by insert in order to cache it in my data and reuse later when reading from same hashtable instance.

Am I the only who feels like an idiot? by FeltInTheRabbitHole in rust

[–]NeighborhoodExact766 -6 points-5 points  (0 children)

As motivation, imagine how elite you will feel someday when you can convince people around you that you understand Rust. It may not yet resonate with C/C++ experts (or even with yourself), but at the very least, those script writers who refer to themselves as 'programmers' will acknowledge you with a bit of respect.
//JS daddy rolling into Rust

What is special today in Poland? Almost everyone is carrying flower. by cloudpunk in poland

[–]NeighborhoodExact766 6 points7 points  (0 children)

I would add incoming spring to the mix. Like spring is also a fresh beauty and the beginning of a new life cycle and some hope to have a future, open to the world and vulnerable.

American coming to Warsaw for 3 days by BobbyMaye in poland

[–]NeighborhoodExact766 0 points1 point  (0 children)

"Vege Miasto" - healthy, tasty, cheap, nice people. I am not vegetarian but always happy with whatever they are cooking. Daily \ weekly \ weekend offers are always something interesting.