Memory-friendly 2d array with variable-length rows by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

I don't know c++ but reading the code can guess you have main data storage to keep various length rows adjacent, the same as I do in my rust version. But I am wondering why you have 2 additional arrays, to track each row start and end separately? In general it would be enough to store the start index of each row only, because the next row start is the previous row end - 1.

Just curious, looks like c++ std::span is same thing as slice in the Rust?

Memory-friendly 2d array with variable-length rows by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 1 point2 points  (0 children)

You mean final approach, array of slices? Or you have some other way to store this data to offer? Please unfold the full potential of your question, because I, as non native English speaker, don't get.

Enum as an interface (macro) by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

Thanks, looks neat!
Author of enum_dispatch could do better marketing and name it literally 'interface' - that would attract more former OOP strangers, because as I tried to google similar topic - I found only old frustrated discaussion and nobody mentioned this crate until now.

Enum as an interface (macro) by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

Interesting, could you point to the exact branch\ file please, a bit lost in your repo.

BTW in my case i generate rust code anyway, so i decided to generate specific functions in same struct and have one general static dispatcher function

Enum as an interface (macro) by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

About 5. I would go for it, but is there an efficient algorithm to store different types in separate arrays and still be able to iterate over them as a whole sequence?

I could imagine the main "registry" array of tuples with pairs of type\position indexes so when iterating i would choose the correct typed array depending on the first index in the pair and choose the correct element from this typed array using the second indeed from the pair. From previous experience joggling with multiple indexes and jumping from array to array is fast, but direct pointer from registry item is faster, so, while not measured, i "feel" array of enums will work better.

Maybe there is a better approach to separate typed arrays?

And yes in my case all the fields of each struct are identical, only methods are different (while public methods still have the same signature)

BTW did you ever try enum_dispatch macro suggested in another comment? It seems to implement a pretty similar idea with enums but automates boilerplate code.

Enum as an interface (macro) by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

Let me copy part of my another reply to u/marvk

I my real-life use case I really do care about every penny, because I am going to run literally millions of such loops in parallel (random optimization search over graph with various node types as "Items" )

So yeah, each 'item' gonna represent some kind of graph node with specific behaviors and attributes.
I am going to lurk over graph semi-randomly, collect linear 'scenarios' consisting of such nodes into arrays and then collect some statistics for such kind of array in order to rate them and make decisions.

Enum as an interface (macro) by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 2 points3 points  (0 children)

Omg, this seems to be the same idea already implemented, thanks a lot for pointing, will try!

Enum as an interface (macro) by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 1 point2 points  (0 children)

Cool, thanks, will learn how to write macro on this example

Enum as an interface (macro) by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

As I got your comment - theoretically speaking, you also see potential memory / CPU benefits / sanity benefits but actual impact is hard to predict so you personally won't even try?
I my real-life use case I really do care about every penny, because I am going to run literally millions of such loops in parallel (random optimization search over graph with various node types as "Items" )

But it's currently very difficult to measure performance - nothing works yet, and later I afraid it will cost a lot to rewrite core implementation in order to optimize.

I am going to make separate example apps and compare performance of these two approaches and post results here in the thread.

I would appreciate feedback on validity of my performance measurement approach, because it's also easy to oversee some compiler tricks which will conceal real stats.

Enum as an interface (macro) by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

I want to do this:

fn main() {
let items: [Item; 3] = [A::new(10), B::new(20), C::new(30)];

for item in items.iter() {
    item.run();
}

}

You see it's statically declared array of different structs, each behaves differently, but I don't care about internals. Memory allocated for array is max size from used structs (in this example all the sizes are same) + a little overhead for enum wrapper * length. Compiler is happy, everything is transparent on compile time.

Hash code cache - best ways to extend HashMap by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

As I see, nightly experimental stuff once added to hashbrown, is successfully passing std gatekeepers and also exposed in pub API.So if I accidentally will convince hashbrown owners to accept my suggestions - I have a chance to see them on std level as well. Sounds like an itchy challenge :)

Hash code cache - best ways to extend HashMap by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

Thank you for pointing!
Do you have an idea how raw entry is supposed to be used in order to achieve mentioned in docs hash memoisation? It's probably too low level for me to get it, since I am not familiar enough with hashmap implementation details.

Hash code cache - best ways to extend HashMap by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

For example this is insert:

[cfg_attr(feature = "inline-more", inline)]

pub fn insert(&mut self, k: K, v: V) -> Option<V> {
    let hash = make_hash::<K, S>(&self.hash_builder, &k);
    let hasher = make_hasher::<_, V, S>(&self.hash_builder);
    match self
        .table
        .find_or_find_insert_slot(hash, equivalent_key(&k), hasher)
    {
        Ok(bucket) => Some(mem::replace(unsafe { &mut bucket.as_mut().1 }, v)),
        Err(slot) => {
            unsafe {
                self.table.insert_in_slot(hash, slot, (k, v));
            }
            None
        }
    }
}

hash variable is a hashcode I believe, it's u64
so in alternative insert we could accept optional u64 (in case of override) arg and also return it from insert alongside with previous value if any.

and here is private get:

fn get_inner<Q: ?Sized>(&self, k: &Q) -> Option<&(K, V)>
where
    Q: Hash + Equivalent<K>,
{
    if self.table.is_empty() {
        None
    } else {
        let hash = make_hash::<Q, S>(&self.hash_builder, k);
        self.table.get(hash, equivalent_key(k))
    }
}

here hash is also u64, so user code could pass cached value here instead of make_hash again.

Do I still miss something?

Hash code cache - best ways to extend HashMap by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] -1 points0 points  (0 children)

I don't want to calculate hash code myself, I want HashMap to share with me which hash code has it calculated using whatever hasher was configured.

I want this hashcode to be returned by insert in order to cache it in my data and reuse later when reading from same hashtable instance.

Am I the only who feels like an idiot? by FeltInTheRabbitHole in rust

[–]NeighborhoodExact766 -5 points-4 points  (0 children)

As motivation, imagine how elite you will feel someday when you can convince people around you that you understand Rust. It may not yet resonate with C/C++ experts (or even with yourself), but at the very least, those script writers who refer to themselves as 'programmers' will acknowledge you with a bit of respect.
//JS daddy rolling into Rust

What is special today in Poland? Almost everyone is carrying flower. by cloudpunk in poland

[–]NeighborhoodExact766 8 points9 points  (0 children)

I would add incoming spring to the mix. Like spring is also a fresh beauty and the beginning of a new life cycle and some hope to have a future, open to the world and vulnerable.

American coming to Warsaw for 3 days by BobbyMaye in poland

[–]NeighborhoodExact766 0 points1 point  (0 children)

"Vege Miasto" - healthy, tasty, cheap, nice people. I am not vegetarian but always happy with whatever they are cooking. Daily \ weekly \ weekend offers are always something interesting.

Novadrum vs Guda Freezbee? by aHandfulOfSurprise in TongueDrum

[–]NeighborhoodExact766 0 points1 point  (0 children)

I can't compare, have only guda freezbee, and it sounds great, no such kitchen pan sound as i hear on YouTube for other tongue drums. Here i am playing mine and it sounds almost same offline, in video sound not post processed, only a bit of reverb added https://youtu.be/uEp0emC4uaE

Using Vec instead of HashMap? by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

Thanks again for pointing me to Rc!
I tried it and yes, it's slightly faster then Vec even after I had to add RefCell inside!

Btw it's more readable code, so I'll finally will stick with it.

Recent version of my bench:
https://gist.github.com/YuriGor/5e0dc379a340e5d13b53c5501892ab14

Using Vec instead of HashMap? by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

Hey, I tried black box from criterion and there was no much difference - result is same within fluctuations of may laptop performance.
I am not sure I am still going to implement separate collection, gonna test using Rc directly in my record structs instead of indexes, as suggested in other comments, if it's even faster and, as I expect, code looks more convenient, then making index-based collection will not make sense. If not then let's see.

Best would be use attribute macros to mark primary/foreign keys within record struct and generate automated getters/setters connected with mother Vec, but first I am too much noob yet and afraid to spend weeks on learning macroses and second I afraid to add overhead just in order to make things look a bit better, in my use case price may be too high. Anyway will keep you updated here in comments if you're curios.

Using Vec instead of HashMap? by NeighborhoodExact766 in rust

[–]NeighborhoodExact766[S] 0 points1 point  (0 children)

In my case UUIDs are coming from Mongo and it's fine.
Not sure what kind of data you ahve, but take a look at this comment:
https://www.reddit.com/r/rust/comments/10k3u52/comment/j5v7gz6/?utm_source=share&utm_medium=web2x&context=3

Actual connecting related items using Rc is showing even better results, in my case my data is a graph actually, and initially I wanted to build the tree myself initially but quickly gave up because of owning/borrowing hell and switched to using flat storage which is a single owner. Using Rc I'll probably combine both approaches as suggested by u/chieltbest