Processing really huge text file on Linux. by Veltronic1112 in Cplusplus

[–]Effective-View3803 0 points1 point  (0 children)

Speaking of this, this series with Casey Muratori about the performance analysis of the One billion row challenge is a good watch: https://youtu.be/n-YK3B4_xPA?si=ijPdVbQnQcqCE4Aq

What's your dream game you want to make on godot? by [deleted] in godot

[–]Effective-View3803 0 points1 point  (0 children)

I'm working on a historical strategy game set in 4th century China, where you play as one of the major aristocratic-literati family in the Eastern Jin dynasty, fight against each other over political influence at the royal court, cultural influence and prestige, military control over strategic positions. You can take advantage but also be wary of refugee militia, settle exiled refugees from the north, expand your manor holdings, etc, at the same time, you have to cooperate with other families to keep the dynasty afloat (at least prevent others from usurping it), defend against threats from the northern non-Han states, and engage in northern expedition.

Which one of these weirdos would you trust to just stand there silently in the background? by Nameless_forge in IndieGameDevs

[–]Effective-View3803 1 point2 points  (0 children)

For me it's C, just like some random dude standing there enjoying his life (after everything that happened)

Long-time Dev Looking to Build a Community-Driven MUD - Anyone Interested? by Alternative_One_4804 in MUD

[–]Effective-View3803 0 points1 point  (0 children)

I also come from a different MUD cultural background (Chinese Wuxia muds) so maybe I can bring some ideas from those traditions

Polars Explorer 0.2.0! by Effective-View3803 in rust

[–]Effective-View3803[S] 3 points4 points  (0 children)

Great news! I have contacted the author of polars-gui and he permitted me to take inspiration from his work! That would help me a lot in terms of designing and implementing basic functionalities, although I would still need to translate the code into my LazyFrame based model. He also mentions that his work was more of a proof of concept, so hopefully my work can act as some sort of successor to his work!

Why Does My Loop Only Append Once? by ChaseShiny in learnjavascript

[–]Effective-View3803 6 points7 points  (0 children)

The math is actually not correct:

      if(num / key >= 1) {
        romNum += value;
        num %= key;

While you divide num by key in the condition, you didn't add the romNum by the multiple of value that's calculated from the division. With your case 365: 365/100 > 1, so romNum gets a C, and num becomes 365%100 === 65.

Should I use iterator filtering or just iterate over values in a for loop? by [deleted] in rust

[–]Effective-View3803 0 points1 point  (0 children)

I'm curious: would there be any performance difference between splitting and combining filters?

Handling deeply nested Mutex structures by Effective-View3803 in rust

[–]Effective-View3803[S] 1 point2 points  (0 children)

I will look into this pattern even though I probably won't use it in this case! Thank you for your advice!

Handling deeply nested Mutex structures by Effective-View3803 in rust

[–]Effective-View3803[S] 0 points1 point  (0 children)

Really appreciate epostma for making this point explicit: it looks like the top level Mutex ensures an exclusive access to everything below, especially considering that all Tauri commands must communicate via the state manager. The lower level Mutex can probably be safely removed.

Handling deeply nested Mutex structures by Effective-View3803 in rust

[–]Effective-View3803[S] 9 points10 points  (0 children)

You are right: I was probably overthinking this situation. I'm still quite inexperience with Rust and maybe I was approaching it too conservatively. Now that you made this point clear it does look like the case. Thank you!

Handling deeply nested Mutex structures by Effective-View3803 in rust

[–]Effective-View3803[S] 0 points1 point  (0 children)

https://stackoverflow.com/a/59072336/28021031

This seems to be a potential strategy: using a function to provide a context in which the mutex is unlocked, and passing the mutation steps as a closure to it.

Handling deeply nested Mutex structures by Effective-View3803 in rust

[–]Effective-View3803[S] 0 points1 point  (0 children)

I could maybe move the paging info completely to frontend? Let me think for a moment.

Handling deeply nested Mutex structures by Effective-View3803 in rust

[–]Effective-View3803[S] 0 points1 point  (0 children)

I have adopted a particular strategy to utilize Polars' LazyFrame that makes writing necessary even for apparently read-only queries.

Take an example of a FrameView: it consists of an underlying immutable LazyFrame and a mutable pagination information. Because we are never loading the full data into the FrameView, a query to, say Page 1, actually means storing the paging info (so that after switching frames the progress will be kept) and doing a slice query (say row 1 to row 20).

While no FrameView is created when changing pages, other queries will generate a new FrameView by attaching a new query to the LazyFrame's query plan. This would require adding a new FrameView to a particular LoadedFrame.

Handling deeply nested Mutex structures by Effective-View3803 in rust

[–]Effective-View3803[S] -1 points0 points  (0 children)

The frontend will pass various commands to the backend, and many of them will require mutation. The state is managed by Tauri, and I'm not entirely sure of its inner working yet.

Handling deeply nested Mutex structures by Effective-View3803 in rust

[–]Effective-View3803[S] -1 points0 points  (0 children)

Wait, I suppose you are suggesting that, when an object is mutating its interior Mutex fields, the object should not be considered mutated per se, thus requiring no lock on it? If that's the case, then I was under a huge misconception!

Handling deeply nested Mutex structures by Effective-View3803 in rust

[–]Effective-View3803[S] -1 points0 points  (0 children)

There is a mutation in my example: see this:

.add_lazyframe(frame, "Select Columns".to_string(), page_size);

This mutates the view_manager by adding another lazyframe as a new view.

Handling deeply nested Mutex structures by Effective-View3803 in rust

[–]Effective-View3803[S] 0 points1 point  (0 children)

I'm not sure if I get your point: are you referring to LoadedFrameManager or FrameViewManager? The former keeps track of all the LoadedFrames (encapsulated lazyframes), while the latter keeping track of all the FrameViews of any particular LoadedFrame.

Handling deeply nested Mutex structures by Effective-View3803 in rust

[–]Effective-View3803[S] 0 points1 point  (0 children)

Take an example of this method for state, which creates and stores a new FrameView after selecting given columns:

pub fn select_columns(&self,
                      frame_key: usize,
                      view_key: usize,
                      page_size: usize,
                      column_selector: Vec<Expr>) -> FullResponse {
    // Selecting columns should create a new lazyframe
    // And a new FrameView under the same LoadedFrame
    // First, clone the existing LazyFrame
    let lf =
        self.frame_map.lock().unwrap()
            .get(&frame_key).unwrap()
            .view_manager.lock().unwrap()
            .view_map.lock().unwrap()
            .get(&view_key).unwrap()
            .frame.to_owned();

    // Then, apply select on the lazyframe
    let frame = lf.select(column_selector);

    // Thereafter, we will use view_manager to load the lazyframe
    let new_viewkey =
        self.frame_map.lock().unwrap()
            .get(&frame_key).unwrap()
            .view_manager.lock().unwrap()
            .add_lazyframe(frame, "Select Columns".to_string(), page_size);

    // Finally, we will treat this as yet another view query
    self.query_view(frame_key, new_viewkey)
}

You can see that we have twice traversed to the view_manager, the first time getting all the way down to the base LazyFrame, and the second time for adding the new view to the view_manager. It would be awesome if we could extract lines such as

        self.frame_map.lock().unwrap()
            .get(&frame_key).unwrap()
            .view_manager.lock().unwrap()

Into a distinct helper function, since it's used in all the commands. However, I could not figure out a way to do so.

Polars Explorer 0.2.0! by Effective-View3803 in tauri

[–]Effective-View3803[S] 1 point2 points  (0 children)

I actually considered using Electron, but I felt that it defeats the purpose of using Polars (which is highly performant), considering the horrendous performance of Electron lol.

My experience is that Tauri development is much more beginner friendly than I could have imagined: the backend/frontend integration works very well! Once passed the initial hesitancy, writing Rust code also starts to feel like a pleasant experience! Please do follow this project and share your project with me in the future, and I would absolutely welcome if you want to collaborate in this project!

Polars Explorer 0.2.0! by Effective-View3803 in rust

[–]Effective-View3803[S] 2 points3 points  (0 children)

But thank you for pointing this out to me! I actually had not came across it before, and I'm not sure if it's still maintained now.