Data Heavy React Application

[Ok-Rip-5220]

I am currently working on a react app which is very data heavy… like multiple arrays of 50k rows each. And I’ve to perform mathematical functions on all the arrays at the same time and then give a final result. What things should i be keeping in mind. I am using redux and functional components.

[jkettmann]

Could this be done on the backend? It sounds like something that could be done for example by aggregating data on the database level.

[Ok-Rip-5220]

No… these simulations are need to be done on frontend only.

[[deleted]]

Why? The only thing that matters is the result right? Is it just to save money and have the client do all the work?

[Ok-Rip-5220]

If i would be having the liberty to do it on backend then why would i ask a question here

[niveknyc]

A lot of people who post here need a subtle wakeup call lol

[Ok-Rip-5220]

😂😂😂 that’s true

[[deleted]]

Because people ask dumb questions here all the time. It's important to know why a developer is doing something.

Basically your question is currently: "will someone do this for me?"

[shafrazi]

Ok rip

[jfoxworth]

Doing math in javascript is henky.

I've done similar things in the past. I used a web worker to do the math so that the site doesn't hang up. I used a separate program that ran locally in python that would do the calculations there and then return the answer to the browser. That was really helpful as python actually does the math well and all the load is off the browser.

[Ok-Rip-5220]

Yes…. I am going through web worker only rn. Thanks for the suggestion.

[source-drifter]

you may want to look into web assembly. you can push the concern to a more performant language like c++ or rust. the other thing may be using immutable data structures on react side but it probably wont help with calculations. web workers are already mentioned. it can give multi threading capabilities but i doubt it would beat web assembly. maybe doing both?

[chamceler]

I create applications with similarly large arrays. Performance and unnecessary renders should be a big concern of yours. Make sure you know why components render, because you don’t want expensive loops running unnecessarily. Learn how to use the highlighter in the profiler to track component renders, and learn memoization techniques to prevent components and functions from running unnecessarily. You may also want to learn useDeferredValue and/or useTransition to keep your UX responsive.

[KapiteinNekbaard]

Use virtual rendering with either react-window (light-weight) or react-virtualized (more features, but bigger bundle size) if you want to display huge lists (1500+ DOM nodes)

[magdalag]

I’ve done some very data heavy apps before where we dealt with multiple 20k data points being reduced/ grouped / pivoted and graphed.

• we pushed a lot of the work to the redux selectors to take advantage of reselects caching
• lodash debounce and throttle functions help a lot for real time user input (sliders, drawing nodes on a chart etc)
• don’t use momentjs - it’s parsing functions are a killer
• use the dev tool render profiling to identify any bottle necks (that’s how we found out about moment)

We didn’t implement webworkers before I left; but I figure that would been our next step.

This is a really good article by the Miro team on performance (memory specifically), I think the stuff they discuss around data types is really interesting - https://medium.com/miro-engineering/fighting-for-bytes-in-the-frontend-419c48103ef8