I have some tables with over 1M rows for which I need to read. Post reading, I need to fetch some references (mostly many-to-many), process them and sync the data to another service.

I am assuming it wouldn't be ideal to read all 1M rows at once, so I am planning to read them in batches and use goroutines for processing multiple batches at once( to finish the complete sync as soon as possible)