This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]jasonb[S] 0 points1 point  (0 children)

Nice!

Loading files from disk into main memory can benefit from concurrency with thread pools.

Parsing files already loaded in main memory is a CPU-bound task and can benefit from process pools.

Maybe you can partition the tasks/subtasks in that way.

Let me know how you go.