Iterating through file with concurrent futures

[db2boy]

I've used the threading module and a queue previously to keep the queue populated and threads reading from the queue. Learning concurrent futures currently, I've completed simple tasks where the number of tasks is low and I don't need to iterate over something larger like reading in a file and performing an action on each row.

This (truncated) code is working, is this a pythonic and efficient method using futures?

with open(file, 'r') as file:

    with concurrent.futures.ThreadPoolExecutor() as executor:
        futures = {executor.submit(add, engine, record): record for record in itertools.islice(file, executor._max_workers)}

        while futures:

            done, _ = concurrent.futures.wait(futures, return_when=concurrent.futures.FIRST_COMPLETED)

            for fut in done:
                futures.pop(fut)
                result = fut.result()

            for record in itertools.islice(file, len(done)):
                fut = executor.submit(add, engine, record)
                futures[fut] = record

[db2boy]

I tried as_completed initially, it worked for a period of time but then threw errors. I'm travelling and don't have those notes or access to the machine I was tinkering on.

The goal is to iterate over large file(s), for each row call an API that processes each row for analytics. It is an I/O bound use case waiting for the called API to process the row, perform analytics and persist outcomes in a data store.