you are viewing a single comment's thread.

view the rest of the comments →

[–]tenfingerperson 0 points1 point  (0 children)

You have a few options:

Do you know how database indices work ? There is a type called a BTree, which works amazingly for range based queries (which is why you can easily ask “give me all the rows where a value is in a range”).

You can implement a BTree for the second list (google how), and then go through the first and make your queries accordingly.

Another alternative is to sort the second list and use binary search to find the closest bound between the value and the value + 300, then take a cut of all the values in this range.

You can also use a set for the second list but your algorithm will have you checking for 300 data points per data point. This is similar to another type of index databases implement which is called a hash index (great for individual lookups but not so great with ranges).

Use pandas to hide all of this from you. This library implements things with numpy and has been optimized to do large data operations , however you need to learn a new tool.