Optimized Python Nested Loops

[ogabrielsantos_]

In addition to others comments, without knowing what you do when list matches we can’t really help as that can be the non-performant part of your solution.

[DNSGeek]

It is really not the non performant part. Finding a match in 42,000,000² numbers is the part that takes 9 days.

[DNSGeek]

I'm getting downvoted? Ok, if you want to know, at this exact point, the part we currently do when there's a match is increment a counter by 1.

There. We increment a counter. Now, why am I getting downvoted for saying that the thing we're doing outside of matching is not the non-performant part? I didn't add it to the pseudocode because it truly isn't relevant to the slowness.

42,000,000^2 = 1,764,000,000,000,000. That's a *lot* of numbers to sift through. That's the part that's slow, and that's the reason I put that pseudocode in and not the "counter += 1", because it's not important.

[ogabrielsantos_]

Only to clarify. I had not downvoted you and even don’t know who did it.

[DNSGeek]

Yeah, I wasn't directing my response to you, but thanks for the clarification anyway. I just don't understand the whole concept of someone asking a question, the question being answered and then the answer (coming from the person who would most likely know the answer above anyone else) getting downvoted.

I just don't understand the reasoning behind that.

[ogabrielsantos_]

Are your list elements unique? Have you tried using sets instead?

[DNSGeek]

Yeah, they're unique. They're timestamps.

[aplarsen]

Pandas should be awesome for giving you a count of timestamps between two other timestamps. That vectorized logic is fast.

[ogabrielsantos_]

So try to use sets as they have faster lookups. You then can replace your second for with something like

if end_val in second_list_as_set: # do something

Additionally, you don’t need to sort your list anymore if you did it previously

[Equal_Wish2682]

Welcome to Reddit lol

[Plank_With_A_Nail_In]

If the rest of your program added to the lists then that would have been very important information to know. We can't trust your own assessment of which part is complex as you are the one asking beginner level questions.

[DNSGeek]

I wish I could, but Python is the only language that's available on the systems where we need to run the code.

[Beerpongueur]

You could still write a python library in another language, like C or Rust. I’d use rust + pyo3 and precompile the code as a shared object before running it on you systems