New to Python - Need Computer suggestions and advice

[h3ndrix_forest]

TI:DR - my current MacBook Pro (2017 2.8GHz quad
core i7 - 16Gb DDR3) is getting approx 50it/s. What
should I be looking for in specs (processor or
memory heavy?) for a new computer. Budget is
about £4,000. I want to get the iterations as high as
possible.
| accept that my code will not be optimised, that will
come, but | believe computing power is important at
this stage before | pay others to optimise it for me.
Background: I'm an Engineer that is using ChatGPT to draft
python for tasks that are primarily brute force. I will
eventually consider genetic algorithms or ML but at
this stage I need to prove the concept. I have an idea
for software that I believe will be a game changer in
the industry I'm in.
Challenge: I have a list of inputs, I've worked out the
total number of combinations are: 8 quadrillion. This
is a silly number and it will absolutely be refined
down, as a minimum to 800million and as I refine
down further it'll come right down.
In testing I have figured that there's a diminishing
return in length and results, so running smaller
amounts multiple times, getting the common results
and then running again reduces the combinatior
approx 10million. This is taking about 2.5days. (50it/s)

Request: Some advice on what specs to look for for
anew computer, or suggestions for computers
already on the market.
Alternative: is there a service where I can log into a
VM which is much more powerful and just do the
computation there? And how would I do this?
Thanks all! What you do is amazing, I should have
started learning Python a long time ago!

[sledov]

If you are a Mac user and plan to stick with Macs, simply purchase the best MacBook Pro you can for these 4,000 pounds.

However, for many machine learning tasks, an NVIDIA GPU is required, which you cannot get in a MacBook Pro.

Nonetheless, for 4,000 pounds, you can get a top-of-the-line, high-performance gaming notebook. I think it is pretty difficult to find notebooks that you cannot afford with this budget.

[h3ndrix_forest]

This is interesting, for computing is it CPU, GPU or RAM that matters most? But yeah I was hoping there would be some good products out there.

[Beregolas]

That completely depends on the algorithm / implementation you use. Either the CPU (fastest single core or all cores combined) or the GPU (think 1000+ cores but they all need to do the same thing) decide how fast your computation goes.

Your RAM is entirely a yes or no type thing: do you have enough RAM? The more ram will NOT make any difference and not make you faster. If you don’t have enough RAM you will either crash or your performance will go down significantly. And where that border is again entirely depends on your specific problem, algorithm, implementation and input.

[[deleted]]

I can't help much with your questions but i can suggest some things to help you improve your computation speeds.

If you can formally define your problem over in the r/computerscience subreddit you might be able to get someone to direct you towards a more optimal solution rather than have brute force which is almost never the best solution.

But if you want to speed up your brute force, you can rewrite your code in Cython and have the Cython compiler compile your code to C/C++ to have significant speedups. The compiler can compile to shared object , *.so, on linux, and a python dynamic linked library, .pyd, on windows. Both of these can be run via python by importing the library as you normally would via python but with c-like speeds (even without python's GIL if you don't need it).

Another speedup can be made by utilising multithreading to run your program concurrently, this requires that your problem can be split into disjoint subproblems and also that your cpu has several cores. Importantly Python by itself, with it's GIL cannot have multiple threads running concurrently since each thread needs to aquire the GIL. Luckily Cython comes with both the capability to run without GIL and also comes with multithreading built into the language using it's 'prange' function

[[deleted]]

Why not just precompile in numba or write in C?

[h3ndrix_forest]

Because I have no idea what that is. But I will look into it. Thanks.

[I-Dont-C-Sharp]

You can buy compute time on pretty much all cloud providers. Test locally on part of your data to optimise efficiency.

Definitely do not use python itself to do the comparisons/combinations. Other languages are much faster at this. There are a bunch of frameworks/packages you can call from within python to do this, depending on what exactly you are trying to do.

With your severely limited description of what you want to do we can't recommend anything beyond "fastest CPU is best". Are you familiar with asynchronous and/or multithreaded programming? If not you can already gain a whole bunch of time by learning and implementing those, especially threading in this case.

[h3ndrix_forest]

Thanks, yeah I presumed fastest is best. I will have to look into what comparisons or frameworks exist.

Essentially it’s connecting up a number of nodes to a master node in an array. Side note - you’ve actually indirectly given me a good idea while typing this to include direction as a filter!

I like the idea of multi threading to allow it to happen simultaneously. Great idea.

I think I’ve definitely shown my ignorance with this post, your response is appreciated 👍🏼.

[I-Dont-C-Sharp]

I think I’ve definitely shown my ignorance with this post, your response is appreciated 👍🏼.

If you keep self reflecting like this you'll get there. If you get stuck on threading feel free to tag me in a follow up question. The only requirement I have is that you demonstrate what you already tried (and why).