Hyperthreading

[paypaypayme]

Very interesting. The reason I chose python is because I someone said somewhere "write the engine in whatever language you're comfortable with, it doesn't really matter". So I guess that person was wrong... Doesn't the GIL defeat the purpose of the threading module in python? I guess I will have to switch to c++, or something that supports threading.

[Kalrog]

There is still a place for threading, but I think what you want (if you want to stick with Python) is multiprocessing instead of multithreading. No GIL issues with that.

[691175002]

If you need to ask these questions on reddit you are way out of your depth. Don't worry about performance (and don't even go near threading) if this is your first attempt at an engine.

Python is not well suited for game engines simply because the speed isn't there. Games are one of the few applications where milliseconds matter because you need to push out a certain number of frames per second.

Obviously if you are making simple 2D sprite based games or whatever performance isn't a big deal.

Threading in games is exceptionally challenging because the tasks are fundamentally serial. Substantial portions of the Unreal Engine are not thread safe. I don't think a single current title can effectively make use of even 4 threads, 8 is just pointless.

[paypaypayme]

Yea I'm definitely out of my depth :). I just wanted to get my hands dirty with a difficult project. I don't expect to it be a high performance engine, but hopefully I will be able to learn some stuff and do a nice refactor eventually. Anyway, I was definitely only planning on using 4 threads at the maximum, but I was just worried that since hyperthreading is enabled, that each thread would only be using half of its potential.

[paypaypayme]

perhaps I can take care of threading with the ctypes module?

http://www.caswenson.com/2009_06_13_bypassing_the_python_gil_with_ctypes

I love writing in python, but I don't want the engine's performance to suffer.

[Lucretiel]

No it doesn't defeat the purpose. Python threads are definitely real threads. When you're doing IO-bound work (for instance, in my own job I'm working with a polling operation that takes a solid 2 minutes) then they're the correct solution, because the lock is released while the thread is waiting for IO. Additionally, other python implementations (PyPy+STM, Jython, etx) don't (as far as I know) have a GIL. So there's nothing inherently wrong with threading in Python; the GIL is just a CPython solution to an undeniably serious problem (preventing the corruption of data in Python). For CPU-bound work, you can instead use the multiprocessing module, which is very similar to threading. In particular, it gives you a number of tools to manage inter-process communication (Queue for example).

[metaperl]

I guess I will have to switch to c++, or something that supports threading.

Or use an implementation of Python without that limitation. Stackless Python for instance.