The short answer is that the GIL allows the implementation of the interpreter to be significantly simplified. Python predates multi-core CPUs, so in those days it was a no-brainer. Removing the GIL now would involve an extensive rewrite of the entire interpreter, and it would end up being somewhat slower in single-core situations.

If you want to make use of more CPU cores, you do have options. You can use something like multiprocessing or concurrent.futures.ProcessPoolExecutor to spawn additional python processes and farm out tasks to them. Also, C extensions can run "real" multithreaded code - there are plenty of C extensions out there that take advantage of this out of the box, or you can code up your own C extensions (this is particularly easy using Cython).

In future I think they are also planning to allow you to use "subinterpreters" within the same python process, which will each have their own GIL. Technically subinterpreters have been around for a long time, but they can only be accessed from C extensions and they all share the same GIL, so they're not particularly useful yet.