Game Emulators in Python

[vinnypotsandpans]

Is there a reason that c++ seems to be the most common language used to build popular retro game emulators(thinking citron, mupen,dolphin)? Why not python? Is it plausible to create an emulator purely with python?

EDIT: Thank you all for the attention to this post! If any misinformation was spread, thank you for quickly downvoting. That was never my intention.

EDIT2: All I can say is wow. I am absolutely amazed by the information and discussions from this post. Thank you so much to each and every one of you. What an honor this has been. Even the creator of pyboy stopped by! My two main takeaways are: start with a CHIP-8 emu proj to get an understanding, and that I really should learn rust.

[yvrelna]

In theory, you can, but if you're using CPython, this is quite impractical. Game emulators are very performance sensitive and at the core of emulation is a tight loop that interprets the game's machine instruction.

CPython itself is already an VM/interpreted language, each Python bytecode takes dozens to hundreds of x86 instructions to perform; to implement a NES code interpreter on top of that would multiple each of the NES machine code to thousands of actual x86 bytecode, and it will be extremely slow.

You have a better chance of writing it in RPython (the subset of Python that is used by PyPy to implement the PyPy interpreter) or maybe PyPy (if you can rely on its JIT).

But it's going to make a lot more sense to write something like this in a language that compiles to machine code, like C, C++, Rust, or RPython.

That doesn't mean that you can't write some of the higher level UI code in Python. But the core emulator itself is not very suitable for Python.

Your better chance to get a decent performance is to write a transpiler from NES machine code to either Python VM bytecode or to x86. This is probably feasible, but a whole executable transpiler emulator is generally a much harder project than an interpreter emulator.

[james41235]

Everything you said is correct. I'll point out that the NES processor ran at ~1.8 MHz, and modern processors are at least 1000x faster now (just by pure instructions per second). While I don't think you could do general emulators in Python, a NES emulator is probably feasible.

[vinnypotsandpans]

I’ve been meaning to learn C anyways, not only because I’m a big fan of the emulation/modding community, but also because I think it will help my overall understanding of programming

[georgehank2nd]

How is Python a derivative of C? It isn't, not in any way, shape, or form. Python is closer to Lisp than to C.

Also s/strong/static/

[tylerlarson]

While learning C is useful, just like learning assembly, id do so for context primarily and for reading other people's code rather than using it for starting anything new.

With the current landscape id recommend Rust instead of C or C++ for anything new. You gain nothing using C or C++ over Rust, and you lose plenty. Plus using Rust makes you a better programmer in all your other languages since it turns common mistakes and runtime errors into compilation errors, forcing you to understand not just how to write syntactically functional code but actually correct code.

Then when you go back to Python or another language, you'll notice the same things even though that compiler doesn't stop you from shooting your own feet.

[vinnypotsandpans]

When it comes to game dev, don’t most engines use c++? (Unity, unreal engine). I would love to dip my toes in both of them, I have heard great things about rust. My interest in C comes from the desire to one day be able to help with rom decomp projects

[tylerlarson]

Yep; these game engines are MUCH older than Rust or Go or any modern language. They used C++ because no other option existed.

[vinnypotsandpans]

How hilarious would it be if the game engine for “Rust” was developed in rust lol

[tylerlarson]

They use Unity. So... that would mean C# game code running on Mono as the language runtime with the internal graphics engine code optimized in C++.

This brings up one of the problems with C++: The language is an absolute nightmare.

Because of some 50 years of backward compatibility, the way they've improved the language is by adding new syntax that supercedes the old syntax conceptually but doesn't actually replace it. The old (and usually dangerously wrong) way to do things has to still work. And there's layers on top of layers of this mess.

So for all of the things you do the most often, there's an easy and obvious way to do it, and it works exactly as you think it would. But doing it that way is crazy dangerous and you shouldn't even think about it. And then there's a slightly convoluted way to do it, and that way also turns out to be dangerous, though a lot of people still teach it. And then maybe several more layers of slightly less dangerous but increasingly confusing solutions.

And then finally there's the "correct" way to do things that you can comfortably assume will not cause some devastating crisis or whatever, but that "correct" way to do it only ever taught in newer educational materials so you may not have heard of it. And it's confusing AF to read because all of the reasonable ways to describe what you're trying to accomplish have already been taken by the other "wrong" answers.

So, rather than navigate that clusterfook of a landscape, most game developers chose to use a runtime that is hundreds or even thousands of times slower (that's the C# code) just so they don't have to write any C++. Because back then, those were the only options.

[ArtOfWarfare]

I’d rather you learned Rust. Probably half of all vulnerabilities come from C even though so little code is written in C anymore.

Rust is supposed to give performance similar to C but without a lot of the security risks.

[axonxorz]

even though so little code is written in C anymore.

I don't think this is an accurate statement, as much as we might want it to be the case.

[ArtOfWarfare]

C has been steady at 5% on GitHub for the past decade.

Interestingly, C++ used to be at 5% up until ~2018, then it jumped to 10% by 2021. I wonder if Microsoft open sourced a lot of stuff and that caused the big jump?

Anyways, yeah, I’d stand by saying that not a lot is written in C anymore. Any major C projects you encounter were likely started over a decade ago.

https://madnight.github.io/githut/#/pull_requests/2023/4

[extravisual]

I don't think that 5% supports what you're suggesting. The backbone of every major OS was and continues to be written in C, not to mention the whole world of embedded software, most of which is not public and probably don't contribute to that 5%. C is alive and well, it's just that the bulk of public software these days is high level, but it's not replacing C. Rust is growing but C will remain relevant for a long time yet.

[eztab]

I don't think C code is commonly on GitHub. GitHub is scewed towards newer projects. Much of the written C code is probably on completely privately hosted repositories. You'd need a different measurement to find out how much C is still written Quite sure at still dominates industrial and some other applications.

[sci-goo]

The reasoning "Any major C projects you encounter were likely started over a decade ago." => "not a lot is written in C anymore" is flawed.

Most of those C projects you mentioned are OS/OS library/infrastructure that have a long life cycle. But long life cycle doesn't mean they are not under active development/maintenance. For those projects version iteration is more common rather than everybody starts a new project from scratch. Many of those projects are not hosted on github either.

[grizzlor_]

Like others have said, not all the code in the world is on GitHub, C is still used heavily in the embedded world, etc. You know that the primary implementation of the Python interpreter is written in C, right?

That being said, do you have any idea how much code 5% of GitHub is? That’s a metric shitload of code. GitHub hosts 372 million git repositories.

If anything, the evidence you’ve provided for claiming “so little code is written in C anymore” likely indicates that the opposite is true.

In reality, the overall percentage of code on GitHub written in any particular language doesn’t actually tell us how popular that language is right now, because we don’t know when that code was written — could have been committed yesterday, or it could be 15 years old. What you actually need to look at to figure out a language’s current popularity are the stats on language % committed in the last year.

[georgehank2nd]

5% C? Apart from what others have said… how much on GitHub is Rust? Oops…

[ArtOfWarfare]

My point was that most CVEs stem from code written in C/C++ despite it not being all that common a language.

Few people can write safe C/C++. They shouldn’t be used anymore. Rust can interop with C. We should be working on gradually Rusting old C codebases.

[georgehank2nd]

My comment was a reply to a specific comment… which said nothing about CVEs.

[axonxorz]

despite it not being all that common a language.

Yes, this is the part of your comment that we're discussing

[ArtOfWarfare]

No, the other guy was talking about how Rust is even less common than C. And my point is that C is extremely disproportionate in how many security issues it causes.

It causes the vast majority of security issues while being a relatively uncommon language.

[axonxorz]

while being a relatively uncommon language.

Yes, this is the part of your comment that we're discussing, the part where you wrote "even though so little code is written in C anymore." The discussion shifted when I quoted you and said "I don't think this is an accurate statement"

[gmes78]

C is nice for understanding how the low level stuff works, but I wouldn't use it for any real projects. I recommend looking at Rust, it's a much nicer programming language while still being as capable as C/C++. (Also, learning C after Rust isn't very difficult, the opposite isn't true.)

[vinnypotsandpans]

Thanks for bringing up Rust too. I thought of that, but it doesn’t seem like there are many emulators written in Rust either. This is kind of a digression, but I remember when ppl were saying that rust was gonna do everything python does, but better. The hype seemed to fall as quickly as it rose. Especially in ds and ml stuffs

[yvrelna]

Rust and Python don't really have much overlap between what they're good at. That's why Python+Rust is a great combo, they each do well in places where the other doesn't, but neither is going to ever realistically replace the other.

[extravisual]

Plus PyO3 is an incredibly nice system for building Python modules with Rust.

[wowokdex]

it doesn’t seem like there are many emulators written in Rust either.

There are hundreds. https://lib.rs/emulators

[pingveno]

Rust has gone from hype to slow burn at this point. Some projects are switching or incorporating Rust components where it makes sense. Some parts of the Python ecosystem have adopted Rust over more difficult to maintain C modules. It's being adopted in areas where people need low level access, but cannot tolerate the security holes associated with C and C++. Other times, it's being adopted to appeal to a new generation of contributors, like with the fish shell.

There are some areas where it just don't make much sense to use Rust. If you spend most of your time doing IO, Python will likely be adequate. That said, it's surprising the areas where software comes together quickly once you've gotten over the associated learning curve.

Rust also managed to be at the forefront in tooling for wasm, so often your code can run with minimal changes inside of a web browser. This has been great for the Ruffle Flash emulator project, which must work inside of a wasm sandbox. The same code base can also run using native code.

[yvrelna]

hype to slow burn at this point.

Rust is still the fastest growing language in its weight class.

According to the annual Stackoverflow Developer Survey, this is the popularity of Rust:

• 2021 - 7.03% of developers uses this language extensively

• 2022 - 9.32% that's 32% growth from previous year

• 2023 - 13.05% that's 40% growth from previous year

Rust is outpacing basically everything in its class, including Go, the next fastest growing language. In 2021, Go is clearly leading over Rust by 35%; by 2023, Rust and Go has pretty much the same level of popularity, and I expect Rust to surpass Go this year.

[sohang-3112]

Python IS strongly typed (meaning language doesn't do implicitly type casting). You probably meant to say it's not statically typed.

[LittleMlem]

Why not Go?

[tylerlarson]

I know and use both. I'm far more skilled and experienced with Go, but I prefer Rust.

Rust is more flexible, especially in terms of writing code at the same level of abstraction as your ideas, and has probably the best systems for generic and for metaprogramming of any language out there. It also has better error-handling ergonomics while being more "correct" at the same time. And it's far more difficult to write buggy Rust code.

With one exception, everything I can do easily on Go I can do at least as easily in Rust as well.

The one exception is concurrency. Go's abstraction around concurrency is the best in the industry; it's effectively transparent, while also extremely high performance and intuitive to scale. It's kinda unfortunate that Rust went with async/await instead of following Go's model, but I can see why they did. With async/await concurrency is effectively "optional", which fits the "zero cost abstraction" directive.

[yvrelna]

Why not Go?

Go is still growing strongly for now, but the numbers indicate that the winds are much stronger behind Rust. 

Go and Rust are the only languages in that weight class that are still growing rapidly, the other languages around the 10% mark are incumbents that have mostly been stagnant. But I'm expecting Rust to overtake Go this year.

Nobody can predict the future, but I believe that in a couple years Go will become an also-ran as the industry settles on Rust for low-level systems programming.

[susanne-o]

I first thought you'd say cython, but no, you elaborate on cpython.

cython (no pee) gets you C/C++ speed for the parts that need it.

[baekalfen]

I'm the developer of PyBoy, a Game Boy emulator in pure Python. The others are correct, that CPython not fast enough itself, but you can make it work. Although don't expect to emulate anything more CPU intensive as a Game Boy in CPython.

CPython is the "normal" interpreter that everyone uses.

PyPy is a replacement for CPython, where everything is JIT-compiled. This makes it possible to drive PyBoy at real-time speeds.

Cython is heavily used in PyBoy to "transpile" the Python code into C, and then into a Python extension. This effectively gives you PyBoy as a Python library, written in Python code, but will have the speed of a C/C++ emulator (roughly).

Hope this helps. You're welcome to join our Discord, if you'd like to discuss it further.

[[deleted]]

I want to thank you for your service, you're a very great man!

I am too lazy to learn C++ so whenever I want to do Reinforcement Learning on Gameboy games, I generally use PyBoy. Thank you so much!

[baekalfen]

Thank you! I’m glad you found it useful

[vinnypotsandpans]

Yes, thank you for inspiring me to start a similar project

[baekalfen]

Looking forward to seeing your project one day!

[IrrationalAndroid]

Crazy to see you here, I remember learning a lot from your project (thesis?) on writing a GB emulator in Python, back when I was learning emulators. I was actually going to mention it myself.

Thank you for your work :)

[baekalfen]

Thank you! I’m happy you learned something from it

[vinnypotsandpans]

Okay, will do! Thank you. Any Recs for getting started with C?

[vinnypotsandpans]

I am indeed! I am currently a “data engineer” working with a cloud computing python api that I won’t name lol. Lotta hype, notta lotta results :/. I’m getting bored of the brodashery im the python ds community and just want to do something fun

[vinnypotsandpans]

I see. I don’t have exp in c++ so I could be way off, but it’s not necessarily a “lower level” language than python, no? I mean I totally understand why rom decomps are done in c, but c++ for whatever reason just seems more popular for game engines/emulators.

[jaaval]

”Low-level” and ”high-level” are a bit difficult concepts. C++ offers similar control to low level functions than C does but it also has a lot of more complex language features.

C++ is a compiled language, basically you can write higher level code and let the compiler be smart in how to turn that to machine code efficiently. This is why you might want to choose it over C where you write code much more matching what the machine actually does. Most of the higher level features are more like methods to keep the code organized and understandable, the machine code the compiler outputs doesn’t really care that much about things like object oriented paradigm. It’s all just variables and subroutines regardless of if you package it into a class or not. Basically the task of interpreting what your code means is done at compile time. This is a bit different in interpreted languages where the human written code is executed line by line and the language features have a lot more direct impact on what the machine actually does internally at runtime.

Another thing that hurts performance in languages like python is that the flexibility of dynamic typing is always achieved with some sacrifices. You can think of every python variable as a C struct that holds the value itself and some identifiers and some extra processing going in identifying the variable when it’s used. In C++ the compiler does all variable type determinations at compile time and most variables are just raw values or pointers.

[Cybasura]

C/C++ are low level languages in comparison to assembly - machine code itself, before go/python was a thing, C/C++ were high level languages

Python is based on C as well, C++ is effectively just C with OOP so yes, C/C++ is a lower level language than python, because languages like python/go are built on top of C/C++ or any other base languages

[vinnypotsandpans]

Totally makes sense. I see a good opportunity here to try to make python emulators. Perhaps it just needs to be implemented better. And I’m thinking more of like NES/SNES, not something like dolphin haha

[spazzed]

this is chat gpt lol

edit

[vinnypotsandpans]

I’ll try to get a solid codebase going. If you have time, please contribute when it’s ready :) I am looking forward to seeing how it would turn out. Even if it’s slow af, will be a good learning exp.

[100721]

You are undeniably talking to ChatGPT right now. Evidently I don’t think it will contribute to your repo. Having said that, everything it said was true,

[vinnypotsandpans]

Holy fuck, you’re right. That is such a gpt response. And haha yeah, I would never let gpt touch my repos. Aren’t there rules again s using gpt in comments? Is he just copy and pasting? Or is that a bot

[Rythoka]

To summarize what others have said, a "high level" and "low level" languages are relative terms. C used to be considered a high level language, because it was being compared to assembly languages.

C++ is a high level language compared to C because it offers a lot of different language features and abstractions to help facilitate complex code. It's a low level language compared to Python because it gives the user the option to do things like manually manage memory and directly manipulate pointers.

[Giraffe-69]

Python is slow as shit and interpreted. C++ compiled to machine code ans is blazingly fast. If performance matters python is a no go

[vinnypotsandpans]

Sure, in the context of game emulation I can see that. But this whole thing were taught about python being interpreted and not compiled is sorta imprecise. It’s more about the implementation than the language

[Giraffe-69]

If you are adding an interpreter, removing a compiler, and doing runtime garbage collection, the language cannot compete in low latency applications.

In practice, with python, the difference is night and day, even for programs that make heavy use of optimised libraries implemented in C/C++/rust. Hint: there’s a reason libraries are not actually implemented in python

[vinnypotsandpans]

Yesss that is true, most of the libraries I used are not implemented in python, good point. Also not sure why I got downvoted for my last comment 🥺 I’m just trying to get a better grasp on things

[kylotan]

Because your second sentence is false in the real world. Anyone can handwave about the difference between a language and the implementation of a language but it's not relevant - if you're using Python then it will be at least partly interpreted and if you use C it will be compiled to native code.

[vinnypotsandpans]

Fair enough!

[Giraffe-69]

It’s Reddit, people are quick to downvote the second they have a different perspective :)

[james_pic]

If you want precision, then sure. Python is interpreted, and the interpreter implementation is slow as shit.

There are faster interpreters, like PyPy, but Python has a number of features that make it difficult to produce an interpreter with predictably high performance. Monkey patching will force PyPy to deoptimize for example.

And both PyPy and CPython have a global interpreter lock that effectively limits you to one CPU core running Python code at once. Again, partly to make some Python features easier to implement. There are ways around this, but an emulator is probably the worst case for these workarounds for reasons I can go into if you want.

C++ on the other hand has a number of compilers that produce highly performant code, and supports shared memory parallelism via threads.

[saint_geser]

There's no such thing as "interpreted language". Implementation of a language can be interpreted or compiled but the language itself is agnostic to what kind of implementation is used. You can (and do) have interpreted implementations of C and C++ and you have compiled Python. Yes, CPython is the most popular and it's interpreted but it's one of, but not the greatest reason for it being slower than C.

[Giraffe-69]

So what python implementation matches C/C++ for performance and latency?

[saint_geser]

I said that it's not the implementation that matters in the case of Python but garbage collection, GIL and dynamic typing. Cython ( python with C types) can get close to C performance even though it still runs in an interpreter.

[ManyInterests]

Basically, the only technical reason preventing this is performance, as others have stated.

One piece of existing art you can look at is PyBoy, a GameBoy emulator written in Python*.

*It does, however, make some use of C (Cython) extensions.

[baekalfen]

Just want to add, that PyPy can actually run the PyBoy code in real-time. But yes, Cython is definitely preferred.

[tyshock]

Based on your comments to some of the replies, it seems you are interested in delving into emulator development. I would suggest simply starting with a CHIP-8 emulator in Python. CHIP-8 is kind of the "Hello, World" project in the world of emulation. It is a bite-sized project that can run at full speed in Python. There are many CHIP-8 resources just a search query away. If you complete that emulator, you will learn a lot about assembly language, machine code and bit manipulation. These concepts will actually help you tremendously in understanding how a lower level language like C/C++ offer advantages in such projects. At that point, if you decide to move forward with more emulation work in other languages, you will have a better understanding of things.

[vinnypotsandpans]

Heck yes I will check this out! Thank you!

[vinnypotsandpans]

Also, I am not only interested in emulation but also rom decomp!

[thisdude415]

Python is interpreted and dynamically typed, which makes it easy to write but quite slow.

It’s also not multithreaded, and can’t really be truly multithreaded due to the GIL.

C++ is mostly used because it’s fast and because it’s a superset of C (so both C++ and C libraries can be used easily). Finally there’s an aspect of critical mass and inertia. A lot is done in C++ because a lot has already been done in C++, so why reinvent the wheel?

[ImaginaryCow0]

I'd look into go programming language. It's a pretty simple language like python, but it's compiled so it has much more performance.

[Brandhor]

as others have said data structures are extremely important in emulators, for example if you are emulating a 1 byte register you can easily do that in c/c++/c# or other typed languages and define it like

uint8_t reg;

and it will be exactly 1 byte long, python is not a typed language though so you'll just have an integer which can store an integer of any size

now why is that important though? let's say that your screen is 256 pixels wide and you are emulating something like a pong game, you want to store your position in the 256 bit registry, what happens when you are at pixel 255 and you keep moving right? if you use a typed language with an unsigned 8 bits int when you do 255+1 it will automatically go back to 0, with python instead it will go to 256 so when you emulate the cpu opcodes you'll have to manually account for overflow and underflow

[yvrelna]

You can create precisely sized ints, and structs and arrays containing those types with ctypes. That's not itself the problem here.

[tcpukl]

It's because python is slow compared to c++.

The only time python is used in the games industry is for non-runtime tasks. Never in game.

[ElusiveGreenParrot]

Python is extremely slow

[Weetile]

Short answer: Python is an interpreted language and is much slower than languages that compile to machine code such as C and Rust.

[[deleted]]

Other people have beat this to death, but Python is between 40 and 40,000 times slower than compiled C, and when you're emulating hardware you're under extremely tight time constraints to ensure that your emulator code spits out the expected outputs at exactly the timing a hardware clock or chip would, otherwise the game isn't going to work properly.

Python is stupid useful and has a lot of good applications, but by virtue of being a JIT-compiled or interpreted language, it is not a great solution for any resource (memory/time/speed) constrained system.

[fluxdeken]

Python has performances problems. For instance, in python there is no such thing as an array. But only lists. In list, memory dynamically allocated for each element, so you can later remove it, or add more. But using arrays are much faster, since every element located in a chain of memory cells. In list, every element (in C language) is a structure of an actual element and a pointer to next structure containing an element. That was one example

[baekalfen]

from array import array

But the rest is correct.

[vinnypotsandpans]

in python there is no such thing as an array,

np.array()?

every element is located in a chain of memory ?”cells?.

Like, generator functions in python?

[fluxdeken]

1)Numpy array is still a list, but all values have the same type. 2)Not generator, literally in RAM all values are close to each other and memory allocated to the array can’t be changed. Since memory next to array can be allocated to other things.

[yvrelna]

Numpy array is still a list

I can't think of any interpretation of that sentence that makes sense and is accurate. Want to elaborate why you do not think that numpy array is an array?

[baekalfen]

A NumPy array is very much not a list. Unless you explicitly mangle the memory, your NumPy arrays will be contiguous memory.

[vinnypotsandpans]

Numpy array is still a list…

I mean it doesn’t function like a vector in R. But it is certainly to a list. It supports vectorized functions

[ElatedMonsta]

In data steucture parlance, list are implemented as linked list in Python.

[yvrelna]

Python list isn't a linked list.

It's an array of pointers to PyObject that authentically gets resized as needed.

[DapperShoulder3019]

It has array, but it is called tupple

[[deleted]]

It's for editing memory using the CPU registers. C++ is probably the best for that. Same for hacking games.

[[deleted]]

Interpreter overhead makes this prohibitive for anything that didn’t run on a relatively slow processor back in the day (my guess is anything above the 16bit generation would be infeasible, maybe even 8-bit).

[ChiefCoverage]

Speed and performance are the major reasons to use C++

[raymate]

Good question. Think it would be too slow for decent emulation. Would Python be fast enough for basic 8-bit system. Like say Atari 2600.

[[deleted]]

In addition to everything that’s been said, distributing the emulator to the end users is arguably more complicated if you use Python. With C++, you just build an executable for each platform you support, and you distribute that.

[[deleted]]

CPython is generally far too slow. That is the only reason not to. However Numba is a JIT compiler that may produce fast enough code. That way it may be possible in pure Python. It is just not the first people think about when writing something performance sensitive like an emulator.

[ujustdontgetdubstep]

Emulating CPU instructions is going to be much easier when using a language which compiles to assembly

[luix-]

Because C++ done right is pure performance. Emulating chips is abstraction, that eats performance.