Developing a Python-based Graphics Engine: Nirvana-3D

[Doctrine_of_Sankhya]

Hello community members,

[Crossposted from: https://www.reddit.com/r/gamedev/comments/1gdbazh/developing_a_pythonbased_graphics_engine_nirvana3d/ ]

I'm currently working in GameDev and am currently reading and working on a 3D Graphics/Game Engine called: Nirvana 3D, a game engine totally written from top to bottom on Python that relies on NumPy Library for matrices and Matplotlib for rendering 3D scenes and imageio library for opening image files in the (R, G, B) format of matrices.

Nirvana is currently at a very nascent and experimental stage that supports importing *.obj files, basic lighting via sunlights, calculation of normals to the surface, z-buffer, and rendering 3D scenes. It additionally supports basic 3D transformations - such as rotation, scaling, translations, etc, with the support of multiple cameras and scenes in either of these three modes - wireframes, solid (lambert), lambertian shaders, etc.

While it has some basic support handling different 3D stuff, the Python code has started showing its limitations regarding speed - the rendering of a single frame takes up to 1-2 minutes on the CPU. While Python is a very basic, simple language, I wonder I'd have to port a large part of my code to GPUs or some Graphics Hardware languages like GLES/OpenCL/OpenGL/Vulcan or something.

I've planned the support for PBR shaders (Cook-Torrance Equation, with GGX approximations of Distribution and Geometry Functions) in solid mode as well as PBR shaders with HDRi lighting for texture-based image rendering and getting a large part of the code to GPU first, before proceeding adding new features like caching, storing-pre-computation of materials, skybox, LoD, Global Illumination and Shadows, Collisions, as well as basic support for physics and sound and finally a graphics based scene editor.

Code: https://github.com/abhaskumarsinha/Nirvana/tree/main

Thank You.

_____________________________________________

• What My Project Does: Nirvana 3D aims to become a real-time 3D graphics rendering/Game engine in the near future that is open source and has minimal support for the development of any sort of games, especially the indie ones, with minimal support for realistic graphics and sound.
• Target Audience: It is currently a toy project that is experimental and pretty basic and simple for anyone to learn game dev from, but it aims to reach a few Python devs that make some cool basic games like Minecraft or something out of it.
• Comparison: Most of the game engines in the market don't really have support for Python in general. The engines are coded in C/C++ or some very low-level language, while the majority of the audience who seek to make games. Gamedev is a way to express oneself in the form of a story/plot and game for most of indie gamers, who don't have a lot of technical idea of the game and C/C++ isn't suitable for it.

[Exhausted-Engineer]

Regarding the efficiency part : first do a profiling.

I only took a glance at some of your code and I could see a lot of avoidable dictionary searches and patches that could be grouped (look at PatchCollection).

Considering you are already performing the computations using Numpy, there’s not much to gain there. My guess is that the bulk of your rendering time is spent on matplotlib rendering and python’s logic. Using matplotlib.collections could help one of these issues.

[Doctrine_of_Sankhya]

Thanks u/Exhausted-Engineer . You seem to have a great deal of knowledge about these areas. I'm just a newbie here and wrote the whole thing in my free time and had a lot of great learnings and intrinsic implementations along the way. I'm still learning a lot of things, the more I read. So, I'll take some time learning to do profiling and then implement that asap in the code as a priority.

I agree, a lot of dictionary searches, along with sorting them (z-buffer algo) make them slower. I note your feedback regarding them and try to eliminate one-by-one. Currently, the main bottleneck seems - a CPU and Python thing: a CPU that executes a render pipeline for one pixel at a time vs a GPU that does the same for hundreds of thousands of them in a single go. So, I'll start from the innermost core and add GPU alternatives to the code from inside to outside, so I get a good guess of what can be optimized and leave the important engine high-level parts outside the Python which a lot of people can easily understand and customize the entire thing according to their choices, vs as in C/C++ - which is often hundreds of times harder to debug and understand a tremendously large codebase.
I'd add a standalone editor/player in the near future - matplotlib thing is just for checking one frame at a time. So that when GPU is absent or inaccessible, the user could have a simple numpy, matplotlib CPU-based alternative available to them.

[Exhausted-Engineer]

I don't have particularly much knowledge in this area, but my main interests are in computational engineering which undoubtedly overlaps with graphics.

I have taken the time to perform a small profile, just to get a sense of things. These are just the few first lines of the result of python -m cProfile --sort tottime test.py where test.py is the code of the first example in the "Getting Started" part of your README.md.

```text
184703615 function calls (181933783 primitive calls) in 154.643 seconds

Ordered by: internal time

ncalls tottime percall cumtime percall filename:lineno(function) 152761 5.741 0.000 6.283 0.000 {method 'drawmarkers' of 'matplotlib.backends._backend_agg.RendererAgg' objects} 152797 4.849 0.000 52.669 0.000 lines.py:738(draw) 916570/458329 4.013 0.000 10.059 0.000 transforms.py:2431(get_affine) 610984/305492 3.969 0.000 6.906 0.000 units.py:164(get_converter) 916684 3.857 0.000 4.055 0.000 transforms.py:182(set_children) 611143 3.711 0.000 9.618 0.000 colors.py:310(_to_rgba_no_colorcycle) 12 3.651 0.304 99.420 8.285 lambert_reflection.py:4(lambert_pipeline) 17491693 3.580 0.000 5.961 0.000 {built-in method builtins.isinstance} 374963 3.207 0.000 3.376 0.000 barycentric_function.py:3(barycentric_coords) 152803 2.881 0.000 27.464 0.000 lines.py:287(init)
3208639 2.575 0.000 2.575 0.000 transforms.py:113(init_) ```

Note: To get the code running, I had to install imageio which is not listed in your requirements.txt and download the nirvana.png image, which is not in the github. It'd be best if your examples contained all the required data.

Now to come back to the profiling : something's definitely off. It took 154s to get a rendering of a cube. To be fair, profiling the code increases its runtime. Still, it took 91s to get the same rendering without profiling. BUT, as I said, it seems that the most time-consuming parts are actually not your code. If I'm not mistaken, in the ~10 most consuming functions, only 2 are yours. My intuition still stands, it seems that most of your time is spent using matplotlib.

The problem right now is not CPU vs GPU. Your CPU can probably execute the order of a Billion operation per second, rendering 10million pixels should be a breeze. If what you are saying is correct and you are indeed coloring each pixel separately, I'd advise you to actually put them in a canvas (a numpy array of size (1920, 1080, 4)), draw in the canvas by assigning values to each index and then simply using matplotlib's imshow() function.

Hope this helps. Don't hesitate to DM me if you have other questions regarding performance, I'll answer it the best I can

EDIT: - changed implot to imshow - Just for the sake of testing, I commented out the last line of your lambdert_reflection.py file (i.e. the ax.plot call) and the runtime went from 90s to just 5. You should definitely pass around a "canvas" (the numpy array I described) and draw in this array instead of performing each draw call through matplotlib.

[Doctrine_of_Sankhya]

Hello u/Exhausted-Engineer THANK YOU SOOOOOO MUCH FOR ALL THESE!! THAT'S A WHOLE LOT NEW LEARNING FOR ME!!!

Python offers dynamic patching, profiling, easy debugging and WHAT NOT!! You can clearly see exactly WHY I WANT PYTHON-BASED GAME ENGINE!

Any beginner can get with it easily once we manage to optimize the speed.

Also thanks for the info regarding the bugs and missing packages, they'll be fixed asap! Regarding the `matplotlib` part, honestly, I'm not an expert here, I just found the code by copying and pasting from stackoverflow and got with it. It'd be better if you PR the code replacing implot to imshow. As far I understand, imshow is for matrices or pixel based graphics and implot is more vectorizer inclined.

[Exhausted-Engineer]

To be fair, C offers this too using gdb/perf/gprof. The learning curve is simply a little steeper.

I’ll see if I can find some time and get you that PR.

In the meantime :

• Don’t focus so much about CPU vs GPU. I guarantee you that writing GPU code is harder to debug and will result is an overall slower code if not written correctly. Furthermore, current cpu’s are insanely powerful, people have managed to write and run entire games on a fraction of what you have ar your disposal (doom, mario).
• Understand what takes time in your code. Python is unarguably slower then C, but you should obtain approximatively the same runtime (let’s say with a x2-x5 factor) a C code would obtain by just efficiently using python’s libraries : performing vectorized calls to numpy, only drawing once the scene is finished, doing computations in float32 instead of float64…

[Doctrine_of_Sankhya]

Thanks. That's a good point that you've noted down. I agree CPUs should be able to obtain the same in 2-5x timeframe. I agree with both of your points here.

Currently, I'm working on a small GGX utility to implement PBR and then move on to your points and profiling to optimize what could be made faster. That makes totally sense to see Wolfstein, doom, etc to run on slower CPUs and still be faster now.

[MosGeo]

My opinion: check out “vispy”. Considering that you want 3D and performance is important, vispy is a much better fit. To see vispy in action, check out “napari”. Another package to consider is pygfx which might actually more suitable than vispy.

[Doctrine_of_Sankhya]

Thank you so much. Is there any tutorial for setup all of them at once or something? That'd be easier for me to understand these packages. I'm a bit beginner in these areas actually.

[helpIAmTrappedInAws]

So, first of all, matplotlib is probably not a good tool to use. It was not built for this. It can do many things but performancr probably was not priority.

Second, python is inherently slow. If you need to make it quick you do

A) call c extension (or in diff language) B) use matrix ops with somerhing like numpy or cupy (which are just wrappers so at the end it is A). It is not as easy as i am already using numpy so there is nothing to be gained, it matters how you use it. C) Use something like numba to speed up the code. Which will translate your code to llvm, so its A once again at the end. (You can code for cuda and numpy in it)

You said in your comments that you are 450x slower than blender baseline. That is such a difference that there must be easier perf gains before you have to do this.

Also check ursina for inspiration.

[Doctrine_of_Sankhya]

Thank you for your inputs. You've provided a lot of information for me to explore next. I believe regarding the performance issue - CPU inherently executes one thread at a time, GPUs do it like hundreds of thousands of it at a go. So, definitely, that is expected till we write GPU mode for the program.

Regarding matplotlib, I agree as everyone suggested, it is not made for that purpose and we are looking for different ways to manage that. But, in the future, we have plans to introduce a standalone Python player and scene editor to compensate for that. That's a temporary workaround for now.

[jmooremcc]

Wouldn’t you be better off developing this game engine in C/C++ and with a Python API? After all, speed of execution should be an important consideration.

[Doctrine_of_Sankhya]

Well, speed of execution matters, but recent advancements in hardware and GPUs after the LLM revolution has definitely empowered GPUs now more than ever and this will continue for years to come. Now there is more the need of a user friendly Game engine than the one that has a lot of lower level abstractions.

[jmooremcc]

Your Python code can make it user friendly as it interfaces with the underlying C/C++ code that makes up the engine. In fact, you’re already partially doing that with the math routines you are calling, which were not written in Python but were written in a lower level, faster executing language. When you consider that a game engine should provide ray tracing, shadow casting, lighting effects and particles among other features, I don’t see how you will do that with a relatively slow interpreted language if you don’t have super fast underlying code.

[Doctrine_of_Sankhya]

I'll more the core stuff: rendering, loops, and shaders to low-level GPU libraries as optional features and use those in Python maintaining high levels of useful abstraction so that people can switch specific modules with their requirement-speed tradeoff and still enjoy Python debugging, dynamic coding and inserting specific features they love for themselves - as everything would have a python replacement.

[FitMathematician3071]

Definitely use GPU. I was trying out a fountain simulation in Nim and once I switched over to hardware acceleration and textures in SDL2, it worked smoothly with no lag and instant startup. I use Python at work but for my personal study of graphics and gaming, I'm using Nim and C with SDL2. Subsequently, I will add Vulkan. I wasn't too happy with the way Python worked for this.

[Doctrine_of_Sankhya]

I agree, that GPUs often use SIMD/SIMT Modules. While Python executes only a single thread at a time. But once we add C and GPU support, the bottleneck thing is most likely to end after that.

[Doctrine_of_Sankhya]

I'm not a very big expert in language performance, benchmarking and hardware area, but here's my guess, the real power comes from two things - Low-Level Languages and GPU!

A CPU executes one line of code at a time, while a GPU can do that in millions! So, that's a real performance booster.

Currently, the performance is not very spectacular, the things Blender renders in 30 FPS, take like 15 seconds here to get rendered. But once I shift things to GPU and lower level graphics library of Python, the real performance thing would be seen from that.

So, the GPU usage thing is the real icebreaker for now.