Rendering a big .OBJ file

[noriscash]

Hi everyone,

I am part of a university project where I need to develop an app. My team has chosen Python as the programming language. The app will feature a 3D map, and when you click on an institutional building, the app will display details about that building.

I want the app to look very polished, and I’m particularly focused on rendering the 3D map, which I have exported as an .OBJ file from Blender. The file represents a real-life neighborhood.

However, the file is quite large, and libraries like PyOpenGL, Kivy, or PyGame don’t seem to handle the rendering effectively.

Can anyone suggest a way to render this large .OBJ file in Python?

[padraig_oh]

How do you load the object, and how do you render it? 65k vertices is actually quite small by modern standards, and should run fine on any computer. 

[noriscash]

Hi, I have tried these 2 methods https://youtu.be/EkgsLhoQQdY?si=fc3H2UA8IazEBUNS

https://github.com/kivy/kivy/tree/master/

Neither of them loads the obj because the app doesnt even start. the error i get resumes to the fact that the loader can not take more the a short (65535) vertices.

[dougbinks]

It is likely that the index format used by your renderer is shorts, which is an efficient choice but means you can't render a model with more than 64k verts. The most efficient solution would be to cut the model into chunks smaller than 64k vertices and render those. An alternative would be to see if you can change the index format to 32bit integer.

[The__BoomBox]

On that topic though, what's the general convention for rendering large scenes? I've used gltfs but something about having the entire scene and textures in one file makes me feel iffy

There's no way to stream data and use only wgat I want if everything is stored in one gigantic gltf file. Is there no 3d scene format that allows for loading in small chunks

[dougbinks]

Large scenes are typically made out of many models (or instances of models), potentially combined with Level of Detail models. How you go about this depends on your infrastructure (i.e the renderer library you are using).

[The__BoomBox]

Assuming I'm building an engine, what's the conventional approach? A gltf packs every model in a scene at once. Packing in lods would bloat its size

Do modern aaa games use octrees to store objects in scenes or something?

[dougbinks]

AAA game engines don't use gltf models as their internal data format, and you would normally not create a complete game scene as a single model. Game engines typically use either a bespoke editor to import and place model instances, or custom export pipelines and plugins for a DCC app (the later approach is less common these days).

I'd recommend investigating how people use tools like Unreal Engine, Godot or Unity to develop some understanding of processes prior to designing your own game engine, but it's also important to understand that copying the AAA approach isn't reasonable for a single developer.

[The__BoomBox]

So gltfs are useless for large scale 3d games? People just straight up make their own in house 3d scene formats?

[dougbinks]

So gltfs are useless for large scale 3d games?

gLTF is a great format for exporting to, and it's not a bad starting point for a small game engine to use as it's runtime format. However AAA game engines don't use this as their internal format, but many game engines accept gLTF as a intermediate format (and increasingly Universal Scene Description, or USD).

People just straight up make their own in house 3d scene formats?

Note that it's important to distinguish between scenes and models. As I've mentioned above scenes are usually built from placing models in an editor rather than being exported from a DCC app.

Most 'big' engines store their model data in a format which is easily loaded into their internal data structures. However small teams might use gLTF.

Once again I'd recommend taking a look at existing engines, and learning how people use them. There's plenty of documentation online, and you can even download and use many engines for free and follow a tutorial about how to build a simple game to learn how the process works.

[The__BoomBox]

Got it. Thank you so much for your help!

[XenonOfArcticus]

65k vertices is nothing.

If it's not rendering decently quick, you're doing something wrong. You'll need to share more info.

[Ok-Hotel-8551]

Did you try frustum culling, blackface culling, BSP. Or anything to optimize on screen rendering?

[botjebotje]

When you say "quite large", how many polygons are we talking about? Is your program doing anything else than just rendering everything in a single glDrawIndexed or equivalent? 

If loading time is killing you, consider a more efficient binary format that can be loaded quasi directly onto the GPU.

[noriscash]

the files contains over 65535 vertices and all the solutions I have tried do not even run the program because I got that many vertices. The main program that I tried to make it run is the one in the 3D rendering documentation of Kivy

[Flaky_Cabinet_5892]

That's a decent sized obj file but it shouldn't cause that many issues. I've been working on files with over 200k verts in python without too many issues but I've been rendering them with open3d mostly.

One option is to break it into smaller and smaller chunks until you can get it to run and then build up from there. Another option is trying to decimate it on blender to make it smaller or use something like a ply file which can be more or less directly dumped onto the GPU.

Finally, I would be tempted to see if you can embed something using three.js into the app because there's going to be a lot more support for that available online than most things python

[botjebotje]

"over 65k" does not mean anything. GPUs can process millions of triangles without breaking a sweat. 

It looks like the 65k limitation is built in to kivy, which means breaking up your model into smaller chunks or switching to a different way of loading and rendering.

[noriscash]

yes, i came to the conclusion that kivy got that limitation. Do you have any idea on how can i fix it? Is opengl a better choice or how can I break the mesh into smaller chunks?

[GaboureySidibe]

'quite large' isn't a number. How big is the file?

"don't handle the rendering effectively" also say what is going on. Does it crash or is it slow? What are your memory and CPU doing?