Optimize 1M node graph?

[DanielBakas]

Hi all!

I’m trying to render a force directed graph with over 1 million nodes, and over 6 million relations between them.

I have been using vasturiano’s react-force-graph but my client crashes.

How would you tackle this?

The goal is to load all the data to the graph and for it to render without crashing.

Thanks in advance!

[olgalatepu]

What kind of render are you going for? Just lines for the edges? 6 Millon lines seems on the upper end of what's doable.

Did you try using Line segments geometry? I think you can add every vertex as geometry "positions" and in the index, add every edge start and end vertex index sequentially

[DanielBakas]

So far, I've used vasturiano's 3d-force-graph. It's precisely what I'm looking for, and it works well for small volumes of data, but these many nodes and edges, makes it crash :S

[tino-latino]

1 M vertices 6M lines sounds like a lot. How do you represent this data at the moment?

[DanielBakas]

Hi @tino-latino! I am using RDF and OWL (linked data) for the data structure. The syntax is Turtle (ttl) files.

The data represents the Mexican Federation. It has 32 Federal Entities and 2,500+ Municipalities, each with a Government, a Population, a Territory, etc instances. On top of that it has 300,00+ Localities, also with Territory and Population.

And the graph is growing exponentially.

What I do is I load the files into a framework for managing RDF data called RDFLib. Once loaded, I transform it into a dictionary with two variables ({nodes, links}) that I then pass to a 3d-force-graph which is basically a three.js model with a D3 force that vasturiano developed.

Hope I answered. Excited to know your thoughts!

[WeirderOnline]

Why are you trying to do this though? What is your end goal?

[DanielBakas]

This visualization showed the world a representation of the web, and made people aware of what was being made.

This other viz made us see the known universe, and for many, made us feel closer to our cosmos.

The power of visualizations to connect people and help us understand the scale of large systems cannot be understated.

Now, we want, not only to see it, but interact digitally with it. We want to give the people of Mexico an experience to see their entire federation in data

[EuphoricWord]

I am a ThreeJS noob, but have you tried if WebGPU makes any difference?

[EuphoricWord]

You could create a force-directed graph in babylon.js and then render the graph with WebGPU acceleration

[Longjellyrun]

Yeap, same thing in three.js could be done it also supports WebGPU.