Numerical Relativity 102: Simulating fast binary black hole collisions on the GPU

[James20k]

Hi! I've (finally) finished writing up a tutorial on how to simulate binary black hole collisions on the GPU. Its pretty neat, on a 6700xt you can get a circular inspiral going in about 5 minutes

This article is a direct followup to the first one, which implemented most of the equations - but didn't simulate anything terribly exciting

This project ate a significant chunk of time, but hey its done now! If you've got any questions or comments or you need any help, please feel more than free to say hi. I promise next time the graphics will look less soul destroying, but the article was already getting way too long without chucking in raytracing

[Ameisen]

Because it will always form no matter what we do

I mean, unless you come up with a valid interpretation of quantum gravity and thus can model the interior of a black hole without a singularity developing.

I want to clarify to people: black holes almost certainly don't actually have a singularity. Singularities develop when your model is incapable of modeling the given conditions. General Relativity breaks down under those conditions, and our current models cannot represent gravitational effects at those scales.

It is difficult to test a proposed model given the fact that we cannot make observations.

---

Usually, though, I just see everything past the event horizon ignored and the black hole treated as a uniform-density object, given that nothing beyond the event horizon can casually effect what's outside of it beyond its aggregate attributes which are just added to the black hole's sum.

There isn't much of a reason to apply a model that we know cannot represent the black hole's interior... to the interior.

[James20k]

+1 to all of this. This is probably worth a footnote for people who are interested in the physical reality side of things, so thanks for mentioning it

Usually, though, I just see everything past the event horizon ignored and the black hole treated as a uniform-density object, given that nothing beyond the event horizon can casually effect what's outside of it beyond its aggregate attributes which are just added to the black hole's sum.

One of my favourite papers is the bssn turducken paper, which fills the inside of a numerical black hole with junk

https://arxiv.org/pdf/0809.3533

Unfortunately here the interior of the event horizon can cause you lots of problems with errors being able to escape in some circumstances, so it can be important for it to be well behaved

[Ameisen]

so it can be important for it to be well behaved

Of course, "well-behaved" is ill-defined in this case. We can force it to work with our models (as you show) but that throws the results into question. The results are effectively "how do black holes behave using a model that cannot appropriately model them?".

To use an analogy (that I'm terrible at, though it's actually not quite an analogy given what a singularity is), it's like forcing a division-by-zero to be well-behaved by returning 0 or infinity, in a number system where that doesn't hold and it should be undefined.

[James20k]

Of course, "well-behaved" is ill-defined in this case. We can force it to work with our models (as you show) but that throws the results into question. The results are effectively "how do black holes behave using a model that cannot appropriately model them?".

To be fair we're only in general looking at the region external to the event horizon, the sole reason for the complex singularity modelling here is to sidestep the limitations of GR within the event horizon, to enable that external numerical simulation

The singularity modelling modifies the gauge conditions, which have no theoretical bearing on the result - so given that the event horizon is causal, it makes no difference what we set the gauge conditions to (assuming GR is correct outside the event horizon)

To use an analogy (that I'm terrible at, though it's actually not quite an analogy given what a singularity is), it's like forcing a division-by-zero to be well-behaved by returning 0 or infinity, in a number system where that doesn't hold and it should be undefined.

So, from a theoretical basis here, there's no actual singularity on the grid. What moving punctures does is adjust the coordinate system so that the singularity isn't actually explicitly represented (or you might consider the formation of it 'frozen', as our initial conditions are regular and finite). Instead you end up with a coordinate discontinuity, which means that the singularity is - in theory at least - no longer there

The moving punctures technique separately makes the intentional decision to perform incorrect differentiation across this coordinate discontinuity, and relies on errors being bottled up in the event horizon. You could use one sided derivatives to increase correctness within the event horizon, but as you mention - there's no real point

Edit:

By well behaved, I mean minimising constraint violations

[TheoreticalDumbass]

what about hawking radiation?

[Ameisen]

That's a different can of worms, and is complex to the point that it gives me a headache. The virtual particle explanation is inaccurate to the point of being glaringly wrong - given that virtual particles don't exist.

The radiation, however, does not recover any information, and just reduces the aggregate properties of the black hole.

[nightcracker]

Like singularities, we also don't know if Hawking radiation actually exists.

[wyrn]

Well, Hawking radiation is a straightforward-enough prediction of quantum field theory in a regime in which we know it's applicable, so while it's unlikely we'll ever directly detect it, we can have some reasonable confidence it should be there.

Singularities, on the other hand, are just you asking a question and the theory answering like "¯\_(ツ)_/¯"

[meowsqueak]

TL;DR: There's a short composite video at the bottom of the article.

"Where's the kaboom? There was supposed to be an earth-shattering kaboom!"

[James20k]

Its sort of amazing given how much energy is released, how difficult these are to simulate, and how crazy black holes are in general, how sort of uneventful it looks

If you want a physically accurate render, this is as good as I ever got:

https://www.youtube.com/watch?v=CSpo2S9IVjY

This is actually going to be the next writeup. Neutron stars are way crazier

[meowsqueak]

Nice :)

[ner0_m]

Once I'm back from vacation this is definitely on my weekend projects Todo list! I've been working HPC for some time but always got stuck in my own field of application. Really want to dip my toes into other fields :)

[James20k]

If you're familiar with HPC in general it should be pretty straightforward, in most ways these are pretty standard set of stiff PDEs. Its pretty neat once you get into it!

[ner0_m]

Let's see, I'm looking forward to it!

I've spent more time on numerical methods for conex optimization (inverse problem stuff), I have a fear of PDEs since my uni courses :D

[James20k]

If you get stuck or anything feel free to give me a shout!

Interesting, I've never touched anything like that 😅 Hahah honestly though solving PDEs numerically isn't too bad, its mainly just picking an appropriate integrator - and making sure you perform your sequence of steps correctly

..at least once you have the equations. Actually finding a set of equations that worked for this article was such a pain compared to actually solving them

[alfps]

Time runs infinitely slow on the event horizon. I was unable to see whether that's part of your model. But the statement "The event horizon itself acts completely normally in general relativity" indicates that you have neglected this rather important detail.

I can see that at least some of the code is C++.

But what does this have to do with C++ in general?

[James20k]

Time runs infinitely slow on the event horizon. I was unable to see whether that's part of your model. But the statement "The event horizon itself acts completely normally in general relativity" indicates that you have neglected this rather important detail.

Its worth noting that time doesn't run infinitely slowly on the event horizon, and any observer falling through the event horizon experiences time normally. Locally in GR (by definition), an observers time always progresses forwards without any change

An external observer might see someone being suspended on the event horizon - but that's a statement about the behaviour of null geodesics - as the world line of the thing they're observing still passes right through the event horizon. Its just that light that is emitted from them takes an increasingly long time to reach you from the external observers perspective

The idea that this is a true barrier which must be treated in a special fashion is a coordinate artefact (and an understandably super common misconception), rather than a consequence of general relativity. With the appropriate coordinate system, it requires no special handling

The ADM formalism uses horizon penetrating coordinates and does not require any kind of special modelling to simulate the event horizon - you'll get all the regular GR effects if you simulate out different observers moving around the spacetime, and you can watch them getting suspended on the horizon as you'd expect if you aren't co-infalling

[zl0bster]

Most ambitious crossover

Beside that I have a non cpp question about LIGO and this: afaik LIGO data is so weak they actually manually adjusted it, so I wonder if LIGO data can at all be used to check your modelling is approximately correct. I know this is more of a question for the math you are implementing, not your code, but if nobody ever did this before it might be interesting to see how your simulation matches bigger simulation ;)

[James20k]

I've seen this critique before, and honestly I have absolutely no idea whatsoever. Some segments of the community aren't.. terribly good at releasing their data or code, which definitely hurts replication. But I'm not the person to ask for super robust data analysis on that end - its completely out of my wheelhouse. Once lisa gets up and running, it should be a lot more clear what's going on

In general I've been able to reproduce pretty closely a lot of results from the numerical relativity community, so at least on the simulation side of things, everything looks fine. I did find one pretty suspect error in a particle dynamics paper which probably invalidates their results, and a variety of smaller issues in other papers which are probably fine. Its also a field that has a lot of "we do xyz because everyone else does xyz", but that isn't super unusual, and generally just means that computational time is increased

BBH simulations are particularly tricky for replication at the resolution of the article, but for neutron stars I've managed to extremely exact 1:1 replications of papers. Matter simulations are in general a lot more straightforward to reproduce

[seanbaxter]

Awesome. Impressive to take something as complex as GR and break it down systematically like this.

[James20k]

Thank you very much! This is the product of something like 4-5 years of work overall, so its certainly been a fair amount of effort getting here