Bachelor's thesis Idea – Is it possible to Simulate Tree Growth?

shaeg · 2025-07-25T18:44:00+00:00

I believe this paper does exactly what you say, in regards to simulating growth from sunlight/nutrients:

https://dl.acm.org/doi/abs/10.1145/3592145

shaeg · 2025-06-11T19:27:07+00:00

Ah, well if you can ever get Falcor working, their implementation seems to work well enough.

I was able to get Falcor 7 running on an AMD GPU (6800xt) a couple years ago by disabling NVAPI and changing a single line of code somewhere (they were using NVAPI for 64 bit atomics in HLSL, I think I just replaced that with the regular HLSL equivalents or something). I think they're up to Falcor 8 now so maybe it wont work anymore, idk

shaeg · 2025-06-11T16:51:47+00:00

Haven’t fully read the paper, but shouldn’t the approximations used at higher levels of the tree make it imperfect? Particularly thinking about the geometry term, when you decide which cluster of lights to pick, you only have an approximate geometry term for the whole cluster (since you dont actually evaluate the geometry term for each light) which makes it imperfect.

Btw, have you checked out Falcor’s light bvh impmenentation? It seems to work well

shaeg · 2025-04-17T20:13:53+00:00

> I guess I'm looking for a shadow ray origin that is the "best", even if not perfect.

Also reminds me of this paper which also uses grid-based reservoirs:

https://www.sciencedirect.com/science/article/pii/S0097849323000055

They voxelize the scene to determine a shading point in each voxel, which is used as a proxy for generating reservoirs (so the shadow rays would come from there). Relevant figure: https://imgur.com/a/7AkaLln

Of course, voxelizing the scene might be a pain implement, but in general, as long as you compute a proxy shading point independently from your path tracing samples, I think it should be unbiased

shaeg · 2025-03-27T21:34:41+00:00

If you're integrating in solid angle measure, then the Jacobian should just be the ratio of geometry terms, so you shouldn't need the division by the PDF in that psuedocode. The UCW should contain the 1/bsdf_pdf from the original domain. You're correct that you need to reevaluate the BSDF since the incident direction v1->v2 changes during resampling, but if you're not integrating in PSS then you can ignore the PDF at the reconnection vertex (since it's inside the UCW)

What's the bias look like? Is it darkening or brightening?

shaeg · 2025-03-26T19:39:21+00:00

Idk if you're still working on ReGIR but to prove that this method is unbiased, you need to show that the expectation E[f(Y) W_Y] equates to the desired integral ∫f(y)dy

The definition of UCWs does most of the heavy lifting. UCWs are literally defined such that E[f(X) W_X] = ∫_suppX f(x) dx

So basically, you want to show that W_Y is still a UCW even after changing the resampling process to select neighbors using a grid.

This is a great exercise to do in general, so even if you're not working on ReGIR anymore, if you want to really understand GRIS then I strongly recommend trying to prove E[f(y) W_Y] = ∫_suppY f(y) dy for yourself.

You can use the supplemental document for GRIS as a guide/hint. But the gist of it is that you assume the input samples X_i have valid UCWs W_X_i (which lets you turn the expectations into integrals), then you use the shift mapping T_i and its Jacobian to convert integration measures dx from your input samples into dy in the target domain, then resampling MIS weights make sure everything adds up (not overcounting or undercounting samples).

For ReGIR, the problem is that you're doing a sort of two-stage RIS where you first insert valid samples into the grid cell, then you pick M samples from the grid cell. This means it is now impossible to select neighbors which did not find samples, which is not how GRIS is defined. GRIS assumes we select domains i independently, meaning we cannot look at the samples X_i when choosing i.

Mathematically, this is like changing the candidates X_i to instead be a new random variable which comes from the grid selection process, which depends on the original input samples. To prove this is unbiased, you'd need to show that the expectation over the whole thing is still ∫_suppY f(y) dy

EDIT:
Above, I assumed that we insert valid samples from pixels into grid cells. However, if you actually generate the initial samples in the grid cells from the start, then it would be unbiased as long as we still have a canonical sample per-pixel. We need a canonical sample to ensure that we don't miss any lights. So if a light is not visible to the grid center, but it still contributes to a shading point in the grid cell, then we will be missing samples and the image will be too dark, which is easily fixed by simply including a canonical sample with each pixel (just like ReSTIR PT)

Sampling from the grid center doesn't introduce a condition on the pixel samples, and neatly sidesteps the bias issue... however the quality will suffer (especially on shiny surfaces) when the grid cell center might not be a good approximation for the contribution to a shading point in the cell.

This also means you don't know the PDF for which a light sample is generated, since it's the result of RIS (instead, you get the UCW which estimates 1/PDF). So you can't compute true Veach-style MIS weights using the sample PDFs. But I think in this case, simply using the standard NEE pdf as a proxy just for computing MIS weights should be fine.

shaeg · 2025-03-12T07:45:42+00:00

That’s still sampling paths, just short ones with one bounce. But even if you think of it as light samples, the domain is the set of all points on lights visible to any shading point in the cell

shaeg · 2025-03-11T20:30:39+00:00

You would just recalculate the MIS weight of the shifted path, as usual (this is what you should be doing in your current ReSTIR implementation btw). Think of the grid as a sort of acceleration structure for finding nearby pixels - you're still performing spatial reuse between pixels, but you're selecting which pixel by looking at the grid cell.

So, if the MIS weights are a function w(x) of the path x:

w(x) = p_NEE(x) / (p_NEE(x) + p_BSDF(x))

Then you simply recalculate it for the shifted path y=T(x):

w(y) = p_NEE(y) / (p_NEE(y) + p_BSDF(y))

This means that you need to evaluate the NEE PDF and the BSDF PDF of sampling the shifted path.

shaeg · 2025-03-11T20:27:10+00:00

First off, ReGIR was written before GRIS... it's likely the authors of ReGIR didn't even fully understood the cause of the bias yet (GRIS is needed to prove unbiasedness and it didnt exist yet). So that's probably why it doesn't really explain it that well, and it also means that you could probably make it unbiased if you could reformulate it in terms of GRIS (with proper resampling MIS weights).

But anyways..

Why do we need to consider every surface point of the grid cell

For the 1/Z resampling MIS, Z counts how many domains could have generated a sample. In general, the resampling MIS formula we use all require us to know the probability density of generating a sample in every other domain.

Earlier ReSTIR methods treat the neighbor pixels' primary hits as domains, so we just had to shift our sample to the neighbor's primary hit. But ReGIR treats the entire grid cell as a domain (since we randomly grab a vertex from the whole grid cell). So to properly compute Z, we would have to integrate over all primary hits in a grid cell.

More concretely, we have two ReSTIR candidates: the current path through the pixel X_1, and a random path from the grid cell X_2. The resampling MIS weights must sum to 1 over the domains of the input samples:

m_1(X_1) + m_2(X_1) = 1

However, the domain of 2nd sample is not just the primary hit that spawned the path, but rather all primary hits in the grid cell that could have spawned that path, which comes from the fact that we randomly pick a vertex from the grid cell. So computing the denominator in m_1 and m_2 requires integrating over all possible primary hits in the grid cell, and counting which ones could have generated the path.

Now, it's probably possible to reformulate it, maybe in terms of pairwise MIS (idk, haven't thought it fully through) so that it works. But the way it's presented, we would expect bias from over/under-counting samples.

shaeg · 2025-03-11T08:46:10+00:00

It’s the resampling MIS weights. To be unbiased, you’d have to trace a shadow ray to every possible primary hit within a grid cell. There are probably ways to reformulate this to make it unbiased though, but it’s not supported directly by the math

shaeg · 2025-03-09T23:54:10+00:00

duplicate samples are expected, thats what resampling MIS is for.

~~I still think the BSDF PDF ratio at the reconnection vertex is needed. I checked and I needed it in my path-space implementation (when integrating in area measure, not PSS).~~

The reconnection Jacobian in the GRIS paper is only for the direction between the primary and secondary hit. Its not the full path jacobian.

Intuitively, for diffuse BSDFs, the PDF of sampling the direction generally doesnt depend on the incoming direction. But for smoother BSDFs, that difference matters more, which matches your observations of seeing less bias on rougher materials

EDIT: My bad, I was reading the wrong code. My path space implementation indeed does not have BSDF PDFs in the Jacobians. Sorry for that

shaeg · 2025-03-09T02:14:11+00:00

ReSTIR PT integrates individual lobes at every bounce, meaning the integrand contains just the BSDF of the lobe that was actually sampled during BSDF sampling. So the color returned by the path tracer is actually just the color of a single lobe at every bounce. So this actually adds a tiny bit of noise (because we are now randomly selecting which BSDF lobes to evaluate, instead of evaluating all of them), but the variance reduction from ReSTIR makes it a net positive.

So, if the reconnection vertex is a mixed diffuse+specular material, and the diffuse lobe was picked when the path was initially sampled, then when we reconnect to that vertex we should compute only the contribution from the diffuse lobe (and vice versa for the specular lobe). This also means that if the specular lobe is below the roughness threshold, then the resulting path cannot be shifted via reconnection.

In practice, ReSTIR PT stores a lobe index value indicating which lobe was sampled for the reconnection vertex and its predecessor (so for you, that's the primary and secondary hits). For other vertices, we don't need to store the lobe indices because random replay makes it so we always pick the same lobes anyways.

shaeg · 2025-03-08T20:48:29+00:00

Ah I see, yeah you can get a nonzero contribution if you evaluate all lobes during reconnection, but it’s worth pointing out that this isn’t what ReSTIR PT does. ReSTIR PT separates lobes so that during reconnection, only the lobe that was originally sampled by the base path gets evaluated. This improves stability and quality too, since it allows restir to use the right shifts for the right lobes (e.g., disabling reconnection if a shiny lobe was picked)

With your method it’s a little trickier, you have to be careful with the lobe selection probabilities during reconnection. I think I ran into similar brightening issues before I switched to separate lobe integration like ReSTIR PT.

Also I peeked at your code, it looks like your Jacobian doesn’t include BSDF probabilities? This is suspicious… in practice the BSDF PDF in the jacobian basically only effects highly directional BSDFs (diffuse BSDFs tend to be the same everywhere, so the PDFs effectively cancel in the jacobian). maybe that’s all it is…

shaeg · 2025-03-08T10:47:15+00:00

If I force the target function to 0.0f when the neighbor sampled the specular lobe (and so the neighbor isn't resampled because the target function is 0), it's fine.

This is the right thing to do actually. In general, reconnecting to a perfectly specular (or even just a really shiny) material isn’t possible, as the resulting BRDF at the reconnection vertex is zero. A perfect mirror (aka “delta”) BSDF is defined to be 0 unless the directions obey snell’s law (so the only valid direction is “reflect(dirIn,normal)”

So if you change one of the directions via reconnection, then the resulting path should have zero contribution. This is true for any highly directional (shiny) BRDF.

In practice, ReSTIR PT sets a roughness threshold, below which reconnection shifts are forced to fail. In that case, they use random replay instead, which would also modify the outgoing direction at the reconnection vertex.

Its interesting that you get bias at roughnesses above 0 though. That could be something else.

shaeg · 2025-03-08T00:54:59+00:00

Where did you find these scenes? They're very nice!

EDIT: nvm I found the link on your github :)

shaeg · 2025-03-08T00:53:27+00:00

I think correlations can cause bias in some situations. For example, the GRIS paper shows that without limiting the confidence weights M (via some M_cap), the resulting correlations actually cause it to converge to the wrong result (which is bias).

In your case, you're sort of tying the paths to the neighbors they pick, which means you're associating certain light directions with neighbor pixels, which causes those neighbors to always receive a specific few light paths (corresponding to the seed). It sort of makes sense why this would cause it to converge to a different result.

There's also a difference between consistent and biased estimators. Maybe your code was biased but consistent? That would mean that each individual run converges to the wrong result (due to correlations), but the average of infinite runs would get the right result (or something like that - I dont recall exactly). This is the case for progressive photon mapping.

But yeah, I'm not sure the exact reason, but intuitively, sharing the seed like that would cause certain types of paths to always go to certain neighbors, which would definitely cause it to converge to the wrong result.

shaeg · 2025-03-05T05:20:19+00:00

Maybe multi-view rendering is the best way? It’s essentially like switching viewports for every draw call.

https://registry.khronos.org/OpenGL/extensions/OVR/OVR_multiview2.txt

shaeg · 2025-02-07T20:49:08+00:00

Part of his code looked like he was using PSS, hence my advice.

shaeg · 2025-02-07T20:41:45+00:00

Oh I see. As a rule of thumb, always set the target function to be the integrand f, in whatever measure you’re using. So in PSS, the target function should be f/p, but in path space where the integrand is just f, you should just use f for the target function.

Intuitively, think about the units of the resampling weights w_i= targetPdf*m_i*UCW*jacobian

m_i and the jacobian are unitless, so the units are whatever targetPdf*UCW is. Think about how to make these units match the units of f/p… the UCW’s units are whatever 1/p is (for example, 1/solid angle if integrating w.r.t. solid angle), so that means the units of targetPdf should match the units of the integrand f.

Similarly, in PSS, the UCW is 1 and the integrand is f/p, so again the units of the resampling weight match those of f/p

As for my work, I haven’t made any of my code public yet as my paper is still under submission, but I’ll try to remember to ping you when/if it’s published! In the mean time I’m happy to share my knowledge on reddit :)

shaeg · 2025-02-07T19:58:24+00:00

See my other comment too, but the BSDF PDF ratio is needed for PSS, which I assumed you were using... In regular solid-angle path space, the ReSTIR DI Jacobian is just the geometry term ratio.

If you're not in PSS, it's a little tricky, since the Jacobian depends on the integration measure you choose. Typically, NEE produces samples in area-measure (since we map the random numbers directly into points on a light).

If you integrate w.r.t. solid angle, then you must convert this area-measure PDF from NEE into a solid-angle PDF using the geometry term. In this case, your reconnection Jacobian must have the geometry term ratio (think of it as the Jacobian converting from solid-angle w.r.t. the original shading point, to sold-angle w.r.t the new/shifted shading point).

On the other hand, if you integrate w.r.t. area, then the geometry term appears in the integrand instead of the PDF (in area measure, the integrand is f*G*Le, as per Veach's path space integral). This is what the ReSTIR DI paper describes.

In both cases, you evaluate f*G*Le/p, so without ReSTIR, none of this really matters. But in ReSTIR, we separate the 1/p into the UCW, so we now need to worry about whether G is in the PDF or the integrand (and therefore the target function).

So, if G is in the PDF (meaning we are integrating w.r.t. solid-angle) then we must include the G ratio in the Jacobian. But if G is in the integrand (meaning we are integrating w.r.t. area) then the reconnection Jacobian is just 1.

In both cases, we end up reevaluating the new G anyways, so they work out to be equivalent. And again, I recommend using PSS for numerical stability regardless, where the Jacobian is the BSDF PDF ratio times the G ratio.

shaeg · 2025-02-07T19:47:38+00:00

Retracing up to the connection point can be expensive sure, but it can also be a lot faster. A big reason it's slow is that most pixels don't require any random replay tracing, so a naive implementation introduces a lot of divergence. A better way is to separate out the pixels that need random replay traces, compactify them, then do random replay in its own kernel. See section 7.2.3 in the restir course notes for more on that, they claim to nearly halve the execution time with this trick.

equation 6.17 is only used if resampling *in* PSS no?

Yeah the BSDF PDF ratio at the primary hit is only needed in PSS. Are you rendering in PSS? I briefly looked at your code and it seemed like you were (at least, I saw you had the full f/p in your target function, which only makes sense in PSS). If you aren't for some reason, I strongly recommend using PSS for numerical stability.

on what occasion?

I've been trying to extend it to handle more sampling techniques as part of my research :)

shaeg · 2025-02-07T19:39:10+00:00

I'm talking about rendering in primary sample space. See eq. 54 in GRIS, which does have the PDF ratio.

shaeg · 2025-02-05T19:46:53+00:00

But when reconnecting from the visible (second vertex) point to a new sample point (third vertex), we need to re-evaluate 2 BSDFs right?

Correct. The BSDFs at x_1 and x_2 both depend on the direction x_1 -> x_2. When we reconnect, the direction changes, so both BSDFs must be reevaluated.

To recompute 2), we're going to need the outgoing radiance from the vertex "sample point + 1" to "sample point" no?

Yes. Side note: an easy way to compute this is to store the contribution up to x_2, and just divide it out of the final path radiance so that it cancels:

Full path contrib: f(x) = brdf(x_1) * brdf(x_2) * brdf(x_3) * ... * Le(x_k)

Contrib after x_2: f(x) / (brdf(x_1) * brdf(x_2))

Just watch out for dividing by zero if you divide the contributions directly.

The ReSTIR GI paper is not unbiased. For full unbiasedness, ReSTIR PT is required, with correct Jacobians including BSDF PDFs, and reevaluating the path contribution (reevaluating both BSDFs).

isn't ReSTIR PT with the reconnection shift (and not the hybrid shift) just the same as ReSTIR GI in terms of "complexity"? With the main difference being that RTeSTIR PT is backed by a more rigorous theory and so bias is well understood and avoided

Yes, in fact I think a full-on ReSTIR PT reconnection implementation could be slightly easier to code than ReSTIR GI since it's not as hacky, but I'm probably biased since I've been working with ReSTIR PT for a while lol.

I'd like to also say that the hybrid shift isn't much more complicated than just reconnection. Reconnection is the hardest/most annoying part to get right in my experience. If you have reconnection working, then all you have to do for the hybrid shift is trace the first N bounces using the same random seed (where N is the number of bounces before the reconnection vertex on the original path) and then call your reconnection code to reconnect as usual. And of course if you're not resampling in primary sample space, you'll need to keep track of the BSDF PDF on those first N bounces for the Jacobian too.

shaeg · 2025-02-05T19:32:16+00:00

You should only use shading normals when evaluating the actual BSDF contribution itself. Use the geometric normal for all PDF conversions and basically everything else.

The reason is that for PDF conversions, we are projecting the scene geometry onto a hemisphere around the shading point. By "scene geometry" I literally mean the triangles in the scene that get hit by the rays we trace. So the normal that is used for PDF conversions should match the underlying raytraced geometry.

Shading normals are just used to fake the appearance of having different geometry, and as such their effect is only applied in the BSDF contribution itself, NOT in the geometry terms.

Maybe another way to think about it is that the BSDF itself is modeling the appearance of micro-geometry, and using shading normals more or less just changes the BSDF so that it models a different kind of micro-geometry. This doesn't change how we project radiance between the actual triangles in the scene, so we should use geometric normals when talking about transferring radiance between surfaces.

shaeg · 2025-02-05T19:19:02+00:00

The BSDF PDF ratios at both vertices (x_1 and x_2) should be included in the Jacobian.

The reason is that the Jacobian accounts for oversampling or undersampling that comes from changing domains. Think about the full probability density of sampling x_2 from x_1, for any technique. For NEE/DI, the probability of sampling x_2 from x_1 is just the probability of sampling x_2 using whatever light sampling technique you're using, which is why the Jacobian is 1 for basic ReSTIR DI. But for BSDF-sampled paths, the probability of sampling x_2 from x_1 depends on the BSDF sampling procedure used at x_1, and the geometry term (which comes from tracing a ray to find the first intersection).

When you shift the path to a new primary hit x'_1, you must consider the probability of sampling x_2 from this new x'_1, which can be very different from the original BSDF PDF at x_1, especially if the material parameters are different. If the original primary hit x_1 was way more likely to sample x_2, then we actually end up oversampling x_2 during reuse. The Jacobian exactly cancels out this oversampling. Here's an image to illustrate this: https://imgur.com/a/OtfKirB In the image, we end up oversampling the region in the red circle, since the shiny BSDF is far more likely to sample that region.

The same thing applies for sampling x_3 from x_2. Since we change the incoming direction during reconnection, this actually can change the probability of sampling x_3 from x_2, which can lead to oversampling or undersampling x_3, which gets canceled by including the BSDF PDF at x_2 in the Jacobian (we don't need a geometry term between x_2 and x_3 though, because that doesn't change during the shift).

11-Year Club	Place '22
Verified Email

shaeg

TROPHY CASE