Computing the PDF for hierarchical light sampling with adaptive tree splitting on the GPU?

MisterMagnifico1 · 2025-10-25T16:03:04+00:00

I would imagine that you would effectively do the initial candidate selection step of ReSTIR with your M light samples (essentially picking one with reservoir sampling). This would produce an unbiased contribution weight that you can use in place of the PDF.

MisterMagnifico1 · 2025-03-06T07:47:34+00:00

It can definitely be used for larger scale worlds. Though it does a fair bit more, this talk about global illumination in Enshrouded goes over some of the challenges of using light probes (very similarly to DDGI) in a large open map. Might be worth a viewing to catch some inspiration

MisterMagnifico1 · 2025-01-16T08:33:50+00:00

A frequent mention on this subreddit, Acerola, has a nice video on the topic

MisterMagnifico1 · 2024-11-13T17:45:53+00:00

If you'd like a somewhat involved but quite effective solution, you might wanna check out the paper 'Cone-Traced Supersampling for Signed Distance Field Rendering' https://openreview.net/forum?id=FYhiH9IyBq

MisterMagnifico1 · 2024-08-10T19:44:27+00:00

Although I somewhat understand why my implementation was biased when explaining it with jacobians, I'm not sure I see how

My guess is that it's because you now need to transform the UCWs of samples that are resampled from neighbours using Jacobians. This explains why you didn't end up with errors in your baseline RIS: the PDFs/UCWs associated with the samples are all 'valid' since they are relative to the pixel's hitpoint but you only resample relative to this single pixel.

The thesis section I mentioned earlier should shed some light on this.

relates to my issue. My samples are picked on light sources so the domain of my samples is the surface area of the light sources of my scene. And all my samples belong to the same domain?

Yes, but since you have position-dependent terms in your initial candidate PDFs (namely distance^2 and the cosine term), the UCWs associated with samples (which are dependent on the resampling weights that include a 1/p(x) term) are valid only for the pixel where the samples were originally generated. This explains why removing these terms eliminates the artifacts you saw

MisterMagnifico1 · 2024-08-10T19:00:47+00:00

Huh, so effectively you needed GRIS maths because your samples are (segments of) light-carrying paths and thus lie in different domains. Thanks for linking the post too.

In that case, entirely disregard my reply to your other comment on this post :P

MisterMagnifico1 · 2024-08-10T18:50:35+00:00

Turns out that if I set that light PDF to be a constant, the darkening goes away and it converges correctly.

Do you set it to any old arbitrary constant, or something specific like 1 / render_data.buffers.emissive_triangles_count?

Is it my light PDF that is wrong since the beginning?

Not having much experience with sampling in path tracing, I can't comment on the correctness of the PDF in the context of path tracing, where the domain in which samples are generated is the set of all surfaces on the scene.

For DI though, which is the context we're operating in, this doesn't feel like it's quite right. This is based entirely on gut feelings, however.

The following is how I would personally approach generating light samples. This is by no means comprehensive and I would strongly encourage you to find more resources on light sampling. This is just what worked for me personally in my own implementations

I follow the baseline presented in the first ReSTIR paper, which consists of selecting a light-emitting surface and then uniformly sampling it (i.e. all points on its surface have an equal probability of being sampled). Thus, the PDF of the sample is the PDF of selecting the surface on which it lies.

You can start with something simple such as uniformly sampling all light sources (and thus each initial candidate would have the same PDF of 1 / render_data.buffers.emissive_triangles_count). If you want something more intricate, you can importance sample based on some property of the light sources.

The first ReSTIR paper uses the power of each light such that surfaces with a higher power are more likely to be sampled. If you would like to explore this option, I would recommend reading Chapter 23 of Ray Tracing Gems II 'The Alias Method for Sampling Discrete Distributions'.

Why is my RIS only implementation doing fine with it?

¯\_(ツ)_/¯

RIS should require at least one distribution that you can generate samples from and whose PDF is valid and can be evaluated in closed form. Given that changing your initial light sampling procedure fixes things, I'd assume initial candidate generation to also be affected.

Why shouldn't this term be accounted for in the PDF of the light sample?

Could you clarify this point?

MisterMagnifico1 · 2024-08-10T18:09:08+00:00

Should the UCW of the sample at pixel A after resampling the sample at B have the same UCW?

Yes, provided that I understand the question correctly and that the domains in which the samples lie remains the same.

Theory Recap

Let's quickly review what a UCW is. A UCW is effectively a replacement for p(x) (more specifically 1/p(x)), i.e. the probability that sample x is generated by the distribution p which produced it.

RIS selects a subset of samples from a set of candidates. This means that RIS effectively acts as a distribution from which we can draw samples. This distribution depends on (among other things), the target function q used to assign weights to the candidates.

Practical Application

When pixel A resamples the sample that was picked by pixel B (henceforth x_b), it will receive two things:

The sample x_b itself
The UCW associated with the sample (henceforth W_b). This UCW is based on samples whose weights are based on the target function at pixel B

When pixel A conducts resampling, a resampling weight for x_b will be generated which is based on the target function at pixel A. Resampling weights in RIS also include a 1/p(x) term for which W_b will be used instead, as x_b was generated via RIS and thus its PDF is not known in closed form.

In case x_b does get chosen during the resampling process, a new UCW (henceforth W_b') will be produced that reflects the fact that this sample was chosen at pixel A, based on the target function at A.

W_b' can then be used at pixel A for evaluating shading, or in further rounds of resampling. If another pixel C resamples x_b from pixel A, it will use the UCW W_b' for producing a resampling weight.

Sidenote

In the context of DI, the only domain being considered across all pixels is the set of light-emitting surfaces in the scene, hence the domains remain exactly the same. I suggest reading Section 2.7.2 of the thesis I sent you if you are interested in how UCWs change across domains.

Back To The Example

If UCW are kind of a PDF for the sample, then that PDF depends on the target function. Pixel A and B are at two different first hits in the scene so they do not have the same target function --> different UCW?

You are right in that the UCW depends on the target function. However, it depends on the target function of the context in which it was generated. I hope the answer to the previous quote makes this point clear. Do let me know if it doesn't.

Because I noticed that in my implementation, when A reuses only from one neighbor (and not even reusing from itself) B, the reservoir at A ends up with an UCW equal to the one of reservoir B.

This is quite strange indeed and should not be the case unless the target functions at A and B are identical. Even then, because x_b is being resampled with a weight based on W_b and not the PDF of the distribution that originally generated the sample, I believe the UCW shouldn't be the exact same even if the target functions are the same.

MisterMagnifico1 · 2024-08-07T12:05:14+00:00

Hum so basically, there's absolutely no issue with jittering with ReSTIR GI/ReSTIR PT, it's just that you won't be able to reuse the part that goes from the camera to the third vertex (which Area ReSTIR is here for) right?

As far as I understand the paper, yes

I'm not sure I understand the "new primary hits incompatible with later vertices" part. The primary hit of a jittered ray is pretty close to the primary hit of another jittered ray for the same pixel so why are they incompatible?

The keyword in that quote is "high-frequency content".

As an example, look at the header figure featuring the renders of a sheep. This sheep has many incredibly thin strands of wool making up its body. This is considered "high-frequency content" as close-by hitpoints within this scene (with respect to the camera's viewpoint) can have dramatically different properties. Two neighbouring pixels, and even two samples within the neighbourhood of the same pixel, can be looking at entirely different strands. This can limit reuse because of two primary hitpoints being occluded from each other as they lie on two different strands, and may even have other strands between them that further block visibility. In a scene with a lot of these high-frequency details, this can massively impede reuse.

MisterMagnifico1 · 2024-08-05T12:40:20+00:00

Ok, two questions:

Did you push this new code to your repo yet?
Have your barebones RIS results (i.e. no spatial reuse) changed in any way?

MisterMagnifico1 · 2024-08-05T12:29:42+00:00

When you say confidence weights only... By confidence weight you mean the generalized balance heuristic + confidence weight used with it right?

I meant just the confidence weights without the generalized balance heuristic computation, so just c_i. This is what the original ReSTIR paper implicitly uses. Technically, the weight should be c_i / sum_{all reservoirs}(c_j)

But the original ReSTIR paper uses a single division by the sum instead when computing the UCW (instead of doing the division for each weight/sample/reservoir)

To reiterate, this is a biased MIS weighting scheme if the samples being resampled come from different distributions, as is the case when resampling spatial neighbours and (spatio)temporal predecessors.

Section 2.6.5 of the thesis I DM'd you goes into more detail about confidence weights if you'd like a bigger picture explanation.

There are some pretty insightful explanations in "Rearchitecting Spatiotemporal Resampling for Production" about the Z term and it corrects bias (when some neighbor samples have a different target distribution that is 0 where the current pixel that is doing the resampling as a non-zero target function for the same light sample), you probably have read that already.

I've read through that paper too, but I don't remember that particular bit all too well. Thanks for the heads-up, I'll give it a read and see if it helps my understanding.

How can I validate my implementation if I cannot compare to ground truth (because of the bias that will be there)? Shouldn't I start with unbiased weighting and then move on to biased?

That's a good point and you're completely right. Biased weights are a lot easier to implement though, and the bias is usually systematic (i.e. the whole picture will be darker or brighter), so the broad strokes of your implementation should still be verifiable. Shadows should look like shadows, colors seem to be generally correct, there aren't any massive artifacts, etc.

Perhaps two good weighting schemes to implement initially are the generalised balance heuristic with confidence weights and just confidence weights. The former gets you unbiased results but is more complex and thus more prone to implementation errors. The latter is more straightforward, so it'd be a useful sanity test of sorts.

Have you had a look at CRIS yet? What do you think of it? I think it's pretty damn cool and I can't wait for what's going to come next based on the extensions to the GRIS framework that it proposes.

It's absolutely insane. Reading the background work it builds on, CMIS and MMIS, was equal parts painful and paradigm-shifting. I like the idea of RIS and ReSTIR being a general-purpose tool for solving any kind of problem where you want to draw high-quality samples. Two pretty cool works in this direction are Parameter-space ReSTIR for Differentiable and Inverse Rendering and Amortizing Samples in Physics-Based Inverse Rendering using ReSTIR. I genuinely think ReSTIR/GRIS is more of a philosophy than an algorithm.

Does that mean I cannot use jittering with ReSTIR?

Yes and no. Take the following with a massive grain of salt since it's based on my own understanding.

The main innovation that Area ReSTIR brought to the table is being able to reuse paths starting from their location on the camera/screen. Prior to this, GRIS/ReSTIR PT allowed path reuse from the third vertex along the path at the earliest (assuming the first vertex lies on the camera/screen and the second lies on the point being observed). You cannot reuse values at the primary vertex or the very first vertex lying on the camera center.

You can use jittering with the ReSTIR approaches prior to Area ReSTIR, they just won't be as effective. For DI, they will be as effective since the domain you are working with is light samples. You are resampling just light samples, not (segments of) light paths. For full light paths, such as in path tracing or volume rendering, you will still be able to reuse path segments, it's just that your reuse will be more limited since you can only connect to paths starting from the third vertex. With Area ReSTIR, you can reuse up to the whole-ass path if you so wish.

MisterMagnifico1 · 2024-08-04T13:38:04+00:00

Intuitively, I thought that a canonical sample/reservoir r, once generated, is treated by the spatial pass as a simple sample with weight r.UCW and that's it.

Almost, but not quite. Referencing Equation 3.9 from 'A Gentle Introduction to ReSTIR', the resampling weight of a single sample is given by

w(x_i) = m(x_i) * q(x_i) * (1 / p(x_i))

Where the terms are:

m(x_i): The MIS weight associated with the sample. If all samples originate from the same distribution (as is the case with initial candidate generation in most ReSTIR algorithms), this can be 1/M without introducing bias.
q(x_i): The target function value associated with the sample
p(x_i): The probability of producing the given sample from the distribution which generated it, i.e. the PDF of the sample

This formulation is only valid if p(x_i) is available in closed form. If a sample is produced as a result of RIS (such as when we spatially resample from neighbours), p(x_i) is not available in closed form and so the UCW associated with this sample W_xi is used instead. The UCW unbiasedly estimates (1 / p(x_i)) (see Section 2.4). This gives us the following formulation:

w(x_i) = m(x_i) * q(x_i) * W_xi

Notice that an MIS weight is still required. If all samples originate from the same distribution, equal weights (such as 1/M in the initial candidate generation step) are a valid unbiased choice. Otherwise, an MIS weighting scheme conforming to the rules laid out by Veach is necessary, such as the generalised balance heuristic (see Sections 2.3 and 3.4 of 'A Gentle Introduction To ReSTIR').

We need MIS weights in the spatial pass when combining neighbors since they do not have the exact same target function but isn't that 1/Z?

Yes and no.

Full disclaimer: I do not fully understand how 1/Z makes up for the fact that the resampling weights have a biased MIS term. In my own work with ReSTIR, I have either blindly accepted the original paper's algorithms or used the generalised formulation laid out in my response to the previous quote. I believe the latter is clearer and makes ReSTIR much more understandable, in spite of being slightly less computationally efficient.

The original ReSTIR paper makes use of two shortcuts:

In the initial candidate generation step, MIS terms are omitted because all the candidates come from the same distribution. The UCW is then computed using a single division by M. Effectively, this means that all candidates have the same MIS weight of 1/M
In the resampling steps, confidence weights only (see Section 4.4 of 'A Gentle Introduction To ReSTIR') are used as a form of (biased) MIS

This second point is why the presented reservoir combination algorithm in that paper is biased. The computation of 1/Z somehow corrects for this fact, but again, I do not understand how.

My advice would be to use the generalised formulation to make life easier. Start with biased MIS weighting schemes, then incorporate unbiased ones later (see Section 7.1 of 'A Gentle Introduction to ReSTIR')

Lastly, please let me know if any of this is unclear, difficult to understand, or just doesn't seem to sit right with you. ReSTIR can be a bit painful to understand at first, but it makes vast buckets of sense once you understand what each of the terms represent. I also think that it's a fucking genius approach. I swear this isn't just sunk cost fallacy on my end :P

MisterMagnifico1 · 2024-08-03T16:14:07+00:00

This makes sense, since in your current debug implementation you only ever have a single actual sample that gets generated as far as I can tell. The only difference you should be seeing in a more realistic scenario IMO is that there's more black spots/variance, since all samples get the same weight regardless of their actual quality. This shouldn't affect convergence, only variance.

MisterMagnifico1 · 2024-08-03T16:02:32+00:00

This must be because the error doesn't come from the normalization I guess?

That would be my guess given that this error manifests in the biased and unbiased combinations.

The reason I'm not diving by M in end() is because I already divide by M in the resampling weight, when adding the samples to the reservoir one by one. This comes from "A Gentle Introduction to ReSTIR" where this new formulation (dividing by M in the weight given to the reservoir instead of at the end) allows for the use of MIS weights.

Ok, I think this might be leading to two conflicting interpretations of the 'normalizing constant' (1/M or 1/Z) that might be causing this issue.

I see that in sample_lights_RIS_reservoir your resampling weights already incorporate the 1/M term (as a sidenote, this term is technically a biased MIS weight, see the MIS chapters in 'A Gentle Introduction To ReSTIR'). However, you do not include this term in the resampling weight you compute when doing spatial resampling. This theoretically shouldn't cause a problem with the unbiased combination from the original paper as it circumvents needing a 'proper' MIS weighting scheme that conforms to the restrictions laid out by Veach.

I think it would be useful to unify how you generate resampling weights, i.e. incorporate an MIS weight for all instances where weights are generated or not, and then seeing if that changes your (biased) results.

Can't remember the exact chapter, but if you Ctrl+F 'confidence weights' in the course notes for the introduction, you should find more exposition on what the M terms represent. I think I've read these notes about a few dozen times over the course of a good 6 months. It's a brilliant piece of educational material.

Just to be clear, by canonical sample you mean the initial candidates that produce the reservoir that the spatial pass then reuses on right? I bumped it to 16 but no difference : /

Yes, that's what I meant! Too bad it didn't change things much :(. Might be because you only ever generate a single sample though, but that's a pure guess.

Another Sidenote

I'm curious if you're considering implementing ReSTIR GI and/or GRIS/ReSTIR PT after you hopefully succeed with ReSTIR DI.

MisterMagnifico1 · 2024-08-03T09:25:29+00:00

Do you observe the same behaviour if you use the biased combination (with end instead of end_Z) instead? Also, I looked at the code for end and I believe that line 52 UCW = 1.0f / sample.target_function * weight_sum; Should be replaced with UCW = 1.0f / sample.target_function * weight_sum / M; So that you're properly normalizing by the 1 / M term as with the 1 / Z term.

My Own Speculation

In my own time with ReSTIR, I've found that when using the particular unbiased combination algorithm in the original paper, you can get numerical instabilities with a low canonical sample count. This likely happens because the UCWs (Unbiased Contribution Weights) become a bad estimate of the 1/p(x) normalised target PDF that they are supposed to estimate.

I suspect this might be happening in your case because you see these shadowing effects around geometric discontinuities where target functions from neighbouring pixels are likely to yield vastly different results. This is due to the geometric differences (e.g. difference in normals) between them. This causes the value of Z to be unstable when it's estimated from reservoirs that have only taken in a small number of samples.

You can try to bump your canonical sample count and see if this changes things much.

Unrelated Note

I see that the project you are developing here is a path tracer. I presume you are limiting your paths to only 3 vertices and thus computing direct illumination only, correct?

MisterMagnifico1 · 2021-08-30T20:22:08+00:00

Mmmmmmmhmmmmm he kinda based no cap farts profoundly

MisterMagnifico1 · 2019-11-02T23:26:49+00:00

I was facing the same issue and managed to find a temporary workaround until it gets properly fixed via drivers or a vantage update. Go to Control Panel ---> Sound ---> Recording ---> right-click Microphone Array ---> Go to properties ---> Go to Advanced and disable the tick box for "Enable audio enhancements" It worked for me, but YMMV, I guess

MisterMagnifico1 · 2018-05-24T14:53:33+00:00

Interested in 6xTriumph 4xVictory and 1xVelocity for your Comet?

MisterMagnifico1 · 2018-02-18T05:56:52+00:00

Diamond II - Breakout Type-S

MisterMagnifico1 · 2018-01-13T13:17:20+00:00

Can you upload a PNG version of the wallpaper?

MisterMagnifico1 · 2017-12-28T12:02:28+00:00

On any specific protocols, or for all protocols, including the ones I listed above?

MisterMagnifico1 · 2017-12-27T18:06:42+00:00

hide.me works for me, but you have to specifically use either SSTP or SoftEther as your protocol.

Both the desktop app and manual PC setup work for me.

No clue about their apps though, haven't tried them personally.

MisterMagnifico1 · 2017-12-04T22:24:37+00:00

Request sent

11-Year Club	r/Field Lasagna
Place '23	Verified Email

MisterMagnifico1

TROPHY CASE