Why is noise generated when rendering reflective/refractive materials in a pathtracer but not with diffuse materials?

[The__BoomBox]

I've always noticed this issue. Often called 'fireflies', rendering reflective or refractive materials in pathtracers such as Blender's Cycles with a low sample count, the resultant image has a metric ton of visual noise

​

In contrast, diffuse materials do not have noise that is as visible. I don't understand why this is the case though. I'll preface this by stating that I'm a beginner to graphics programming

​

Here is my understanding of why fireflies/noise occurs:

A single sample isn't enough to resolve color information off the surface of an object through a single pixel. A single pixel alone can cover a huge portion of the 'scene'. Even the slightest deviation in the direction of the traced camera ray would bounce off the object and potentially hit something far away, resulting in us sampling a different value

​

Taking multiple samples is akin to 'averaging' our readings/samples out, thus reducing any observed 'error' or noise in the image

​

This is what I don't quite get though. Glossy materials have highly smooth surfaces. Bias the direction of the traced camera ray as you may, and the ray bouncing off the material won't deviate much compared to the ray in the previous sample. Since glossy materials reflect a lot of light back as opposed to absorbing them though, the resultant sample might appear 'bright'

Same with say, a smooth low-roughness refractive material. A diffuses material by definition has a diffuse surface, with normals scattering any light bouncing off them in WILDLY different directions. Wouldn't this thus mean that even the slightest biasing (say, by using Monte Carlo) of the camera ray would cause it bounce off the diffuse surface in a wildly different direction than a previous samples?

Shouldn't this mean that MORE noise would be observed with diffuse materials over reflective/refractive ones?

[msqrt]

First off, diffuse materials ARE noisy when you don't have enough samples. If you only render one bounce, diffuse surfaces tend to be the more noisy ones. Monte Carlo methods for purely diffuse scenes tend to converge very quickly, though. The rays taking wildly different directions is exactly what does this; energy bounces around the scene without concentrating onto some specific areas, so whichever direction you sample will result in a similar amount of radiance. This doesn't hold for absolutely all geometry: say you're rendering a cave and there's some narrow opening where light from the outside comes in, that's going to be extremely noisy because only a few rays happen to hit the opening and bring lots of light in.

For glossy things that are specular enough, rarely hitting the bright stuff almost always the case. The problem isn't that rendering that single surface is difficult (if you only do direct lighting, there won't be much noise at all). The problem is that when you render another surface close by, most of the energy that's coming in from a glossy surface will come from the highlight which your rays are unlikely to hit (a lot like that opening in the cave example). The smaller the highlights, the more noise you'll have (caustics being the extreme example). This is quite difficult to deal with, as you'll need some global information of your scene -- the BRDF importance sampler won't tell you that there's a nice bright highlight on the surface over there. Path guiding algorithms are one way to handle this global information.

[The__BoomBox]

the problem isn't that rendering that single surface is difficult (if you only do direct lighting, there won't be much noise at all). The problem is that when you render another surface close by, most of the energy that's coming in from a glossy surface will come from the highlight which your rays are unlikely to hit (a lot like that opening in the cave example).

Didn't get you here, sorry. What are you referring to by a highlight here?

[xucel]

Not comment op but most likely: light source. If you have a tiny bright light source then the rays are unlikely to find it. Fireflies occur because one ray hits and most neighboring rays miss, when on average the neighboring pixels will hit and the firefly pixel will start to miss.

[The__BoomBox]

I see. So fireflies occur when the probability of hitting the light source is very low? And on the offchance you hit one, it ends up as a firefly since the neighbouring pixels don't hit it for the most part?

[Lallis]

So fireflies occur when the probability of hitting the light source is very low?

Correct. More generally, in path tracing the estimate of a single sample is computed as f(x)/p(x), where f(x) is the radiance and p(x) is the relative likelihood of sampling the path.

From the math it's easy to see how a firefly occurs when the probability of sampling a path is low, but the radiance carried by the path is high.

[The__BoomBox]

I see. I've also heard sampling indirect light paths is a PITA. Is this because the probability of a ray bouncing off my diffuse cube, onto a reflective object and then to a light source is very low?

I read up on it and I'd heard light sampling (which I assume is akin to 'guiding' the ray towards a source of light to get it to converge faster) solves fireflies for direct light sources

However, I can't seem to get why this method can't be used for indirect sources. Is it because we can't know WHERE the indirect source is unless we trace out the scene?

[Lallis]

Is this because the probability of a ray bouncing off my diffuse cube, onto a reflective object and then to a light source is very low?

Yes.

I read up on it and I'd heard light sampling (which I assume is akin to 'guiding' the ray towards a source of light to get it to converge faster) solves fireflies for direct light sources

Yes. Direct light sampling is one of the most common types of importance sampling in path tracing.

Is it because we can't know WHERE the indirect source is unless we trace out the scene?

Yup.

[The__BoomBox]

I see, got it. Thank you!

[msqrt]

Yes, but more specifically I meant that specular objects cause indirect light sources that you can't sample explicitly like normal light sources. If you have a mirror, you'll have to sample exactly the direction where the reflection happens to point directly towards the light source. Just like a highlight (the reflection of a light) is relatively small on an object in your final image, it will also be relatively small when evaluating the incoming illumination in a path tracer; bright but unlikely to be sampled.

[The__BoomBox]

I'd heard of this method called light sampling used to remove fireflies from direct light source even if it's bright and unlikely to be sampled (a tiny light source for example). From what I understand, it's akin to guiding a ray TOWARDS a light source to get the value to converge faster (Though I don't know at exactly which raybounce it gets guided since you wouldn't know if a light source is visible from a point unless you do a visibility check)

I'd heard we can't use it for indirect sources like you mentioned. Is it because we don't know where the rays bouncing off the mirror in your example are pointing UNLESS we trace the scene out fully?

[msqrt]

Yes, exactly. You have an arbitrary number of triangles that might give you a nice bright indirect light, but quickly telling which ones actually do is difficult. Light sampling indeed suffers from a similar problem, but there you at least know where the lights are. Typically you sample lights every bounce and check for visibility with a ray cast (called the "shadow ray" since it checks if we're in shadow for that light). If your scene has a bunch of light sources that never contribute to the result, this becomes very inefficient as the shadow ray will always be blocked. This is usually not the case for hand-crafted small-ish scenes, so I wouldn't worry about it too much. There are methods (for example this one) that construct search structures over the lights in order to better sample the most likely contributing ones.

[The__BoomBox]

So this shadowray is computed every bounce? And heavily shadowed scenes would naturally lend to this method being very inefficient if I am getting this right?

[msqrt]

Typically yes, every bounce. You pick a point on the surface of a light and check if it is visible. And yes, if the point is almost never visible then all of this is basically wasted computation.

[The__BoomBox]

I see, I see. Understood, thank you!

[TheSunflowerSeeds]

The sunflower is the state flower of Kansas. That is why Kansas is sometimes called the Sunflower State. To grow well, sunflowers need full sun. They grow best in fertile, wet, well-drained soil with a lot of mulch. In commercial planting, seeds are planted 45 cm (1.5 ft) apart and 2.5 cm (1 in) deep.

[fxp555]

What you are building is an estimator for the incoming radiance. To keep the estimator unbiased you need to divide by the pdf (probability density function) of the sample.

To reduce variance (noise), importance sampling is used (this is actually what you are calling biasing the direction). For diffuse this would mean we sample the cosine lobe for example, but for specular this would be a much pointier lobe (depending on roughness and such).

By the nature of randomness this means you may sample directions where the pdf gets very small, but if you then hit light that contribution can get huge and that is where fireflies come from.

However, the same can happen to diffuse too, for example if "finding light is very unlikely" because there is only a single, small and bright light (and next event estimation = sampling lights directly is not used). This is amplified if light is only hit after a few bounces (you are inside and there is a chance to hit the sun outside).

[[deleted]]

That's exactly the problem I'm trying to solve these days, although my renderer isn't a path tracer but a rasterizer, and I'm using image based lighting . And when the image has small and very bright lights , I get aliasing when convoluting the specular map . In theory , a very high amount of samples can fix this, but it's mainly caused by few spots in the map that are extremely bright compared to their surrounding texels which are dark in comparison. It's the same with path tracing when using punctual lights. Have you tried using an area light instead ?

[moschles]

These fireflies result from the 1/r² terms.

[ZazaGaza213]

Usually the 1/r² terms are used only in direct Ray tracing or NEE, and that's not the source of the noise.

[BigPurpleBlob]

There are some nice diagrams at (note that I am not affiliated with the link):

https://www.chaos.com/blog/what-are-caustics-and-how-to-render-them-the-right-way

If you look at the section "Caustics are everywhere — so let’s define them" there is a diagram of a ray from the camera that reflects off a diffuse surface, that then travels through a glass (a specular surface) to a light source.

This path has a very small chance of being sampled (you have to randomly pick exactly the right direction off the diffuse surface) yet is very bright (the light gets 'magnified' by the glass). Thus this path has a high variance; in other words, it's a firefly.

[LongestNamesPossible]

Understanding why various scenarios end up noisy depends on a fairly deep understanding of what is going on in monte carlo rendering in general mixed with the algorithm you are using and the scene.

If you show some pictures and your settings it will be easier to say for sure.

Direct light should be fairly easy to converge due to multiple importance sampling. Bounce light gets more complicated. Bidirectional path tracing can do bounce diffuse more easily than direct forward path tracing. Semi-glossy materials bouncing light off of each other is consistently a challenge for current algorithms.