Results vary vs sample count while they should not

[leroyvn]

When rendering a simple scene with the path tracer, my results vary with the number of samples. The scene is composed of a square surface with the default diffuse BSDF illuminated by an environment emitter (constant or directional) (see below a render at 32 SPP with the constant emitter).

The code I used is on this gist. I'm using a perspective camera with a single pixel and a very small field of view to make sure that I only see a small part of the square and no background. With both emitters, I vary the number of SPP by powers of 10 up to 1e8. The theoretical solution is rho (the surface's reflectance) for the constant emitter case and rho / pi for the directional emitter case. Results are as follows:

emitter      spp    li
------------ ------ -----------
constant     1e+00  0.256591797
constant     1e+01  0.50390625
constant     1e+02  0.484130859
constant     1e+03  0.492675781
constant     1e+04  0.499267578
constant     1e+05  0.5
constant     1e+06  0.500488281
constant     1e+07  0.505371094
constant     1e+08  1.0
------------ ------ -----------
directional  1e+00  0.159179688
directional  1e+01  0.159179688
directional  1e+02  0.159179688
directional  1e+03  0.159179688
directional  1e+04  0.159179688
directional  1e+05  0.159179688
directional  1e+06  0.157592773
directional  1e+07  0.153076172
directional  1e+08  0.25

Or, as a plot:

Depending on the number of samples taken, the incoming radiance at the sensor varies. I'm surprised to see something like that since the value returned by either emitter is constant. I'm actually also surprised to see that the results plateau with good accuracy until 100.000 samples for the directional case and do not for the constant case. Discrepancies become high when reaching 100 million SPP.

Any idea of what could be the cause of that?

System configuration

Platform: macOS 10.15.4
Compiler: Apple clang version 11.0.0 (clang-1100.0.33.17)
Python version: Python 3.7.5 :: Anaconda, Inc.
Mitsuba commit hash: bad308d
Variant used: scalar_rgb

[wjakob]

Just a quick thought: accumulating many samples and then dividing by the number will be numerically imprecise in single precision for such large sample counts. Have you tried a double precision build?

[leroyvn]

Not yet, I'll do it!

[leroyvn]

Using the scalar_rgb_double variant yields the following results:

emitter      spp    li
------------ ------ -----------
constant     1e+00  0.256591797
constant     1e+01  0.50390625
constant     1e+02  0.484130859
constant     1e+03  0.492675781
constant     1e+04  0.499267578
constant     1e+05  0.5
constant     1e+06  0.5
constant     1e+07  0.5
constant     1e+08  0.5
------------ ------ -----------
directional  1e+00  0.159179688
directional  1e+01  0.159179688
directional  1e+02  0.159179688
directional  1e+03  0.159179688
directional  1e+04  0.159179688
directional  1e+05  0.159179688
directional  1e+06  0.159179688
directional  1e+07  0.159179688
directional  1e+08  0.159179688

It looks more normal to me (totally normal actually for the directional sensor) but I still need 100k+ samples to get really close to the theoretical value with the constantsensor.

[wjakob]

Great — these new numbers don’t strike me as unusual. It’s standard convergence behavior in a Monte Carlo context where we start out with a given amount of error for 1 sample and then roughly need 100x more samples to get 1 more digit of accuracy.

The key to improve convergence is to have good importance sampling schemes to start with so that even the 1-sample estimate is already quite good. In this case, what is probably happening is that the constant environment sampler is a bit naïve: it samples the whole sphere uniformly instead of only the hemisphere above the shading point (even better would be a cosine-weighted distribution over the hemisphere). I think you’d see exactly the same convergence behavior if you implemented this simple sampling scheme for a planar setup in NumPy or Matlab. (Should be possible in 3-4 lines of code if you're interested in giving this a try).

Improving the directional sampling strategy is of course a point that we could talk about. Or are you mainly just concerned with overall correctness?

[leroyvn]

Hi @wjakob, thank you for this answer. Correctness is indeed my main concern in life but I'm not worried about it in that particular case :) I'd be however happy to improve the directional sampling routine, would it be possible. But I guess this should go to another issue?

[leroyvn]

I'll close this issue anyway, I think we've been to the end of the problem.