Make Models be affected by Fog

[laurensdijkstra]

glTF Models are currently not affected by Fog. This creates a highly unrealistic image. It would be desirable to have Models be affected by Fog just like the Terrain Primitives are.

[ptrgags]

This feature has two parts: fog culling (tiles in the distance for horizon views should use lower-resolution tiles) and fog rendering (Blend the model color with the atmosphere color like terrain does.

Fog culling is already implemented in CesiumJS, but not enabled by default. It goes by the term dynamicScreenSpaceError. I find this a bit confusing, and the documentation could be written in a clearer manner.

This is an unfamiliar part of the 3D Tiles code for me, so I explored the code a bit yesterday and now can explain the parameters better:

Parameter	Effective range of values	Description
dynamicScreenSpaceError	{true, false}	The switch that turns fog culling on/off.
dynamicScreenSpaceErrorFactor	>=0.0 px	Fog culling works by reducing the computed screen space error (SSE) in pixels for tiles far away from the camera for horizon views. This factor is the maximum SSE adjustment. For example, if this is the default of 4.0, then tiles far away from the camera will be SSE = tile.SSE - 4 px. Increasing this value is the easiest way to increase the intensity of the fog culling's effects. Setting this to 0.0 has the same effect as turning off fog culling. Furthermore, this should be restricted to positive values, as negative values will cull tiles near the camera
dynamicScreenSpaceErrorDensity	>=0.0	The adjusted SSE falls off from the original value like a bell curve as distance from the camera increases, to simulate fog being more intense in the distance. Increasing this density parameter makes the bell curve have a sharper peak, which simulates thicker fog. Setting this value to 0.0 Has the same effect as turning off fog culling.
dynamicScreenSpaceErrorHeightFalloff	[0.0, 1.0] (a percentage of the height range)	Fog culling depends on the camera height such that the effect is largest when the camera is near "street level" and gradually falls off to no effect when the camera is above the data. This falloff value is a percentage of the height range of the data1. When the camera is below the falloff point, fog culling will have the strongest effect. Between the falloff point and the maximum height, fog culling will gradually taper off. Above the maximum height, fog culling has no effect.

Another observation: Fog culling rolls off with the camera angle. It has a maximum effect when the camera is facing the horizon, and no effect when the camera is facing straight up or straight down.

Here's a diagram of what the fog culling SSE adjustment looks like:

Here's what the density parameter does:

Here's what the SSE Factor parameter does:

And here's an animation that shows how culling is strongest when the camera is low, but rolls off as the height increases past the falloff point.

Footnotes

1. The height range that is computed depends on the the bounding volume of the dataset. For smaller tilesets with a box/region bounding volume, this is based on the min/max height. For large world-scale bounding volumes, an approximate height range is computed relative to the ellipsoid. ↩

[ptrgags]

Before we enable fog culling by default, we'd like to do some performance testing to evaluate how well it works and what parameters would be the best defaults.

For the first round of tests, I'm using Google Photorealistic 3D Tiles to test how well this feature helps reduce tiles loaded for world-scale datasets. This involves

1. Measure initial tiles loaded time and tiles loaded/selected for the dataset without fog culling
2. Measure it again with the default fog culling settings
3. Try increasing just the density parameter to have the maximum effect (I'm going to try setting it rather high at 0.1)
4. Try increasing just the SSE factor parameter to simulate somewhat aggressive fog without going crazy. I'm going to try 32 px. This is twice the default maximumSSE threshold, and causes a noticeable difference in the background of landscape views.

I've already started some of these measurements, I will post the data and a summary once I have the rest of the numbers.

[ptrgags]

P3DT Fog Performance Test Results

I ran the above tests for comparison. I used 5 different areas of Google P3DT, and for each area 3 views (aerial, street level, top-down) for comparison.

Here's an example HTML file for one of the test setups. I adjusted the parameters in the tilesetOptions object and then refreshing the page as needed).
performance-example-setup.html.txt

Here's a summary of the time to first load, averaged over all the different views:

Configuration	Average time to first load (s)	% difference from no fog
No Fog	7.989	N/A
Fog enabled with defaults	5.678	-28.93%
Fog with SSE Factor = 32	3.658	-54.22%
Fog with density = 0.1	5.403	-32.37%

For the full data (which includes other metrics like number of tiles loaded), see this CSV file:
Dynamic SSE Perf Testing - P3DT.csv

[ptrgags]

Here's some visual comparisons of the data in CesiumJS vs Google Earth to compare how much is culled. It's hard to get the camera views exactly the same since the camera controls are so different, but here's something roughly similar:

CesiumJS P3DT	Google Earth	Comments
		CesiumJS seems to load more tiles in the distance.
		Here Google Earth has a bit more detail in the distance
		Google Earth has a bit more detail in the distance

So overall, it seems like dynamicScreenSpaceErrorFactor: 32 does create somewhat intense fog culling, but not extremely so.

[ptrgags]

Another thing we wanted to check: Does fog culling serve as a workaround for horizon culling for world-scale tilesets? Freeze frame will help here:

With fog off:

With fog on, (SSEFactor 32):

As I suspected, it doesn't completely replace horizon culling (as tiles on the other side of the globe still render), but it does reduce the LODs loaded. Not perfect, but definitely an improvement!

[ggetz]

Thanks @ptrgags for the research!

1. There does appear to be a significant performance gain from implementing fog, so we'll likely move forwards with the process
2. Based on the screenshot comparisons, we may want to do a bit of tweaking to the default parameters to get the best balance between performance and visual fidelity, but we're definitely in the right ballpark
3. It sounds like horizon culling, or even outright culling a tiling which is completely in fog, may provide and additional benefit, so we may want to take a look at that next after this is completed

[ptrgags]

I started prototyping a very barebones FogPipelineStage to add some code to the model shader to see how the czm_fog() function works. I was happily surprised that it (+ AutomaticUniforms + Fog) automatically handles applying fog and even the camera tilt.

Here's Google P3DT with czm_fog() applied (with a hard-coded false color for now) compared to a similar view of San Francisco with Cesium World Terrain. The comparison here is how much of the model is in fog with default settings:

There's still several more details to determine:

• Figuring out how to pass all the necessary variables for computing the atmosphere color to the model shader, as this needs to be passed to the fog function.
• Determining when exactly to enable/disable fog. Certainly there's scene.fog.renderable, but from the Globe shader there are quite a few defines that impact what code runs.
• It looks like the Globe shader also gives the option to do the fog calculations in the vertex color as a performance/visual quality tradeoff
• The globe shader also has several other details about when lighting is/isn't applied, tonemapping, etc. I need to learn what's important for fog in general and what parts (if any) are globe-specific.

[ptrgags]

More notes from exploring the existing globe shader and a discussion with @lilleyse:

• The sky atmosphere, ground atmosphere and fog currently are enabled and configured independently.
• Dynamic lighting (based on sun position over time) affects fog too.
• see this Sandcastle for some of the relevant settings for turning off the atmosphere but keeping fog (while enabling dynamic lighting
• To get the fog to use the atmosphere color, there are several uniforms in AtmosphereCommon.glsl that need to be added to the Model shader. These uniforms can be found in the Globe (for ground atmosphere/fog for terrain) and SkyAtmosphere, but neither are accessible from Model.
• One thought is to make these AutomaticUniforms, so the Globe settings will also apply to 3D Tiles fog.
• Longer term, we could even combine the SkyAtmosphere parameters with the Globe so they only need to be configured in one place. But this is out of scope. See Proposal: Consolidate SkyAtmosphere and Globe atmosphere settings? #11681