Experiment setup to determine sane LOD values for tree (groups)

[Garanas]

Description of the proposed changes

Now that we have #6752 to bake properties we have the ability to bake in the LOD values that we want. And since #6751 is merged in we can safely assume that all players will start with the 'High' settings for LODs. Users that take the 'Extreme' setting are on their own.

With that we have everything we need to restart the discussion in #6715.

This pull request provides a standardized test that we can use to inspect the behavior and compare it between hardware configurations. We have to make some assumptions.

• We assume that individual trees and tree groups should have the same LOD to minimize the intel that is leaked through the fog of war when units move through tree groups.
• We assume that the map that is being played is Seton's Clutch. We also assume that this map is representative.
• We assume that players do not play optimal, and therefore do not reclaim tree groups. As a result there will be a lot of broken tree groups because of units traversing them.
• We assume that breaking half the tree groups is representative of this.

With these assumptions we can get some data. The tree groups are broken automatically. There's two hotkeys with a standardized camera configuration:

Then you can open the console using ~. You can open the statistics by typing ShowStats. If you open the entity count tab then you can see there's quite a few props. If you open the frame tab then you can see the current time (in milliseconds) it takes to render a frame. This is a combination of he Lua code being run (in the UI thread, each frame) and the rendering of what is on screen. You can also see the current frames per second (fps).

Then the experiment can start:

• Set game option to for level of detail in the graphics tab to 'High'.
• Change the weight factor of tree groups in blueprint-props.lua. Ideally we check all values ranging from 2.6 up to 14.0 in steps of 2.0.
• (Re)start the map (default hotkey is CTRL + F10)
• Open the statistics.
• For each hotkey with fixed camera positions
• • Write down the frame time.

Additional context

Note that FAForever/FA-Binary-Patches#115 also exists. It does not solve the problem of this pull request - if you increase the LOD by too much it will still just lag. but it may (instead) give players the ability to scale the LOD values of the entities (units, projectiles or props) that the player is most interested in.

I'm not sure if being able to scale LOD values of props, units and/or projectiles is a healthy game option to have. It's super specific and technical. Not a lot of players will understand that.

[Garanas]

Hardware

CPU: 7950x3d
GPU: 7800 XT
RAM: 4x32gb at 3600MT/s

Data

Base line when completely zoomed out: ~1.5ms

• 2.6 (current values on FAF game type)

• • camera at 370: ~9.0ms (half of trees are culled)
• • camera at 470: all trees are culled
• • camera at 570: all trees are culled
• • camera at 670: all trees are culled
• • camera at 770: all trees are culled

• 6.0 (roughly matches the value of old 'extended graphics' and the new 'extreme' value for level of detail)

• • camera at 370: ~13ms (all trees are visible)
• • camera at 470: ~15ms (all trees are visible)
• • camera at 570: ~9.0ms (half of trees are culled)
• • camera at 670: all trees are culled
• • camera at 770: all trees are culled

• 10.0

• • camera at 370: ~13ms (all trees are visible)
• • camera at 470: ~15ms (all trees are visible)
• • camera at 570: ~16ms (all trees are visible)
• • camera at 670: ~17ms (all trees are visible)
• • camera at 770: all trees are culled

• 14.0 (Roughly values of Steam)

• • camera at 370: ~13ms (all trees are visible)
• • camera at 470: ~15ms (all trees are visible)
• • camera at 570: ~16ms (all trees are visible)
• • camera at 670: ~17ms (all trees are visible)
• • camera at 770: ~17.5ms (all trees are visible, if you translate a bit north it hits 19ms)

[Garanas]

@Basilisk3 / @lL1l1 / @Hdt80bro / @BlackYps / @speed2CZ / @4z0t / @GodFuper (yes, you make pull requests so here we go 😄 ) and maybe @clyf, and @Dhomie - given that you all have a development environment, please take five minutes to test the changes in question and make a similar comment to #6761 (comment). That should give us enough data to make a decision.

[lL1l1]

Hardware

CPU: 7800x3d
GPU: RTX 4080 Super
RAM: 2x32gb @ 6000 MT/s CL30

Settings

Set SC_FrameTimeClamp 0 so FPS is not limited.
Notice I set the shadow resolution to max, which is a recent new setting and may affect the results.

Moholog is open but nothing is being posted in there (debug spew is off).

ACU spawn is in north rock, so it does not affect the trees being viewed in the southwest corner (south air).

UI Mods (some of them do auto selections which create ~1ms lag spikes intermittently, and I've ignored those spikes):

info: Active mods in session:
info:         "Selection Deprioritizer"       v06 (bc7731b6-1416-11ea-8d71-362b9e155667) by partytime, changes by CodingSquirrel and Myxir
info:         "Attack Radius Displays Firing Randomness" v03 (080b1e56-6628-42ca-9ab9-e0df5fea0ad8) by Nomander
info:         "UI mod tools"                  v12 (ui-mod-tools-4z0t-v12)                by 4z0t
info:         "ZeP_MiniMapZoom"               v02 (5F593E02-FEF3-487C-916F-D6020903054B) by Ze_PilOt
info:         "Penguin's Icon Mod"            v02 (3ccc4eca-0cb1-11ec-ac65-a37599393411-02) by Emperor_Penguin
info:         "Selection Cost UI"             v09 (2018eaac-e8ed-11e4-b02c-1681e6b44c9)  by Eternal-
info:         "Better chat"                   v14 (better-chat-4z0t-v14)                 by 4z0t
info:         "Rings For All"                 v04 (rings-for-all-v04)                    by 4z0t, Nomander
info:         "Fix Weapon Tooltips"           v09 (4a3d885f-28ff-4244-8a53-66a3545d661b) by Divran, Nomander
info:         "Strategic Rings"               v09 (c28a8edc-33e2-466c-b318-f8c36601e5c0) by GreenSubmarine
info:         "Advanced Target Priorities Tweaked" v06 (6b38a7e2-4370-4437-8d48-0743616e4e20) by Strogo, Nomander
info:         "Reveal Positions (legal)"      v04 (128a0b16-a4d0-4bf2-8f84-935de62715e8) by Myxir, Eternal-, Nomander, Erador
info:         "ChatBeepLite"                  v03 (fd2a78da-790f-4ba4-a6a9-063521c0e548) by Uveso and arma473
info:         "5iver Nukem"                   v04 (d36d33e5-3927-499f-8990-dc86fee30962) by ThwagmoStar
info:         "Notifications v5.2"            v05 (0faf3443-1122-633s-ya-V00000005002)   by Myxir
info:         "Engineer Alt Selection"        v01 (engineer-alt-selection-v01)           by 4z0t
info:         "Missile Launch Hotkey"         v01 (3a2d9c15-48fb-4c79-b823-fd83421cb83a) by clyf
info:         "SharedMouseNM"                 v03 (eternal-pack-shared_mouse_2)          by Eternal-, Tweaked by Nomander
info:         "EcoManagerCBTNM"               v09 (c37fc150-ce96-11ee-EcoManagerCBT-v9NM) by Crotalus, CheeseBerry, Nomander
info:         "UI Party"                      v15 (022E3DB4-9C00-5ED7-9876-4866D316E015) by Anihilnine, with contributions (technical help / ideas / I stole their code) from Zock, Domino, Myxir, yorick, Sir Prize, Crotalus, Coding Squirrel, Morax, Speed2, Hotbuild, camelCase, HUSAR_PL, MaCielPL, tatsu, sheeo, icedreamer, JoonasTo, Heaven, Fast-Thick-Pants
info:         "Common Mod Tools"              v01 (zcbf6277-24e3-437a-b968-Common-v1)    by Crotalus
info:         "Supreme Score Board2 NMEdit"   v01 (e295d108-900a-4c98-a3ff-1d668c7d8bef) by HUSSAR
info:         "Specific Target Priorities"    v03 (Specific-Target-Priorities-v03)       by 4z0t
info:         "MexUpgradeFix"                 v01 (02f284b4-137f-11ed-840a-372sdhd83770) by SpikeyNoob
info:         "High Eco Context Templates"    v01 (88d73038-1756-4e79-8199-e471f4f7b32f) by Nomander
info:         "Share Condition Popup"         v01 (casdb00a-0ca5-11ec-bf8a-346f5a12eff26) by Emperor_Penguin

Data

Base line when completely zoomed out: ~1.15ms

• 2.6 (current values on FAF game type)

  • camera at 370: ~3.7ms (almost all trees are culled)
    

  • camera at 470: ~1.4ms (all trees are culled)

  • camera at 570: ~1.3ms (all trees are culled)

  • camera at 670: ~1.2ms (all trees are culled)

  • camera at 770: ~1.2ms (all trees are culled)

• 6.0 (roughly matches the value of old 'extended graphics' and the new 'extreme' value for level of detail)

  • camera at 370: ~12.6ms (all trees are visible)
  • camera at 470: ~14.9ms (all trees are visible)
  • camera at 570: ~9.1ms (half of trees are culled)
  • camera at 670: ~1.2ms (all trees are culled)
  • camera at 770: ~1.2ms (all trees are culled)

• 10.0

  • camera at 370: ~12.6ms (all trees are visible)
  • camera at 470: ~14.9ms (all trees are visible)
  • camera at 570: ~16.1ms (all trees are visible)
  • camera at 670: ~16.1ms (all trees are visible)
  • camera at 770: ~1.9ms (all trees are culled) (this is weird)

• 14.0 (Roughly values of Steam)

  • camera at 370: ~13.6ms (all trees are visible)
  • camera at 470: ~14.8ms (all trees are visible)
  • camera at 570: ~16.1ms (all trees are visible)
  • camera at 670: ~17.0ms (all trees are visible)
  • camera at 770: ~17.4ms (all trees are visible)
    • Settings shadow resolution from 2048 to 512: ~17.2ms
    • Setting shadows off or setting shadow LOD below 770: ~9.8ms

[BlackYps]

Video settings are the same as @lL1l1 to make comparisons easier.
I also started taking note of the frame times with disabled shadows. Interestingly, changing the shadow size has literally zero effect on the frame times, as does switching between low, medium or high shadow fidelity.

Hardware

CPU: Ryzen 5600X (but in a virtual machine, so performance is a bit lower than normal)
GPU: RTX 3060 Ti
RAM: 21 GB (this is also a result of the virtualization. Hardware RAM is 32GB)

Data

Base line when completely zoomed out: ~2.4ms

• 2.6 (current values on FAF game type)

  • camera at 370: ~12ms (half of trees are culled)
    For me it looks like this. What @lL1l1 posted is how it looks for me with a value of 2.0. Are you sure that you used 2.6?
    

  • camera at 470: ~2.1ms (all trees are culled)

  • camera at 570: ~2.1ms (all trees are culled)

  • camera at 670: ~2.0ms (all trees are culled)

  • camera at 770: ~1.8ms (all trees are culled)

• 6.0 (roughly matches the value of old 'extended graphics' and the new 'extreme' value for level of detail)

  • camera at 370: ~20ms (all trees are visible)
  • camera at 470: ~24ms (all trees are visible)
  • camera at 570: ~13ms (exactly half of trees are culled)
  • camera at 670: ~1.9ms (all trees are culled)
  • camera at 770: ~1.9ms (all trees are culled)

• 10.0

  • camera at 370: ~20ms (all trees are visible) shadow off: ~9ms
  • camera at 470: ~24ms (all trees are visible) shadow off: ~11ms
  • camera at 570: ~26ms (all trees are visible) shadow off: ~13ms
  • camera at 670: ~26ms (all trees are visible) shadow off: ~15ms
  • camera at 770: ~2.5ms (all trees are culled) shadow off: ~2.1ms

• 14.0 (Roughly values of Steam)

  • camera at 370: ~20ms (all trees are visible) shadow off: ~9ms
  • camera at 470: ~24ms (all trees are visible) shadow off: ~11ms
  • camera at 570: ~27ms (all trees are visible) shadow off: ~14ms
  • camera at 670: ~28ms (all trees are visible) shadow off: ~15ms
  • camera at 770: ~29ms (all trees are visible) shadow off: ~16ms

I think we can reduce the amount of measurement points to two passes with 14, one with shadows and one without, and another with 1.0 where all trees are culled for comparison.
The frame times stay the same when the trees are visible, regardless of the exact lod value. (Which is not surprising.)

[BlackYps]

What I find really interesting is the big perfomance drain of the shadows. We should collect some more data about this, but it seems that people could dial in their settings with this tradeoff, where they can decide if they would rather always have shadows or a long tree visibility distance.

[lL1l1]

I tried testing it again and at first got the same result as before, but after disabling UI party's alternative start sequence I got your result. What's odd is that after re-enabling it, I still don't get the old result.

[Garanas]

6.0 (roughly matches the value of old 'extended graphics' and the new 'extreme' value for level of detail)

• camera at 370: ~20ms (all trees are visible)
• camera at 470: ~24ms (all trees are visible)
• camera at 570: ~13ms (exactly half of trees are culled)
• camera at 670: ~1.9ms (all trees are culled)
• camera at 770: ~1.9ms (all trees are culled)

That you @BlackYps immediately hit more than 16 ms/frame right off the bat with decent hardware (even if it runs in a virtual machine) is terrifying!

I also started taking note of the frame times with disabled shadows. Interestingly, changing the shadow size has literally zero effect on the frame times, as does switching between low, medium or high shadow fidelity.

It feels to me that for shadows there is a second batch of (less optimized) draw calls.

[BlackYps]

That you @BlackYps immediately hit more than 16 ms/frame right off the bat with decent hardware (even if it runs in a virtual machine) is terrifying!

I wouldn't read too much into it unless we have more benchmarks to compare. The virtualization leaves me with roughly 80% of the single core performance compared to bare metal in the cpu-z benchmark, but I don't know how the load in the game might differ from this artificial test, especially regarding the cpu cache.
What I do see nicely is that the the framerate bogs down when I hit 100% load on the core that has to manage the render thread, which illustrates that it's indeed a draw call issue.

[lL1l1]

Update for me since I fixed the camera LOD issue (I still have no idea how it happened, but re-enabling UI mods fixed it):

• 2.6 (current values on FAF game type)
  • camera at 370: ~8.4ms

[Garanas]

I was hoping to have some more data. But this will do.

What if I make a pull request where I push the the value to 10, and at the same time we merge in #6763 ?

[lL1l1]

Since we have the new extreme LOD option, wouldn't it be a bit much to bump it from 2.6 to 10?

[Garanas]

The extreme option is there for those with the best hardware. And if you have that (as in my case) then you should still be fine, as long as shadows do not render indefinitely.

Let's continue this in #6851