smarter cache eviction

[DocJade]

tier zero is leaps and bounds better than it used to be, but every bit counts

https://en.wikipedia.org/wiki/Cache_replacement_policies
for ideas

https://danluu.com/2choices-eviction/
On low hit rates, random evictions can be better than LRU

Maybe more agressively trim tier 1? Keep track of how recently a block has been read or updated?

Prefer to not evict block 0 to prevent tossing headers?

[DocJade]

tmorrow, will fill in how many swaps a factorio copy took, and how many swaps it took to load (after restarting filesystem to load from a blank slate)

will be used as a basis for benchmarking improvements.

[DocJade]

Would prefer to avoid randomness (or at least, randomly seeded randomness) to make benchmarks consistent

[DocJade]

11900 swaps to upload factorio. Yikes.
18171 swaps to load the game to the main menu.
18255 main menu, then opening the save

1024 disks used.

[DocJade]

Increased cache to 8 disks, new cache strat for swapping.

Upload factorio: 2566
Start game: 15656 (need to turn off multithreading!
Load save: 15735.

[DocJade]

Load factorio: 10589 (single threaded)
Load save: 10673

Cooked.

[DocJade]

idea

[DocJade]

#65 did not seem to help, but i think this is due to single threaded loading not working properly

[DocJade]

Multi threading flag is borked, needs to be set in the config.ini

Load factorio: 8931 (actually single threaded)
Load save: 9009

[DocJade]

rsync to copy in alphabetical order:

Upload factorio: 2533

Start FS (load all directory items): 819
Load Factorio: 7547
Load save: 7632

This is as good as it's gonna get. Don't have time for deeper ideas

[DocJade]

New read ahead:
Upload Factorio: 2506

Start FS: 762
Load Factorio: 12715 (!!! BAD!)
load save not relevant

With the new read-ahead we really suffer on random reads it seems, since the data is not purely sequential unlike the bad apple testing.

Increasing cache size again... Sad, but ~23MB of cache is still quite small, right?

[DocJade]

when we do cache tier cleanup, we should try cleaning up the current disk first (not the blocks we need to write, but the pure read blocks) since we could always read those in again without swapping. Swapping is the highest evil.