(LedgerStore) Use rocksdb_fifo as the blockstore directory when FIFO is used

[yhchiang-sol]

Summary of Changes

To avoid mixing the use of different shred storage types, each shred storage type
will have its blockstore in a different directory.

This PR still keeps the RocksFifo setting hidden. The default ShredStorageType and
blockstore directory are still RocksLevel and rocksdb.

Will follow-up with PRs on making FIFO option public in ledger-tool and validator.

Test Plan

• Added a new test to verify the existence of rocksdb-fifo directory when FIFO compaction is used.
• Updated existing test to verify the current setting still store ledger under rocksdb directory.
• Manually ran ledger_cleanup_test with both level and fifo compaction and verified the resulting ledger.
• Ran a validator with this PR.

[yhchiang-sol]

The effect is also verified by running the ledger_cleanup benchmark with FIFO enabled:

[keelar] (1046.sol) ~/solana/yhchiang-sol/core/farf/ledger/core/tests/ledger_cleanup.rs-345-9kJ9DaJ29wc6YJFXTQQNqiPziDxbSwnLvbARcz8AnGwT $ ls
rocksdb_fifo
[keelar] (1046.sol) ~/solana/yhchiang-sol/core/farf/ledger/core/tests/ledger_cleanup.rs-345-9kJ9DaJ29wc6YJFXTQQNqiPziDxbSwnLvbARcz8AnGwT $

Will follow up with more tests on GC.

[yhchiang-sol]

Manually test with #23272 by specifying the same ledger path but with different --rocksdb-shred-compaction values. Observe the ledger correctly keep both rocksdb and rocksdb-fifo:

/mnt/disks/ledger/mainnet $ ls -l
total 828
srw------- 1 yueh_hsuan_solana_com yueh_hsuan_solana_com      0 Feb 23 22:20 admin.rpc
drwxrwxr-x 2 yueh_hsuan_solana_com yueh_hsuan_solana_com  69632 Feb 23 23:01 calculate_accounts_hash_cache
-rw-rw-r-- 1 yueh_hsuan_solana_com yueh_hsuan_solana_com 606488 Feb 23 23:01 contact-info.bin
-rw-r--r-- 1 yueh_hsuan_solana_com yueh_hsuan_solana_com 132347 Mar 16  2020 genesis.bin
-rw-rw-r-- 1 yueh_hsuan_solana_com yueh_hsuan_solana_com  20144 Feb 23 22:01 genesis.tar.bz2
-rw-rw-r-- 1 yueh_hsuan_solana_com yueh_hsuan_solana_com      0 Feb 23 22:01 ledger.lock
drwxrwxr-x 2 yueh_hsuan_solana_com yueh_hsuan_solana_com   4096 Feb 23 22:19 rocksdb
drwxrwxr-x 2 yueh_hsuan_solana_com yueh_hsuan_solana_com   4096 Feb 23 23:01 rocksdb_fifo
drwxrwxr-x 2 yueh_hsuan_solana_com yueh_hsuan_solana_com   4096 Feb 23 22:01 tmp-genesis

[codecov]

Codecov Report

Merging #23236 (be6e638) into master (2996f1f) will increase coverage by 0.0%.
The diff coverage is 97.5%.

❗ Current head be6e638 differs from pull request most recent head 562737b. Consider uploading reports for the commit 562737b to get more accurate results

@@           Coverage Diff            @@
##           master   #23236    +/-   ##
========================================
  Coverage    81.3%    81.3%            
========================================
  Files         571      572     +1     
  Lines      155738   155826    +88     
========================================
+ Hits       126682   126810   +128     
+ Misses      29056    29016    -40     

[mvines]

lgtm once steve's comment is addressed

[steviez]

LGTM!

Pull request has been modified.