Stress/Crash Test for OptimisticTransactionDB

[jaykorean]

Context:
OptimisticTransactionDB has not been covered by db_stress (including crash test) like TransactionDB.

Summary:

1. Adding the following gflag options to to test OptimisticTransactionDB

• use_optimistic_txn: When true, open OptimisticTransactionDB to test
• occ_validation_policy: OccValidationPolicy::kValidateParallel = 1 by default.
• share_occ_lock_buckets: Use shared occ locks
• occ_lock_bucket_count: 500 by default. Number of buckets to use for shared occ lock.

2. Opening OptimisticTransactionDB and NewTxn/Commit added per use_optimistic_txn flag in db_stress_test_base.cc
3. OptimisticTransactionDB blackbox/whitebox test added in crash_test.mk

Please note that the existing flag use_txn is being used here. When use_txn == true and use_optimistic_txn == false, we use TransactionDB (a.k.a. pessimistic transaction db). When both use_txn and use_optimistic_txn are true, we use OptimisticTransactionDB. If use_txn == false but use_optimistic_txn == true throw error with message "You cannot set use_optimistic_txn true while use_txn is false. Please set use_txn true if you want to use OptimisticTransactionDB".

Test Plan:

Crash Test
Serial Validation

export CRASH_TEST_EXT_ARGS="--use_optimistic_txn=1 --use_txn=1 --use_put_entity_one_in=0 --occ_validation_policy=0"
make crash_test -j

Parallel Validation (no share bucket)

export CRASH_TEST_EXT_ARGS="--use_optimistic_txn=1 --use_txn=1 --use_put_entity_one_in=0 --occ_validation_policy=1 --share_occ_lock_buckets=0" 
make crash_test -j

Parallel Validation (share bucket)

export CRASH_TEST_EXT_ARGS="--use_optimistic_txn=1 --use_txn=1 --use_put_entity_one_in=0 --occ_validation_policy=1 --share_occ_lock_buckets=1 --occ_lock_bucket_count=500" 
make crash_test -j

Stress Test

./db_stress -use_optimistic_txn -threads=32

[pdillinger]

Good start. Doesn't look like this is connected up to the transaction testing code that is guarded by FLAGS_use_txn, though that could be put in a next PR. We will need to essentially audit all those places and decide which apply to OTDB. You can decide how to resolve mixing --use_txn and --use_optimistic_txn options, or not mixing, and whether it makes sense to keep checking FLAGS_use_txn or to introduce another variable for transactions in general.

[jaykorean]

wondering why we shouldn't parallelize. (copied from other tests as I assume the requirement is the same for OTDB, but I'd love to learn)

[facebook-github-bot]

@jaykorean has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[facebook-github-bot]

@jaykorean has updated the pull request. You must reimport the pull request before landing.

[pdillinger]

LGTM! (Be sure to re-import)

[facebook-github-bot]

@jaykorean has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[facebook-github-bot]

@jaykorean merged this pull request in 17d5200.