[BUG] upgrading from v0.22 to v0.23-rc1 fails - buckets table no longer compatible

[tserong]

Describe the bug

Run s3gw v0.23-rc1 with a datastore created by v0.22. s3gw will fail to start with one of the following errors.

If the datastore is empty:

2023-11-20T20:31:35.371+1100 7f66d8a05f40  0 Migrating legacy database file /scratch/s3gw/22/s3gw.db -> /scratch/s3gw/22/sfs.db
2023-11-20T20:31:35.401+1100 7f66d8a05f40  0 Done migrating legacy database. Continuing startup with /scratch/s3gw/22/sfs.db
2023-11-20T20:31:35.401+1100 7f66d8a05f40  1 db user version: 4
2023-11-20T20:31:35.471+1100 7f66d8a05f40  0 rgw main: ERROR ACCESSING SFS METADATA. Tables: [ buckets ] are no longer compatible.
2023-11-20T20:31:35.471+1100 7f66d8a05f40 -1 Couldn't init storage provider (RADOS)

If the datastore contains some objects:

2023-11-20T20:34:27.046+1100 7f81fa6cdf40  0 Migrating legacy database file /scratch/s3gw/22/s3gw.db -> /scratch/s3gw/22/sfs.db
2023-11-20T20:34:27.073+1100 7f81fa6cdf40  0 Done migrating legacy database. Continuing startup with /scratch/s3gw/22/sfs.db
2023-11-20T20:34:27.073+1100 7f81fa6cdf40  1 db user version: 4
2023-11-20T20:34:27.100+1100 7f81fa6cdf40 -1 [SQLITE] (787) abort at 46 in [DROP TABLE "buckets"]: FOREIGN KEY constraint failed
2023-11-20T20:34:27.100+1100 7f81fa6cdf40  0 rgw main: ERROR ACCESSING SFS METADATA. Metadata database might be corrupted or is no longer compatible
2023-11-20T20:34:27.100+1100 7f81fa6cdf40 -1 Couldn't init storage provider (RADOS)

Why is it trying to drop the buckets table in the second case?

To Reproduce

Steps to reproduce the behavior:

To get the first error:

1. Run v0.22.0: docker run -p7480:7480 -v/scratch/s3gw/22:/data quay.io/s3gw/s3gw:v0.22.0
2. Hit CTRL-C to terminate s3gw
3. Run chown -R tserong:users /scratch/s3gw/22 because you're about to be running s3gw from a source checkout as tserong (obviously replace your username as appropriate)
4. Run s3gw v0.23.0-rc1 from a source checkout against the same datastore as above (.../build/bin/radosgw...)
5. Observe that it fails to start with "ERROR ACCESSING SFS METADATA. Tables: [ buckets ] are no longer compatible."

To get the second error:

1. Run v0.22.0: docker run -p7480:7480 -v/scratch/s3gw/22:/data quay.io/s3gw/s3gw:v0.22.0
2. Create a bucket and an object (e.g. s3cmd mb s3://foo ; echo test | s3cmd put - s3://foo/test)
3. Hit CTRL-C to terminate s3gw
4. Run chown -R tserong:users /scratch/s3gw/22 because you're about to be running s3gw from a source checkout as tserong (obviously replace your username as appropriate)
5. Run s3gw v0.23.0-rc1 from a source checkout against the same datastore as above (.../build/bin/radosgw...)
6. Observe that it fails to start with "[SQLITE] (787) abort at 46 in [DROP TABLE "buckets"]: FOREIGN KEY constraint failed" and "ERROR ACCESSING SFS METADATA. Metadata database might be corrupted or is no longer compatible"

Expected behavior

s3gw migrates the database and starts up just fine.