allow configuring multiple object store backends

[icewind1991]

Allow configuring multiple object store backends to allow further load balancing and easier migration.

Configuration

Multiple object stores are configured by setting the objectstore system config to an array of object storage configurations instead of a single configuration. This array can contains both configuration items and references to other configuration keys (which functions as an alias).

When a the configured bucket for a user is an alias, the target of the alias is stored as the users chosen configuration instead to make migrations easier.
The "default" option is required to be an alias to prevent instances from accidentally being setup in a way that makes migration difficult.

By default everything is put in the "default" bucket, objects for the root storage are put in the "root" bucket if it exists ("default" if it doesn't), the backend for users can be set with a user preference option (occ user:setting <user> homeobjectstore objectstore <config key>).

Existing objectstore and objectstore_multibucket configurations are handled transparently.

Migration example

Migrating users to a new s3 server could be done with the following steps:

• create a new object store configuration
• update the default reference to the new configuration.

all new users will now be stored on the new s3 server, existing users can now be migrated one-by-one with minimal downtime

• copy over the objects for a single user to the new s3 server
• update the objectstore config key for the user

example configuration:

[
  'objectstore' => [
    'default' => 'new',
    'new' => [
      'class' => 'OC\\Files\\ObjectStore\\S3',
      'arguments' => [
        'multibucket' => true,
        'bucket' => 'nextcloud-',
        'hostname' => 'objectstore2',
        ...
      ],
    ],
    'old' => [
      'class' => 'OC\\Files\\ObjectStore\\S3',
      'arguments' => [
        'bucket' => 'nextcloud',
        'hostname' => 'objectstore1',
        ...
      ],
    ],
    'root' => [
      'class' => 'OC\\Files\\ObjectStore\\S3',
      'arguments' => [
        'bucket' => 'nextcloud-root',
        'hostname' => 'objectstore1',
        ...
      ],
    ],
  ],
]

For a use case where all objects in the root storage is put in "objectstore1/nextcloud-root", users created before the objectstore migration have the "homeobjectstore.objectstore" config option set to "old" and have their objects stored in "objectstore1/nextcloud" which all users created afterwards have their objects stored in the "objectstore2/nextcloud-XX" buckets.

todo

• command to rename configurations
  • create "<new>" objectstore configuration as a copy from "<old>"
  • for all users who are currently using "<old>" set the objectstore config key to "<new>"
• command to migrate users to a different config
  • same as multibucket_migrate but for objectstore configs instead of buckets
• command to list users by configuration

[come-nc]

That’s not an appid ⚠️
What about using core?

[icewind1991]

Having a separate "scope" makes it much easier to work with the values in bulk (see for example the query in the rename command).

Fwiw: homeobjectstore is already used as an appid for storing the bucket for the user.

[salmart-dev]

I'm not familiar with the object store, but the logic here makes sense in accordance to the PR's description.