NG: Configuration

[BrianBland]

Let's define the approach by which users will configure the next-gen registry:

First, some use cases:

1. I want to run a simple registry in container form with a default storage driver (local filesystem or S3) and want to provide the relevant parameters.
2. I want to configure a persistent registry with many fine-tuned parameters.
3. I want to manage multiple registries and manage my configurations in one place.

Current solutions:

1. Parameters can be provided via environment variables at runtime. Example shown below:

   docker run \
            -e SETTINGS_FLAVOR=s3 \
            -e AWS_BUCKET=mybucket \
            -e STORAGE_PATH=/registry \
            -e AWS_KEY=myawskey \
            -e AWS_SECRET=myawssecret \
            -e SEARCH_BACKEND=sqlalchemy \
            -p 5000:5000 \
           registry
   

2. Edit the config.yml and provide it to the container with a shared volume

3. Same as 2, but with an NFS volume or a synchronized directory(?)

Proposed solutions:

1. Keep the same behavior with environment variables. We may want to do a better job namespacing configuration variables for storage drivers though. For instance, the s3 driver, we provide AWS_BUCKET and AWS_KEY, but perhaps these should be S3_BUCKET and S3_AWS_KEY or S3_KEY.
2. We should probably still use a YAML configuration file, although we should be more conscious of versioning, as to avoid requiring a rewrite of configuration files on each version change. The registry could continue to use old configuration file versions up to a point, as long as we're explicit in specifying the configuration version.
3. Allow the registry to use a URL to point at a YAML configuration, as hosted on a web server somewhere. This allows us to update the configuration in one place and have the changes propagate to all registries using this configuration endpoint (great for mirroring setups).

[wking]

On Wed, Oct 22, 2014 at 10:02:49AM -0700, Brian Bland wrote:

Proposed solutions:

These all sounds good to me, but I'd try and decouple the storage
config settings (S3 parameters) from registry config settings
(pointers to the S3 backend). I see no reason to have the storage
stuff be in-process with the registry, when libchan makes it easy to
run the storage in a separate process (see #630 / #643). In that
case, we could just have the registry connect to an existing (possibly
external) storage service, or spin up the storage service as a new
subprocess pointing at the same config the registry used. In either
case, it would be nice to have sufficient namespacing in the
registry's YAML config that the storage config could live in the same
file.

registry:
storage:
streaming: s3://my-s3-service.example.com/
atomic: redis://my-redis-service.example.com/
log-level: debug
log-handler: syslog:///dev/log
debug-endpoints: true
…

s3:
bucket: foo
key: bar
log-level: warning
log-handler: file:///var/log/s3.log
…

redis:
database: redis://username:password@redis.example.com:6379/0
…

[dmp42]

+1

Better namespacing is definitely something to get right from the get go.

About split configuration, and entirely split process, let people do that, but that shouldn't be the default.

About env variables mapping, this needs to be detailed more.

Do we want the previous behavior?

foo: _env:SOME_FOO:default_value

Or do we want implicit interpolation on any conf variable?

foo:default_value and FOO sourcing?

How do we elegantly map nested names to env:

foo:
   bar: baz

would that "map" to:

• FOO_BAR
• FOO.BAR
• other?

About "remote" configuration files support (http, whatever), do we want that in the core, or should that be the responsibility of a deployment script to copy it over locally?

[wking]

On Wed, Oct 22, 2014 at 11:16:17AM -0700, Olivier Gambier wrote:

About split configuration, and entirely split process, let people do
that, but that shouldn't be the default.

If folks want an in-process storage backend, they'd just replace the
remote URL with a marker to the compiled-in service:

registry:
storage:
streaming: s3

Which would say “use the compiled-in s3 storage, and pass it the
parsed config so it can grab what it needs (e.g. the s3: block).” Or:

registry:
storage:
streaming: s3+subprocess:///path/to/s3?--option=value

to launch an S3 subprocess if you don't want to use a persistent
service.

Or do we want implicit interpolation on any conf variable?

This sounds good to me.

How do we elegantly map nested names to env:

foo:
   bar: baz

would that "map" to:

• FOO_BAR
• FOO.BAR

No dots [1]. FOO_BAR should be unambiguous. I don't expect
collisions like:

foo:
bar: …

foo-bar: …

About "remote" configuration files support (http, whatever), do we
want that in the core, or should that be the responsibility of a
deployment script to copy it over locally?

I'm fine if the deployment script copies it down, but it should be
part of the stock registry image so folks don't have to create
special-case sub-images to get this behavior.

[1]: From POSIX 2013 2:

Environment variable names used by the utilities in the Shell and
Utilities volume of POSIX.1-2008 consist solely of uppercase
letters, digits, and the ( '_' ) from the characters
defined in Portable Character Set and do not begin with a digit.

[BrianBland]

As for the versioning scheme, we should probably use something similar to Semantic Versioning, although I can't think of a need for the patch version when it comes to defining a configuration schema.

This leaves us with a major and minor version (X.Y), where minor version increases indicate the addition of a new optional parameter, and any deletions, name changes, or type changes will require a major version increase.

The version value should be declared as a top level parameter in the configuration file like so: version: 1.8. We'll convert it to a string internally to maintain the guarantee that 1.9 < 1.10 < 1.11

[wking]

On Wed, Oct 22, 2014 at 05:51:06PM -0700, Brian Bland wrote:

As for the versioning scheme, …

This sounds great to me.

[BrianBland]

No dots [1]. FOO_BAR should be unambiguous. I don't expect
collisions like:
foo:
  bar: …
foo-bar: …

If we make the assumption that there is no ambiguity here, and also that environment variables cannot distinguish between upper and lower case, then perhaps we should disallow hyphens and underscores in configuration key names. foo-bar should be replaced by foobar, which leads us to the following example:

version: 0.1

registry:
  loglevel: info # REGISTRY_LOGLEVEL=info
  storage:
    s3: # REGISTRY_STORAGE=s3
      region: us-east-1 # REGISTRY_STORAGE_S3_REGION=us-east-1
      bucket: my-bucket # REGISTRY_STORAGE_S3_BUCKET=my-bucket
      rootpath: /registry # REGISTRY_STORAGE_S3_ROOTPATH=/registry
      encrypt: true # REGISTRY_STORAGE_S3_ENCRYPT=true
      secure: false # REGISTRY_STORAGE_S3_SECURE=false
      accesskey: SAMPLEACCESSKEY # REGISTRY_STORAGE_S3_ACCESSKEY=SAMPLEACCESSKEY
      secretkey: SUPERSECRET # REGISTRY_STORAGE_S3_SECRETKEY=SUPERSECRET
      host: ~
      port: ~

In the above example, I showed the respective environment variables which would alternatively provide the configuration in the YAML file. The driver name and the key-value pairs of its parameters specify the arguments to NewDriverClient as seen in #643 at https://github.com/docker/docker-registry/pull/643/files#diff-c8c4532a664949dfe479be7d227b2807R23 (This map is of type map[string]string and not map[string]interface{} because environment variables are provided only as strings)

[wking]

On Wed, Oct 22, 2014 at 06:42:02PM -0700, Brian Bland wrote:

If we make the assumption that there is no ambiguity here, and also
that environment variables cannot distinguish between upper and
lower case, then perhaps we should disallow hyphens and underscores
in configuration key names.

I'm fine with assuming there's no ambiguity even with hyphens or
underscores in the config names (I'd replace hyphens with underscores
to find the environemt variable name). Personally, I prefer hyphens
to underscores in situations like this (YAML-parsing) that can can
handle them. That's a really low-grade preference though, so I'll be
happy if the convention we eventually decide on is underscores.

registry:
loglevel: info # REGISTRY_LOGLEVEL=info
storage:
s3: # REGISTRY_STORAGE=s3

I'm still pushing for a distinction between atomic and streaming
storage though ;) 1.

[visualphoenix]

+1 for underscores. +1 for semantic versioning.

[wking]

On Wed, Oct 22, 2014 at 05:51:06PM -0700, Brian Bland wrote:

The version value should be declared as a top level parameter in the
configuration file like so: version: 1.8. We'll convert it to a
string internally to maintain the guarantee that 1.9 < 1.10 < 1.11

Looking over #652, I think we should just use ‘version: '1.8'’ to make
it obvious that we're parsing it as a string. Or better yet,
‘version: [1, 8]’ so the YAML library can handle the parsing (and
error reporting) for us.

[dmp42]

@BrianBland +1 for underscores - it makes it more clear to people what they need to look for inside the configuration file.

And +1 on everything else.

[dmp42]

Implemented with #652