[Compose file] allow names for non-external volumes

[dnephin]

Proposed syntax:

volumes:
  data:
    name: something
    driver: drivername

This will fix a bunch of issues:

• docker stack deploy failed if volume name with special character like "@" used in the compose file  moby/moby#33529
• Docker Compose Variable substitution - can't use a variable in a yaml/compose key moby/moby#33540
• templated volume name syntax not supported in compose v3 format? moby/moby#30770

Probably others I haven't found yet.

We should also deprecate external: name: foo (external: bool would remain undeprecated).

cc @shin- What do you think?

Also, should we go ahead and do this with secrets, configs and networks as well, to keep them consistent?

[dnephin]

cc @vdemeester as well

[shin-]

Seems reasonable. external is ignored in docker stack anyway, right?

[xask00]

In reference to moby/moby#30770

Will this change make templated names for volumes possible in the compose YML file used by docker stack deploy like below -

volumes:
  data:
    name: {{.Service.Name}}-{{.Task.Slot}}-vol
    driver: cloudstor:aws

equivalent of the command line -

docker service create \
  --replicas 5 \
  --name ping2 \
  --mount type=volume,volume-driver=cloudstor:aws,source={{.Service.Name}}-{{.Task.Slot}}-vol,destination=/mydata \
  alpine ping docker.com

[dnephin]

Yes, that is one of the problems that will be fixed by this change.

[xask00]

@dnephin thanks for confirming

[eliaoggian]

I Think this fix will be great! When will this be implemented?

Thanks!

[shouze]

@eliaoggian yes see #306 need to wait for 3.4 version of the schema with some other changes. I guess we can't expect a GA until 17.09 @dnephin right?

[vieux]

@dnephin @thaJeztah should this one be closed ?

[dnephin]

Yes, this issue is closed, fixed by #306

[alculquicondor]

I'm trying to use version 3.4 but it doesn't seem to work.

The relevant configuration is:

services:
    hdfsdata:                                                                                                                                                                                                                           
        image: quay.io/geomesa/hdfs:geomesa-1.3.2-accumulo-1.8.1                                                                                                                                                                        
        command: data                                                                                                                                                                                                                   
        depends_on:                                                                                                                                                                                                                     
            - hdfsname                                                                                                                                                                                                                  
        environment:                                                                                                                                                                                                                    
            - HADOOP_MASTER_ADDRESS=hdfsname                                                                                                                                                                                            
        volumes:                                                                                                                                                                                                                        
            - hdfsdata:/data/hdfs                                                                                                                                                                                                       
        deploy:                                                                                                                                                                                                                         
            mode: global                                                                                                                                                                                                                
            endpoint_mode: dnsrr

volumes:
    hdfsdata:
        name: '{{.Service.Name}}-{{.Task.Slot}}'
        driver: cloudstor:aws
        driver_opts:
            backing: relocatable
            size: 20

However, only a single volume is created with the task.slot from the container in the manager and all tasks attempt to attatch to it. Am I doing something wrong?

[shouze]

It depends if the docker release you run on aws, which one are you running?

[alculquicondor]

Here

~ $ docker version
Client:
 Version:      17.09.0-ce
 API version:  1.32
 Go version:   go1.8.3
 Git commit:   afdb6d4
 Built:        Tue Sep 26 22:39:28 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.09.0-ce
 API version:  1.32 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   afdb6d4
 Built:        Tue Sep 26 22:45:38 2017
 OS/Arch:      linux/amd64
 Experimental: true