Can't set size of volumeClaimTemplate using a workflow variable

[danxmoran]

Checklist:

• I've included the version.
• I've included reproduction steps.
• I've included the workflow YAML.
• I've included the logs.

What happened:

Running Argo 2.4.2, trying to use a workflow parameter to set the size of a volumeClaimTemplate fails with:

error unmarshaling JSON: quantities must match the regular expression '^([+-]?[0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$'

The workflow is never submitted.

What you expected to happen:

I expect the workflow to validate successfully and be submitted.

How to reproduce it (as minimally and precisely as possible):

Try to submit this workflow:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: templated-volume-size-
spec:
  entrypoint: whalesay-volume
  arguments:
    parameters:
    - name: volume-size
      value: 1Gi
  volumeClaimTemplates:
  - metadata:
      name: workdir
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: "{{workflow.parameters.volume-size}}" 
  templates:
  - name: whalesay-volume 
    container:
      image: docker/whalesay:latest
      command: [sh, -c]
      args: ["echo generating message in volume; cowsay hello world | tee /mnt/vol/hello_world.txt"]
      volumeMounts:
      - name: workdir
        mountPath: /mnt/vol

Environment:

• Argo version:

$ argo version
argo: v2.4.2
  BuildDate: 2019-10-21T18:39:52Z
  GitCommit: 675c66267f0c916de0f233d8101aa0646acb46d4
  GitTreeState: clean
  GitTag: v2.4.2
  GoVersion: go1.11.5
  Compiler: gc
  Platform: darwin/amd64

• Kubernetes version :

$ kubectl version -o yaml
clientVersion:
  buildDate: "2019-09-18T14:51:13Z"
  compiler: gc
  gitCommit: 67d2fcf276fcd9cf743ad4be9a9ef5828adc082f
  gitTreeState: clean
  gitVersion: v1.15.4
  goVersion: go1.12.9
  major: "1"
  minor: "15"
  platform: darwin/amd64
serverVersion:
  buildDate: "2019-11-26T00:40:25Z"
  compiler: gc
  gitCommit: a6ba43f5a24ac29e631bb627c9b2a719c4e93638
  gitTreeState: clean
  gitVersion: v1.15.4-gke.22
  goVersion: go1.12.11b4
  major: "1"
  minor: 15+
  platform: linux/amd64

[danxmoran]

It looks to me like this is another instance of #703. #1687 fixed the problem for pod specs, but the volume claim template might have been left out?

[willgleich]

For those who stumble upon here. I reached out in slack and @danxmoran is utilizing a workaround

The work-around we eventually landed on was using a resource step to manually provision the PVC as the 1st step of the workflow The argument-injection works when building those manifests

[alexec]

We would need something like podSpecPatch to fix this - sadly non trivial.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[withnale]

Can I clarify something with this... One of the things that is nice about volumeClaimTemplates is that these are known by the workflow and are cleaned up correctly whatever the state of the pipeline - if it fails or succeeds.

Is the same true of the workaround? If I use a resource step, do I need to make sure the cleanup step will trigger whether the pipeline succeeds or fails, or does argo know the resources it created and automatically cleans them up at the end?

[sherifabdlnaby]

This should be trivial and natively supported in Argo Workflows IMO, especially for Data Workloads.
Our use-case is that we use Argo Workflow to Dump and Transform Databases, and we use Workflow Template to achieve it.

These Databases vary in size greatly, we are forced to use PVC Size = max(All Databases) which is very ineffecient in our case.

[caelan-io]

Reopening as a missing feature reported by @fhochleitner and related to discussion thread #9738