support variable volume list

[parisni]

Currently Spel and env variables allows to dynamically generate the content of a given string element.

We need more complex scenario such as :

1. Mount a volume depending on the result of a Spel
2. Mount variable list of volumes, depending on the result of a Spel

As for 2. We would mount one volume per group. Let's say user has groups [project1, project2] then we would mount both [/data/project1, /data/priject2]

Glad to contributeon this.

[LEDfan]

This is already possible. When ShinyProxy parses a list that support SpEL, it will ignore any value that is either empty or is an empty list. For example, the following will be parsed to just a single value:

      container-volumes:
        - null
        - []
        - "/tmp/test:/tmp/test"

In addition, ShinyProxy allow to have nested lists (but only one level). Both parsing features make it possible to achieve your scenario:

1. have the SpEL expression return null or an empty list:

   container-volumes: "#{ true == true ? '/tmp/test:/tmp/test' : null}"

   or

   container-volumes: "#{ true == true ? '/tmp/test:/tmp/test' : {}}"

2. use multiple spel expressions

    container-volumes:
       - "#{true == true ? '/tmp/test:/tmp/test' : null}"
       - "#{true == false ? '/tmp/abc:/tmp/abc' : null}"

I'm aware that these specific features are not fully documented on the website, I'll update it soon.

[parisni]

Thanks !

Definitely I understand this covers my first point.

Not sure on the second point which is about dealing with projects (ie:
users in same project to share volumes). Let me be more clear.

This one will produce at most 2 volumes:

 container-volumes:
    - "#{true == true ? '/tmp/test:/tmp/test' : null}"
    - "#{true == false ? '/tmp/abc:/tmp/abc' : null}"

I d'like to be able to generate a list of volume from a spel:

 container-volumes: #{spel-syntax-to-dynamically-produce-a-list}

would expand into 0 to n volumes according to the groups spel result.

This would be fine for volumes, but also for any primitive list such as
app parameters - below would allow one to login a given project they
belong.

  parameters:
    definitions:
      - id: project
        display-name: Data project
        description: The project you belong to

    value-sets:
      - values:
          project: "#{groups.^[#this.substring(0,6) == 'PROJECT_'].toLowerCase()}"
        access-control:
          groups: "#{groups.^[#this.substring(0,6) == 'PROJECT_'].toLowerCase()}"

[parisni]

I made a tiny hugly patch which allows to "flatmap" the volumes.
Now given this users and volumes:

  users:
    - name: jeff
      password: password
      groups: [mathematicians, scientists]

[...]
      container-volumes:
        - "/tmp/jupyter/#{proxy.userId}/work:/home/jovyan/work,/tmp/jupyter/#{proxy.userId}/work2:/home/jovyan/work2"
        - "#{listToCsv('/tmp/jupyter/<replacement>/foo:/home/jovyan/<replacement>', groups)}"

I get those folders in jupyter (from two volume entries, 4 volumes are eventually bound):

The listToCsv spel allows to join multiple volume into one, later it is flattened. This allows the support of dynamic volumes number, based on spel.

diff --git a/src/main/java/eu/openanalytics/containerproxy/backend/docker/DockerEngineBackend.java b/src/main/java/eu/openanalytics/containerproxy/backend/docker/DockerEngineBackend.java
index c669303..4c69984 100644
--- a/src/main/java/eu/openanalytics/containerproxy/backend/docker/DockerEngineBackend.java
+++ b/src/main/java/eu/openanalytics/containerproxy/backend/docker/DockerEngineBackend.java
@@ -60,12 +60,9 @@ import java.net.MalformedURLException;
 import java.net.URI;
 import java.net.URL;
 import java.nio.channels.ClosedChannelException;
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
+import java.util.*;
 import java.util.function.BiConsumer;
+import java.util.stream.Collectors;
 import java.util.stream.Stream;

 @Component
@@ -132,7 +129,7 @@ public class DockerEngineBackend extends AbstractDockerBackend {

             spec.getNetwork().ifPresent(hostConfigBuilder::networkMode);
             spec.getDns().ifPresent(hostConfigBuilder::dns);
-            spec.getVolumes().ifPresent(hostConfigBuilder::binds);
+            spec.getVolumes().ifPresent(vol -> hostConfigBuilder.binds(((List<java.lang.String>)vol).stream().map(volu -> Arrays.asList(volu.split(","))).flatMap(List::stream).collect(Collectors.toList())));
             hostConfigBuilder.privileged(isPrivileged() || spec.isPrivileged());
             spec.getDockerIpc().ifPresent(hostConfigBuilder::ipcMode);

diff --git a/src/main/java/eu/openanalytics/containerproxy/spec/expression/SpecExpressionContext.java b/src/main/java/eu/openanalytics/containerproxy/spec/expression/SpecExpressionContext.java
index 21c785c..f0ab838 100644
--- a/src/main/java/eu/openanalytics/containerproxy/spec/expression/SpecExpressionContext.java
+++ b/src/main/java/eu/openanalytics/containerproxy/spec/expression/SpecExpressionContext.java
@@ -38,6 +38,7 @@ import org.springframework.security.ldap.userdetails.LdapUserDetails;
 import java.util.Arrays;
 import java.util.Collections;
 import java.util.List;
+import java.util.stream.Collectors;

 @Value
 @EqualsAndHashCode(doNotUseGetters = true)
@@ -145,6 +146,16 @@ public class SpecExpressionContext {
         return Arrays.stream(allowedValues).anyMatch(it -> it.trim().equalsIgnoreCase(attribute.trim()));
     }

+    /**
+     * Returns a pipe separated value of the pattern repeated n times, where n is the values size.
+     */
+    public String listToCsv(String pattern, String... values) {
+        if (!pattern.contains("<replacement>")){
+            throw new RuntimeException("the pattern shall contain at least one <replacement>");
+        }
+        return Arrays.stream(values).map(v -> pattern.replaceAll("<replacement>", v)).collect(Collectors.joining(","));
+    }
+
     public SpecExpressionContext copy(Object... objects) {
         return create(toBuilder(), objects);
     }

[LEDfan]

Hi, what you are trying to achieve is already possible. ShinyProxy already flattens nested lists and SpEL Collection Projection allows you to "loop" over a list (https://docs.spring.io/spring-framework/reference/core/expressions/language-ref/collection-projection.html), therefore you can do something like:

users:
    - name: jack
      password: password
      groups:
        - test1
        - test2
        - test3
        - test
specs:
    - id: rstudio
      container-image: openanalytics/shinyproxy-rstudio-ide-demo:2023.06.0_421__4.3.1
      container-volumes: "#{groups.!['/tmp/volumes/' + #this.toLowerCase() + ':' + '/volumes/' + #this.toLowerCase()]}"
      container-env:
        DISABLE_AUTH: true
        WWW_ROOT_PATH: "#{proxy.getRuntimeValue('SHINYPROXY_PUBLIC_PATH')}"
      port: 8787

Once launches it will have the following volumes:

root@afeaced47587:~# ls -l /volumes/
total 0
drwxr-xr-x 2 root root 40 Oct 28 15:35 test1
drwxr-xr-x 2 root root 40 Oct 28 15:35 test2
drwxr-xr-x 2 root root 40 Oct 28 15:35 test3
drwxr-xr-x 2 root root 40 Oct 28 15:35 test4

And as mentioned in my previous post, you can even have multiple of these lists:

container-volumes: 
  - "#{groups.!['/tmp/volumes/' + #this.toLowerCase() + ':' + '/volumes/' + #this.toLowerCase()]}"
  - "#{groups.!['/tmp/volumes/' + #this.toLowerCase() + ':' + '/volumes2/' + #this.toLowerCase()]}"

root@857cf7bd9277:/# ls -l /volumes
total 0
drwxr-xr-x 2 root root 40 Oct 28 15:35 test1
drwxr-xr-x 2 root root 40 Oct 28 15:35 test2
drwxr-xr-x 2 root root 40 Oct 28 15:35 test3
drwxr-xr-x 2 root root 40 Oct 28 15:35 test4
root@857cf7bd9277:/# ls -l /volumes2/
total 0
drwxr-xr-x 2 root root 40 Oct 28 15:35 test1
drwxr-xr-x 2 root root 40 Oct 28 15:35 test2
drwxr-xr-x 2 root root 40 Oct 28 15:35 test3
drwxr-xr-x 2 root root 40 Oct 28 15:35 test4

[parisni]

Many thanks. Sounds like a good example in your documentation !