support variable volume list

Currently Spel and env variables allows to dynamically generate the content of a given string element.

We need more complex scenario such as :

Mount a volume depending on the result of a Spel
Mount variable list of volumes, depending on the result of a Spel

As for 2. We would mount one volume per group. Let's say user has groups [project1, project2] then we would mount both [/data/project1, /data/priject2]

Glad to contributeon this.

This is already possible. When ShinyProxy parses a list that support SpEL, it will ignore any value that is either empty or is an empty list. For example, the following will be parsed to just a single value:

      container-volumes:
        - null
        - []
        - "/tmp/test:/tmp/test"

In addition, ShinyProxy allow to have nested lists (but only one level). Both parsing features make it possible to achieve your scenario:

have the SpEL expression return null or an empty list:

container-volumes: "#{ true == true ? '/tmp/test:/tmp/test' : null}"

or

container-volumes: "#{ true == true ? '/tmp/test:/tmp/test' : {}}"

use multiple spel expressions

 container-volumes:
    - "#{true == true ? '/tmp/test:/tmp/test' : null}"
    - "#{true == false ? '/tmp/abc:/tmp/abc' : null}"

I'm aware that these specific features are not fully documented on the website, I'll update it soon.

Thanks !

Definitely I understand this covers my first point.

Not sure on the second point which is about dealing with projects (ie:
users in same project to share volumes). Let me be more clear.

This one will produce at most 2 volumes:

 container-volumes:
    - "#{true == true ? '/tmp/test:/tmp/test' : null}"
    - "#{true == false ? '/tmp/abc:/tmp/abc' : null}"

I d'like to be able to generate a list of volume from a spel:

 container-volumes: #{spel-syntax-to-dynamically-produce-a-list}

would expand into 0 to n volumes according to the groups spel result.

This would be fine for volumes, but also for any primitive list such as
app parameters - below would allow one to login a given project they
belong.

  parameters:
    definitions:
      - id: project
        display-name: Data project
        description: The project you belong to

    value-sets:
      - values:
          project: "#{groups.^[#this.substring(0,6) == 'PROJECT_'].toLowerCase()}"
        access-control:
          groups: "#{groups.^[#this.substring(0,6) == 'PROJECT_'].toLowerCase()}"

I made a tiny hugly patch which allows to "flatmap" the volumes.
Now given this users and volumes:

  users:
    - name: jeff
      password: password
      groups: [mathematicians, scientists]

[...]
      container-volumes:
        - "/tmp/jupyter/#{proxy.userId}/work:/home/jovyan/work,/tmp/jupyter/#{proxy.userId}/work2:/home/jovyan/work2"
        - "#{listToCsv('/tmp/jupyter/<replacement>/foo:/home/jovyan/<replacement>', groups)}"

I get those folders in jupyter (from two volume entries, 4 volumes are eventually bound):

The listToCsv spel allows to join multiple volume into one, later it is flattened. This allows the support of dynamic volumes number, based on spel.

diff --git a/src/main/java/eu/openanalytics/containerproxy/backend/docker/DockerEngineBackend.java b/src/main/java/eu/openanalytics/containerproxy/backend/docker/DockerEngineBackend.java
index c669303..4c69984 100644
--- a/src/main/java/eu/openanalytics/containerproxy/backend/docker/DockerEngineBackend.java
+++ b/src/main/java/eu/openanalytics/containerproxy/backend/docker/DockerEngineBackend.java
@@ -60,12 +60,9 @@ import java.net.MalformedURLException;
 import java.net.URI;
 import java.net.URL;
 import java.nio.channels.ClosedChannelException;
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
+import java.util.*;
 import java.util.function.BiConsumer;
+import java.util.stream.Collectors;
 import java.util.stream.Stream;

 @Component
@@ -132,7 +129,7 @@ public class DockerEngineBackend extends AbstractDockerBackend {

             spec.getNetwork().ifPresent(hostConfigBuilder::networkMode);
             spec.getDns().ifPresent(hostConfigBuilder::dns);
-            spec.getVolumes().ifPresent(hostConfigBuilder::binds);
+            spec.getVolumes().ifPresent(vol -> hostConfigBuilder.binds(((List<java.lang.String>)vol).stream().map(volu -> Arrays.asList(volu.split(","))).flatMap(List::stream).collect(Collectors.toList())));
             hostConfigBuilder.privileged(isPrivileged() || spec.isPrivileged());
             spec.getDockerIpc().ifPresent(hostConfigBuilder::ipcMode);

diff --git a/src/main/java/eu/openanalytics/containerproxy/spec/expression/SpecExpressionContext.java b/src/main/java/eu/openanalytics/containerproxy/spec/expression/SpecExpressionContext.java
index 21c785c..f0ab838 100644
--- a/src/main/java/eu/openanalytics/containerproxy/spec/expression/SpecExpressionContext.java
+++ b/src/main/java/eu/openanalytics/containerproxy/spec/expression/SpecExpressionContext.java
@@ -38,6 +38,7 @@ import org.springframework.security.ldap.userdetails.LdapUserDetails;
 import java.util.Arrays;
 import java.util.Collections;
 import java.util.List;
+import java.util.stream.Collectors;

 @Value
 @EqualsAndHashCode(doNotUseGetters = true)
@@ -145,6 +146,16 @@ public class SpecExpressionContext {
         return Arrays.stream(allowedValues).anyMatch(it -> it.trim().equalsIgnoreCase(attribute.trim()));
     }

+    /**
+     * Returns a pipe separated value of the pattern repeated n times, where n is the values size.
+     */
+    public String listToCsv(String pattern, String... values) {
+        if (!pattern.contains("<replacement>")){
+            throw new RuntimeException("the pattern shall contain at least one <replacement>");
+        }
+        return Arrays.stream(values).map(v -> pattern.replaceAll("<replacement>", v)).collect(Collectors.joining(","));
+    }
+
     public SpecExpressionContext copy(Object... objects) {
         return create(toBuilder(), objects);
     }

Hi, what you are trying to achieve is already possible. ShinyProxy already flattens nested lists and SpEL Collection Projection allows you to "loop" over a list (https://docs.spring.io/spring-framework/reference/core/expressions/language-ref/collection-projection.html), therefore you can do something like:

users:
    - name: jack
      password: password
      groups:
        - test1
        - test2
        - test3
        - test
specs:
    - id: rstudio
      container-image: openanalytics/shinyproxy-rstudio-ide-demo:2023.06.0_421__4.3.1
      container-volumes: "#{groups.!['/tmp/volumes/' + #this.toLowerCase() + ':' + '/volumes/' + #this.toLowerCase()]}"
      container-env:
        DISABLE_AUTH: true
        WWW_ROOT_PATH: "#{proxy.getRuntimeValue('SHINYPROXY_PUBLIC_PATH')}"
      port: 8787

Once launches it will have the following volumes:

root@afeaced47587:~# ls -l /volumes/
total 0
drwxr-xr-x 2 root root 40 Oct 28 15:35 test1
drwxr-xr-x 2 root root 40 Oct 28 15:35 test2
drwxr-xr-x 2 root root 40 Oct 28 15:35 test3
drwxr-xr-x 2 root root 40 Oct 28 15:35 test4

And as mentioned in my previous post, you can even have multiple of these lists:

container-volumes: 
  - "#{groups.!['/tmp/volumes/' + #this.toLowerCase() + ':' + '/volumes/' + #this.toLowerCase()]}"
  - "#{groups.!['/tmp/volumes/' + #this.toLowerCase() + ':' + '/volumes2/' + #this.toLowerCase()]}"

root@857cf7bd9277:/# ls -l /volumes
total 0
drwxr-xr-x 2 root root 40 Oct 28 15:35 test1
drwxr-xr-x 2 root root 40 Oct 28 15:35 test2
drwxr-xr-x 2 root root 40 Oct 28 15:35 test3
drwxr-xr-x 2 root root 40 Oct 28 15:35 test4
root@857cf7bd9277:/# ls -l /volumes2/
total 0
drwxr-xr-x 2 root root 40 Oct 28 15:35 test1
drwxr-xr-x 2 root root 40 Oct 28 15:35 test2
drwxr-xr-x 2 root root 40 Oct 28 15:35 test3
drwxr-xr-x 2 root root 40 Oct 28 15:35 test4

Many thanks. Sounds like a good example in your documentation !

Navigation Menu

Repository navigation

support variable volume list #88

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

support variable volume list #88

Description

Activity

LEDfan commented on Sep 17, 2024

parisni commented on Sep 17, 2024

parisni commented on Sep 19, 2024

LEDfan commented on Oct 28, 2024

parisni commented on Nov 14, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions