Tool language for defining and requiring specific secrets

[mvdbeek]

I think for the tool case it's a little unfortunate that all of this needs to be defined by admins. It'd be nice if a tool author could say

<requirements>
    <requirement type="secret" inject_as_env="key_for_service">
        <!-- i've just copied an example from user_preferences_extra_conf.yml -->
        <interface name="access_token" label="Dropbox access token" required="true"/>
    </requirement>
</requirements>

And the tool form would prompt the user to set it up in their extra preferences and perhaps grant the tool permission to access this particular secret. Resolution of the secret could then be hierarchical and look at the various possible sources.

Originally posted by @mvdbeek in #17498 (comment)

[bernt-matthias]

This sounds like a good idea, also for testing #17452

Wondering how to avoid accidental usage of the same token by different tools (there will be many tools using "password") while still allowing for reuse of tokens by different tools.

[hexylena]

100% for this, it was my original complaint about the way it was defined, it should always have been part of the tool interface and aggregated by Galaxy.

Tools should, not do that. Perhaps we add a linter.
Per my 2016 comment, if we add a mandatory 'namespace' parameter, that could solve it. Authors would naturally choose a keyword related to their tool, I would choose something like apollo and then username and password wouldn't conflict.

[hexylena]

We could then also easily expose in the UI that this tool will use secrets (good for users to be aware when their credentials are being accessed) and make that more traceable in job details/logs that a secret was accessed.

[mvdbeek]

Wondering how to avoid accidental usage of the same token by different tools (there will be many tools using "password")

😆 that's the same problem the extra user preferences approach has now. I think this isn't necessarily a problem, the user should simply be asked which secret should be used ?

[khaled196]

Hello,

I am working on a tool that requires credentials to log in to the MongoDB server created by the user or his institution. galaxyproject/tools-iuc#6028

I would appreciate it if I get some help in understanding the current status of this ongoing work.

Examples of current tool configurations using similar annotations.

Documentation on how to modify the XSD schema for Galaxy.

Guidelines on accessing and modifying the user-preference store.

Frontend development documentation for Galaxy's user interface.

[mvdbeek]

You would have to implement this, maybe start with modifying your tool so you can read secrets from extra preferences (see https://github.com/galaxyproject/tools-iuc/tree/main/tools/pyega3 for an example), then start building out a layer such as what I suggested in the opening post so that admins don't need to define the user preferences manually. Given that abstraction you can then build out a user interface to prompt users at the tool form layer.

There's a small example in https://training.galaxyproject.org/training-material/topics/dev/tutorials/core-contributing/tutorial.html for backend and frontend components, but of course these things change faster than we can update the documentation. There are Makefile targets and Readmes in the code for the most important things to get you started.

[khaled196]

Hi @mvdbeek

Thank you for the example, I have a basic idea now, I applied it to the tool and it's happy with it.

but I have a question about

<requirements>
    <requirement type="secret" inject_as_env="key_for_service">
        <!-- i've just copied an example from user_preferences_extra_conf.yml -->
        <interface name="access_token" label="Dropbox access token" required="true"/>
    </requirement>
</requirements>

the tools weren't happy about it at all in the linting.

is it because it is missing from the locale.js file

my credentials are

 <configfiles>
            <configfile name="credentials"><![CDATA[
            #set $db_auth_source = $__user__.extra_preferences.get('beacon2_account|db_auth_source', "")
            #set $db_user = $__user__.extra_preferences.get('beacon2_account|db_user', "")
            #set $db_password = $__user__.extra_preferences.get('beacon2_account|db_password', "")
            #if $db_user == "" or $db_password == "" or $db_auth_source == "":
                #set $db_auth_source = "admin"
                #set $db_user = "admin"
                #set $db_password = "adminpass"
            #end if
            {
                "db_auth_source": "$db_auth_source",
                "db_user": "$db_user",
                "db_password": "$db_password"
            }
            ]]></configfile>
    </configfiles>

[mvdbeek]

the tools weren't happy about it at all in the linting.

yes, that's the part you'll have to implement. linting checks your tool xml against the tool schema, and of course you'll have to parse those values in the tool interface and build up something that would prompt users to fill in the secrets.