Skip to content

Conversation

@davelopez
Copy link
Contributor

@davelopez davelopez commented Oct 23, 2025

Closes #20904

This PR adds custom validation to template variables in file source and object store templates, enforcing input constraints with clear error messages.

Features

Three validator types:

  • regex: Pattern matching with optional negation
  • length: String length bounds (min/max)
  • range: Numeric value bounds (min/max)

Key characteristics:

  • Validators are optional and can be chained
  • Skip validation for empty optional fields
  • Run on both frontend (immediate feedback) and backend (security)
  • Provide clear, customizable error messages

Example Usage (included in lib/galaxy/files/templates/examples/production_s3fs.yml)

variables:
  bucket:
    label: Bucket Name
    type: string
    validators:
      - type: regex
        expression: '^(?!\s)(?!.*\s$).*$'
        message: "Bucket name cannot have leading or trailing whitespace"
      - type: regex
        expression: '^.*[^/]$'
        message: "Bucket name cannot end with a slash"
Screenshot from 2025-10-24 13-04-02 Screenshot from 2025-10-24 13-06-06

This prevents common configuration mistakes and improves the user experience when setting up custom storage connections.

How to test the changes?

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

@jmchilton
Copy link
Member

These don't use the pre-existing tool_util validators but the schema for declaring them looks compatible. It is intention that the schema is the same I assume? I wish that code had been reusable but if the schema is the same and there is test coverage I think this is fine to not reuse that stuff.

@davelopez
Copy link
Contributor Author

Good point! I was not aware of the pre-existing tool_util validators, but I guess Claude tried to keep consistency with them when suggesting the schema.
I will try to reuse them, thanks for pointing it out!

@davelopez davelopez marked this pull request as draft October 24, 2025 09:09
@davelopez davelopez marked this pull request as ready for review October 24, 2025 11:07
@davelopez
Copy link
Contributor Author

The existing tool_util validators are now reused. Thanks again for catching that!

@davelopez davelopez force-pushed the user_templates_custom_validation branch from 3ee6558 to 413c579 Compare October 27, 2025 16:48
@jmchilton
Copy link
Member

It might be the case that galaxy-util is Python 3.7 compatible and galaxy-tool-util-models isn't - or you maybe need to have galaxy-util depend on galaxy-tool-util-models. Not sure but I could imagine both being a problem.

_______________ ERROR collecting galaxy/util/config_templates.py _______________
galaxy/util/config_templates.py:81: in <module>
    class BaseTemplateVariable(StrictModel):
galaxy/util/config_templates.py:85: in BaseTemplateVariable
    validators: Optional[Sequence[AnySafeValidatorModel]] = None
E   TypeError: 'ABCMeta' object is not subscriptable
________ ERROR collecting tests/util/test_config_template_validation.py ________
tests/util/test_config_template_validation.py:17: in <module>
    from galaxy.util.config_templates import (
galaxy/util/config_templates.py:81: in <module>
    class BaseTemplateVariable(StrictModel):
galaxy/util/config_templates.py:85: in BaseTemplateVariable
    validators: Optional[Sequence[AnySafeValidatorModel]] = None
E   TypeError: 'ABCMeta' object is not subscriptable
________ ERROR collecting tests/util/test_config_template_validation.py ________
tests/util/test_config_template_validation.py:17: in <module>
    from galaxy.util.config_templates import (
galaxy/util/config_templates.py:81: in <module>
    class BaseTemplateVariable(StrictModel):
galaxy/util/config_templates.py:85: in BaseTemplateVariable
    validators: Optional[Sequence[AnySafeValidatorModel]] = None
E   TypeError: 'ABCMeta' object is not subscriptable

@davelopez
Copy link
Contributor Author

Oops! I completely missed those test failures because I thought they were part of the known package test failures and didn't even check them 😅

Thanks for noticing! I will take a look 👍

@davelopez davelopez marked this pull request as draft October 30, 2025 13:50
Introduces a flexible validator system for template variables, enabling regex, length, and range checks with customizable messages.
Introduces a mechanism for attaching and evaluating custom validators (such as regex patterns) to template variables in form handling logic. Enhances client-side validation to catch invalid input earlier and provide immediate feedback, improving user experience and reducing unnecessary server requests.
Introduces regex-based validators to prevent leading/trailing
whitespace and trailing slashes in bucket names, reducing
misconfiguration risks and improving user input reliability.
Moves Sequence import to typing to avoid "'ABCMeta' object is not subscriptable" issues.

Co-authored-by: Nicola Soranzo <nicola.soranzo@earlham.ac.uk>
@davelopez davelopez force-pushed the user_templates_custom_validation branch from 413c579 to 0f8aa5f Compare October 30, 2025 14:56
@davelopez davelopez marked this pull request as ready for review October 31, 2025 10:37
@davelopez
Copy link
Contributor Author

Looks like @nsoranzo suggestion worked! Only unrelated test failures now :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

user file sources fail path construction if trailing slash included in s3 bucket

3 participants