Implement Data Landing Requests

[jmchilton]

Builds on #20288 (merged). Implements #19940.

The idea is just like we do for upload (the __DATA_FETCH__ tool). For a data import tool, we have a special API that wraps the tool version of the API for data import and complying to the fetch_data.py schema - I've mirrored that here for tool landings to create a data landing API.

A data landing create request returns to a tool landing object. When the tool landing object is "claimed", if it is for the data fetch tool a separate UI is rendered to visually display an editable version of the data request UI.

The import interface breaks each data fetch "target" section into a table with a slightly different interface based on whether the target's destination is hdas or an hdca.

The nice thing about the tabular interface is it can be used to display collections or dataset and we can adapt the columns based on the metadata being specified by the request. So that type isn't fixed in the table - the request included at least one type so all the entries have types.

If the request specified other metadata - there would be other columns. Here are some examples of included alternative sets of metadata in the request:

Here are examples with group tags, obscure upload options, and genome:

The tables are also capable or representing high dimensional tables:

There are some options still not implemented, esoteric collection types not as readily tabularized, and library functionality not implemented. I don't think many will be tempted to use these options - but the UI works just fine for these options - it is just the annotated version of the request is shown and the UI is just used for reviewing it before import.

The view request lets the researcher view the request for any target and if the tables have been tweaked by the researcher - a message about this being the original request will be shown:

I feel like I could tweak the UI a ton more and implement more validation - but this component is a bit obscure still so maybe only worth a deep polish after it gets some use?

How to test the changes?

(Select all options that apply)

• Instructions for manual testing are as follows:
  1. All the samples I've shown above are now included as a single huge upload request that can be created using the landing.py client script using the follow the command:
  2. . .venv/bin/activate; PYTHONPATH=lib python lib/galaxy/tool_util/client/landing.py -g http://localhost:8081 -p upload
  3. This will output some text like: "Your customized form is located at http://localhost:8081/tool_landings/e732a8c0-fc94-4afc-8eec-582b10d87f40" - just go that browser to view the landing form and click import to import a bunch of junk into your history.

License

• I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

[jmchilton]

Retrying failed Selenium tests in case lib/galaxy_test/selenium/test_uploads.py::TestUploads::test_upload_composite_dataset_pasted_data is related. Seems like it might be.

[mvdbeek]

Awesome again, thank you so much @jmchilton! I'll get this deployed to test for some hands-on experience.

[mvdbeek]

Thanks so much @jmchilton, I think we'll build many cool integration for this! @bgruening @wm75 I think this could be a viable alternative for (async) data source tools.

[bgruening]

Indeed! Thanks a lot @jmchilton! Can I ask for some documentation, example code etc ... I have a few DBs lined up that showed interest in using it :)

[mvdbeek]

The sample code usage is described in "How to test these changes". We'll implement a version for BRC in the coming days