Webapp file ingestion changes

[Mesh-ach]

Functionality to list available bronze datasets and upload selected CSVs into GCS bucket without manual uploads.

changes

• Added GET /institutions/{inst_id}/input/bronze-datasets to list available .csv datasets in the institution’s Databricks bronze volume root (no subfolders), filtered by VALID_BRONZE_FILE_RE.
• Added POST /institutions/{inst_id}/input/import-from-bronze to import a selected bronze dataset into the institution’s GCS bucket under unvalidated/ using the existing signed-upload URL flow (same overwrite checks as manual uploads).
• Added unit tests covering the new endpoints and mocked Databricks/GCS interactions.

context

• Today users must upload input datasets from the UI into GCS before they can run inference.
• This change enables a “dataset select” workflow where datasets already present in Databricks bronze can be selected and pulled into the existing GCS validation/batching/inference pipeline, eliminating manual uploads while keeping downstream behaviour unchanged.

questions

No questions at this time

---

• To see the specific tasks where the Asana app for GitHub is being used, see below:
  • https://app.asana.com/0/0/1212480373505400