Skip to content
This repository was archived by the owner on Sep 3, 2022. It is now read-only.
This repository was archived by the owner on Sep 3, 2022. It is now read-only.

Enable running the same pipeline N times, but with different parameters each time #659

@rajivpb

Description

@rajivpb

The scenario is described in more detail here:
http://engineering.pmc.com/2017/03/playing-around-with-apache-airflow-bigquery-62/

This is possible using the pipeline object model in pydatalab (i.e we could write a cell that repeatedly called generate_airflow_spec for different dicts of 'parameters').

However, no current way of doing this with %%bq pipeline (i.e. interactively from within Datalab) is currently known.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions