Closed
Description
Description
Consider adding parallel traffic routing in order to create shadow pipelines. A shadow pipeline consists of having 2 models running in parallel, one serving the requests while the other does some other task, while not serving the request (e.g. model evaluation to keep track of the performance)
Motivation
Can be useful for complex MLOps pipelines to keep track of model performance.
Additional context
- Istio mirroring docs
- ingress-nginx mirroring docs
- Seldon supports this feature, through ingress routing (see here).
Checklist
- Can you have multiple primaries and a shadow pipeline? What are the possible combinations of multiple primaries and shadow pipelines? Should we support all combinations or only a few to maintain a nice user experience?
- Add a field to indicate which branch will be the primary and which branch(es?) will be mirror
- Add validations for the designed UI
- Update telemetry and UserStr functions accordingly
- Update docs