Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[data] Add label to indicate if operator is backpressured (ray-projec…
…t#47095) ## Why are these changes needed? During the execution of an operator, display whether or not an operator is backpressured. This will more easily surface to the viewer if there are portions of their dataset execution that are bottlenecking the other operations. When backpressured the display will change from `Map(g): 3 active, 22 queued, [cpu: 0.3, objects: 768.0MB]: : 0.00 row [02:14, ? row/s` to `Map(g): 3 active, 22 queued, BACKPRESSURED, [cpu: 0.3, objects: 768.0MB]: : 0.00 row [02:14, ? row/s`. ### Examples: #### Typical example ```python import ray import time def f(x): time.sleep(0.1) return x ray.data.range(1000).map(f).map(f, num_cpus=0.1).materialize() ``` https://github.com/user-attachments/assets/8283fdff-d94f-43c3-a9f0-c929924c441f #### Backpressure example ```python import ray import time def f(x): time.sleep(0.1) return x def g(x): time.sleep(10000000) return x ray.data.range(1000).map(f).map(g, num_cpus=0.1).materialize() ``` https://github.com/user-attachments/assets/d263b8ee-8dd2-4ec6-bd45-c55941c45253 --------- Signed-off-by: Matthew Owen <mowen@anyscale.com>
- Loading branch information