Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[core][dag] Add ascii based CG visualization #48315

Open
wants to merge 24 commits into
base: master
Choose a base branch
from

Conversation

Bye-legumes
Copy link
Contributor

@Bye-legumes Bye-legumes commented Oct 29, 2024

Why are these changes needed?

Add ascii based CG visualization so we can have better view of the CG.

Related issue number

Close
#48190

Example

codes

import ray
from ray.dag.input_node import InputNode
from ray.dag.output_node import MultiOutputNode

ray.init()
@ray.remote
class Actor:
    @ray.method(num_returns=3)
    def return_three(self, x):
        return x, x + 1, x+2

    def echo(self, x):
        return x
    
    @ray.method(num_returns=2)
    def return_two(self,x):
        return x, x + 1

a = Actor.remote()
b = Actor.remote()
with InputNode() as i:
    o1, o2, o3 = a.return_three.bind(i)
    o4 = b.echo.bind(o1)
    o5 = b.echo.bind(o2)
    o6, o7 = b.return_two.bind(o3)
    dag = MultiOutputNode([o4, o5, o6, o7])

compiled_dag = dag.experimental_compile()

# Get the DOT source
res = compiled_dag.visualize(format='ascii')
print(res)






#compiled_dag = dag.experimental_compile()
# Call the visualize method
#compiled_dag.visualize(filename=filename, view=False)
#res = compiled_dag.execute(1)
#print(ray.get(res))
# Assert that the visualization file exists
#assert os.path.exists(f'{filename}.png'), "Visualization file was not created."

# Clean up
#compiled_dag.teardown()

output

Nodes Information:
            0 [label="Task 0  InputNode"]
            1 [label="Task 1  Actor: 54777d... Method: return_three"]
            2 [label="Task 2  ClassMethodOutputNode[0]"]
            3 [label="Task 3  ClassMethodOutputNode[1]"]
            4 [label="Task 4  ClassMethodOutputNode[2]"]
            5 [label="Task 5  Actor: c927c9... Method: echo"]
            6 [label="Task 6  Actor: c927c9... Method: echo"]
            7 [label="Task 7  Actor: c927c9... Method: return_two"]
            8 [label="Task 8  MultiOutputNode"]
            9 [label="Task 9  ClassMethodOutputNode[0]"]
            10 [label="Task 10  ClassMethodOutputNode[1]"]

            Edges Information:
            0 ---> 1 [label=SharedMemoryType]
            1 ---> 2 [label=SharedMemoryType]
            1 ---> 3 [label=SharedMemoryType]
            1 ---> 4 [label=SharedMemoryType]
            2 ---> 5 [label=SharedMemoryType]
            3 ---> 6 [label=SharedMemoryType]
            4 ---> 7 [label=SharedMemoryType]
            5 ---> 8 [label=SharedMemoryType]
            6 ---> 8 [label=SharedMemoryType]
            9 ---> 8 [label=ChannelOutputType]
            10 ---> 8 [label=ChannelOutputType]
            7 ---> 9 [label=SharedMemoryType]
            7 ---> 10 [label=SharedMemoryType]

            Experimental Graph Built:
            0:InputNode
            |
            1:Actor_54777d:return_three
            |---------------------------->|---------------------------->|                                                  # noqa
            2:Output[0]                   3:Output[1]                   4:Output[2]                                        # noqa
            |                             |                             |                                                  # noqa
            5:Actor_c927c9:echo           6:Actor_c927c9:echo           7:Actor_c927c9:return_two                          # noqa
            |                             |                             |---------------------------->|                    # noqa
            |                             |                             9:Output[0]                   10:Output[1]         # noqa
            |<----------------------------|-----------------------------|-----------------------------|                    # noqa
            8:MultiOutputNode

Checks

  • [√] I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • [√] I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • [√] Unit tests
    • [√] Release tests
    • This PR is not tested :(

Signed-off-by: zhilong <zhilong.chen@mail.mcgill.ca>
Signed-off-by: zhilong <zhilong.chen@mail.mcgill.ca>
Signed-off-by: zhilong <zhilong.chen@mail.mcgill.ca>
Signed-off-by: zhilong <zhilong.chen@mail.mcgill.ca>
@Bye-legumes
Copy link
Contributor Author

@ruisearch42 Hi, can you check this. I just add another method based on ascii and the previous visualization keeps the same

@ruisearch42 ruisearch42 self-assigned this Oct 30, 2024
Copy link
Contributor

@ruisearch42 ruisearch42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution.

I think ASCII visualization only scales to a certain graph size and complexity and would like to get an idea what types of graphs work well for this implementation and what would break.

python/ray/dag/compiled_dag_node.py Outdated Show resolved Hide resolved
python/ray/dag/tests/experimental/test_accelerated_dag.py Outdated Show resolved Hide resolved
python/ray/dag/tests/experimental/test_accelerated_dag.py Outdated Show resolved Hide resolved
python/ray/dag/compiled_dag_node.py Outdated Show resolved Hide resolved
python/ray/dag/compiled_dag_node.py Outdated Show resolved Hide resolved
python/ray/dag/compiled_dag_node.py Outdated Show resolved Hide resolved
python/ray/dag/compiled_dag_node.py Outdated Show resolved Hide resolved
python/ray/dag/compiled_dag_node.py Outdated Show resolved Hide resolved
python/ray/dag/compiled_dag_node.py Outdated Show resolved Hide resolved
python/ray/dag/tests/experimental/test_accelerated_dag.py Outdated Show resolved Hide resolved
python/ray/dag/tests/experimental/test_accelerated_dag.py Outdated Show resolved Hide resolved
@Bye-legumes
Copy link
Contributor Author

Hi, @ruisearch42, I just made some modification based on your comments. I think it looks better now. While I prefer to return the sting instead of print so we can run the tests.

Bye-legumes and others added 8 commits November 7, 2024 12:57
Signed-off-by: zhilong <zhilong.chen@mail.mcgill.ca>
Signed-off-by: zhilong <zhilong.chen@mail.mcgill.ca>
Signed-off-by: zhilong <zhilong.chen@mail.mcgill.ca>
Signed-off-by: zhilong <zhilong.chen@mail.mcgill.ca>
Signed-off-by: zhilong <zhilong.chen@mail.mcgill.ca>
Signed-off-by: zhilong <zhilong.chen@mail.mcgill.ca>
Signed-off-by: zhilong <zhilong.chen@mail.mcgill.ca>
Copy link
Contributor

@ruisearch42 ruisearch42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just had some ideas to make the viz easier to read. After that I will probably have a final round of review for merging.

python/ray/dag/tests/experimental/test_accelerated_dag.py Outdated Show resolved Hide resolved
python/ray/dag/tests/experimental/test_accelerated_dag.py Outdated Show resolved Hide resolved
python/ray/dag/tests/experimental/test_accelerated_dag.py Outdated Show resolved Hide resolved
python/ray/dag/tests/experimental/test_accelerated_dag.py Outdated Show resolved Hide resolved
python/ray/dag/compiled_dag_node.py Outdated Show resolved Hide resolved
Bye-legumes and others added 7 commits November 11, 2024 10:39
Signed-off-by: zhilong <zhilong.chen@mail.mcgill.ca>
Signed-off-by: zhilong <zhilong.chen@mail.mcgill.ca>
Signed-off-by: zhilong <zhilong.chen@mail.mcgill.ca>
Signed-off-by: zhilong <zhilong.chen@mail.mcgill.ca>
Signed-off-by: zhilong <zhilong.chen@mail.mcgill.ca>
Signed-off-by: zhilong <zhilong.chen@mail.mcgill.ca>
@Bye-legumes
Copy link
Contributor Author

I just had some ideas to make the viz easier to read. After that I will probably have a final round of review for merging.

I just make some fix, can you check these? Thx!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants