Implements a distributed partitioner that is lazier than find_distributed_partitions

[kaushikcfd]

• Only materialized nodes can be inputs/outputs to a partition
• Non-materialized nodes that are used in multiple partitions are duplicated to uphold the lazy behavior of the expressions
• Implementation includes a working examples/distributed_partition_sends_asap.py which demonstrates the output for the partitioner.

Draft because:

• Cleanup
• Add tests
• Pass CI
• Nomenclature incoherence: Users/Successors/Predecessors

[kaushikcfd]

With this branch:

2022-03-20 17:46:37,399 - INFO - pytato.partition - find_partition: Split 5058 nodes into 3 parts, with [2024, 1556, 1633] nodes in each partition

With main:

2022-03-20 17:48:13,940 - INFO - pytato.partition - find_partition: Split 4521 nodes into 131 parts, with [2001, 8, 5, 6, 7, 56, 44, 340, 8, 101, 149, 6, 56, 7, 54, 8, 5, 8, 
47, 8, 11, 96, 47, 54, 47, 4, 54, 8, 11, 7, 9, 4, 47, 56, 56, 37, 149, 4, 20, 56, 8, 96, 47, 5, 37, 11, 2, 149, 8, 8, 13, 7, 20, 7, 6, 138, 20, 4, 8, 54, 7, 4, 7, 149, 39, 47
, 56, 371, 44, 26, 23, 54, 8, 12, 20, 4, 17, 47, 7, 65, 4, 8, 5, 54, 5, 6, 38, 8, 85, 56, 47, 6, 8, 16, 2, 8, 37, 56, 47, 7, 20, 6, 3, 37, 85, 11, 89, 54, 4, 31, 8, 10, 2, 44
, 8, 3, 8, 8, 5, 4, 17, 61, 47, 8, 9, 8, 4, 54, 8, 33, 7] nodes in each partition.

That's for CNS 2D-DG operator, written at https://github.com/illinois-ceesd/drivers_y2-isolator.

[inducer]

Whoa, nice job! It sure sounds like the lame annotations-based approach of #275 won't be needed now.

[inducer]

Thanks for working on this! I enjoyed reading this, and I still think the approach is really clever. There's a number of things flagged here, but nothing fundamental. Good to go once these are addressed!

[inducer]

Unsubscribing... @-mention or request review once it's ready for a look or needs attention.

[inducer]

This looks great! Thanks for working on it.