Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-organize project structure to separate PyTorch dependencies from core project. #2542

Merged
merged 13 commits into from
Nov 3, 2023

Conversation

stellaraccident
Copy link
Collaborator

@stellaraccident stellaraccident commented Nov 2, 2023

This is a first step towards the structure we discussed here: https://gist.github.com/stellaraccident/931b068aaf7fa56f34069426740ebf20

There are two primary goals:

  1. Separate the core project (C++ dialects and conversions) from the hard PyTorch dependencies. We move all such things into projects/pt1 as a starting point since they are presently entangled with PT1-era APIs. Additional work can be done to disentangle components from that (specifically LTC is identified as likely ultimately living in a projects/ltc).
  2. Create space for native PyTorch2 Dynamo-based infra to be upstreamed without needing to co-exist with the original TorchScript path.

Very little changes in this path with respect to build layering or options. These can be updated in a followup without commingling directory structure changes.

This also takes steps toward a couple of other layering enhancements:

  • Removes the llvm-external-projects/torch-mlir-dialects sub-project, collapsing it into the main tree.
  • Audits and fixes up the core C++ build to account for issues found while moving things. This is just an opportunistic pass through but roughly ~halves the number of build actions for the project from the high 4000's to the low 2000's.

It deviates from the discussed plan by having a projects/ tree instead of compat/. As I was thinking about it, this will better accommodate the follow-on code movement.

Once things are roughly in place and the CI passing, followups will focus on more in-situ fixes and cleanups.

@stellaraccident stellaraccident marked this pull request as ready for review November 2, 2023 20:56
@stellaraccident stellaraccident merged commit 6961f0a into main Nov 3, 2023
5 checks passed
stellaraccident pushed a commit that referenced this pull request Nov 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants