Migrate the unified compiler to Catalyst #2199

mudit2812 · 2025-11-17T19:27:55Z

This PR migrates all source code for the unified compiler to Catalyst.

Changes:

Add python_interface submodule to frontend/catalyst.
Add python_interface submodule to frontend/test/pytest.
Remove references to pennylane.compiler.python_compiler.
Add a utils.py file to frontend/test/pytest to house data that is used by multiple test files.

Remaining work:

Update the cookbook and associated notebooks to use catalyst.python_interface instead of pennylane.compiler.python_compiler.
Migrate alll changelog pertaining to the unified compiler from PennyLane's changelog.

Questions for reviewers:

Should I build wheels in this PR?
Should xdsl and xdsl-jax be hard dependencies?

Future work:

Add xDSL universe. This is needed to integrate lit testing for the unified compiler with the existing Catalyst infrastructure (see below).
Integrate unified compiler testing with Catalyst's testing infrastructure. This will involve removing all unit/integration lit tests from the frontend/test/pytest folder and adapting them so that they can be moved to the frontend/test/lit folder.
Begin the process of having documentation for the unified compiler on pennylane.ai.
Improve test coverage for the unified compiler.
Add tests and GitHub workflows to validate parity between xDSL and MLIR dialects.

[sc-99039] [sc-98172]

paul0403 · 2025-11-17T19:46:09Z

Oh, the +20k lines 🤖

mudit2812 · 2025-11-17T19:56:12Z

Oh, the +20k lines 🤖

I'm not really sure what the review procedure should be for this PR. Beyond linting and updates to which modules we're referencing (pennylane.compiler.python_compiler -> catalyst.python_interface), there aren't any changes.

…neAI/catalyst into migrate-unified-compiler

…_in_dim verify methods

mudit2812 · 2025-11-18T18:29:21Z

.codecov.yml

+ignore:
+  - "frontend/catalyst/python_interface"


So far, we have been omitting the unified compiler's source code from test coverage reports. I have open PRs in pennylane to work towards improving coverage, but until then, we need to keep omitting the unified compiler from coverage reports.

mudit2812 · 2025-11-18T18:29:57Z

.github/workflows/check-catalyst.yaml

+        # Install graphviz for testing the mlir-op-graph integration
+        sudo apt-get install -y graphviz
+        python3 -m pip install graphviz


If it is preferred to install graphviz in a different manner, please let me know.

We should be able to get a green make tests locally when installing Catalyst from source by following the installation guide. The problem with this approach is that it installs graphviz for ci tests but locally you would get an error if you don't have it installed. So I think we should either add graphviz to the requirements.txt and update the installation guide or completely decouple the python_interface tests from current Catalyst test suit, so it wouldn't be run in a regular call to make tests. I am fine with both but maybe the latter is more in line with previous discussions about making python_interface independent from Catalyst. What do you think?

Another solution is that instead of letting those tests fail, I could just pytest.importoskip("graphviz"). Thoughts? It wouldn't decouple visualization tests from catalyst CI, but there won't be any failures.

That could work, but I just realized we want to include the visualization module in the release right? Then maybe should we actually have the related tests run in the CI or the wheels or both? Also after adding the pytest.importoskip("graphviz") let's trigger wheels to make sure the tests there don't fail.

codecov · 2025-11-18T18:55:54Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.72%. Comparing base (a5aa5ec) to head (ca1aa3a).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2199      +/-   ##
==========================================
+ Coverage   97.55%   97.72%   +0.16%     
==========================================
  Files          93       93              
  Lines       11180    11180              
  Branches     1065     1065              
==========================================
+ Hits        10907    10926      +19     
+ Misses        208      192      -16     
+ Partials       65       62       -3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

mudit2812 · 2025-11-18T18:56:08Z

Should I include wheel builds for this PR?

mudit2812 · 2025-11-26T15:41:52Z

frontend/catalyst/python_interface/dialects/stablehlo/dynamism.py

+    irdl_options = [ParsePropInAttrDict()]
+
+    # pylint: disable=too-many-branches
+    def verify_(self):


@mehrdad2m CodeFactor was complaining about DynamicBroadcastInDimOp.verify_ being too complex, so I broke it down into multiple smaller methods. Let me know if the split makes sense to you.

mudit2812 · 2025-11-26T15:42:35Z

frontend/catalyst/python_interface/dialects/stablehlo/reduction.py

+
+    # pylint: disable=no-member
+    # pylint: disable=too-many-branches
+    def verify_(self):


@mehrdad2m same thing here, CodeFactor was complaining about the method being too complex.

**Context:** Migrates the `chore/xdsl-utils` branch from PennyLane. **Description of the Change:** Renames the visualization module to inspection and improves several xDSL parsing utilities within the python interface. **Benefits:** Simplifies and augments existing utility functions within the ~visualization~ inspection module **Possible Drawbacks:** **Related GitHub Issues:** [sc-104851]

mehrdad2m

I noticed there are still some remaining references to the Python compiler, so I added some suggestions for that.

frontend/catalyst/python_interface/__init__.py

frontend/catalyst/python_interface/dialects/catalyst.py

frontend/catalyst/python_interface/dialects/mbqc.py

frontend/catalyst/python_interface/dialects/qec.py

frontend/catalyst/python_interface/doc/unified_compiler_cookbook.rst

frontend/catalyst/compiler.py

frontend/test/pytest/python_interface/dialects/test_stablehlo_dialect.py

frontend/test/pytest/device/test_decomposition.py

frontend/test/pytest/python_interface/inspection/test_draw_unified_compiler.py

mehrdad2m · 2025-12-01T19:08:42Z

.github/workflows/check-catalyst.yaml

+        # Install graphviz for testing the mlir-op-graph integration
+        sudo apt-get install -y graphviz
+        python3 -m pip install graphviz


We should be able to get a green make tests locally when installing Catalyst from source by following the installation guide. The problem with this approach is that it installs graphviz for ci tests but locally you would get an error if you don't have it installed. So I think we should either add graphviz to the requirements.txt and update the installation guide or completely decouple the python_interface tests from current Catalyst test suit, so it wouldn't be run in a regular call to make tests. I am fine with both but maybe the latter is more in line with previous discussions about making python_interface independent from Catalyst. What do you think?

…neAI/catalyst into migrate-unified-compiler

mudit2812 · 2025-12-03T15:35:21Z

I have confirmed locally in a Python environment without xdsl and xdsl-jax that importing catalyst and using qjit does not raise any errors due to xdsl not being installed.

mudit2812 added 2 commits November 17, 2025 10:48

Migrate unified compiler to Catalyst

df8b97b

Fixing migration artifacts; linting

9536707

mudit2812 added the unified compiler Pull requests for the integration with xDSL label Nov 17, 2025

Fix CI errors; Ignore unified compiler in coverage reports

1453aae

mudit2812 and others added 22 commits November 17, 2025 15:11

Test out how graphviz is installed

efc0f88

Lint some more

10cd749

Merge branch 'main' into migrate-unified-compiler

27de22d

[skip ci] Skip CI

f7aa2b6

[skip ci] Skip CI

012f58b

Merge branch 'migrate-unified-compiler' of https://github.com/PennyLa…

f0a0c90

…neAI/catalyst into migrate-unified-compiler

Try installing graphviz with apt

c98755e

Fix codefactor complaints

bb5bd00

Merge branch 'main' into migrate-unified-compiler

1448472

Try appeasing codefactor again

5e745f5

Try appeasing codefactor once again

0409933

Pylint suppression

e786a12

Reduce complexity of stablehlo.reduce and stablehlo.dynamic_broadcast…

4ae10ff

…_in_dim verify methods

Fix some failures

dc41a7b

Try change to graphviz installation

58b13e4

Merge branch 'main' into migrate-unified-compiler

4e5f904

Try installing graphviz with pip

637d2cf

Merge branch 'main' into migrate-unified-compiler

091c5e1

Try installing graphviz with both apt and pip

64e0fd4

Add utils file to remove conftest imports

dc15d3a

Merge branch 'main' into migrate-unified-compiler

e7e33d6

Remove unused imports

f919e8d

mudit2812 commented Nov 18, 2025

View reviewed changes

mudit2812 and others added 4 commits November 18, 2025 14:24

Migrate all changelog entries from PennyLane

6961324

Merge branch 'main' into migrate-unified-compiler

0fdbb46

Add EOF new line to .codecov.yml

a868bc3

change changelog entry slightly

8f00b2f

mudit2812 requested a review from mehrdad2m November 18, 2025 21:17

Merge branch 'main' into migrate-unified-compiler

22a6e6e

mudit2812 commented Nov 26, 2025

View reviewed changes

mudit2812 and others added 2 commits November 26, 2025 07:43

Merge branch 'main' into migrate-unified-compiler

b4a321d

Remove reference to 'remove-chained-self-inverses'

4cbe02a

mudit2812 requested review from andrijapau and dime10 November 26, 2025 21:32

mudit2812 and others added 2 commits November 27, 2025 14:03

Remove leftover references to pennylane.compiler.python_compiler

c5f08c3

mehrdad2m reviewed Dec 1, 2025

View reviewed changes

frontend/test/pytest/python_interface/dialects/test_stablehlo_dialect.py Outdated Show resolved Hide resolved

mehrdad2m reviewed Dec 1, 2025

View reviewed changes

frontend/test/pytest/device/test_decomposition.py Outdated Show resolved Hide resolved

mehrdad2m reviewed Dec 1, 2025

View reviewed changes

mudit2812 and others added 3 commits December 1, 2025 12:53

Merge branch 'main' into migrate-unified-compiler

3395581

Remove more references to python_compiler

a69c9ba

Address more code review comments

9d208f9

mudit2812 requested a review from mehrdad2m December 1, 2025 21:30

mudit2812 and others added 5 commits December 1, 2025 17:22

Fix jit

8ce6f96

Fix jit again

664d24a

Merge branch 'main' into migrate-unified-compiler

e07a332

Fix line-too-long

f9035c0

Merge branch 'migrate-unified-compiler' of https://github.com/PennyLa…

ca1aa3a

…neAI/catalyst into migrate-unified-compiler

		ignore:
		- "frontend/catalyst/python_interface"

Migrate the unified compiler to Catalyst #2199

Are you sure you want to change the base?

Migrate the unified compiler to Catalyst #2199

Conversation

mudit2812 commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paul0403 commented Nov 17, 2025

Uh oh!

mudit2812 commented Nov 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mudit2812 commented Nov 18, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mehrdad2m left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mudit2812 commented Dec 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

mudit2812 commented Nov 17, 2025 •

edited

Loading

codecov bot commented Nov 18, 2025 •

edited

Loading