Create a frontend UI for users to specify quantum compilation pipelines

[paul0403]

Context:
We wish to create a frontend pipeline function for users to specify what circuit transformation passes to run in the dictionary form.
https://app.shortcut.com/xanaduai/epic/67492/p1-a-ui-is-available-for-users-to-specify-quantum-compilation-pipelines-from-python

Description of the Change:
A new API decorator function catalyst.passes.pipeline can be applied to qnodes. The decorator takes in a dictionary specifying what circuit transformation passes to run on the qnode with what options, and adds the passes to the transform_named_sequence.

In addition, qjit now takes in a kwarg circuit_transform_pipeline which expects the same pipeline dictionary. This will apply the pipeline to all qnodes in the qjit.

To test the pipeline, we added the boilerplate for a merge_rotation pass with a pass option. The pass is currently empty and does nothing. The merge_rotation boilerplate pass is not exposed as a user-facing API, and only exists for the purpose of testing the pipeline.

Benefits:
User can specify pass pipelines.

Possible Drawbacks:
There are two items of improvements possible:

1. The target qnode of the pass is recorded by name. This is not optimal. The quantum scope work will likely put each qnode into a module instead of a func.func ... attributes {qnode}. When that is in place, the qnode's module can have a proper attribute (as opposed to discardable) that records its transform schedule, i.e.

module_with_transform @name_of_module {
  // transform schedule
} {
  // contents of the module
}

This eliminates the need for matching target functions by name.

2. The number of qjit kwargs is maybe too many. As of now we implement the syntax specified by the epic, but there could be an alternate design.

[sc-67520]

[paul0403]

@rmoyard The epic description says the zne mitigation pass should go into the catalyst.passes module. I wanted to keep the old usage so in passes.py I just created a wrapper that calls the underlying mitigate_with_zne (the currently existing one).

The spirit of the epic is that the passes.py module should control what passes to run. Therefore I turned off lower-mitigation in the default pipeline and moved it into the apply-transform-sequence (see #911). The behavior is the same as before:

• for functions not under mitigate_with_zne, nothing happens. Originally because the pass in the default pipeline ignores them, now because the pass is not run at all
• for functions under mitigate_with_zne, the pass runs. Originally because the pass runs in the default pipeline, now because mitigate_with_zne registers the pass in the transform sequence.

Thoughts on this?

Update: we decided to exclude ZNE

[paul0403]

Marking ready for review to check CI.

Two remaining items:

1. Do we really need to include ZNE? The pipeline was only meant for peephole circuit transformations We (@rmoyard and myself) decided to exclude ZNE @josh146
2. Make local configurations override global configurations

[mehrdad2m]

I have some thoughts/questions about the design of the pipeline. Something that is worth clearing up is the definition of a "pass". My understanding of a compiler optimization pass is a transformation from one representation of the code to another without changing its functionality with hope that the transformed version is more optimized (runs faster, better memory usage, ...).
With this definition, I think there should be a distinction between things like cancel_inverses and mitigate_with_zne and only the former should be considered as a compiler optimization pass.
My question is then would the user want to run a performance related pipeline (that includes things like cancel_inverses) on individual functions within one qjit call?
My instinct is that the answer is mostly no in which case we can just tie the pipeline to the qjit (similar to how clang runs O2 or O3 on the whole module) instead which makes the implementation nicer and simpler. This would be something like the pipelines option of qjit except that instead of replacing the whole compilation pipeline, it modifies the list of QUANTUM_COMPILATION_PASS passes.
In some specific cases that we want a function to go through a certain optimization pass, we can still use the individual decorators that would work similar to pragma hints in c++. Any thoughts?

If the answer to the question is yes then ignore this comment :)

[mehrdad2m]

Very nice work 🚀

[paul0403]

Summary of results from discussion:

1. The current scope of the transform dialect is "function passes that need no processing in the frontend". This excludes ZNE.
   Note that the pipeline feature has no strict dependency on ZNE. We can merge in the pipeline; if we decide to include ZNE we can always add it later.

2. (future work) Devices can have their own default pipeline, i.e. as a device implementor, I want to run pass X, Y, Z on programs that target my device. Users don't need to set it themselves neither on the qjit, nor qnode

3. (future work) The quantum scope work will likely put each qnode into a module instead of a func.func ... attributes {qnode}. When that is in place, the qnode's module can have a proper attribute (as opposed to discardable) that records its transform schedule, i.e.

module_with_transform @name_of_module {
  // transform schedule
} {
  // contents of the module
}

This eliminates the need for matching target functions by name.

[rmoyard]

Thanks, here are some comments

[rmoyard]

No action here, but @josh146 @isaacdevlugt this will soon beat the number of qnode's arguments.

[rmoyard]

Great job @paul0403 ! Some UI to figure out but happy to approve as a first step 👍