Update dialect overview diagram #2072

mortbopet · 2021-11-02T16:27:17Z

Seeing as quite a few new dialects and conversions have entered CIRCT since this diagram was made, I think it's due for an update! Let me know what you think, and if you have anything you'd like to have added/removed.

note: I'll keep the below preview updated wrt. feedback in the comments.

Seeing as quite a few new dialects and conversions have entered CIRCT since this diagram was made, I think it's due for an update! Let me know what you think, and if you have anything you'd like to have added/removed.

lattner · 2021-11-02T17:31:32Z

This is extremely timely. I'm going to suck this into the CIRCT keynote at the devmtg in two weeks :-)

lattner · 2021-11-02T17:32:30Z

One node that could be added is "CHIRRTL" which is the predecessor the High FIRRTL. The FIR Parser can produce all of CHIRRTL, High, Mid, and Low FIRRTL.

youngar · 2021-11-02T17:33:18Z

From a FIRRTL perspective:

There is only one FIRRTL dialect, so it probably makes sense to squash High+Med+Low into one box which would make the diagram a lot simpler
We don't handle FIRRTL in EmitVerilog, so the line from Lower FIRRTL -> EmitVerilog can be removed

updated: We are missing a some dialects: seq, comb, esi, msft Not sure how they fit in the diagram though. It might make some sense to document the python CDE.

Fabian has some notes about some changes to LLHD.

mortbopet · 2021-11-02T19:15:55Z

How about basing this on a .dot file? We might loose a bit on how nice the graph looks, but i think it could remove a bit of friction for making people update the graph on a regular basis.

From something like the following:

Digraph G {
  // MLIR dialects
  subgraph cluster_mlir {
    label = "Upstream MLIR";
    SCF [fillcolor="lightblue"]
    Standard [fillcolor="lightblue"]
    Arith [fillcolor="lightblue"]
  }

  // ===== Nodes =====

  subgraph cluster_circt {
    label = "CIRCT"
    // CIRCT dialects
    Calyx
    Handshake
    FIRRTL
    BLLHD [label="Behavioral LLHD"]
   ...

    // Internal tools
    EmitVerilog [fillcolor="darksalmon"]
    FIRRTLParser [label="FIRRTL Parser" fillcolor="darksalmon"]
    Scheduling [fillcolor="darksalmon"]
  }

  // External tools
  Moore [shape=octagon fillcolor="gold"]
  ...  
  // File formats
  FIRFile [label="FIR file" fillcolor="yellowgreen" shape=note]
  ...

  // ===== Connections =====
  // Dialect conversions
  SCF -> Calyx
  SCF -> StaticLogic [style=dashed]
  Handshake -> FIRRTL
  BLLHD -> SLLHD
  ...


  // Tool flows
  Scheduling -> StaticLogic [dir=both]
  FIRFile -> FIRRTLParser
  FIRRTLParser -> FIRRTL
  FIRRTL -> EmitVerilog
 ...
}
```

teqdruid · 2021-11-02T21:23:41Z

I don't mind the graphviz output, though it is messier. Manually placing some of the nodes could make it clearer (IIRC graphviz supports X/Y coordinates). I don't much care if we move to graphviz. Is it possible to add a legend to the graphviz diagram? That's important.

Notes:

MSFT has an ExportTcl exporter.
ESI can produce Cap'nProto for cosim communication.
EmitVerilog has been renamed to ExportVerilog

teqdruid · 2021-11-02T21:27:04Z

Also, using 'ortho splines' mode would better match the previous image: https://graphviz.org/docs/attrs/splines/

maerhart · 2021-11-02T21:34:10Z

Quick comment:

There should also be an arrow from Comb to LLHD Sim
LLHD Sim is actually part of CIRCT (both LLHDToLLVM and Engine) so it might make sense to put it inside the box

mikeurbach · 2021-11-02T22:01:42Z

I think a refresh is great in general, and I support checking in a .dot file as well as the rendered diagram.

One quick note on the Scheduling + StaticLogic side: we are moving towards doing the scheduling based on Affine dialect rather than SCF, since upstream tools provide the level of detail scheduling needs only for Affine out of the box. So that would be a dotted line from Affine to StaticLogic. We may do progressive lowering through SCF internally, or we may just created the pipeline directly from Affine.

I'll leave it to @maerhart and @fabianschuiki to drill into the LLHD parts, but my impression is that work is moving away from converting to the LLHD dialect, and instead supporting the core dialects in the simulator directly.

mortbopet · 2021-11-02T22:33:00Z

Thank you for the inputs, I've tried to update the diagram accordingly. @teqdruid I'll try to work in a legend; it's quite tricky to do directly in GraphViz without messing up the layout... but I'm thinking that we can just embed an image of the legend inside a node somewhere. Regarding using ortho lines, I'd say it yields worse results that the other possible spline types:

Another interesting possibility with this diagram is to embed links on both nodes (link to Dialect docs) and edges (link to conversion pass), providing a visual way of navigating the proejct.

teqdruid · 2021-11-02T22:46:27Z

Yep -- ortho is ugly.

Can you push your dot file? I'd like to use a modified version in some slides I'm prepping now.

teqdruid · 2021-11-03T05:17:32Z

Thanks for the dot!

I also think that PyCDE deserves a mention as a frontend which is in active development and about to be in use. It can theoretically produce any dialect but HW, Comb, Seq, MSFT are the ones it currently targets with ESI planned.

fabianschuiki · 2021-11-03T06:12:37Z

Thanks for updating this @mortbopet! And @mikeurbach is right: @maerhart has been removing a lot of ops from LLHD in favor of the core dialects. "Structural LLHD" and "Behavioural LLHD" are now basically just "LLHD", and the simulator takes in a mixture of HW/Comb/Seq and LLHD.

teqdruid · 2021-11-03T09:56:41Z

docs/dialects.dot

+  PyCDE -> HW
+  PyCDE -> Comb
+  PyCDE -> Seq
+  PyCDE -> LLHD


PyCDE doesn't directly produce LLHD... though it'd be interesting to expose some of LLHD's timing constructs through PyCDE.

Thanks for adding it!

Would you prefer a dotted edge between PyCDE and LLHD, or no edge? I guess there's also a danger in adding hypothetical edges all over, cluttering up the graph 🤔.

mortbopet · 2021-11-04T21:14:07Z

Regarding the legend, i think the best looking choice will be to just have it as a separate image above the generated image. Trying to embed it into the dot file seems to be messing too much with the layout engine.

seldridge · 2021-11-05T01:09:46Z

Color nit: I would like to use "better" colors for this. In the past, I've used Colorbrewer as I didn't trust myself to pick good colors.

For four colors that are print friendly and colorblind safe, we could use:

#a6cee3
#1f78b4
#b2df8a
#33a02c

There are more options if we don't need to be colorblind safe, e.g.:

#7fc97f
#beaed4
#fdc086
#ffff99

Or:

#8dd3c7
#ffffb3
#bebada
#fb8072

seldridge · 2021-11-05T01:36:34Z

Would you be able to add Chisel as an external tool? It would be sufficient to just have it as a tool with no inputs.
For accuracy, FIRRTL Dialect also lowers to SV. This will make the diagram even more cluttered. 😬

Purely for reference, below is the graphviz I've used for this previously. This is a cartoon compared to what you're showing here, however! This is also a very Chisel-centric view.

One approach that may compact things further is to use sub-clusterings and have arrows start/stop at the cluster boundary. This may hurt accuracy, which I feel is important with this diagram, here as opposed to the cartoon I've used to explain the project. (You can see an example of this in the graphviz below.)

graphviz source

digraph A {
  compound=true
  graph [ranksep=0.25 dpi=300 rankdir=TB]

  {
    node [shape=note]
    "Foo.scala"
    "Foo.fir" "Foo.anno.json"
    "Foo.v"
  }

  {
    node [style=filled shape=box width=1]
    Chisel [label="Chisel\nFront End" fillcolor="#fff5eb"]
    FIRRTL [label="FIRRTL\nDialect" fillcolor="#e5f5e0"]
    HW [label="HW" fillcolor="#fcbba1"]
    Seq [label="Seq" fillcolor="#fcbba1"]
    Comb [label="Comb" fillcolor="#fcbba1"]
    SV [label="SV" fillcolor="#fcbba1"]
    Handshake [label="Handshake"]
    ESI [label="ESI"]
    Tensorflow [label="Tensorflow"]
    Vector [label="Vector"]
    LLVM [label="LLVM"]
    Calyx [label="Calyx"]
    // LLHD [label="LLHD"]
  }

  subgraph cluster_circt {
    label="CIRCT"
    FIRRTL SV Calyx
    // LLHD

    subgraph cluster_rtl_dialects {
      label= "RTL Dialects"
      HW Seq Comb
    }

    subgraph cluster_timing_insensitive_dialects {
      label= "Timing Insensitive Dialects"
      Handshake ESI
    }
  }

  subgraph cluster_mlir {
    label="MLIR Dialects"
    Tensorflow Vector LLVM
  }

  Vector -> Handshake [lhead=cluster_circt ltail=cluster_mlir style=dashed minlen=2]

  "Foo.scala" -> Chisel -> {"Foo.fir" "Foo.anno.json"}

  {"Foo.fir" "Foo.anno.json"} -> FIRRTL
  "Foo.fir" -> {Handshake ESI} [style=invis]
  {FIRRTL Handshake ESI Calyx} -> Seq [lhead=cluster_rtl_dialects minlen=2]
  FIRRTL -> SV
  // Seq -> LLHD [ltail=cluster_rtl_dialects minlen=2]
  Seq -> "Foo.v" [ltail=cluster_rtl_dialects minline=2]
  SV -> "Foo.v"
}

seldridge

This looks awesome.

Some nits about how to DRY out the graphviz. 😄

docs/dialects.dot

mortbopet · 2021-11-05T09:18:30Z

@seldridge thank you for the tips! I've incorporated it into the preview at the top 👍 .

lattner · 2021-11-05T16:26:31Z

This is looking really great to me

seldridge · 2021-11-05T17:52:36Z

docs/dialects.dot

+  }
+
+  subgraph cluster_circt {
+    label = "CIRCT"


A labeljust="r" would get this out of the way of the SCF/Arith arrows.

teqdruid · 2021-11-05T20:16:30Z

I omitted the CIRCT box in the diagrams I modified for my talk, instead putting non-CIRCT stuff in boxes. It made the diagram significantly better.

…

On Fri, Nov 5, 2021 at 10:52 AM Schuyler Eldridge ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In docs/dialects.dot <#2072 (comment)>: > + ranksep=0.5 // vertical compression + compound=true + + // MLIR dialects + subgraph cluster_mlir { + label = "Upstream MLIR"; + node [fillcolor="#beaed4"] + SCF Affine + subgraph cluster_std_arith_dialect { + label = "" + Standard Arith + } + } + + subgraph cluster_circt { + label = "CIRCT" A labeljust="r" would get this out of the way of the SCF/Arith arrows. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2072 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALNXYGU2KP2WG72GJKSBADUKQRX7ANCNFSM5HG4I2OQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

mortbopet · 2021-11-08T09:26:07Z

@teqdruid I agree that the CIRCT box might not be necessary and constrains the layout too much... but i do think we loose a bit on distinguishing what is part of CIRCT and what is not.

What did you put in those non-circt boxes? A well defined box could be fir/chisel/py/.sv.vhd/moore but other than that, boxes seem to constrain the layout too heavily.

mikeurbach · 2021-11-08T19:27:40Z

@mortbopet the diagram @teqdruid is referring to is much more abstract than what you are working on here. See below for what we included in the talk.

I do agree that removing the CIRCT box could help with layout. For the diagram we are discussing here, what if we removed the CIRCT box, and added some invisible edges to try to help line up the layout? That's what I did for the below diagram from the same talk. There are invisible edges from both Standard and SCF to Scheduling. It's still more abstract than what you are doing here, but I think the same idea holds.

teqdruid · 2021-11-08T19:56:52Z

the dot format does allow specifying x, y coordinates instead of invisible edges.

…

On Mon, Nov 8, 2021 at 11:27 AM mikeurbach ***@***.***> wrote: @mortbopet <https://github.com/mortbopet> the diagram @teqdruid <https://github.com/teqdruid> is referring to is much more abstract than what you are working on here. See below for what we included in the talk. [image: Screenshot from 2021-11-08 12-22-43] <https://user-images.githubusercontent.com/1182100/140804690-65c7c655-8709-4712-afd2-c4d6ec5f0590.png> I do agree that removing the CIRCT box could help with layout. For the diagram we are discussing here, what if we removed the CIRCT box, and added some invisible edges to try to help line up the layout? That's what I did for the below diagram from the same talk. There are invisible edges from both Standard and SCF to Scheduling. It's still more abstract than what you are doing here, but I think the same idea holds. [image: Screenshot from 2021-11-08 12-26-07] <https://user-images.githubusercontent.com/1182100/140804978-54117607-e5ae-46d5-989a-9472cb61ad62.png> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2072 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALNXYGBFYARPB5ASEFDQDTULAQDPANCNFSM5HG4I2OQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

mikeurbach · 2021-12-09T17:06:35Z

Since several of us are now using this dot based diagram in talks, can we merge this PR? I think it is already a great starting point.

mortbopet · 2021-12-09T18:21:56Z

Since several of us are now using this dot based diagram in talks, can we merge this PR?

Fine by me; i think the current version is fine as well - if someone wants to spend some time on manually layout'ing it by adding coordinates, i am not opposed to that either (that will -most likely- not be me, though 😉).

lattner · 2021-12-10T17:52:20Z

@mortbopet do you have commit access or need help landing this?

mortbopet · 2021-12-10T21:26:12Z

@lattner i do; merged it for now and then we'll take it from there.

[Draft] Update dialect overview diagram

7a5bd53

Seeing as quite a few new dialects and conversions have entered CIRCT since this diagram was made, I think it's due for an update! Let me know what you think, and if you have anything you'd like to have added/removed.

mortbopet requested review from mikeurbach, lattner, stephenneuendorffer and teqdruid November 2, 2021 16:27

mortbopet marked this pull request as draft November 2, 2021 16:35

mortbopet added 3 commits November 2, 2021 22:50

Add a dialect .dot file

dd2d188

Base dialect diagram on a graphviz graph

69f85fa

Formatting

19829b1

mortbopet added 3 commits November 3, 2021 09:26

Some more updates

b8d65d4

Add legend

67b8a53

Add some links

5f2efd5

mortbopet force-pushed the update_dialectsimg branch from 3686df3 to 5f2efd5 Compare November 3, 2021 09:53

teqdruid reviewed Nov 3, 2021

View reviewed changes

mortbopet marked this pull request as ready for review November 3, 2021 20:11

mortbopet changed the title ~~[Draft] Update dialect overview diagram~~ Update dialect overview diagram Nov 3, 2021

Add a legend

b7ed00c

seldridge reviewed Nov 5, 2021

View reviewed changes

docs/dialects.dot Outdated Show resolved Hide resolved

docs/dialects.dot Outdated Show resolved Hide resolved

docs/dialects.dot Outdated Show resolved Hide resolved

docs/dialects.dot Outdated Show resolved Hide resolved

dry things up a bit

ff2545e

mortbopet force-pushed the update_dialectsimg branch from 7f6d99c to ff2545e Compare November 5, 2021 09:16

dry things up a bit more

72ecc07

seldridge reviewed Nov 5, 2021

View reviewed changes

mortbopet added 2 commits November 8, 2021 09:28

Calyx native label

d82c0f9

There is no HLS cluster

080a1af

mortbopet merged commit 56bc7b5 into main Dec 10, 2021

jopperm mentioned this pull request Apr 26, 2022

Update dialect overview diagram #2976

Merged

mortbopet deleted the update_dialectsimg branch May 17, 2022 07:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update dialect overview diagram #2072

Update dialect overview diagram #2072

mortbopet commented Nov 2, 2021 •

edited

Loading

lattner commented Nov 2, 2021

lattner commented Nov 2, 2021

youngar commented Nov 2, 2021 •

edited

Loading

mortbopet commented Nov 2, 2021

teqdruid commented Nov 2, 2021

teqdruid commented Nov 2, 2021

maerhart commented Nov 2, 2021

mikeurbach commented Nov 2, 2021

mortbopet commented Nov 2, 2021

teqdruid commented Nov 2, 2021

teqdruid commented Nov 3, 2021

fabianschuiki commented Nov 3, 2021

teqdruid Nov 3, 2021

teqdruid Nov 3, 2021

mortbopet Nov 3, 2021

mortbopet commented Nov 4, 2021

seldridge commented Nov 5, 2021

seldridge commented Nov 5, 2021

seldridge left a comment

mortbopet commented Nov 5, 2021

lattner commented Nov 5, 2021

seldridge Nov 5, 2021

teqdruid commented Nov 5, 2021 via email

mortbopet commented Nov 8, 2021 •

edited

Loading

mikeurbach commented Nov 8, 2021

teqdruid commented Nov 8, 2021 via email

mikeurbach commented Dec 9, 2021

mortbopet commented Dec 9, 2021

lattner commented Dec 10, 2021

mortbopet commented Dec 10, 2021

Update dialect overview diagram #2072

Update dialect overview diagram #2072

Conversation

mortbopet commented Nov 2, 2021 • edited Loading

lattner commented Nov 2, 2021

lattner commented Nov 2, 2021

youngar commented Nov 2, 2021 • edited Loading

mortbopet commented Nov 2, 2021

teqdruid commented Nov 2, 2021

teqdruid commented Nov 2, 2021

maerhart commented Nov 2, 2021

mikeurbach commented Nov 2, 2021

mortbopet commented Nov 2, 2021

teqdruid commented Nov 2, 2021

teqdruid commented Nov 3, 2021

fabianschuiki commented Nov 3, 2021

teqdruid Nov 3, 2021

Choose a reason for hiding this comment

teqdruid Nov 3, 2021

Choose a reason for hiding this comment

mortbopet Nov 3, 2021

Choose a reason for hiding this comment

mortbopet commented Nov 4, 2021

seldridge commented Nov 5, 2021

seldridge commented Nov 5, 2021

seldridge left a comment

Choose a reason for hiding this comment

mortbopet commented Nov 5, 2021

lattner commented Nov 5, 2021

seldridge Nov 5, 2021

Choose a reason for hiding this comment

teqdruid commented Nov 5, 2021 via email

mortbopet commented Nov 8, 2021 • edited Loading

mikeurbach commented Nov 8, 2021

teqdruid commented Nov 8, 2021 via email

mikeurbach commented Dec 9, 2021

mortbopet commented Dec 9, 2021

lattner commented Dec 10, 2021

mortbopet commented Dec 10, 2021

mortbopet commented Nov 2, 2021 •

edited

Loading

youngar commented Nov 2, 2021 •

edited

Loading

mortbopet commented Nov 8, 2021 •

edited

Loading