Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docs generation from cells help output #4332

Merged
merged 68 commits into from
Oct 16, 2024
Merged

Conversation

KrystalDelusion
Copy link
Member

@KrystalDelusion KrystalDelusion commented Apr 12, 2024

Generate rst files for the cells library, similar to how the command reference is generated. Python pre-parse allows for a bit more flexibility.

ReadtheDocs preview available at https://yosyshq.readthedocs.io/projects/yosys/en/docs-preview-cellhelp/

@KrystalDelusion KrystalDelusion force-pushed the krys/help_docs branch 2 times, most recently from 4b6286b to 9295daa Compare May 2, 2024 22:24
@KrystalDelusion KrystalDelusion marked this pull request as ready for review September 16, 2024 08:03
@jix
Copy link
Member

jix commented Sep 16, 2024

A few things I noticed, including some nits that might not need to be addressed:

  • On the top-level "Internal cell library" page, there is still the "TODO brief overview of internal cell library"
  • On the "Unary operators" page:
    • it says "Note that $reduce_or and $reduce_bool actually represent the same logic function. But the HDL frontends generate them in different situations". AFAICT verific doesn't generate either, so maybe this should be more specific and say that "read_verilog" does this?
    • It says $pos is a buffer, which currently is true, but when we add a $buf cell, it should only be used for arithmetic unary +, which is the same as a buffer except when modeling x-propagation where unary + will be treated as an arithmetic operation in that any single x input bit will produce all x outputs. Not sure if anything about the documentation should be changed now though, as it correctly documents how things are treated currently.
  • On the "Binary operators" page:
    • is $modfloor really known as “remainder”? I know that terminology of the different rounding behaviors is a big mess, but if there's any correlation I'd expect it to be the other way around with modulo always being non-negative for a positive divisor and remainder corresponding to the truncating variant. I do like that the page lists examples and that the cells state the invariants that hold, so no matter the expected terminology there shouldn't be any confusion.
    • the descriptions for $shift and $shiftx seem to be swapped
  • On the "Multiplexers" page:
    • $bmux and $bwmux have no description, I think it should at least mention that $bmux is a wide mux that instead of a one-hot in the case of $pmux, uses a binary encoding / an address to select an input. For $bwmux it should mention that it is a bitwise mux and maybe mention the equivalent logic term
  • I didn't fully review the "Memories" page as I know that there are some subtleties that I don't quite remember and that I'd have to look up in the yosys sources.
  • I didn't review the "Finite state machines", "Coarse arithmetics", "Arbitrary logic functions" and "Specify rules" pages as I'm not familiar enough with the corresponding cell types.
  • The "Formal verification cells" page is still all TODOs

@KrystalDelusion
Copy link
Member Author

The $modfloor thing really threw me as well, because remainder should never be negative; but there are languages that follow the given semantics (Python, for example, gives the same results when using the % operator)... Will change it to at least be more specific about 'other languages' to hopefully be less ambiguous.

Allows for more expressive code when constructing help messages for cells.
Will also move extra logic in parsing help strings into the initial python parse instead of doing it in the C++ at export time.
Include Source file and line number in SimHelper struct, and use it for verilog code caption in rst dump.
Also reformat python string conversion to iterate over a list of fields instead of repeating code for each.
Since `simcells.v` uses consistent formatting we can handle it specifically to help tidy up sphinx warnings about the truth tables, and instead chuck them in a code block which when printing to rst.
Also has the side effect that rst code blocks can be added manually with `//- ::` followed by a blank line.
Generate in a temp directory and use `rsync -rc` to only update rst files that have changed.  This prevents sphinx from having to re-generate every cmd/cell page any time the git sha changes.
Also change cmd gen to match.
Fix `$macc` page.
Subclass the command reference code in order to support smart references to the internal cells.
Use new `autoref` role when using single backticks. Allows automatic mapping to a cmd ref or a cell ref.
- Drop `cell_code` and instead map code lookups to the `cell_help` dict.
- Add helper functions to struct for checking and getting the right cell.
- Add `CellType` for cell to `write_cell_rst` function declaration in
  preparation for use in future.
- Iterate over `yosys_celltypes.cell_types` when exporting cell rst files,
  reporting errors for any cells defined in `cell_types` but not
  `cell_help_messages`.
Use autodocs to perform cell reference docs generation instead of generating rst files directly.
e.g.
```
.. autocell:: simlib.v:$alu
   :source:
   :linenos:
```
Drop word_other block since it raises a warning, which will cause the RTDs build to fail.
Explicitly assign $_TBUF_ to `gate_other` and remove catch if a cell has no group.
Tags are added to the list of properties when exporting to `cells.json`.
Also tag as x-aware cells and add titles.
These two files can now be safely .gitignore'd.
Properties are both an option:
```
.. cell:def:: $add
   :properties: is_evaluable
```
and a field:
```
.. cell:def:: $eqx

   :property x-aware:
   :property is_evaluable:
```

Properties as an option appear in the property index: linking a given property to all cells with that property; while properties as a field display with the cell.
Also a few extra cell help texts.
Add properties page, move cell_gate and cell_word under a singular cell_index along with properties.  Fix links accordingly.

Also drop x-aware and x-output todos since they are resolved.
Also put property lists *after* cell description.
- Unswap shift/shiftx
- Add brief overview to cell lib
- Clarify $div cell B input
- Clarify unary operators
- What is $modfloor
Also making ver2 cell descriptions consistently spaced.
$lut and $sop were missed in the rebase, and $buf is new to main since the last rebase.
It somehow got lost in the rebase.
@KrystalDelusion KrystalDelusion merged commit f137509 into main Oct 16, 2024
33 checks passed
@KrystalDelusion KrystalDelusion deleted the krys/help_docs branch October 16, 2024 16:01
@widlarizer
Copy link
Collaborator

This also added unused CellType members, which is a bit risky to expose before implementing. I discovered it because I was looking at adding is_symmetrical. I'll avoid touching it if this is something you're working on, let me just suggest that we could move from std::vector to std::set and then set members by checking intersections:

for (auto type : unary_ops)
	setup_type(type, {ID::A}, {ID::Y}, true, symmetrical_ops.count(type));

@KrystalDelusion
Copy link
Member Author

@widlarizer it was more a proof of concept because I don't actually know which cells have those properties, just that they were useful properties to report 😅 Feel free to change it by adding others or commenting out the unused ones or whatever, just make sure that kernel/register.cc write_cell_rst is updated to match 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants