Maintenance of core parts of the embedded / Cortex-M ecosystem

[japaric]

A.K.A. @japaric is a bottleneck for development :-)

There are critical components of the embedded / Cortex-M ecosystem that are currently being
maintained by me. Unfortunately I don't have the bandwidth to keep up with everything so issues and
PRs have been piling up in some repos.

This issue is a call for maintainers (if you want to help leave a comment!) and a place to discuss
how to organize maintainership duties.

First, these are the crates that should be maintained by more people:

• embedded-hal

Abstractions for things like Serial, SPI, I2C, etc. to build generic drivers that work with AVR, ARM
Cortex-M, MSP430, RISCV, etc. microcontrollers.

There's a bunch to do here as some abstractions are missing and async support is currently
nonexistent. Most of the work to do here is ensuring that the established process for adding new
traits is followed and that discussions move towards consensus.

Quite a few people have volunteered to help and are now collaborators so I think this one is well
covered for now.

• svd2rust

Tool used to transform CMSIS SVD files (XML files that describe the hardware registers of a
device) into code for low level manipulation of hardware registers. Although the SVD spec was
created by ARM svd2rust is being used for both ARM Cortex-M and MSP430 devices. In the later case
TI provides DSLite files (their version of SVD files) and those are converted to SVD files using the
dslite2svd tool.

It's expected that svd2rust will also grow AVR support -- Atmel has its own description file
format: atdf -- using the atdf2svd approach. And there are XSVD (JSON version of SVD files) files
for RISCV -- I don't think they are official though? -- so svd2rust could also grow support for
RISCV via a xsvd2svd tool.

Unfortunately svd2rust has grown into an unmaintainable mess. The test suite takes a lot of time
to run (it takes longer than the CI limit) and it only checks if the generated code still compiles
-- it provides no feedback about whether the generated code exposes a certain interface or not. The
underlying parser lacks error handling and simply panics on errors. Also, svd2rust doesn't give
you helpful error messages when your SVD file has some problem -- this is an important feature to
have when manually editing SVD files.

So both svd2rust and the underlying svd parser need to be refactor (read: rewritten) to make
them more modular and testable. I think the approach to take here is a refactor like the one
proposed in rust-embedded/svd#37. Basically we should be able to test code generation on parts of a SVD
file. For example that <enumeratedValues>..</enumeratedValues> produces this enum Foo { .. }
and this API impl Foo { fn is_variant(&self) -> bool { .. } }, etc. This will let us write unit
tests that can replace the huge existing tests that only check for compilation errors. And while you
are at it you can add error handling to the parser and nice error messages to svd2rust.

This is a huge task but it will make it easier to continue the development of svd2rust. There a
bunch of open issues related to SVD files not being supported; the refactor should make it easier to
fix those issues.

If you want to tackle this let me know in the comments!

---

Then we have core cortex-m crates.

• cortex-m

This repo provides access to Cortex-M specific features like access to registers, safe wrappers around
assembly instructions and access to core peripherals.

One old issue in this repo is providing a svd2rust generated API to access the core peripherals.
The current API has been manually written and is not as type safe as a svd2rust one would be.
Fixing this requires writing or putting together an SVD file that contains only core peripherals,
running svd2rust over it and committing the code to the cortex-m repo.

• cortex-m-rt

This crate takes care of communicating the memory layout of a program to the linker and providing
implementations of language items that are required to build a no_std program.

There are a few missing features in this crate: support for placing functions / interrupt handlers /
the whole program in RAM, relocation of the vector table, etc. that need a design. It may be useful
to coordinate with the representatives of the other embedded targets to see if we can provide a
similar user interface for these features.

• cortex-m-quickstart

This crate is used as a Cargo project template for writing Cortex-M applications.

Every now and then a user figures a new way to misfollow the instructions and encounters a new error
message. These errors should be documented in the troubleshooting guide.

---

Regarding organization I think it may make sense to move these core components under the
rust-embedded org and then create, say, a cortex-m team in charge of maintaining the cortex-m
crates. As the AVR, MSP430 and RISCV communities grow they may also want to move core components
into the rust-embedded org and create their own maintainer teams. Thoughts on this?

cc @ryankurte @Emilgardis might be interested in picking up svd2rust development
cc @dylanmckay (AVR) @pftbest (MSP430) @dvc94ch (RISCV) see last paragraph, also svd2rust support

[ryankurte]

I like the idea of moving components into rust-embedded (and increasing the bus factor with that org 😂).

I'd like to see a code of conduct (something line the contributor covenant and the contacts and policy to support it, as well as contribution guidelines (ie. how to pr, policy on breaking changes and versioning, adoption of projects by rust-embedded etc.) at the org level.

Also willing to pick up some of rust2svd/svd-rs(/embedded-hal) development/support/maintenance, and to update japaric/svd#37 to get it merged as a first step in the refactor if that's useful.

[dylanmckay]

@japaric

It's expected that svd2rust will also grow AVR support -- Atmel has its own description file
format: atdf -- using the atdf2svd approach. And there are XSVD (JSON version of SVD files) files
for RISCV -- I don't think they are official though? -- so svd2rust could also grow support for
RISCV via a xsvd2svd tool.

I have already written a library to parse ATDF files into data structures, which should be fairly easy to plug in.

It is [almost being] used in the ruduino library to implement a trait-based AVR-specific HAL. It is being used in another library that defines all registers/IOs for every AVR device.

[dylanmckay]

I like the idea of moving components into rust-embedded (and increasing the bus factor with that org 😂).

Agree strongly. I have mixed feelings on whether this should be done for AVR before it actually gets merged into upstream Rust though.

I've tried to be quite careful to manage expectations with the AVR fork - there's been quite a bit of interest, but the experience is still quite buggy. I think that moving some of the AVR stuff to the rust-embedded org might give off the impression that the AVR experience is well supported, where in fact things such as having optimisations disabled on a nontrivial project make it quite likely to have an LLVM assertion error.

I think there's a lot of value in bringing the embedded projects into one organisation. It'd definitely help the embedded experience become a lot more seamless, especially with strong collaboration on things such as HALs.

[hannobraun]

@japaric I'm happy to help out here and there with the cortex-m crates, but I don't have the bandwidth to drive any major initiatives.

Fixing this requires writing or putting together an SVD file that contains only core peripherals,
running svd2rust over it and committing the code to the cortex-m repo.

I did some digging. I already knew about these incomplete SVD files, so at least there must have been some interest by ARM at some point.

I found this issue, specifically this comment, which states that core SVD files exist now, referring to some Keil pack. I found this download page, and I believe the pack that the comment refers to is "ARM V2M-MPS2 Board Support PACK for Cortex-M System Design Kit" under the "Keil" heading.

I downloaded the file (despite the name it's just a zip file; add ".zip" to the name and you should be able to open it), and yes, it contains SVD files for all the Cortex-Ms. I don't know if they're any good, and I don't know what the EULA says about using them, but I guess at the very least it makes sense to check them out.

Regarding organization I think it may make sense to move these core components under the
rust-embedded org and then create, say, a cortex-m team in charge of maintaining the cortex-m
crates.

👍

[docbrown]

@hannobraun It looks like the SVDs in that pack are for some kind of abstract Cortex-M device. They include peripherals (timer, GPIO, serial engine, etc) that aren't common to all Cortex-M designs. Maybe they're just copied directly from the Cortex-M System Design Kit?

@japaric If a SVD needs to be written from scratch, I can try to put one together directly from the Technical Reference Manuals so that there's no problem with licensing. Unless someone is already doing that.

[hannobraun]

@docbrown Ah, too bad. And I'm guessing we're not allowed to modify it (though I haven't read the EULA).

[dvc94ch]

So moved all riscv crates to a rust-riscv org. Should help find riscv crates, contributing etc. If someone wants to collaborate I can add you to the org.
There is also a riscv-crates repo that creates a cargo workspace to make hacking on riscv crates easier - rust and llvm forks are updated.
Some of these crates could be moved to rust-embedded once they're more mature.

[docbrown]

@hannobraun Yeah, it's kind of a minefield. There's the EULA (which appears to only reference example code and drivers, but IANAL), a 3-clause BSD(?) license from ARM on the SVDs themselves, and then another license from ARM applied to the code generated from the SVD (the licenseText element). And if the SVDs were just copied from the reference design in the Cortex-M System Design Kit, then there's likely a EULA for that, too (which we can assume Keil is in compliance with, but the kit doesn't appear to be publicly available to verify that). Writing an SVD from scratch seems like the safest option, IMO.

[hannobraun]

@docbrown

Writing an SVD from scratch seems like the safest option, IMO.

I'd try to contact ARM first, to try and clarify the license. That might not come to much, but it probably won't hurt either. But if you're willing to write an SVD from scratch, I'm not trying to stop you! Just offering suggestions.

[Emilgardis]

I'd be happy to help maintaining svd2rust. I'm also glad that were taking this direction because the pressure and responsibility has been something I've noticed to be far to big for a one-man team.

[japaric]

@Emilgardis @ryankurte

Thanks for volunteering!

I have just sent you collaborator invites for both japaric/svd and japaric/svd2rust. Those two
are in a worse state than the cortex-m repos and I don't want to block development on those until we
figure out an org structure.

@ryankurte

I'd like to see a code of conduct (..) as well as contribution guidelines (..) at the org level.

Great idea! Any suggestion on how to make it visible at the org level? I don't think you can have
pinned repos at the org level. Though it seems you can have some description text at
https://github.com/$ORG

@dylanmckay

I have already written a library to parse ATDF files into data structures

Awesome! I know that @kunerd and ecstaticmorse_ (IRC nickname) are interested in svd2rust support
for AVR and/or a atdf2svd tool. You should all synchronize efforts.

[scowcron]

@dylanmckay @japaric re: ATDF

atdf2rust really would be nice to have, even for micros in the ATSAM families (Cortex-M). Atmel's SVD files are non-compliant and don't play nicely with svd2rust. I'd love to be able to generate code for those without have to hack svd2rust to ignore problems in the SVDs.

The other significant pain point I had working with svd2rust was lack of register group support, but I see that there's progress on pulling in svd v0.6.0, so I won't harp too much on that.

[kunerd]

@dylanmckay @japaric: ATDF

@ecstatic-morse created this atdf parser repo, which also can parse .atdf files into svd2rust structs, if I remember correctly. From my point of view it would make sense to separate the code generation part from the description file parsing part, so that for example svd2rust and atdf2rust tools can be created by using the same code generation base, either as a lib or by adding different executables to svd2rust. I think I can offer some time to support on this.

@dylanmckay: btw - thanks for the great work on the avr-rust fork