Determine an idiomatic way of sharing or transferring resources between Interrupt and User contexts

[jamesmunns]

See discussion on this thread: rust-embedded/bare-metal#15

Edit: This may end up being more than one solution, particularly "moving" vs "sharing", and for different levels of "sharing" guarantees".

[adamgreig]

I can see at least a few different use cases:

Sharing a variable between the main thread and only one interrupt handler

This is for something like a semaphore the ISR can signal to tell the main thread to perform an action, or some data received by the ISR that it wants to send to the main thread, or the main thread and the ISR sharing a buffer that the main thread fills and the ISR drains. The main thread can never pre-empt the ISR, and since no other ISR shares the variable, it should be possible for the ISR to get direct exclusive non-panicking access. The main thread would need to disable the ISR (but only that one ISR) to get safe exclusive access.

The variable might be statically initialised (in the simplest case), or might have to be early-runtime initialised. We might be able to support statically initialised variables first and only later have a good solution for runtime-initialised: you can always use an Option<T> to bridge the gap.

Sharing a variable between the main thread and one or more interrupt handlers

This is the much harder case where any ISR might access the variable and might pre-empt another ISR, which seems like it leaves us with either whole-program analysis a la RTFM or critical sections a la the current Mutex dance.

Moving a variable from the main thread to one interrupt handler

Main needs to initialise or obtain some variable (typically a peripheral instance) and only the ISR needs to access it. By definition this is not a statically initialised variable but rather something initialised at runtime then moved. Once moved you'd want the ISR to have direct and non-panicking access.

Anything else? I think even just addressing the simplest version of the first use case (statically initialised, shared between main and one ISR only) would be a huge win.

[jamesmunns]

Re: Moving a variable; it would be nice to have a way to move the variable back if we shut down the interrupt. This could be useful when swapping different interrupts throughout the run time, but probably not strictly necessary for a first useful approach. Additionally this approach is useful if you are using something like BBQueue where you have SPSC guarantees already, and you just need to give one or more producers/consumers to the interrupt handler so it can fill/drain events/data as necessary (without a mutex or semaphore).

Re: one or more interrupt handlers; yeah, I think we need the RTFM approach to avoid deadlock guarantees, but it might be useful to have something like Shared for this usecase, with a big red "you must BYO guarantees about deadlock avoidance", or somehow never allow more than one resource handle to be held at any time.

Re main thread and only one interrupt handler; I think you nailed it. I would say one possible interesting item that this point would allow would be something like ping-pong/double buffers, where the main thread can move the buffer from 1->2, and the interrupt can move the buffer from 2->1 in a safe way with shared memory.

[adamgreig]

it might be useful to have something like Shared for this usecase, with a big red "you must BYO guarantees about deadlock avoidance"

Sure. It's already deadlock-free on Cortex-M since Mutex requires a CS which can't be interfered with. Honestly at that point we're not far off just suggesting using static mut and pretending you're dealing with C and people can do their own analysis re pre-emption or deadlocking; no different to what you'd have to do in C. I'd really like to sort out the simpler cases before worrying about replacing what RTFM already does well for large/complicated scenarios.

[jamesmunns]

@adamgreig Ah sorry, you are correct, in the other thread we talked about the case where only one interrupt would be disabled, and I wasn't thinking of the "total critical section" case we currently have.

Honestly I think having something like an arc_singleton!() would handle the first two cases, and should be possible once MaybeUnInit lands in 1.32. (Essentially a Mutex<T> that is statically allocated similarly to the singleton!() macro or the bbq!() macro in BBQueue).

[therealprof]

Actually the current Mutex dance also allows moving stuff out of the Mutex and into a "static" variable in the interrupt handler allowing to protect it from external access without having to lock the resource.

[eddyp]

Honestly I think having something like an arc_singleton!() would handle the first two cases

What would be the underlying hw mechanism on which it should be implemented?

Because if we're talking about single core and there is an interrupt involved, anything else but a way to mask/disable the interrupt doesn't seem to cut it.

/confused

Still, is this supposed to be a mechanism only for single core?

[perlindgren]

Adding to the discussion. We are working on a multicore extension of RTFM (prototype has been up and running and under evaluation for some months already).

The most straightforward approach is to have Tasks and Resources associated to cores by the programmer, allowing shared resources only within a single domain. This might sound too restrictive, but given that message passing is implemented using lock free queues, we only need to ensure that atomicity across cores are enforced. In practice this allows zero cost data propagation between cores. Not exactly sure haw this arc_singleton would fit into this picture though....

As mentioned in other issue (#15), we would be happy to see actual examples where RTFM does not fit Your bill of embedded programming, and from there suggest and/or develop suitable patterns. (And I don't agree that hiding a potentially panicking Mutex behind a newtype would be a better abstraction than the guaranteed race- and deadlock free access you get from RTFM).

Best regards/
Per

[HarkonenBade]

@perlindgren Is there any meaningful way to make RTFM not require as much macro magic? Even if it means driving some upstream development? As the thing that presents me with the most issue from RTFM is the use of macros to generate a new DSL within rust, I would favour something that was more purely expressed in regular rust syntax.

[jamesmunns]

@perlindgren I think the rub here is that I would like to support users who do, and do not use RTFM. I could believe that use of RTFM, or a similar tool which has whole-program visibility, is perhaps to only way to guarantee zero cost overhead towards safe code.

However, I think there is still value in a low-cost, yet safe set of abstractions that could be used outside of the context of RTFM.

If you believe it is only valuable to develop Embedded Rust in the context of RTFM, I might suggest that you submit an RFC making that a primary/official goal of the Working Group.

[jamesmunns]

@eddyp Yeah, my suggestion for arc_singleton!() was perhaps overly bold. What I had in mind was a statically allocated mutex that would be initialized on first access. Sort of like lazy_static!() + the cortex-m::Mutex.

This likely would only be safe across a single core.

[japaric]

Solutions that don't depend on procedural macros, off the top of my head:

A. Signaling from ISR to main

• static _: AtomicBool

  • Upside: multi-core safe (if correct Ordering is used)
  • Downside(correctness): global static variable, anyone can modify it
  • Downside(portability): doesn't work on ARMv6-M, a CAS loop is needed in main

• Signal

  • What: Use bit-banding to pack 32 AtomicBools in a single word.
  • Unclear: is bit-banding multi-core safe? (IDK)
  • Downside(portability): doesn't work on ARMv6-M for the same reasons as AtomicBool

B. Sharing between ISR and main

• static _: spin::RwLock<T>

  • Upside: multi-core safe
  • Upside: doesn't block all interrupts
  • Downside(overhead): unnecessary overhead when used from ISR
  • Downside(portability): doesn't work on ARMv6-M for the same reasons as AtomicBool
  • Downside(correctness): may deadlock
  • Downside(correctness): global static variable

• static _: bare_metal::Mutex<RefCell<T>>

  • Downside(safety): not multi-core safe (it should not even implement Sync)
  • Downside(overhead): unnecessary overhead when used from ISR
  • Downside(correctness): always blocks all interrupts
  • Downside(overhead+correctness): can panic
  • Downside(correctness): global static variable

C. Moving from main to ISR

• Dynamic interrupt handlers

  • What: Move semantics, _: Send requirement
  • Upside: compiler enforced access control / no global access
  • Downside(overhead): Extra memory overhead per handler, plus if let branch
    and dynamic dispatch on each ISR
  • Downside(ergonomics): Need some sort of allocator to move the captures into
    "leaked" memory at runtime (this is equivalent to creating static mut
    variables at runtime)
  • Downside(correctness): allocator can run out of memory (this is a bug
    (programmer error) that should be easy to fix though)

• static _: spin::RwLock<Option<T>>, static _: bare_metal::Mutex<RefCell<Option<T>>>

  • These don't have proper move semantics; they are just runtime initialized
    versions of (B)
  • Extra downside: will panic if one tries to use the static before initializing it

[japaric]

@HarkonenBade

I would favour something that was more purely expressed in regular rust syntax.

RTFM is expressed in regular Rust syntax; attributes, which cortex-m-rt also uses
plenty of, are regular Rust syntax; if you can rustfmt something then it's
regular Rust syntax.

With crate level attributes you can reduce the number of required annotations by
introducing inference; though, that's more macro magic, not less. For example,
moving from main to ISR could look like this:

#![app]

// runtime initialized static
// (you can't get rid of this because there's no static-level type interference)
static mut SERIAL: Serial = (); // or `= UNINIT` (w/e syntax you prefer)

fn init() {
    // ..

    // initialize the static
    SERIAL = Serial::new();
}

fn main() -> ! {
    loop {
        // ..
    }
}

#[interrupt]
fn USART0() {
    // "move" into this ISR (you only get `&mut Serial`, though)
    let serial = SERIAL;

    // do stuff with `serial`
}

Is there any meaningful way to make RTFM not require as much macro magic?

Even if it means driving some upstream development?

The DSL is used to express "before, after" constraints and ownership / sharing
of static variables. These only make sense in the bare metal domain (where
non-reentrant interrupt handlers exist), so a DSL is the right way to express
this, IMO.

As the syntax / features are not general purpose I don't see them ever being
integrated into the language. Unless we are talking about adding some --dsl
flag to rustc; though I don't really see that ever happening either.

[therealprof]

Downside(correctness): always blocks all interrupts

That is not necessarily a downside and certainly not a correctness problem. Also one could move the protected resource into the ISR upon first use if the ISR is supposed to be the exclusive owner to get rid of the critical section if performance really is an issue.

Extra downside: will panic if one tries to use the static before initializing it

I don't have any problems with deterministic panics. Even better would be if the compiler could figure it out and warn about it.

[HarkonenBade]

@HarkonenBade

I would favour something that was more purely expressed in regular rust syntax.

RTFM is expressed in regular Rust syntax; attributes, which cortex-m-rt also uses
plenty of, are regular Rust syntax; if you can rustfmt something then it's
regular Rust syntax.

With crate level attributes you can reduce the number of required annotations by
introducing inference; though, that's more macro magic, not less. For example,
moving from main to ISR could look like this:

#![app]

// runtime initialized static
// (you can't get rid of this because there's no static-level type interference)
static mut SERIAL: Serial = (); // or `= UNINIT` (w/e syntax you prefer)

fn init() {
    // ..

    // initialize the static
    SERIAL = Serial::new();
}

fn main() -> ! {
    loop {
        // ..
    }
}

#[interrupt]
fn USART0() {
    // "move" into this ISR (you only get `&mut Serial`, though)
    let serial = SERIAL;

    // do stuff with `serial`
}

Is there any meaningful way to make RTFM not require as much macro magic?

Even if it means driving some upstream development?

The DSL is used to express "before, after" constraints and ownership / sharing
of static variables. These only make sense in the bare metal domain (where
non-reentrant interrupt handlers exist), so a DSL is the right way to express
this, IMO.

As the syntax / features are not general purpose I don't see them ever being
integrated into the language. Unless we are talking about adding some --dsl
flag to rustc; though I don't really see that ever happening either.

Ok, that syntax is something I feel much more comfortable with, I think i was mostly being thrown off by the weird const stuff in the current version of RTFM.