[strict provenance] Provide a way to "create allocations at a fixed address"

[RalfJung]

This issue is part of the Strict Provenance Experiment - #95228

On some platforms it makes sense to just take a hard-coded address, cast it to a pointer, and starting working with that pointer -- because there are external environment assumptions that say that certain things are provided at certain addresses.

This is perfectly fine for Rust as long as that memory is entirely disjoint from all the memory that Rust understands (static globals, stack allocations, heap allocations). Basically we can think of there being a single hard-coded provenance for "all the memory that is disjoint from the Abstract Machine", and that is the provenance we would like to use for these pointers created from hard-coded addresses. These restrictions make this operation a lot easier to specify than from_exposed_addr. (Remember: from_exposed_addr is outside of Strict Provenance. The goal of this issue is to provide a way to write such code while following Strict Provenance.)

In the spirit of the Strict Provenance APIs, that means we probably want a function that does this, and that we can attach suitable documentation to. There are some open questions for the syntax and semantics of that function:

• What should it be called? make_alloc, assume_alloc, hard_coded_alloc? I am leaning towards something with "alloc" because this function is basically like an allocator, except that you tell it at which address to allocate and you have to promise that that is Okay To Do.
• It should definitely take a usize for the address and return *mut T. Should it also take a size, saying how large this assumed allocation is?
• Should each invocation return a fresh provenance, or should they all return the same provenance? Probably the latter; using a fresh provenance comes with a bunch of restrictions that I doubt such low-level code will be happy about.
• (other things I have not thought of)

Cc @Lokathor who keeps mentioning this usecase every time I want to ban int2ptr casts. ;)
Tagging WG-embedded since that's where this kind of stuff mostly happens (AFAIK)

[RalfJung]

Until such a function exists, one can use from_exposed_addr_mut for this purpose. The "provenance for all things outside the Abstract Machine" is definitely exposed and can hence be picked up by that function.

[Lokathor]

One thing I'd like to mention at the start is that hardware addresses should not be modeled as "owned" by anything inside Rust. They're generally volatile, and generally what you read from them might change without Rust doing anything on its end. At best we should think of them as Rust sharing the address with some external force.

Because of this, I think a name like "alloc" would be a misstep. People have the notion/understanding that when you alloc some memory you own it while it's live, and then you (or some drop glue) explicitly does a "free", and and it's dead. That's not really how hardware access works at all, so for a new situation we pick a new name.

Particularly, lots of existing and well working rust code doesn't bother with a "free" step at all when using hardware pointers. You just make up a pointer that you know is good, you read or write, and then you forget the pointer, and someone else might even be holding a pointer to that same location while you're doing all this, and it's all fine. If a person wants to build an ownership-style API on top of this using the borrow checker they can do that (there's several such examples), but that's separate from the base requirements of the abstract machine interacting with the hardware.

[clarfonthey]

Perhaps directly labelling these addresses as volatile might be the best way to go? This would also help for FFI as well, since you could make the distinction between an address for something that is "owned" inside the Rust code until passed back via FFI, or something that could concurrently be modified outside of Rust, by hardware or another thread.

[RalfJung]

@Lokathor what you say also confirms my thoughts above that we want to use the same provenance for all calls to this function, and not generate a fresh one. That provenance is assumed to be exposed from the start, so it is basically public and can change any time control is given to other code (including via another thread).

Using volatile accesses is a separate concern; Rust assumes for all pointers that if you do 2 non-atomic loads immediately after one another, they will give the same result -- there is no good way to opt-out of that assumption just for a specific provenance; this needs a more explicit marker at the relevant access. (I.e., making it volatile.)

Perhaps directly labelling these addresses as volatile might be the best way to go?

There is no such thing as a volatile address. Being volatile is the property of an access.
If you want to ensure that all accesses to an address are volatile, that's fine, but this can't be automatic or else Rust would have to assume that any raw pointer access might have to be volatile.

There are plenty of discussions around the semantics of volatile, their interaction with reference types, and so on; I would like to keep them out of here. This thread is solely about the name, signature, and provenance interaction of the function that is intended to replace those casts.

[Lokathor]

so it is basically public and can change any time control is given to other code (including via another thread).

Nit: a hardware address can change even without control going to other code.

It could be modeled that the hardware is another thread and that the hardware thread has started before the abstract machine does (similar to how static values are initialized before the abstract machine begins), if that suits you.

There is no such thing as a volatile address. Being volatile is the property of an access.

But that doesn't matter at all? There's no such thing as a wrapping integer, wrapping is a property of an integer operation. And yet we have a "wrapping" type for our integers that makes all the normal operations for the type into the wrapping version. It wouldn't be a world breaker to just have a volatile pointer type (mostly for addresses outside the AM, but not exclusively) where *ptr is a read_volatile or write_volatile operation instead of a standard read or write operation. And then it would be pretty clear in a function signature "oh this pointer isn't the normal kind of pointer".

EDIT: and I don't think that you can talk about hardware addresses usefully without touching on the subject of volatile because the special volatile sauce is how things go from "unsynchronized concurrent access between a program thread and the hardware thread, aka a data race, aka UB" to being "totally normal operation of the hardware".

[RalfJung]

Nit: a hardware address can change even without control going to other code.

I am well aware. That's what volatile is for.

And I think we can very easily not make this the 50th thread about volatile. Just focus on the provenance concern. They are entirely orthogonal. We do not have to co-design this with a volatile pointer type, and you are not helping this cause by tying all these issues in one big knot that is impossible to tackle incrementally. Please focus on one issue at a time.

[RalfJung]

There are plenty of cases where you want access at a particular address that has nothing to do with volatile. For example, you might write a kernel and know that the firmware put some data structure with important info at a hard-coded address.

So, this API certainly should not be tied to volatile in any way. Just like ownership/borrowing, APIs for convenient (and maybe even safe) volatile access can be built on top of this lower-level primitive. That's why I marked the previous posts about volatile as off-topic.

[adamgreig]

I'll raise this at the next embedded WG meeting (tues 28th 20:00 CEST) to see if anyone else has thoughts.

It should definitely take a usize for the address and return *mut T. Should it also take a size, saying how large this assumed allocation is?

I'd think so, we will often have a pointer to a large struct or array or quite possibly an entire memory region that is only for MMIO with fixed addresses (e.g. on Cortex-M, 0x4000_0000 through 0x5FFF_FFFF is all peripheral MMIO).

[RalfJung]

I'd think so, we will often have a pointer to a large struct or array or quite possibly an entire memory region that is only for MMIO with fixed addresses (e.g. on Cortex-M, 0x4000_0000 through 0x5FFF_FFFF is all peripheral MMIO).

And then would you want it to be UB to use that pointer outside the given range? Or what would the exact consequence be of setting a particular size?

[adamgreig]

If it didn't take a size, would you assume a single word provenance or the whole memory, like an escaped pointer? I was imagining you'd want to be able to restrict it to the smallest usable provenance, which is generally known statically or easily bounded to a memory region.

If there's not really any advantage to the compiler to knowing the possible valid range of the pointer, then I suppose just taking the address and giving it a whole-memory provenance is a simpler API.

My guess for a typical embedded use-case is you'd create one of these pointers for each instance of a peripheral on your chip, so each has a non-overlapping size of say <100 words, and probably create them on-demand each time you accessed the peripheral.