Automatically making immutable const data reside in program memory

[gergoerdi]

It would be useful to have a way of marking a string constant (a &'static [u8]?) explicitly to ensure it is put into PROGMEM. For reference, it can be done in C by using the PROGMEM macro on the definition of a const uint8_t[]:

const uint8_t font_rom[] PROGMEM = {
    0xF0, 0x90, 0x90, 0x90, 0xF0, 0x00, 0x00, 0x00
    };

and then you can use pgm_read_byte to read it one by one.

Out of scope: Automatically promoting static values to PROGMEM.

[dylanmckay]

I personally am a fan of this, although here's AVR-GCC's reasoning for not doing this.

Many users bring up the idea of using C's keyword const as a means of declaring data to be in Program Space. Doing this would be an abuse of the intended meaning of the const keyword.

const is used to tell the compiler that the data is to be "read-only". It is used to help make it easier for the compiler to make certain transformations, or to help the compiler check for incorrect usage of those variables.

For example, the const keyword is commonly used in many functions as a modifier on the parameter type. This tells the compiler that the function will only use the parameter as read-only and will not modify the contents of the parameter variable.

const was intended for uses such as this, not as a means to identify where the data should be stored. If it were used as a means to define data storage, then it loses its correct meaning (changes its semantics) in other situations such as in the function parameter example.

[dylanmckay]

Also, for anybody reading, in LLVM all pointers are annotated with address spaces, and so we don't need to use anything special like pgm_read_byte - any load/store through a pointer will automatically use the correct instructions to read from RAM or program memory.

[gergoerdi]

FWIW, currently a &'static [u8] is already compiled into a .text section:

Rust:

static THE_BYTES : &'static [u8] = include_bytes!("../font.img");

LLVM IR:

@byte_str.0 = internal unnamed_addr constant [64 x i8] c"\F0\90\90\90\F0\00\00\00 `  p\00\00\00\F0\10\F0\80\F0\00\00\00\F0\10\F0\10\F0\00\00\00\90\90\F0\10\10\00\00\00\F0\80\F0\10\F0\00\00\00\F0\80\F0\90\F0\00\00\00\F0\10 @@\00\00\00", align 1

AVR asm:

	.type	byte_str.0,@object      ; @byte_str.0
	.section	.rodata,"a",@progbits
byte_str.0:
	.asciz	"\360\220\220\220\360\000\000\000 `  p\000\000\000\360\020\360\200\360\000\000\000\360\020\360\020\360\000\000\000\220\220\360\020\020\000\000\000\360\200\360\020\360\000\000\000\360\200\360\220\360\000\000\000\360\020 @@\000\000"
	.size	byte_str.0, 64

(which ends up in .text).

However, accessing it doesn't go through lpm:

Rust:

    for &x in THE_BYTES {
        spi::sync(x);
    }

LLVM IR (of the relevant part):

  %1 = getelementptr inbounds i8, i8* %0, i16 1
  %2 = load i8, i8* %0, align 1

AVR asm:

        ldi     r26, lo8(byte_str.0)
        ldi     r27, hi8(byte_str.0)
        ldi     r16, lo8(byte_str.0+64)
        ldi     r17, hi8(byte_str.0+64)
LBB3_1:                                 ; %bb6
                                        ; =>This Inner Loop Header: Depth=1
        ld      r24, X+

[dylanmckay]

I think we should be able to do something here, what rules should define a value going into program memory?

I'm thinking that all const values should go into program memory.

Regarding statics, I'm not sure if it'd be safe putting immutable statics into program memory. Statics are supposed to be usable from FFI, and it would be an easy mistake to declare an external static in C that is actually defined in Rust and read from it, expecting it to be in RAM, but it is actually in program memory.

[shepmaster]

AFAIK, Rust's main concern with const vs static is that a static is guaranteed to be able to have its address taken. consts are closer to a C #define in that they may be inlined.

[shepmaster]

what rules should define a value going into program memory?

If we can't come up with a rule, I do think some kind of attribute allowing both opt-in and opt-out would be a good start. If we notice a pattern, that could be encoded in the compiler.

[dylanmckay]

Have made this issue specifically about the automatic promotion of static data to program memory by the Rust compiler itself.

If we want an explicit syntax for it (such as an attribute), that is now at #84

[dylanmckay]

Oh, I also spent some time working on this a while back, it doesn't look like I wrote anything about it.

I started work on modifying the trans crate so that it would mark immutable statics as addrspace(1). I found it to be quite tricky, and I also got a bit confused because by the looks of it, there's two different translation implementations - one for the old compiler middle-end and one for the MIR backend. IIRC there was some shared code between both so it was tricky to find out what to change. The story on that might be better now that some time has passed though.

Doing this won't be too much work, but it's not as trivial as initially thought.

[dylanmckay]

Thinking about this more, we can only do this on devices that support the lpm instruction.

[dylanmckay]

I think it would be a good idea to do this by

• Create a new AVR backend pass, PromoteConstsToProgramMemory. This pass can be explicitly initialized by the frontend, and is not a part of the default pipeline.
• Run it with Rust
• The pass itself should be a NOP if the target MCU does not support lpm
• If the target does support LPM, move all constant globals to address space 1, update all uses to address space 1

This way, the frontend doesn't need to know AVR specific quirks.

[rubberduck203]

I don’t know that I would try to automatically determine where a value should be stored based on static or const. Putting something in PROGMEM seems like the job for an attribute.