STM32 AnalogIn data conversion bug

[aizvorski]

This code appears to be used in the analogin_read_u16() functions in all TARGET_STM32:

    // 12-bit to 16-bit conversion
    value = ((value << 4) & (uint16_t)0xFFF0) | ((value >> 8) & (uint16_t)0x000F);

It is supposed to convert (right-aligned) 12bit to (left-aligned) 16bit. Starting from 0x0123 it produces 0x1231. This seems like a bug, the correct value should be 0x1230.

(Regardless of what the correct value is though, it should probably not duplicate one of the original half-bytes.)

[0xc0170]

cc @bcostm @adustm

[bcostm]

We checked the linearity recently and this formula works as expected. What other formula do you suggest ?

[aizvorski]

@bcostm Okay, I understand the purpose of the formula now. I thought you'd just change the alignment, but you're also expanding the range 0 .. 0xFFF to 0 .. 0xFFFF. It's a pretty good approximation for that, it will be accurate to +/-1 of the correct expansion.

Exploring a little bit:

#include <math.h>
#include <stdio.h>
#include <stdint.h>
void main()
{
    for (uint16_t value = 0; value < 0x1000; value++)
    {
        uint16_t value_converted = ((value << 4) & 0xFFF0) | ((value >> 8) & 0x000F);
        uint16_t value_converted_exact = lround(value * (float)0xFFFF / (float)0xFFF);
        printf("%04x %04x %04x\n", value, value_converted, value_converted_exact);
    }
}

produces output like this:

...
0087 0870 0870
0088 0880 0880
0089 0890 0891
008a 08a0 08a1
...
00fe 0fe0 0fe1
00ff 0ff0 0ff1
0100 1001 1001
0101 1011 1011
...
0198 1981 1981
0199 1991 1991
019a 19a1 19a2
019b 19b1 19b2
...

The plot of the difference between approximate and exact value looks like this:

This is probably close-enough. Sorry, I don't have a better (fast) formula.

[bcostm]

No problem. Thanks for sharing your analysis. Yes I think the actual formula is good enough in term of precision vs code size.