Add menu options for EEPROM size selection #261

dbuezas · 2023-03-30T20:25:29Z

What do you think about this @LaZsolt ?

Relates to #243

dbuezas · 2023-03-30T20:32:04Z

Board setup for 0 EEPROM if the lib is not included

if EEPROM.h is not included, the eeprom emulation registers are never initialised and the board is therefore (I think) setup to emulate zero EEPROM

int lgt_eeprom_size(){
  if ( ECCR & 0x40 ) return  ( 1020 << ( ECCR & 0x3 ) );
  return 0;
}

void setup() {
  Serial.begin(115200);
  Serial.print("lgt_eeprom_size:");
  Serial.println(lgt_eeprom_size());
  delay(1000);
}
void loop() { }

Prints: lgt_eeprom_size: 0
When setting up EEPROM Emulation to 0 KB, compilation shows: Sketch uses 1940 bytes (6%) of program storage space. Maximum is 31744 bytes. (I came up with this value by adding 1024*2 to the old upload.maximum_size)

If the lib is imported

#define USE_LGT_EEPROM_API
#include <EEPROM.h>

void setup() {
  Serial.begin(115200);
  lgt_eeprom_init(EEPROM_SIZE); // this is the constant set through the menu
  Serial.print("lgt_eeprom_size: ");
  Serial.print(lgt_eeprom_size());
  Serial.println();
}

void loop() { }

Prints lgt_eeprom_size: 8160 (or whatever is set in the menu )

dbuezas · 2023-03-30T20:33:13Z

lgt8f/cores/lgt8f/main.cpp

@@ -102,10 +102,6 @@ void lgt8fx8x_init()
 // store ivref calibration value (According to the LGT8F328D databook, if VCAL is rewritten, the value of VCAL1 can be lost.)
 	_VCAL_1_ = VCAL1;

-// enable 1KB E2PROM 


@LaZsolt The EEPROM.h library will set this to 1 by default when imported anyway.

If the lib is not imported, the ECCR register will stay at zero, which means "I won't use the EEPROM"

Incompatible idea. #261 (comment)

dbuezas · 2023-03-30T20:35:48Z

@dwillmore @jg1uaa @jayzakk, your feedback (and that of anybody else) very welcome :)

Also, I need some help writing a test that:

fills the program part of the flash
fills EEPROM
tests if everything is ok and the EEPROM reads correctly and doesn't overwrite the program

Any willing volunteer? :)

dbuezas · 2023-03-30T21:11:10Z

Uhm... I just read in our wiki that the bootloader is stored right after the 2 kb reserved for the EEPROM, this whole thing may be cursed...

I don't find any explicit offset set in platforms.txt though

LaZsolt · 2023-03-31T06:42:50Z

the bootloader is stored right after the 2 kb reserved for the EEPROM

Not true. The bootloader is stored before the 2kB reserved EEPROM space. So

making the EEPROM space configurable via menu would add another 2k

not true.

The factory burned bootloader started at address 0x7400 and the emulated EEPROM start address is 0x7800 which ending at the end of the FLASH.

Ergo for the different emulated EEPROM size require different bootloader.

All in all if the EEPROM size menu exist it is only could reseve ROM space for such a type of EEPROM emulation software which not using the hardware emulation. In this case the bootloader must start at address 0x7C00 and e.g. 1kB emulater EEPROM start at 0x7400

LaZsolt · 2023-03-31T06:45:24Z

Edit:

the bootloader is stored right after the 2 kb reserved for the EEPROM

Not true. The bootloader is stored before the 2kB reserved EEPROM space.

LaZsolt · 2023-03-31T07:49:24Z

Somehow I unmarked the draft attribute this pull request. But I don't know how.

dbuezas · 2023-03-31T08:47:26Z

Somehow I unmarked the draft attribute this pull request. But I don't know how.

it's on the top right. Marked as draft again

dbuezas · 2023-04-01T19:01:44Z

Ok, I'm thinking about closing the PR then.

What if one is using an ISP? There's no bootloader in that case so maybe adding a new board type for it would allow using all the space

SuperUserNameMan · 2023-04-02T08:58:14Z

BTW, I did not follow the impact of the recent updates, but if one of you change the address of the bootloader according to the EEPROM size picked in the menu, please update the https://github.com/dbuezas/lgt8fx/wiki/EEPROM-of-the-LGT8F328p-:-the-important-things-you-need-to-know

SuperUserNameMan · 2023-04-02T09:31:26Z

Uhm... I just read in our wiki that the bootloader is stored right after the 2 kb reserved for the EEPROM, this whole thing may be cursed...

As mentioned by @LaZsolt , when i wrote the wiki, the bootloader was actually stored BEFORE the EEPROM :

bootloader from 0x7400 to 0x77ff (1kB)
EEPROM from 0x7800 to 0x7fFF (2kB)

Extending the EEPROM at runtime would overwrite the bootloader, and might also overwrite a part of the program space.

Setting the EEPROM to 0 by default, would leave 2kB of unused blank space AFTER the bootloader, unless you relocate the bootloader.

Here is something I did not think about when i rewrote the EEPROM library : maybe it is possible to swap the bootloader and the EEPROM.

It is possible to modify the EEPROM library to protect the 2 last 1kB pages in which would be relocated the bootloader.
In consequence, we would waste 1kB (since the bootloader is 1kB long), but the size of the EEPROM could be extended without overwriting the bootloader.
Also, setting the EEPROM to 0 by default would leave 3kB of extra space for the program.

edit : and when the EEPROM is enabled, the unused 1kB page just before (or just after) the bootloader could be used as a emulated-fuse space. (as it is possible to write once into each blank cells without triggering destructive page swap)

dbuezas · 2023-04-02T10:22:06Z

@SuperUserNameMan you are back! :) I sent you Collaborator rights. This allows you to merge PRs and allows ppl in comments to link you even if you are not part of the thread yet.

I did not follow the impact of the recent updates

Nothing related to the bootloader or eeprom except this PR.

maybe it is possible to swap the bootloader and the EEPROM.

It probably is, but this will make this core incompatible with the default bootloader these boards typically ship with. Most likely a bad idea.

Extending the EEPROM at runtime would overwrite the bootloader

I don't understand this: the bootloader is in lower ram than the emulated EEPROM space. Does the EEPROM grow backwards?

I think the only option available would be to make an option to not reserve EEPROM space specifically to be used without the bootloader (i.e while burning via ISP). WDYT?

SuperUserNameMan · 2023-04-02T11:02:03Z

@SuperUserNameMan you are back! :) I sent you Collaborator rights. This allows you to merge PRs and allows ppl in comments to link you even if you are not part of the thread yet.

My availability might be unreliable for major contribution, though i might still be of use regarding EEPROM questions as long as my memory is not outdated.

maybe it is possible to swap the bootloader and the EEPROM.

It probably is, but this will make this core incompatible with the default bootloader these boards typically ship with. Most likely a bad idea.

~~Do we have access to the bootloader source-code ?~~
~~Isn't the bootloader burned at each sketch upload ?~~
(edit : indeed these two questions show how rusty i became ...)

Extending the EEPROM at runtime would overwrite the bootloader

I don't understand this: the bootloader is in lower ram than the emulated EEPROM space. Does the EEPROM grow backwards?

Yes, it is extended backwards so that everything remains within the 32kB memory space.

I think the only option available would be to make an option to not reserve EEPROM space specifically to be used without the bootloader (i.e while burning via ISP). WDYT?

Yes, i think you could set the size of the EEPROM to 0 by default, so when the sketch is uploaded using an ISP, the whole 32KB of flash is available for the sketch.

And when uploading a sketch using the bootloader, setting the size of the EEPROM to 0 by default should have no impact other than wasting 2kB of space at the end of the flash. (however, you might want to check if it does not interfere with the initialization code of the EEPROM library)

SuperUserNameMan · 2023-04-02T11:17:27Z

I don't understand this: the bootloader is in lower ram than the emulated EEPROM space. Does the EEPROM grow backwards?

Do you mean the bootloader is copied into the RAM ?

LaZsolt · 2023-04-02T11:51:02Z

The bootloader runs from flash started at address 0x7400. From these address the code can be read with the pgm_read_byte().

But the bootloader code also readabe from RAM address started at 0xB400.

LaZsolt · 2023-04-02T12:01:59Z

Crazy idea, but if the EEPROM emulated by software it could be jump up the bootloader code.

Another crazy idea, for that case when the EEPROM size is 0, is that define a fixed adressable read only 1k data block with starting address is the same as the bootloader then the compiler could avoid to put any code here, but the last 2k became useable.

dbuezas · 2023-04-02T12:39:59Z

Very interesting!
I don't see any offsets in the upload scripts, so it must be the bootloader the one that adds the offset.
If I'm correct, it means that if one uses EEPROM in a sketch uploaded via ISP, the source code would ve overwritten. Right?

SuperUserNameMan · 2023-04-02T12:59:16Z

If I'm correct, it means that if one uses EEPROM in a sketch uploaded via ISP, the source code would ve overwritten. Right?

Unless the compiler is aware of the maximum available memory, yes.

SuperUserNameMan · 2023-04-02T13:17:54Z

Crazy idea, but if the EEPROM emulated by software it could be jump up the bootloader code.

Yes, i was thinking about the same.
I think it is possible to tell to the EEPROM library to avoid the two 1kB pages encompassing the bootloader. (as they work in pair, we must skip 2kB (edit : the EEPROM libray should just skip the before last 1KB page, and the hardware EEPROM module will convert it to a 2KB skip. See next messages bellows )

We then would have EEPROM with theoretical sizes of 1KB, 3KB or 7KB.

number of 1KB pages of the EEPROM	theoretical size of the EEPROM	actual available bytes and emulated continuous space	occupied space into main Flash memory	beginning address into Flash memory
0	0	0	0	n/a
1	1024	1020	2048	0x7800
2	n/a	n/a	n/a	0x7000
3	3072	3060	8192	0x6000
7	7168	7140	16384	0x4000
default	1024	1020	2048	0x7800

Instead of :

number of 1KB pages of the EEPROM	theoretical size of the EEPROM	actual available bytes and emulated continuous space	occupied space into main Flash memory	beginning address into Flash memory
0	0	0	0	n/a
1	1024	1020	2048	0x7800
2	2048	2040	4096	0x7000
4	4096	4080	8192	0x6000
8	8192	8160	16384	0x4000
default	1024	1020	2048	0x7800

Another crazy idea, for that case when the EEPROM size is 0, is that define a fixed adressable read only 1k data block with starting address is the same as the bootloader then the compiler could avoid to put any code here, but the last 2k became useable.

If it is possible to tell the compiler to skip the bootloader, that's a good idea !

SuperUserNameMan · 2023-04-02T13:24:08Z

Oh wait, I think i need to refresh my memory about how the EEPROM is emulated before misleading you on that topic.

SuperUserNameMan · 2023-04-02T13:48:18Z

Well if you want to skip the bootloader, you just have to hack :
uint16_t lgt_eeprom_continuous_address_to_real_address( uint16_t address )

This function is used to recalculate the EEPROM address so that the library automatically skips the last 4 bytes of each 1KB pages of EEPROM.
It is thus possible to tell it to skip the before last 1KB page of the EEPROM.

Note that the EEPROM library is not aware of the hardware page-swap algorithm which consume 2 Flash pages per 1KB EEPROM page.

Also note that the EEPROM library does NOT think in term of the Flash memory space : it does not care about the actual address into the Flash memory, because it is handled by the hardware EEPROM module which will convert the EEPROM space addresses into Flash space addresses.

So when the EEPROM library will skip the before last 1KB page of its EEPROM space, the hardware module will actually skip 2KB (2 pages) of the Flash memory space (from 0x7000 to 0x7fFF ). Which should preserve the bootloader (and leave 1KB of unused flash memory)

wollewald · 2023-07-01T09:37:30Z

Hi, I have seen this is closed. I have read it, but still I do not understand if there is a way to get completely rid of the EEPROM in order to use the full 32 KB flash for program code.

LaZsolt · 2023-07-01T16:00:54Z

I think if the program uploaded using an ISP then all the 32 KB flash will be available for program code.

wollewald · 2023-07-01T16:25:46Z

Thanks for the reply.

I tried uploading via LArduinoISP, it does not seem to change anything:

I also tried to allocate 0 KB for the EEPROM using:

ECCR = 0x80;
ECCR = 0;

It had no effect. Same with:

lgt_eeprom_init(0);

LaZsolt · 2023-07-01T16:51:13Z

The displayed size of uploadable code size comes from here:

lgt8fx/lgt8f/boards.txt

Line 19 in df986b5

328.upload.maximum_size=29696

It can be exceeded but we get a warning.

wollewald · 2023-07-01T17:15:29Z

I am used from the AVR Arduinos that this max. value is flexible. E.g. if you use the Optiboot bootloader the value is smaller compared to what you get when you don't use it. If it's just a fixed value which uses the default setting of 1 KB EEPROM this explains my confusion. Displaying the real max available flash would be helpful.
Many thanks!!!!

dbuezas · 2023-07-01T17:28:40Z

So displaying the full 32KB? We can do that if the community agrees

wollewald · 2023-07-01T18:10:04Z

The main reason for my question is that some people asked if they could use the Arduino Optiboot for the LGT8F328P, because they think it's smaller than the bootloader for the LGT8F328P. Like me, they just at value you get during upload.

LaZsolt · 2023-07-01T18:26:07Z

So displaying the full 32KB? We can do that if the community agrees

I think displaying the real available flash will not be easy because uploadig with ISP cannot be detected. I will study the Arduino core solution.

An Optiboot support wish for LGT8F328P: Optiboot/optiboot#359
The current LGT8Fx's bootloader version is made from a very old Optiboot version.
I think if someone could make the short version of the Optiboot for LGT8Fx, then it could be bigger than the ATmega version because of different flash programmig method.

wollewald · 2023-07-01T18:43:22Z

Just to be sure that I have understood it correctly: it's just the bootloader that takes ~ 3KB and this does not include the 2 KB which are needed for the 1 KB EEPROM?

And thanks for the link!

LaZsolt · 2023-07-01T19:16:33Z

The LGT8F328P bootloader less than 1 kB.

wollewald · 2023-07-02T09:03:02Z

< 1KB - that's great.

If it's difficult to implement a method to display the exact flash which is available, then some documention how the 29696 bytes are calculated and under which conditions this value applies would be helpful. I am by far not so deep in this matter like you, but still I would say I am not an absolute beginner. I simply thought I did something wrong when I changed the size of the EEPROM and the max available flash size didn't change. Therefore, there are probably quite a number of people who have similar problems. I also have not yet finally understood if there is flash reserved for the EEPROM by default, even if I do not include EEPROM.h. From what I read now I would assume that there are roughly 31 KB Flash available if I don't use the EEPROM and 29696 bytes if I use 1 KB of EEPROM. Sorry, to bother you with all these questions, but would really like to understand this.

dbuezas · 2023-07-02T11:04:57Z

From the wiki article about EEPROM written by @SuperUserNameMan, the flash layout is:

[0, Program..., ...bootloader..., ...EEPROM, 32000]

Trying to use more than 1k of eeprom (taking 2kb of flash) will overwrite the bootloader, because the eeprom "grows backwards".

This means that the only way to have bigger programs is to forego the bootloader (upload via isp).

wollewald · 2023-07-02T11:34:55Z

Hi, this is helpful. Sorry to be so tenacious, but there is still one thing I am not sure of:

"By default, the size of this emulated EEPROM is set to 1KB, and it can be changed at run-time to 0KB, 1KB,2KB, 4K or 8KB."

Does the default (1KB EEPROM) only apply if I include EEPROM.h? In other words: If I use the bootloader and I don't include EEPROM.h, will no flash be reserved for the EEPROM and therefore I should have ~31KB flash?

LaZsolt · 2023-07-02T13:50:37Z

Displaying the real max available flash would be helpful.

I checked the AVR Arduino core.

It looks like no possibile way to displaying different available flash size when uploading with ISP or with bootloader.
Normally you don't want to overwrite the bootloader when using ISP for upload so the available flash size is the same as in normal case. 29696 bytes.
(Using ISP for upload there is possibilities to overwrite the bootloader and the emulated EEPROM space. In this case the full flash size available for program, but you get a warning message.)
The emulated EEPROM space is always reserved regardless of whether it is used or not. In other words the bootloader won't jump to the end of flash when 0 KB EEPROM is set in runtime.
(The reserved emulated EEPROM space is at the end of program flash memory. The bootloader precedes the reserved emulated EEPROM space.)

So in my opinion no need to change the displayed maximum upload size.

In summary: The bootloader stored in a fixed location in program flash memory followed by emulated EEPROM space. If EEPROM space changed bigger than 1 KB the bootloader will be overwritten after writing some bytes to the EEPROM spaces. If someone want to use bigger (or smaller) EEPROM space the bootloader must be relocated. This relocation method is not yet invented.

wollewald · 2023-07-02T14:23:10Z

Very clear - that answers my question perfectly. Thank you!

The other good news: no more questions from side! Thanks for your patience.

dbuezas · 2023-07-02T15:32:51Z

So to use the full 32kb:

Make a program that uses 32Kb
Upload via ISP
Ignore warnings
Don't attempt to write anything to EEPROM

Right?

LaZsolt · 2023-07-02T15:37:11Z

Yes.

dwillmore · 2023-07-02T16:00:26Z

And that goes for ATMEL and LGT?

…

On Sun, Jul 2, 2023 at 11:37 AM LaZsolt ***@***.***> wrote: Yes. — Reply to this email directly, view it on GitHub <#261 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACPEX7FOOJZGR2AFE3XOOTDXOGITDANCNFSM6AAAAAAWNZA6DY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

dbuezas · 2023-07-02T16:34:11Z

No, ATMEL has real EEPROM independent of flash

Add menu options for EEPROM size selection

f72beac

dbuezas commented Mar 30, 2023

View reviewed changes

dbuezas requested review from LaZsolt, jayzakk and dwillmore March 30, 2023 20:35

dbuezas marked this pull request as draft March 30, 2023 20:37

dbuezas assigned dbuezas and unassigned dbuezas Mar 30, 2023

dbuezas mentioned this pull request Mar 30, 2023

EEPROM library on logging principle (from STM32) #254

Open

LaZsolt marked this pull request as ready for review March 31, 2023 07:03

dbuezas marked this pull request as draft March 31, 2023 08:47

dbuezas closed this Apr 2, 2023

dbuezas reopened this Apr 2, 2023

dbuezas closed this Apr 15, 2023

dbuezas deleted the Menu/eeprom_size branch June 14, 2023 07:25

dbuezas mentioned this pull request Jul 31, 2023

LGT8F328P program size #295

Open

Add menu options for EEPROM size selection #261

Add menu options for EEPROM size selection #261

Conversation

dbuezas commented Mar 30, 2023

dbuezas commented Mar 30, 2023

Board setup for 0 EEPROM if the lib is not included

If the lib is imported

dbuezas Mar 30, 2023

Choose a reason for hiding this comment

LaZsolt Mar 31, 2023 • edited Loading

Choose a reason for hiding this comment

dbuezas commented Mar 30, 2023

dbuezas commented Mar 30, 2023

LaZsolt commented Mar 31, 2023 • edited Loading

LaZsolt commented Mar 31, 2023

LaZsolt commented Mar 31, 2023

dbuezas commented Mar 31, 2023

dbuezas commented Apr 1, 2023

SuperUserNameMan commented Apr 2, 2023

SuperUserNameMan commented Apr 2, 2023 • edited Loading

dbuezas commented Apr 2, 2023

SuperUserNameMan commented Apr 2, 2023 • edited Loading

SuperUserNameMan commented Apr 2, 2023

LaZsolt commented Apr 2, 2023

LaZsolt commented Apr 2, 2023

dbuezas commented Apr 2, 2023

SuperUserNameMan commented Apr 2, 2023

SuperUserNameMan commented Apr 2, 2023 • edited Loading

SuperUserNameMan commented Apr 2, 2023

SuperUserNameMan commented Apr 2, 2023 • edited Loading

wollewald commented Jul 1, 2023

LaZsolt commented Jul 1, 2023

wollewald commented Jul 1, 2023

LaZsolt commented Jul 1, 2023

wollewald commented Jul 1, 2023

dbuezas commented Jul 1, 2023

wollewald commented Jul 1, 2023

LaZsolt commented Jul 1, 2023

wollewald commented Jul 1, 2023

LaZsolt commented Jul 1, 2023

wollewald commented Jul 2, 2023

dbuezas commented Jul 2, 2023 • edited Loading

wollewald commented Jul 2, 2023

LaZsolt commented Jul 2, 2023

wollewald commented Jul 2, 2023

dbuezas commented Jul 2, 2023

LaZsolt commented Jul 2, 2023

dwillmore commented Jul 2, 2023 via email

dbuezas commented Jul 2, 2023

LaZsolt Mar 31, 2023 •

edited

Loading

LaZsolt commented Mar 31, 2023 •

edited

Loading

SuperUserNameMan commented Apr 2, 2023 •

edited

Loading

SuperUserNameMan commented Apr 2, 2023 •

edited

Loading

SuperUserNameMan commented Apr 2, 2023 •

edited

Loading

SuperUserNameMan commented Apr 2, 2023 •

edited

Loading

dbuezas commented Jul 2, 2023 •

edited

Loading