Increase in binary size with 2.4.0

[xoseperez]

Description

Using a simple blink sketch shows an increment of ~25Kb in binary size. It is problematic when you are running out of space for OTA, for instance, or with 512Kb boards (I know there are not many of these now).

Sample sketch below is 226304 bytes with 2.3.0 and 251168 bytes with 2.4.0. That's 24864 (an 11%) bytes more to do the same.

Anyone else is having this problem? Is this a known "issue"? Is the code base that larger with 2.4.0? Is there a way to reduce binary size with 2.4.0?

Hardware

Hardware: Any
Core Version: 2.4.0 vs 2.3.0

Settings in IDE

Module: Generic ESP8266 Module
Flash Size: 1MB
CPU Frequency: 80Mhz/any
Flash Mode: dout/any
Flash Frequency: 40Mhz/any
Upload Using: N/A
Reset Method: N/A

Sketch

#define PIN 2

void setup() {
    pinMode(PIN, OUTPUT);
}

void loop() {
    digitalWrite(PIN, HIGH);
    delay(1000);
    digitalWrite(PIN, LOW);
    delay(1000);
}

[xoseperez]

A more complex project shows the same problem. Memory usage also gets worst. This is the data for ESPurna framework:

                        |Cache IRAM |Init RAM   |R.O. RAM   |Uninit RAM |Flash ROM  | Binary size
--
2.3.0 - ESPurna Core   	|29789      |900        |2764       |30232      |238595     | 276192
2.4.0 - ESPurna Core 	|27519      |1292       |3536       |30712      |275891     | 312384
--
2.3.0 - ESPurna Default	|30289      |1032       |8856       |31648      |405443     | 449744
2.4.0 - ESPurna Default |27811      |1432       |9612       |32144      |443347     | 486352

[arendst]

Look at the bright side: iram usage is a lot less allowing more use of interrupt driven code.

As for overall code size, mine grew by 22k too but I saw it coming using the rc versions and provide therefore a smaller binary to OTA load first before the full version can be OTA loaded.

It always gets bigger....

[devyte]

@xoseperez the original ESP-01 with 512KB stopped being supported by Espressif a long time ago. Most other repos no longer support it, e.g.: nodemcu froze a release branch for it, and don't support it in newer releases. You just can't fit much on it. It is now usable mostly for static applications, i.e.: no OTA.
Like the previous comment says, the binary size will usually grow due to various reasons, e.g.:

• the SDK may get bigger
• we switched from lwip1.4 to lwip2, which is bigger
• we're using more STL code and templates over C-style pointer handling to favor code robustness over code size
  One of the reasons for the -rc releases is to provide users with an overview of what the release itself will look like, which gives you time to adapt.

I can't think of anything sane that can be done to reduce the binary size. I will leave this issue open for a while in case someone has some out-of-the-box idea, but for the most part, users should have migrated from 512KB ESP01s to the 1MB (at least) ESP01s quite some time ago.

[xoseperez]

@devyte Yes, I don't think anyone is running or pushing new hardware with just 512Kb but there is a lot of 1Mb modules out there and OTA is a must. I noticed the issue with RC but couldn't find a solution at the moment and now that 2.4.0 is out I was looking for ideas. I'm buying @arendst solution, I'm sure he knows what he's doing.

[Raphiki44]

Hello guys
I also noticed this by passing on 2.4.0.
I use ESP12E for a big project and my sketch just went from 290Ko to 330Ko. Not a big deal for lot of people, but a real probleme for me.
I have a SIMCOMM SIM808 GSM module that I use to emit datas trough mobile network with my ESP8266 as main controller.
I use OTA trough the SIM808 to fetch the bin file but the built-in memory of the SIM808 doesn't exceed 300Ko. So I can't fetch the bin file in one time and this cause real problems on my architecture

Is there any way I can make the SDK or other libs lighter ?

[jgrulich]

Not only bigger bin size is the issue of the 2.4.0 version. I've tried to use it for ESP-radio project, which is very system demanding and speed sensitive and found, that 2.4.0 is much, much slower, need more system resources and the WiFi connection is not stable at all.

[earlephilhower]

@Raphiki44 To save pgm space you can try completely disabling asserts (tools->debug level->noasserts). There's also a LWIP 2 patch coming down which will free an add'l ~600 bytes. That'll get you some of the way there, at least.

@jgrulich If you're doing streaming, try the Tools->LWIP Variant->v2 MSS 14xx option or the LWIP 1.4 version. My own audio AAC/MP3 streaming decoder is quite unhappy w/MSS=5xx, but at 1460 in either LWIP 1.4 or 2.0 it seems fine.

To closer pinpoint the differences, I'd suggest doing a readelf -a and then comparing both segment sizes and individual function sizes to see where PMEM is being used in the same sketch in 2.3.0 and 2.4.0.

[jgrulich]

@earlephilhower thank you for the points. I've already tested all the possible combinations of the LwIP variants, but seems that it is not the root-cause of the connection issues, not only, in the EPS-radio project.
It looks me like there is probably some issue in the WiFi library, because whole the connecting behavior is somehow slowed down, it need more than double time to connect to AP and when the stream is connected, the WiFi connection sometimes falls down and hard reset is mandatory, or the data-stream is loosing data.

[earlephilhower]

@Raphiki44 Any update?

I've just tried compiling my largest app ( https://github.com/earlephilhower/psychoplug ) under git head (2.4.0++) and 2.3.0 release and see my generated .BIN files smaller in 2.4.0+lwipv2...

• 2.4.0 + LWIP v2 + MSS 546 = 417,296
• 2.4.0 + LWIP v1.4 Precompiled = 425,264
• 2.4.0 + LWIP v1.4 Opensource = 425,264
• 2.3.0 = 425,248

This code uses the ESP/Arduino TCP and UDP WiFi, HTTPUpdateServer, Wire, and EEPROM. NTP, DNS serving, and other bits are implemented in my own code, so those Arduino libs are not used.

Maybe you could compare the libs you're using in your app and see if there's something different in yours? The bloat could be a single library under esp8266/libraries and not the core under esp8266/core.

[earlephilhower]

#4172 has some contradicting info from @pidloop

Looks like floating point libs are now used by the ESP8266 core, as well as strtod/dtoa routines. Maybe more, but those were the first ones I saw.

[Raphyyy]

Hi and thank you.
I use PlatformIO as IDE.
I set my configuration file as followed here : http://docs.platformio.org/en/latest/platforms/espressif8266.html#using-arduino-framework-with-staging-version
This normally fetch the latest dev Arduino ESP8266 core.

If I downgrade to 2.3.0 (using the repo https://github.com/platformio/platform-espressif8266.git#v1.5.0 which fetch the 2.3.0 of the ESP8266 Arduino core) I get 292Ko.
If I use this repo : https://github.com/platformio/platform-espressif8266.git#feature/stage my bin file grow to 341Ko.

I use many many libs in my project, here are the list :
Arduino.h
Wire.h
EEPROM.h
ESP8266WebServer.h
Ticker.h
SoftwareSerial.h
Time.h
MD5Builder.h
ESP8266HTTPUpdateServer.h
ESP8266WiFi.h
ESP8266HTTPClient.h
ESP8266httpUpdate.h
And many other string/bin io manipulation libs
My source code (lib exclude) is 800Ko sized.

[earlephilhower]

The closed #4172 issue has the details, but for posterity looking at the compilation of sample code that @pidloop posted, the difference doesn't seem to be largely due to the ESP core or libs, but in the inclusion of newlib. Because of the full-featured printf (actually, it's nano-printf, but compared to ets_printf() it's amazing) you're pulling in much more code, including some of the floating point libs because this new printf() includes %f parsing.

@Raphyyy it would be helpful in order to pinpoint differences between 2.3 and 2.4 to do a readelf -a to dump the function names and sizes between the 2 ELF files and attach them.

readelf -a  your-project.elf | sort -r -k3 -n

With that info you can compare side-by-side a) what new functions are linked in 2.4 and if they're relatively large, b) if any older functions blew up in size, and c) where those functions are coming from..

[emontnemery]

@earlephilhower nano-printf can be built with and without float support. can both versions be made available?

[earlephilhower]

@emontnemery , that's a question for @igrr. It's not a big thing to do, but it involves yet another selection in the boards.txt and build process.

There are other bits of newlib. though, beyond printf which will increase flash size so this isn't going to magically decrease the space needed back to 2.3.x days...

[igrr]

Yes, it is possible to add such a switch. Maybe this can be done using boards.txt generator (so that the switch is not shown by default, but one can re-generate boards.txt locally to enable it).

[d-a-v]

Then we would have to versions of the libc ?

[igrr]

With 'nano' newlib it is possible to have just one version of libc, and enable/disable fp formats by adding a linker argument. I'll post the necessary change today.

[igrr]

Essentially, need to remove -u _printf_float -u _scanf_float from compiler.c.elf.flags in platform.txt to disable FP formatting support.

[d-a-v]

I can handle the board generator.

[earlephilhower]

As a data point, in a blank sketch without the printf/scanf_float I see a code savings of ~17K bytes from GIT head.

void setup() {
  printf("test %d\n", 123);
  int a; scanf("%d", &a);
}
void loop() {
}

Default: 258,995
W/o printf_float, scanf_float 241,983

[d-a-v]

What would be the default in platform.txt, with or without floats ?

[igrr]

With, I think, because it produces correct behavior. Trading off correctness for code size should be an option.

[Raphyyy]

Hi and sorry for the late answer.
Here is the output of readelf -a your-project.elf | sort -r -k3 -n on 2.3.0 and 2.4.0 :
out_readelf-2.3.0.log
out_readelf-2.4.0.log

Bin size on 2.3.0 : 292Ko
Bin size on 2.4.0 : 332Ko

I tried to change my platform.txt file with this one : https://github.com/arendst/Sonoff-Tasmota/blob/c290af914c565255864a33c6c33bb82d8e12eea8/arduino/version%202.4.0/platform.txt

But this change nothing, bin size remains the same.

[earlephilhower]

@Raphyyy , you need to restart the ide completely after changing platform.txt for it to read the new settings (i.e. no -u printf_float/etc.) and then rebuild. I can still see printf_float and scanf_float in the 2.4 log you attached, and the largest new functions are related to that (strtod/etc.)

[Raphyyy]

I'm on PlatformIO, I just found out that PlatformIO doesn't care about platform.txt. Any tricks ?
I will try on Arduino IDE tomorrow if I can't find any way.
Thanks

[emontnemery]

@Raphyyy This is working for PlatformIO: arendst/Tasmota#1694
Documentation here: http://docs.platformio.org/en/latest/projectconf/advanced_scripting.html#projectconf-advanced-scripting

Edit: It would be really nice if linker option was --undefined=_printf_float instead of -u _printf_float, then it should be possible to simply do:

[env:xxxx]
build_unflags = --undefined=_printf_float

in platformio.ini instead of this script trickery.

[d-a-v]

@emontnemery are you referring to current master or current pull-request (link) ?

[Raphyyy]

Thanks a lot @emontnemery
So here is my bin size :

• With _printf_float and _scanf_float : 340 272 octets
• Without : 333 648 octets

So I lost ~7Ko (which is nice) but not 17Ko as announced

[d-a-v]

@emontnemery you fooled me :)

xtensa-lx106-elf-gcc: error: unrecognized command line option '--undefined=_printf_float'

[emontnemery]

@d-a-v It's a linker option, not an option for the compiler.

[d-a-v]

-build.float=-u _printf_float -u _scanf_float
+build.float=-Wl,--undefined=_printf_float -Wl,--undefined=_scanf_float

Would that help ?

[d-a-v]

Closing in favour of #3740 and #4478