ESP32-S3 - Random crashes when reading/writing flash (ESPTOOL-608)

[mrdude2478]

Operating System

Windows 10 64 bit

Esptool Version

Release v4.4

Python Version

Python 3.11

Chip Description

ESP32-S3

Device Description

Lilygo-T-Dongle S3

Hardware Configuration

Nothing attached to the board, onboad LED and TFT screen are attached/pre-soldered to GPIO pins.

How is Esptool Run

Windows tool

Full Esptool Command Line that Was Run

python esptool.py -b 460800 --port COM3 --chip esp32s3 read_flash 0x0 0x1000000 T-Dongle-S3.bin

Esptool Output

No response

More Information

Flash read at various baud rates, esptool randomly crashes at various percentages during the read/write cycle. Also during writing the flash seems to have more success flashing using the No Stub option.

Other Steps to Reproduce

Tried reading and writing using these commands.

Backup:
python esptool.py -b 460800 --port COM3 --chip esp32s3 read_flash 0x0 0x1000000 T-Dongle-S3.bin

Flash:
python esptool.py -b 460800 --port COM3 write_flash --flash_freq keep 0x000000 T-Dongle-S3.bin

Using no-stub option, gives more success but takes hours to read and mostly fails trying to read the full 16MB flash.

I Have Read the Troubleshooting Guide

• I confirm I have read the troubleshooting guide.

[radimkarnis]

Hi @mrdude2478,
please try the latest dev version by running pip install esptool==4.5.dev3. The v4.4 version doesn't include the latest improvements for the S3.

[mrdude2478]

Sorry for the late reply, I had lent out the dongle to a friend and just got it back. I am still getting the same random read errors with 4.5.dev3.

Here's the output:

C:\Users\MrDude\AppData\Local\Programs\Python\Python311\Scripts>esptool.py.exe -b 460800 --port COM3 --chip esp32s3 read_flash 0x0 0x1000000 T-Dongle-S3.bin
esptool.py v4.5.dev3
Serial port COM3
Connecting...
Chip is ESP32-S3 (revision v0.1)
Features: WiFi, BLE
Crystal is 40MHz
MAC: f4:12:fa:41:32:c8
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
2965504 (17 %)
A fatal error occurred: Corrupt data, expected 0x1000 bytes but received 0xffc bytes

and another try:

C:\Users\MrDude\AppData\Local\Programs\Python\Python311\Scripts>esptool.py.exe -b 460800 --port COM3 --chip esp32s3 read_flash 0x0 0x1000000 T-Dongle-S3.bin
esptool.py v4.5.dev3
Serial port COM3
Connecting....
Chip is ESP32-S3 (revision v0.1)
Features: WiFi, BLE
Crystal is 40MHz
MAC: f4:12:fa:41:32:c8
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
536576 (3 %)
A fatal error occurred: Corrupt data, expected 0x1000 bytes but received 0xf03 bytes

Note: If flashing via Arduino IDE, I never get an error flashing, but with esptool, I usually get an error, So com port and PC settings are fine.

Regards.

[radimkarnis]

Note: If flashing via Arduino IDE, I never get an error flashing, but with esptool, I usually get an error, So com port and PC settings are fine.

But you don't read flash with Arduino, only write to it - so, does flashing directly with esptool also work?
If not, it's a regression between esptool versions (the one you use bundled with Arduino and v4.5.dev3 you've downloaded manually.)

If writing flash works, we can focus solely on the read_flash operation.
Here are some things to try:

1. Try with the --no-stub argument
2. Try lowering the baud rate, e.g. -b 115200

[mrdude2478]

Thanks, with --no-stub I get more success (I already said this in the first post), but it takes a very long time - over an hour to read, and sometimes crashes with the same errors. I have also tried reading at various baud rates all the way down to 9600. I am aware you don't read flash with Arduino, only write - but I never get any errors writing with Arduino, whereas with esptool I do. If you look at the output I posted I clearly states that I am using "esptool.py v4.5.dev3" as per your advice. I know you are trying to help, I am just pointing out the issues though with ESP32-S3, I know it's a new-ish chip, maybe the issue is something to do with timing during read cycles???

[radimkarnis]

Thanks, with --no-stub I get more success (I already said this in the first post)

Sorry, I overlooked that.

If you look at the output I posted I clearly states that I am using "esptool.py v4.5.dev3"

But I am asking about the version of esptool in Arduino - it actually also uses esptool.py to flash the chip. So if flashing works in Arduino but doesn't when esptool is downloaded directly, there must be a regression between these two versions. If we can locate the version when this regression got introduced, it will be easy to fix.

[mrdude2478]

This has been added to boards manager:
https://raw.githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_index.json

Installed board:
Boards Manager - esp32, Espressif Systems version 2.0.6 Installed

Arduino output says - esptool.py v4.2.1 during flashing.

Does that help?

[radimkarnis]

Arduino output says - esptool.py v4.2.1 during flashing.

Yes, this is what I am asking about. So this means there was a change somewhere between v4.2.1 and v4.4 that is causing flashing/reading to fail in your environment.

In that case, a quick fix for you is to just use esptool v4.2.1 (pip install esptool==4.2.1).

We can try to identify the exact change that introduced the regression. Would you be willing to help us with that?

If yes, you would have to install esptool in development mode. Please, let me know if you can do that.

[mrdude2478]

Thanks for the reply, I installed the latest build in dev mode - this is the output when trying to dump:

Requirement already satisfied: filelock<4,>=3.4.1 in c:\users\mrdude\appdata\roaming\python\python311\site-packages (from virtualenv>=20.10.0->pre-commit->esptool==4.6.dev0) (3.9.0)
Installing collected packages: esptool
  Attempting uninstall: esptool
    Found existing installation: esptool 4.6.dev0
    Uninstalling esptool-4.6.dev0:
      Successfully uninstalled esptool-4.6.dev0
  Running setup.py develop for esptool
Successfully installed esptool-4.6.dev0

C:\msys64\home\MrDude\PC\esptool>python esptool.py -b 460800 --port COM3 --chip esp32s3 read_flash 0x0 0x1000000 T-Dongle-S3.bin
esptool.py v4.6-dev
Serial port COM3
Connecting...
Chip is ESP32-S3 (revision v0.1)
Features: WiFi, BLE
Crystal is 40MHz
MAC: f4:12:fa:41:32:c8
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
81920 (0 %)
A fatal error occurred: Corrupt data, expected 0x1000 bytes but received 0xfde bytes

C:\msys64\home\MrDude\PC\esptool>python esptool.py -b 460800 --port COM3 --chip esp32s3 read_flash 0x0 0x1000000 T-Dongle-S3.bin
esptool.py v4.6-dev
Serial port COM3
Connecting...
Chip is ESP32-S3 (revision v0.1)
Features: WiFi, BLE
Crystal is 40MHz
MAC: f4:12:fa:41:32:c8
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
761856 (4 %)
A fatal error occurred: Corrupt data, expected 0x1000 bytes but received 0xfa5 bytes

C:\msys64\home\MrDude\PC\esptool>

[mrdude2478]

Update: I have uninstalled python, cleaned the registry, removed all python environmental variables, removed all orphaned python files from my AppData/local folder & roaming folders + temp files, carried out a disc cleanup, rebooted my PC and checked to see if anything python related exists - so I have purged my system of python and anything python related.

Then I ran the latest esptool.exe posted yesterday - and the problem still persists even after everything is cleaned???

C:\Users\MrDude\Desktop\lilygo\Flash-dump\tools>esptool -b 460800 --port COM3 --chip esp32s3 read_flash 0x0 0x1000000 T-Dongle-S3.bin
esptool.py v4.5
Serial port COM3
Connecting...
Chip is ESP32-S3 (revision v0.1)
Features: WiFi, BLE
Crystal is 40MHz
MAC: f4:12:fa:41:32:c8
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
5173248 (30 %)
A fatal error occurred: Corrupt data, expected 0x1000 bytes but received 0xf27 bytes

Maybe it's the onboad 16MB flash, it just seems wierd I can always flash successfully with Arduino and the baudrate uses 921600 with that as well, where it fails with esptool.exe.

I will leave python uninstalled for now, and follow instructions from this point.

Also I find it weird why 4.2.1 esptool.py works in Arduino, yet 4.2.1esptool.exe gives this error?

C:\Users\MrDude\Desktop\lilygo\Flash-dump\tools>esptool -b 460800 --port COM3 --chip esp32s3 read_flash 0x0 0x1000000 T-Dongle-S3.bin
esptool.py v4.2.1
Serial port COM3
Connecting....
Chip is ESP32-S3
Features: WiFi, BLE
Crystal is 40MHz
MAC: f4:12:fa:41:32:c8
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.

A fatal error occurred: Invalid head of packet (0x6F): Possible serial noise or corruption.