Skip to content

Espressif: Update to ESP-IDF v5.4.1 #10191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dhalbert opened this issue Mar 27, 2025 · 14 comments · May be fixed by #10267
Open

Espressif: Update to ESP-IDF v5.4.1 #10191

dhalbert opened this issue Mar 27, 2025 · 14 comments · May be fixed by #10267
Assignees
Labels
espressif applies to multiple Espressif chips
Milestone

Comments

@dhalbert
Copy link
Collaborator

Espressif: Update to ESP-IDF v5.4

@dhalbert dhalbert added the espressif applies to multiple Espressif chips label Mar 27, 2025
@dhalbert dhalbert added this to the 10.0.0 milestone Mar 27, 2025
@AbortRetryFail
Copy link

I encountered build problems with the espressif port this morning. It looks like ESP-IDF's tools include overcomplicated fragile Python dependency check scripts. Is this related?

$ make -j6 BOARD=lilygo_tdeck
- Verbosity options: any combination of "steps commands rules", as `make V=...` or env var BUILD_VERBOSE
-- Building ESP-IDF components for target esp32s3
-- Checking Python dependencies...
The following Python requirements are not satisfied:
Requirement 'setuptools<71.0.1,>=21' was not met. Installed version: 78.1.0
To install the missing packages, please run "install.sh"
Diagnostic information:
    IDF_PYTHON_ENV_PATH: /home/abortretryfail/.espressif/python_env/idf5.3_py3.13_env
    Python interpreter used: /home/abortretryfail/.espressif/python_env/idf5.3_py3.13_env/bin/python
Constraint file: /home/abortretryfail/.espressif/espidf.constraints.v5.3.txt
Requirement files:
 - /home/abortretryfail/software/circuitpython/ports/espressif/esp-idf/tools/requirements/requirements.core.txt
Python being checked: /home/abortretryfail/.espressif/python_env/idf5.3_py3.13_env/bin/python
CMake Error at esp-idf/tools/cmake/build.cmake:368 (message):
  Failed to run Python dependency check.  Python: python, Error: 255
Call Stack (most recent call first):
  esp-idf/tools/cmake/build.cmake:502 (__build_check_python)
  esp-idf/tools/cmake/project.cmake:710 (idf_build_process)
  CMakeLists.txt:12 (project)


-- Configuring incomplete, errors occurred!
make: *** [Makefile:559: build-lilygo_tdeck/esp-idf/config/sdkconfig.h] Error 1

A temporary fix was pip install 'cryptography<43' 'setuptools<71.0.1

@eightycc
Copy link
Collaborator

@AbortRetryFail There are known python package version conflicts between CircuitPython and ESP-IDF. In a nutshell, you'll need to run esp-idf/install.sh a second time after you've pip'ed the CircuitPython dependencies. Here's my recipe for building Espressif: https://gist.github.com/eightycc/197a12ffb747e21aa72bdedb2cade818

@eightycc eightycc self-assigned this Apr 8, 2025
@eightycc eightycc changed the title Espressif: Update to ESP-IDF v5.4 Espressif: Update to ESP-IDF v5.4.1 Apr 8, 2025
@eightycc
Copy link
Collaborator

eightycc commented Apr 8, 2025

ESP-IDF 5.4.1 is the current release version. If there's no objection, I'll fit our local commits onto upstream tag v5.4.1.

@tannewt
Copy link
Member

tannewt commented Apr 8, 2025

Yes, please rebase changes on https://github.com/adafruit/esp-idf/tree/circuitpython-v5.3.2 onto a new circuitpython-v5.4.1 branch. I'll invite you as a repo collaborator there.

@eightycc eightycc linked a pull request Apr 17, 2025 that will close this issue
@eightycc
Copy link
Collaborator

ESP-IDF does more strict checking for valid use of CONFIG_ flags. It now checks that all used flags are defined somewhere in the chain of Kconfig files for the target processor. This applies not just to flags that are defined, but also to flags that are commented out. Although CP does not use Kconfig, these checks are still performed. CP config files all pass, except when building for the ESP32H2. CP configures this for a 48 MHz flash spi clock, but according to Kconfig this isn't a valid setting. The valid setting are 16, 32, and 64. Is this a typo in CP or a setting we want?

I've added support to ESP32H2 Kconfig for a 48 MHz clock.

@eightycc
Copy link
Collaborator

eightycc commented Apr 17, 2025

Symbol ESP_ROM_ELF_DIR is not getting set, resulting in CI failure on all ESP boards. Oddly, this error does not occur building on my local machine. This is a known problem, see espressif/esp-idf#15035.

@eightycc
Copy link
Collaborator

The symbol __atomic_test_and_set is missing in ESP-IDF v5.4.1 causing several boards to fail to build. This is a known regression in v5.4: espressif/esp-idf#15167. Will workaround by adding the commit referenced by the issue to branch circuitpython-v5.4.1 of our ESP-IDF fork.

@eightycc
Copy link
Collaborator

eightycc commented Apr 21, 2025

With a DEBUG=1 build flashed onto a Feather Huzzah ESP32, double faulting early in startup:

configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:6592
load:0x40078000,len:16684
load:0x40080400,len:4
load:0x40080404,len:4268
entry 0x40080658
I (29) boot: ESP-IDF v5.4.1-6-gf50ec8ecdb 2nd stage bootloader
I (29) boot: compile time Apr 21 2025 09:39:10
I (29) boot: Multicore bootloader
I (32) boot: chip revision: v3.0
I (35) qio_mode: Enabling default flash chip QIO
I (39) boot.esp32: SPI Speed      : 80MHz
I (43) boot.esp32: SPI Mode       : QIO
I (46) boot.esp32: SPI Flash Size : 4MB
I (50) boot: Enabling RNG early entropy source...
I (54) boot: Partition Table:
I (57) boot: ## Label            Usage          Type ST Offset   Length
I (63) boot:  0 nvs              WiFi data        01 02 00009000 00005000
I (70) boot:  1 otadata          OTA data         01 00 0000e000 00002000
I (76) boot:  2 ota_0            OTA app          00 10 00010000 00200000
I (83) boot:  3 user_fs          Unknown data     01 81 00210000 001f0000
I (89) boot: End of partition table
I (93) esp_image: segment 0: paddr=00010020 vaddr=3f400020 size=4ba18h (309784) map
I (179) esp_image: segment 1: paddr=0005ba40 vaddr=3ff80060 size=0001ch (    28) load
I (180) esp_image: segment 2: paddr=0005ba64 vaddr=3ffb0000 size=045b4h ( 17844) load
I (188) esp_image: segment 3: paddr=00060020 vaddr=400d0020 size=120420h (1180704) map
I (492) esp_image: segment 4: paddr=00180448 vaddr=3ffb45b4 size=00904h (  2308) load
I (493) esp_image: segment 5: paddr=00180d54 vaddr=40080000 size=17d8ch ( 97676) load
I (526) esp_image: segment 6: paddr=00198ae8 vaddr=400c0000 size=00060h (    96) load
I (526) esp_image: segment 7: paddr=00198b50 vaddr=50000ff0 size=00c00h (  3072) load
I (543) boot: Loaded app from partition at offset 0x10000
I (543) boot: Disabling RNG early entropy source...
I (553) cpu_start: Multicore app
I (561) cpu_start: Pro cpu start user code
I (561) cpu_start: cpu freq: 240000000 Hz
I (561) app_init: Application information:
I (561) app_init: Project name:     circuitpython
I (566) app_init: App version:      10.0.0-alpha.2-79-g22519d88b8
I (572) app_init: Compile time:     Apr 21 2025 09:39:03
I (577) app_init: ELF file SHA256:  19ff33c14...
I (581) app_init: ESP-IDF:          v5.4.1-6-gf50ec8ecdb
I (586) efuse_init: Min chip rev:     v0.0
I (590) efuse_init: Max chip rev:     v3.99 
I (594) efuse_init: Chip rev:         v3.0
I (598) heap_init: Initializing. RAM available for dynamic allocation:
I (604) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (609) heap_init: At 3FFBE418 len 00021BE8 (134 KiB): DRAM
I (614) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (620) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (625) heap_init: At 40097D8C len 00008274 (32 KiB): IRAM
Guru Meditation Error: Core  0 panic'ed (Double exception). 

Core  0 register dump:
PC      : 0x40097c3a  PS      : 0x00040536  A0      : 0x80058d65  A1      : 0x3ffe3ad0  
A2      : 0x00000001  A3      : 0x00000003  A4      : 0x00000000  A5      : 0x00000000  
A6      : 0x00000000  A7      : 0x00000000  A8      : 0x00000001  A9      : 0x00000000  
A10     : 0x00000000  A11     : 0x00000003  A12     : 0x3ffe3ad0  A13     : 0x3f41489c  
A14     : 0x3ffb9b64  A15     : 0x3ffae97c  SAR     : 0x0000001f  EXCCAUSE: 0x00000002  
EXCVADDR: 0x3f41488c  LBEG    : 0x4000c46c  LEND    : 0x4000c477  LCOUNT  : 0x00000000  


Backtrace: 0x40097c37:0x3ffe3ad0 0x40058d62:0x3ffe3b30 0x40058d62:0x00000000 |<-CORRUPTED




ELF file SHA256: 19ff33c14

CPU halted.

@dhalbert
Copy link
Collaborator Author

dhalbert commented Apr 21, 2025

With a DEBUG=1 build flashed onto a Feather Huzzah ESP32, double faulting early in startup:

I am getting the same problem on the tip of main for Feather Huzzah32, so this appears not to be a v5.4.1 problem. I'll do a bisect between there and alpha.2, which works OK.

@dhalbert
Copy link
Collaborator Author

dhalbert commented Apr 21, 2025

With a DEBUG=1 build flashed onto a Feather Huzzah ESP32, double faulting early in startup:

I am getting the same problem on the tip of main for Feather Huzzah32, so this appears not to be a v5.4.1 problem. I'll do a bisect between there and alpha.2, which works OK.

It turned out this double faulting crash only occurs with DEBUG=1 on main and 10.0.0-alpha.2 for Feather Huzzah ESP32. It also occurs with a DEBUG=1 build on main and 10.0.0-alpha.2 are fine without DEBUG=1.

@eightycc
Copy link
Collaborator

@dhalbert My earlier thought that ESP-IDF component vfs was not needed was incorrect. After studying the doc and the code I now see that it is used internally by IDF for console and socket support. Debugging is also a learning experience.

Through a process of setting hardware breakpoints, I've narrowed the point of failure down to a binary routine in the ESP32 ROM. The routine itself is __swsetup_r at 0x40058cc8. It is getting invoked by this code in sp_newlib_init_global_stdio:

void esp_newlib_init_global_stdio(const char *stdio_dev)
{
    if (stdio_dev == NULL) {
        _GLOBAL_REENT->__cleanup = NULL;
        _REENT_SDIDINIT(_GLOBAL_REENT) = 0;
        __sinit(_GLOBAL_REENT);
        _GLOBAL_REENT->__cleanup = esp_cleanup_r;
        _REENT_SDIDINIT(_GLOBAL_REENT) = 1;
    } else {
        _REENT_STDIN(_GLOBAL_REENT) = fopen(stdio_dev, "r");
        _REENT_STDOUT(_GLOBAL_REENT) = fopen(stdio_dev, "w");
        _REENT_STDERR(_GLOBAL_REENT) = fopen(stdio_dev, "w");
#if ESP_ROM_NEEDS_SWSETUP_WORKAROUND
        /*
        - This workaround for printf functions using 32-bit time_t after the 64-bit time_t upgrade
        - The 32-bit time_t usage is triggered through ROM Newlib functions printf related functions calling __swsetup_r() on
          the first call to a particular file pointer (i.e., stdin, stdout, stderr)
        - Thus, we call the toolchain version of __swsetup_r() now (before any printf calls are made) to setup all of the
          file pointers. Thus, the ROM newlib code will never call the ROM version of __swsetup_r().
        - See IDFGH-7728 for more details
        */
        extern int __swsetup_r(struct _reent *, FILE *);
        __swsetup_r(_GLOBAL_REENT, _REENT_STDIN(_GLOBAL_REENT));
        __swsetup_r(_GLOBAL_REENT, _REENT_STDOUT(_GLOBAL_REENT));
        __swsetup_r(_GLOBAL_REENT, _REENT_STDERR(_GLOBAL_REENT));
#endif /* ESP_ROM_NEEDS_SWSETUP_WORKAROUND */
    }
}

The second call to __swsetup_r is the failing call.

@dhalbert
Copy link
Collaborator Author

This looks similar to the code in https://github.com/adafruit/nina-fw/blob/dfc0c299719891002a8b99c187c6532f2d1ce8f9/main/sketch.ino.cpp#L64 I mentioned in discord, which I had to turn off because it was causing an early crash. Might be worth looking in esp-idf issues to see if there are similar reports.

@eightycc
Copy link
Collaborator

@dhalbert The blob you reference tickles exactly the same spot: early initialization of /dev/console in the IDF vfs component. This makes me think that the subsequent double fault out of ROM routine __swsetup_r is a secondary effect of damage already done in the earlier fopen. I'm taking another scan through ESP-IDF issues with this in mind.

@eightycc
Copy link
Collaborator

Here's the exact issue, now closed: espressif/esp-idf#7980. I'm now looking for something stale CP's ESP-IDF config.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
espressif applies to multiple Espressif chips
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants