Uploading fails on macOS Big Sur

[jaredrcleghorn]

Basic Infos

• This issue complies with the issue POLICY doc.
• I have read the documentation at readthedocs and the issue is not addressed there.
• I have tested that the issue is present in current master branch (aka latest git).
• I have searched the issue tracker for a similar issue.
• If there is a stack dump, I have decoded it.
• I have filled out all fields below.

Platform

• Hardware: ESP-12
• Core Version: 2.7.4
• Development Env: Arduino CLI
• Operating System: macOS Big Sur

Settings in IDE

• Module: NodeMCU 1.0 (ESP-12E Module)
• Flash Mode: ?
• Flash Size: 4 MB
• lwip Variant: v2 Lower Memory
• Reset Method: ?
• Flash Frequency: ?
• CPU Frequency: 160 MHz
• Upload Using: SERIAL
• Upload Speed: 115200

Problem Description

Uploading fails on Big Sur. This seems to be due to the fact that this core depends on Python 3.7.2-post1; Big Sur support was not added until 3.9.1. Specifically, the ctypes.util.find_library function does not work in previous version of Python on Big Sur, and pyserial uses this function to import the IOKit and CoreFoundation libraries. You can confirm that this is the source of the problem by observing that the following change to tools/pyserial/serial/tools/list_ports_osx.py fixes the issue:

# hotfix for Big Sur
#iokit = ctypes.cdll.LoadLibrary(ctypes.util.find_library('IOKit'))
#cf = ctypes.cdll.LoadLibrary(ctypes.util.find_library('CoreFoundation'))
iokit = ctypes.cdll.LoadLibrary('/System/Library/Frameworks/IOKit.framework/IOKit')
cf = ctypes.cdll.LoadLibrary('/System/Library/Frameworks/CoreFoundation.framework/CoreFoundation')

(I did not come up with this fix; I found it on the Arduino Forum: https://forum.arduino.cc/index.php?topic=702144.0.) Is upgrading to Python 3.9.1 possible/feasible?

MCVE Sketch

Here is the sketch that I am trying to upload, but this problem seems entirely sketch-independent:

#include <ESP8266WebServer.h>
#include <ESP8266WiFi.h>

ESP8266WebServer server(80);

void setup() {
	// Set up a soft access point.
	WiFi.softAP("Auto Pet Feeder");

	server.on("/", [] { server.send(200, "text/plain", "hello, world"); });

	// Start the server.
	server.begin();
}

void loop() {
	server.handleClient();
}

Debug Messages

Waiting for upload port...
"/Users/jaredcleghorn/Library/Arduino15/packages/esp8266/tools/python3/3.7.2-post1/python3" "/Users/jaredcleghorn/Library/Arduino15/packages/esp8266/hardware/esp8266/2.7.4/tools/upload.py" --chip esp8266 --port "/dev/cu.usbserial-0001" --baud "115200" ""  --before default_reset --after hard_reset write_flash 0x0 "/var/folders/_t/k4768zfs5s36r1prntr_rh_w0000gn/T/arduino-sketch-D7C198CF93F1C3CEBC6240B2A283108F/auto-pet-feeder.ino.bin"
pyserial or esptool directories not found next to this upload.py tool.
Error during Upload: uploading error: uploading error: exit status 1

[earlephilhower]

Can you please give the latest git version a try and report back? We've update pyserial and esptool, both which might be able to compensate for Apple breaking serial ports w/o having to find a new portable Python3.

[jaredrcleghorn]

Looks like this was indeed fixed in pySerial 3.5. I have not been able to successfully install the core using git version with the Arduino CLI, but I will install the IDE, try the latest git version, and see if it works.

[jaredrcleghorn]

I followed the instructions in the docs to install the latest git version, but I am getting the following error when I try to compile:

xtensa-lx106-elf-g++: error: unrecognized command line option '-std=gnu++17'

Any idea how to fix this? Seems related to 979bbfe.

[jaredrcleghorn]

Never mind, looks like it was being caused by the Boards Manager version also being installed. I can confirm that uploading works in the latest git version. 👍

[wjm0913]

@jaredrcleghorn Hello, I have the same problem as you. Can you tell me how to operate it？

Thank you very much！

[jaredrcleghorn]

@wjm0913 Are you using the Arduino CLI or IDE? And how did you install the core?

[wjm0913]

@jaredrcleghorn
I am a beginner, not too understanding！ IDE ！！

[wjm0913]

@jaredrcleghorn
Thank you, I have found a solution on the forum!
https://forum.arduino.cc/index.php?topic=702144.0

[MRezaNasirloo]

I have the same issue, what is the correct way to fix this, I've installed everything from Arduino IDE

[Peter8200]

This is not working for me. I have Big Sur version 11.3.1 but I don't have the "/System/Library/Frameworks/IOKit.framework/IOKit" directory installed. And no idea why this is not on my disk. Does anybody have an idea how to have this directory installed ? Is this related to the latest version ? I need to be able to upload again to the esp01.