-
Notifications
You must be signed in to change notification settings - Fork 13.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Core 3.1.1 - ESP8266HTTPClient.h getString() empty or truncated - read Timeout #8871
Comments
Are there 10secs or only the default 5secs between these two lines ? |
It's 10 sec 19:23:02.769 -> [ENTS]: HTTP_CODE: 200 |
Can you restart with SSL+HTTP_CLIENT debug option enabled ? |
Same ERROR - Extract of full debug print (For full debug print, se end of this comment
DEBUG extract CODE: Full Serial DEBUG print |
I tried to reproduce this bug without success. |
Thanks for continuing to work on this issue.
Can it be something in the dataproviders data that is not expected or?
I cannot share my Token but use this guide to obtain your own to continue test.
You can register and obtain API token free of charge, and here is the procedure:
1. Go to https://transparency.entsoe.eu/ and click "Login" in upper right corner.
2. A new window opens, click on "Register" at the bottom. Fill in your email and select a password, which must be at least 14 character long and must contain at least one special character (other than letter, digit or '@').
3. You'll receive an email requesting you to confirm activation. Click the link in the email, and you're logged in.
4. So far so good - normal stuff.
Now you need to send them an email to request API token, that information is somewhat hidden in paragraph 2 on this page:<https://transparency.entsoe.eu/content/static_content/Static%20content/web%20api/Guide.html>
Send an email to ***@***.*** with “Restful API access” in the subject line. Indicate the email address you entered during registration in the email body. The ENTSO-E Helpdesk will make their best efforts to respond to your request within 3 working days.
5. They will normally respond next working day, I guess 3 days is to be on the safe side.
When you receive answer from them, there will be a button on your "My account settings" page. Click it to generate API token.
Regards
Thorbjörn
|
I think I'm wrestling with the same issue; I have a simple Get request that pulls back less than 800 bytes. Under v3.0.2 of the library it works fine. Upgrade to the latest (3.1.1) and it's truncating the result (which initially made it look like a JSON parsing error). Reverted back to 3.0.2 and it works fine again. Very basic code doing the work - even ChatGPT can spit this out ;)
Under the 3.0.2 version, that rawJSON variable will hold the complete JSON string. Under 3.1.x the string is truncated or missing entirely. Full source is here: https://github.com/VeryKross/BGBuddy |
context switch race happening? if tcp client is disconnected, tls engine is treated as disabled. if engine is disabled, we are not connected. available() would tell whether remaining data is available. Arduino/libraries/ESP8266WiFi/src/WiFiClientSecureBearSSL.cpp Lines 264 to 267 in 84a59aa
Arduino/libraries/ESP8266WiFi/src/WiFiClientSecureBearSSL.cpp Lines 260 to 262 in 84a59aa
Arduino/libraries/ESP8266WiFi/src/WiFiClientSecureBearSSL.cpp Lines 252 to 258 in 84a59aa
|
Let me know if there is a problem getting the TOKEN for tests. |
Building w/ Not sure if above is related, since we check both available() and connected() in HTTP client code. |
Is the above related to my original issue? Did you get and test with your own TOKEN and the Entsoe server? |
Yes to both. Debug opts above should also dump ClientContext transfers info. |
Can you explain how to use const String& instead of String - do you mean "String payload" should be "const String payload"? Debug data is enourmous and I get Exception(29) - tried to capture some here: 00:11:11.159 -> [ENTS]: Today url:https://web-api.tp.entsoe.eu/api?securityToken=ab4797bd-8010-4078-9325-4f3af043410f&documentType=A44&in_Domain=10Y1001A1001A46L&out_Domain=10Y1001A1001A46L&periodStart=202303202300&periodEnd=202303212300 |
Disable extra stuff like MDNS? :u... logs, and its responder callbacks seem to happen simultaneously with the request. |
So it should be... or? |
const String& payload = http.getString(); |
Code now: with const String& payload and also prints ESP.getFreeHeap() This is repeats as an infinite loop until device crashes after timeout 10s Debug print ---- removed same reallocation repeats..... 17:25:56.165 -> ' ---- removed same reallocation repeats..... 17:26:01.251 -> ' |
After minimizing memory use, the ESP8266 still does not get the data with getString() but it does not crash 16:11:43.323 -> [SYS ]: BOOT>> |
Appreciate if we can get this solved asap, it is causing me lots of trouble. ... or is there another way - maybe a workaround to get extract the price data from Entose? THANKS for getting this solved soon! |
Thanks – I will wait for your green light 😊 once you solved it. I tried also using stream with 128bytes chunks and that works ok. Let me know if I can assist you with more info. |
Hi again, sorry for asking - but how can I get what I need ínto my Arduino IDE 1.8.16 ? |
https://arduino-esp8266.readthedocs.io/en/latest/installing.html#using-git-version |
@TobbeG You can also try the unofficial nightly release for Arduino IDE. It is located at https://d-a-v.github.io/esp8266/Arduino/index.html under version 0.0.2 You can then update timeouts (during handshake or normal operations) also use ssl ram usage optimization in your code as added in this example. I have now no issue reading entsoe data with a sketch similar to the above one. |
Are these fixes part of new core release 3.1.2 ? |
No but there is an esp8266/Arduino nightly release (v0.0.2) which is v3.1.2 + something for you to try, described in my previous post. |
@TobbeG Did you have a chance to try the proposed changes ? |
Not yet. I changed my code to use streaming and it works fine for now. |
Basic Infos
Platform
Settings in IDE
Problem Description
When I make https.GET not the all data received gets read into payload.
ESP8266Http library returms read Timeout
https.setTimeout(10000); does not help
Sending same query in web browser, reply is received within 2 seconds always.
Full XML response is around 3700 bytes, but only part of it is in payload (ie. some 1200 bytes but it varies from 0...3700)
despite that HTTP_CODE is 200 and [HTTP-Client] read chunk len is 3701
CODE:
### Debug printed with myown and HTTP_CLIENT DEBUG
The text was updated successfully, but these errors were encountered: