Service error in Home Assistant if more than 2 segments are added

[syndikatss]

What happened?

On version wled 0.14, when adding more than 2 segments to the home assistant while turning on the segments, a service error appears (Could not call the light/turn_on service. unexpected character: line 1 column 851 (char 850)

To Reproduce Bug

Add more than 2 segments

Expected Behavior

Operation of more segments

Install Method

Binary from WLED.me

What version of WLED?

WLED 0.14.0

Which microcontroller/board are you seeing the problem on?

ESP8266

Relevant log/trace output

Logger: homeassistant.components.websocket_api.http.connection
Source: components/websocket_api/commands.py:226
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 21:43:25 (46 occurrences)
Last logged: 22:44:35

[139642857590848] invalid literal: line 1 column 854 (char 853)
[139642857590848] unexpected character: line 1 column 855 (char 854)
[139642117643584] unexpected end of data: line 1 column 1097 (char 1096)
[139642117643584] unexpected character: line 1 column 852 (char 851)
[139642130458816] unexpected character: line 1 column 851 (char 850)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 226, in handle_call_service
    await hass.services.async_call(
  File "/usr/src/homeassistant/homeassistant/core.py", line 2012, in async_call
    response_data = await coro
                    ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2049, in _execute_service
    return await target(service_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 235, in handle_service
    return await service.entity_service_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 876, in entity_service_call
    response_data = await _handle_entity_call(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 948, in _handle_entity_call
    result = await task
             ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/light/__init__.py", line 591, in async_handle_light_off_service
    await light.async_turn_off(**filter_turn_off_params(light, params))
  File "/usr/src/homeassistant/homeassistant/components/wled/helpers.py", line 28, in handler
    await func(self, *args, **kwargs)
  File "/usr/src/homeassistant/homeassistant/components/wled/light.py", line 208, in async_turn_off
    await self.coordinator.wled.segment(
  File "/usr/local/lib/python3.11/site-packages/wled/wled.py", line 515, in segment
    await self.request("/json/state", method="POST", data=state)
  File "/usr/local/lib/python3.11/site-packages/backoff/_async.py", line 151, in retry
    ret = await target(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/wled/wled.py", line 211, in request
    response_data = await response.json()
                    ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/aiohttp_client.py", line 72, in json
    return await super().json(*args, loads=loads, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1120, in json
    return loads(stripped.decode(encoding))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
orjson.JSONDecodeError: unexpected character: line 1 column 851 (char 850)

Anything else?

Sorry if the text looks stupid, I wrote it through a translator

Code of Conduct

• I agree to follow this project's Code of Conduct

[blazoncek]

Can you provide the JSON HomeAssistant is getting?

[syndikatss]

Can you provide the JSON HomeAssistant is getting?

Tell me how to do it?

[blazoncek]

Use Wireshark.
BTW if possible upgrade ESP to ESP32 as ESP8266 struggles with RAM (and HomeAssistant connectivity).

[d2inco]

Having same issue; one WLED setup with a single segment works fine. The other two WLED setups that both have 3 segments (plus the Main conrol) exhibit the same/similar problem when turning on/off, of
orjson.JSONDecodeError: unexpected end of data: line 1 column 1056 (char 1055)
I do not see a way to turn on debugging to get the JSON to be captured within home assistant (nor do i see exactly which command it was sent so that i could try to do it myself with a CURL command); side comment: it would be nice if the HA integration provided a debug toggle.

Besides Wireshark, which isn't feasible here, is there another method to get HA to dump the JSON and/or commands?

[blazoncek]

it would be nice if the HA integration provided a debug toggle.

That's a request for HA team.

[saruter]

Hi,
i have the same problem since updated to wled 0.14 and using Homeassistant. I did some wireshark dumps and it seems that the JSON-Response from the wled-device ist not complete. The json in the response from the wled device is not closed properly.

Example:
HomeAssistant turns off a WLED Device with 0.14 with 4 Segments:

POST /json/state HTTP/1.1
Host: 192.168.179.172
User-Agent: HomeAssistant/2023.9.3 aiohttp/3.8.5 Python/3.11
Accept: application/json, text/plain, */*
Accept-Encoding: gzip, deflate
Content-Length: 21
Content-Type: application/json

{"on":false,"v":true}HTTP/1.1 200 OK
Content-Length: 1436
Content-Type: application/json
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: *
Access-Control-Allow-Headers: *
Connection: close
Accept-Ranges: none

{"on":false,"bri":81,"transition":7,"ps":-1,"pl":-1,"nl":{"on":false,"dur":60,"mode":1,"tbri":0,"rem":-1},"udpn":{"send":false,"recv":true,"sgrp":1,"rgrp":1},"lor":0,"mainseg":0,"seg":[{"id":0,"start":0,"stop":28,"len":28,"grp":1,"spc":0,"of":0,"on":true,"frz":false,"bri":255,"cct":127,"set":0,"n":"unten","col":[[255,234,219,226],[0,0,0,0],[0,0,0,0]],"fx":97,"sx":255,"ix":225,"pal":32,"c1":128,"c2":128,"c3":16,"sel":true,"rev":false,"mi":true,"o1":false,"o2":false,"o3":false,"si":0,"m12":0},{"id":1,"start":28,"stop":76,"len":48,"grp":1,"spc":0,"of":0,"on":true,"frz":false,"bri":255,"cct":127,"set":0,"n":"rechts","col":[[255,234,219,226],[0,0,0,0],[0,0,0,0]],"fx":97,"sx":255,"ix":225,"pal":32,"c1":128,"c2":128,"c3":16,"sel":true,"rev":false,"mi":true,"o1":false,"o2":false,"o3":false,"si":0,"m12":0},{"id":2,"start":76,"stop":104,"len":28,"grp":1,"spc"":1,"spc":0,"of":0,"on":true,"frz":false,"bri":255,"cct":127,"set":0,"n":"oben","col":[[255,234,219,226],[0,0,0,0],[0,0,0,0]],"fx":97,"sx":255,"ix":225,"pal":32,"c1":128,"c2":128,"c3":16,"sel":true,"rev":false,"mi":true,"o1":false,"o2":false,"o3":false,"si":0,"m12":0},{"id":3,"start":104,"stop":152,"len":48,"grp":1,"spc":0,"of":0,"on":true,"frz":false,"bri":255,"cct":127,"set":0,"n":"links","col":[[255,234,219,226],[0,0,0,0],[0,0,0,0]],"fx":97,"sx":255,"ix":225,"pal":32,"c1":128,"c2":128,"c3":16,"sel":true,"rev":false,"mi":true,"o1":false,"o2":false,"o3":false,"si":0,"

As you can see there is no proper closing of the json-string in the response of the wled-device which I think leads to the error messages in Home Assistant beause it is not a valid json response which gets parsed in Home Assistant.

Example of a WLED Device with one segment and version 0.13.3 which does not generate error messages in Home Assistant:

POST /json/state HTTP/1.1
Host: 192.168.179.142
User-Agent: HomeAssistant/2023.9.3 aiohttp/3.8.5 Python/3.11
Accept: application/json, text/plain, */*
Accept-Encoding: gzip, deflate
Content-Length: 37
Content-Type: application/json

{"seg":[{"on":true,"id":0}],"v":true}HTTP/1.1 200 OK
Content-Length: 379
Content-Type: application/json
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: *
Access-Control-Allow-Headers: *
Connection: close
Accept-Ranges: none

{"on":false,"bri":255,"transition":7,"ps":16,"pl":-1,"nl":{"on":false,"dur":60,"mode":1,"tbri":0,"rem":-1},"udpn":{"send":false,"recv":false},"lor":0,"mainseg":0,"seg":[{"id":0,"start":0,"stop":150,"len":150,"grp":1,"spc":0,"of":0,"on":true,"frz":false,"bri":255,"cct":127,"col":[[255,183,74],[0,0,0],[0,0,0]],"fx":0,"sx":128,"ix":128,"pal":0,"sel":true,"rev":false,"mi":false}]}

Here the json in the response is valid.

Hope that helps for further debugging..

Regards
saruter

[blazoncek]

@saruter thanks. Unfortunately it still does not show why.
Can you trace and post separate entries for requests and responses? In raw Wireshark format, no editing.

If you have a PacketSender you can also use that to capture response in entirety.

[saruter]

@blazoncek
I have attached a pcap-file containing a turn on and turn off of the wled-device running 0.14. It is configured with 4 segments.

The pcap file was recorded on the Home Assistant Host System with the following command tcpdump -i br0 host 192.168.179.172 -w wled-off-to-on.pcap

Infos about IPs and content:
192.168.179.172: WLED-Device
192.168.179.43: Home Assistant Server
First HTTP-Post Request for turning WLED on in packet 56, turning WLED off in packet 82
Also included is a GET /presets.json in packet 4 and a GET /json/si in packet 37

When turning the WLED-Device on or off in Home Assistant the following error is displayed:
orjson.JSONDecodeError: unexpected character: line 1 column 862 (char 861)

When looking at the response of the WLED-Device at first I observed that the json-string seems not to be ended/closed correctly. When looking closer at the characters in the json response which seems to raise the error message in Home Assistant I looked closer at the characters at char 861 and observed that there are two colons in a row "spc"::1:

{"id":2,"start":76,"stop":104,"len":28,"grp":1,"spc"::1,"spc":0,"of"

wled-off-to-on.zip

Hope that helps...

[blazoncek]

I have yet to review packet capture but this looks to me as a known issue with AsyncWebServer corrupting JSON text buffer although I've only seen JSON corruption on ESP8266.

What happens if you submit the same packet using PacketSender application? What I mean is reducing the amount of packets/requests submitted to ESP to a single one.

[saruter]

I just tried with Packetsender:
Sending HTTP Post for turning off: Response:
{"on":false,"bri":81,"transition":7,"ps":-1,"pl":-1,"nl":{"on":false,"dur":60,"mode":1,"tbri":0,"rem":-1},"udpn":{"send":false,"recv":true,"sgrp":1,"rgrp":1},"lor":0,"mainseg":0,"seg":[{"id":0,"start":0,"stop":28,"len":28,"grp":1,"spc":0,"of":0,"on":true,"frz":false,"bri":255,"cct":127,"set":0,"n":"unten","col":[[255,234,219,226],[0,0,0,0],[0,0,0,0]],"fx":97,"sx":255,"ix":225,"pal":32,"c1":128,"c2":128,"c3":16,"sel":true,"rev":false,"mi":true,"o1":false,"o2":false,"o3":false,"si":0,"m12":0},{"id":1,"start":28,"stop":76,"len":48,"grp":1,"spc":0,"of":0,"on":true,"frz":false,"bri":255,"cct":127,"set":0,"n":"rechts","col":[[255,234,219,226],[0,0,0,0],[0,0,0,0]],"fx":97,"sx":255,"ix":225,"pal":32,"c1":128,"c2":128,"c3":16,"sel":true,"rev":false,"mi":true,"o1":false,"o2":false,"o3":false,"si":0,"m12":0},{"id":2,"start":76,"stop":104,"len":28,"grp":1,"spc"":1,"spc":0,"of":0,"on":true,"frz":false,"bri":255,"cct":127,"set":0,"n":"oben","col":[[255,234,219,226],[0,0,0,0],[0,0,0,0]],"fx":97,"sx":255,"ix":225,"pal":32,"c1":128,"c2":128,"c3":16,"sel":true,"rev":false,"mi":true,"o1":false,"o2":false,"o3":false,"si":0,"m12":0},{"id":3,"start":104,"stop":152,"len":48,"grp":1,"spc":0,"of":0,"on":true,"frz":false,"bri":255,"cct":127,"set":0,"n":"links","col":[[255,234,219,226],[0,0,0,0],[0,0,0,0]],"fx":97,"sx":255,"ix":225,"pal":32,"c1":128,"c2":128,"c3":16,"sel":true,"rev":false,"mi":true,"o1":false,"o2":false,"o3":false,"si":0,"

Sending HTTP Post for turning on, response:
{"on":true,"bri":81,"transition":7,"ps":-1,"pl":-1,"nl":{"on":false,"dur":60,"mode":1,"tbri":0,"rem":-1},"udpn":{"send":false,"recv":true,"sgrp":1,"rgrp":1},"lor":0,"mainseg":0,"seg":[{"id":0,"start":0,"stop":28,"len":28,"grp":1,"spc":0,"of":0,"on":true,"frz":false,"bri":255,"cct":127,"set":0,"n":"unten","col":[[255,234,219,226],[0,0,0,0],[0,0,0,0]],"fx":97,"sx":255,"ix":225,"pal":32,"c1":128,"c2":128,"c3":16,"sel":true,"rev":false,"mi":true,"o1":false,"o2":false,"o3":false,"si":0,"m12":0},{"id":1,"start":28,"stop":76,"len":48,"grp":1,"spc":0,"of":0,"on":true,"frz":false,"bri":255,"cct":127,"set":0,"n":"rechts","col":[[255,234,219,226],[0,0,0,0],[0,0,0,0]],"fx":97,"sx":255,"ix":225,"pal":32,"c1":128,"c2":128,"c3":16,"sel":true,"rev":false,"mi":true,"o1":false,"o2":false,"o3":false,"si":0,"m12":0},{"id":2,"start":76,"stop":104,"len":28,"grp":1,"spc"::1,"spc":0,"of":0,"on":true,"frz":false,"bri":255,"cct":127,"set":0,"n":"oben","col":[[255,234,219,226],[0,0,0,0],[0,0,0,0]],"fx":97,"sx":255,"ix":225,"pal":32,"c1":128,"c2":128,"c3":16,"sel":true,"rev":false,"mi":true,"o1":false,"o2":false,"o3":false,"si":0,"m12":0},{"id":3,"start":104,"stop":152,"len":48,"grp":1,"spc":0,"of":0,"on":true,"frz":false,"bri":255,"cct":127,"set":0,"n":"links","col":[[255,234,219,226],[0,0,0,0],[0,0,0,0]],"fx":97,"sx":255,"ix":225,"pal":32,"c1":128,"c2":128,"c3":16,"sel":true,"rev":false,"mi":true,"o1":false,"o2":false,"o3":false,"si":0,"

So it seems to be the same here, but I found out that the problematic characters are always happening in Segment json with id 2 and it is a little different when turning on and off but has two times the "spc" string there. That's not happening in the other json for the other segments:

Problematic json when turning off:
"spc"":1,"spc":0,"of":0,
When turing on:
"spc"::1,"spc":0,

[blazoncek]

I've tried several times (on each of ESP8266, ESP32, ESP32-C3, ESP32-S2) and I was not able to reproduce.
My segment set-up varied from 2 to 12.

@saruter can you contact me on Discord and I'll send specially built binary.

[rjbudke]

This is happening to me on HA 10.3 and WLED 14. I had a light (Shelf Light) that had three segments and when I tried to update to WLED 14 via the Home Assistant GUI, it failed out so I reflashed WLED directly onto the device (8266). I renamed the light to "Shelf Light" and re-added it to HA, and since then it's been giving the error mentioned a few times. I didn't know if anyone else having the error had also re-added an existing light.

[Habbie]

I ran into this as well, and can reproduce it with curl:

peter@syracuse:~$ curl -D - -s -H 'Content-Type: application/json' -X POST http://192.168.1.240/json/state -d '{"seg":[{"bri":161,"on":true,"id":2}],"v":true}'
HTTP/1.1 200 OK
Content-Length: 1623
Content-Type: application/json
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: *
Access-Control-Allow-Headers: *
Connection: close
Accept-Ranges: none

{"on":true,"bri":128,"transition":7,"ps":-1,"pl":-1,"nl":{"on":false,"dur":60,"mode":1,"tbri":0,"rem":-1},"udpn":{"send":false,"recv":true,"sgrp":1,"rgrp":1},"lor":0,"mainseg":2,"seg":[{"id":0,"start":0,"stop":8,"len":8,"grp":1,"spc":0,"of":0,"on":false,"frz":false,"bri":255,"cct":127,"set":0,"col":[[255,160,0],[0,0,0],[0,0,0]],"fx":0,"sx":128,"ix":128,"pal":0,"c1":128,"c2":128,"c3":16,"sel":false,"rev":false,"mi":false,"o1":false,"o2":false,"o3":false,"si":0,"m12":0},{"id":1,"start":0,"stop":2,"len":2,"grp":1,"spc":0,"of":0,"on":true,"frz":false,"bri":214,"cct":127,"set":0,"col":[[255,0,21],[0,0,0],[0,0,0]],"fx":0,"sx":128,"ix":128,"pal":0,"c1":128,"c2":128,"c3":16,"sel":false,"rev":false,"mi":false,"o1":false,"o2":false,"o3":false,"si":0,"m12":0},{"id":2,"start":2,"stop":4,"len":2,"grp":1,"spc":0,"of":0,"on":true,"frz":false,"bri":161,"cct":127,"st":127,"set":0,"col":[[128,255,0],[0,0,0],[0,0,0]],"fx":0,"sx":128,"ix":128,"pal":0,"c1":128,"c2":128,"c3":16,"sel":true,"rev":false,"mi":false,"o1":false,"o2":false,"o3":false,"si":0,"m12":0},{"id":3,"start":4,"stop":6,"len":2,"grp":1,"spc":0,"of":0,"on":true,"frz":false,"bri":255,"cct":127,"set":0,"col":[[255,255,96],[0,0,0],[0,0,0]],"fx":0,"sx":128,"ix":128,"pal":0,"c1":128,"c2":128,"c3":16,"sel":false,"rev":false,"mi":false,"o1":false,"o2":false,"o3":false,"si":0,"m12":0},{"id":4,"start":6,"stop":8,"len":2,"grp":1,"spc":0,"of":0,"on":true,"frz":false,"bri":255,"cct":127,"set":0,"col":[[168,255,209],[0,0,0],[0,0,0]],"fx":0,"sx":128,"ix":128,"pal":0,"c1":128,"c2":128,"c3":16,"sel":false,"rev":false,"mi":false,"o1":false,"o2":false,"o3":false,"si":0,"peter@syracuse:~$

As you can see, the JSON is truncated. The Content-Length does match what was returned. peter@syracuse at the end is my terminal prompt, so there's no newline at the end of the response either.

WLED v0.14.0 "Hoshi", installed today, on "esp8266 3.1.2 (2)" (it's a Wemos D1 Mini).