Expected comma or object end but found invalid token at character *** during json load

[PureTryOut]

I've been getting this error since switching from Xubuntu to Gentoo. The strange thing is that this happened to me on my Arch Linux desktop before, but switching to Gentoo there actually fixed it.
I have no clue what causes this at all. Wiping existing Matrix settings and starting over doesn't help.

While this error happens, Weechat becomes unresponsive and spikes to 100% CPU load. I'm guessing the JSON is too long for it too handle or something... The *** is actually a number, but it changes depending on the JSON it loads so I don't think it's really relevant here.

[PureTryOut]

Finally found out the error, it appears to be my own damn fault.
Instead of using full paths when linking matrix.lua to the proper directories, I used relative paths. Meaning that if Weechat was not run from the same directory again (which of course it wasn't) it couldn't resolve the link and would completely break. It now works fine.

[PureTryOut]

So this didn't appear to be it... To this day this issue still happens.
Does anyone have any clue?

[erdnaxeli]

I had a similar issue. The JSON error was complaining about a float which was in the return of an initial sync (I played a lil with account_data). But the JSON was perfectly valid…

I never saw this error again so far.

[timorl]

Not sure if this is the same issue, but I fixed my JSON parsing problem by setting LC_ALL to C:

LC_ALL=C weechat

The problem was with parsing numbers which were not the last field in an object -- they would be then followed by a comma, which would be parsed as part of the number within some locales. If any of you had a non-english locale set, this might have been the same problem.

[erdnaxeli]

Wow 😮

According to cjson documentation, this should be handled:

Lua CJSON uses strtod and snprintf to perform numeric conversion as they are usually well supported, fast and bug free. However, these functions require a workaround for JSON encoding/parsing under locales using a comma decimal separator. Lua CJSON detects the current locale during instantiation to determine and automatically implement the workaround if required.

I could not reproduce it (and I haven't see the bug in the script recently). What is your local?

[timorl]

My locale is pl_PL.utf8 . Right now I'm running weechat with only LC_NUMERIC set to C and this is quite a reasonable workaround.

As far as I can tell the script uses the system-wide cjson which in my case is version 2.1.0. The version didn't change in ages, and neither did my locale, so I have no idea why it suddenly propped up. I've been using the script for some months without issue, before it broke a couple of days ago.

[craftyguy]

I'm also seeing this with locale set to en_US.utf-8 (actually LC_ALL is unset but other vars are). Setting LC_NUMERIC=C seems to work as @timorl suggested.

[craftyguy]

Actually, seems like this is still a problem. Setting LC_ALL=C doesn't help it either.

[craftyguy]

A quick hack to get around this is to comment out line 534 in the script by changing it to:

        -- mprint(('error\t%s during json load: %s'):format(js, stdout))

Normally I do not like 'big hammer' hacks like this, but it's better than having weechat hang for several minutes every time I try to view the weechat buffer.

[balanceofcowards]

I can confirm this bug. For me, the problematic character is the dot in the number 0.0 in the following snippet:
{"m.favourite": {"order": 0.0}}
My locale is:
LANG=de_DE.UTF-8 (...) LC_NUMERIC="de_DE.UTF-8" (...) LC_ALL=
Calling weechat via
LC_ALL=C weechat
or
LC_NUMERIC=C weechat
works fine as a workaround.

Strangely, everything worked fine this morning and broke only an hour ago -- but I suppose this is due to the rare occurrence of floating point numbers: I seem to have only this single value in a json response of ~200k characters.

@craftyguy: Commenting out the error reporting line does not fix the error. For me, the bug prevents the buffer list to be loaded. Skipping the error message still means I can't access the matrix network.

[balanceofcowards]

@erdnaxeli already pointed out the relevant section of the lua-cjson documentation, which further says that:

Lua CJSON should be reinitialised via cjson.new if the locale of the current process changes.

(This apparently happens in this case). The problem goes away if line 532 of matrix.luais changed from:
local success, js = pcall(json.decode, stdout)
to:
local localejson = json.new()
local success, js = pcall(localejson.decode, stdout)

You got a respective pull request: #135