Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

解析[]byte出现乱码(同时并不希望自动base64decode) #244

Closed
pwtitle opened this issue Feb 27, 2018 · 4 comments
Closed

解析[]byte出现乱码(同时并不希望自动base64decode) #244

pwtitle opened this issue Feb 27, 2018 · 4 comments

Comments

@pwtitle
Copy link

pwtitle commented Feb 27, 2018

type Message struct {
URL string json:"url"
Body []byte json:"body"
Strategy []int json:"strategy"
Method string json:"method"
// Headers []Header json:"headers"
Handler string json:"handler,omitempty"
Referer string json:"referer,omitempty"
Headers map[string]string json:"headers"
}

.....
body := []byte({"url":"abc.com","method":"PUT","body":"asdasdfsdfsfsfasfd==","strategy":[10,11,12,13,14,15,16,17,18,19,20],"headers":[],"referer":null,"handler":"f331db77be59f18eab9ada6924e04a32"})

var json = jsoniter.ConfigCompatibleWithStandardLibrary
extra.RegisterFuzzyDecoders()
err := json.Unmarshal(body, &message)
if err != nil {
	fmt.Println(err);
}
fmt.Printf("\r\n%s\r\n", message.Body);

......
结果是:j�Z���u����}

如果
body := []byte({"url":"abc.com","method":"PUT","body":"eyJ1c2VybmFtZSI6InRlc3RlciIsInBhc3N3b3JkIjoiMTIzNDU2In0=","strategy":[10,11,12,13,14,15,16,17,18,19,20],"headers":[],"referer":null,"handler":"f331db77be59f18eab9ada6924e04a32"})

则结果为:{"username":"tester","password":"123456"}

但是我并不想jsoniter帮我自动解码呀。

另外如果body的内容是比较长的base64内容,这返回:

main.Message.Method: Body: decode base64: illegal base64 data at input byte 230, error found in #10 byte of ...|wIn19fX0=","method":|..., bigger context ...|0aW9uIjoiMC4wMCIsInJldmVudWUiOiIyODAwLjAwIn19fX0=","method":"POST","strategy":[30,300],"headers":{"C|...

这个错误。

@pwtitle
Copy link
Author

pwtitle commented Feb 27, 2018

body := []byte(`{"url":"abc.com","method":"PUT","body":"eyJldmVudCI6Imxpbmt0b3VyLm9yZGVyLnBhaWQiLCJlbnYiOiJwcm9kdWN0aW9uIiwiY3JlYXRlZCI6IjIwMTgtMDItMjdUMDM6MTc6NTQrMDA6MDAiLCJwYXJhbXMiOnsib2lkIjoiNTM1NyIsInN0YXR1cyI6IlBBSUQiLCJvcmRlciI6eyJvaWQiOiI1MzU3IiwicGlkIjoiOTQyMyIsIm5hbWUiOiLoi4\/mooXlspsgTGV0J3MgUmVsYXggU1BBIOOAkOWkmuWll+mkkOWPr+mAieOAkSIsImJ1eWVyIjp7InVpZCI6IjkyNSIsInVzZXJuYW1lIjoiamlucGFpQEFHSEMuYWdlbnQiLCJuYW1lIjoi5YyX5Lqs6YeR54mM5Zu96ZmF5peF6KGM56S+5pyJ6ZmQ5YWs5Y+4KOWLv+WIoOWLv+aUue+8ieaciCIsIm1vYmlsZSI6IjE4NjUyMDM2NTkwIiwiZW1haWwiOiIxMDQzODI4OTc2QHFxLmNvbSw4MDYwMTg4MEBxcS5jb20sMzU2NTY0MDA2N0BxcS5jb20sMzI5MTg0OTE2M0BxcS5jb20sMjE3NDQ2MjQ4MEBxcS5jb20iLCJ3ZWNoYXQiOm51bGwsImxvZ28iOm51bGwsImNhbGxiYWNrIjpudWxsLCJyb2xlIjoiQUdFTlQiLCJjdXJyZW5jeSI6IlRIQiIsInRvdGFsIjoiMjgwMC4wMCIsInJlZnVuZCI6IjAuMDAiLCJjb21wZW5zYXRpb24iOiIwLjAwIiwicmV2ZW51ZSI6IjI4MDAuMDAifSwic2VsbGVyIjp7ImlkIjoiMTg1IiwiY29kZSI6IkFHSEMiLCJjdXJyZW5jeSI6IlRIQiIsIm5hbWUiOiJBTElCQUJBIEdST1VQIEhPTERJTkcgQ08uLExURCIsInNob3J0Ijoi6aOO5ZCR5qCHIiwiZW1haWwiOiJyc3ZuQGZ4LXRyaXAuY29tIiwiY2FsbGJhY2siOiIiLCJsb2dvIjoiXC8yMDE3MDkzMFwvMGRcLzE1MDY3NDM2MDEucG5nIiwid2VjaGF0IjpudWxsLCJlbWVyZ2VuY3kiOiIiLCJ0b3RhbCI6IjI4MDAuMDAiLCJyZWZ1bmQiOiIwLjAwIiwiY29tcGVuc2F0aW9uIjoiMC4wMCIsInJldmVudWUiOiIyODAwLjAwIn19fX0=","strategy":[10,11,12,13,14,15,16,17,18,19,20],"headers":[],"referer":null,"handler":"f331db77be59f18eab9ada6924e04a32"}`)

以上内容,解析的结果是:
main.Message.Referer: Headers: Strategy: []int: readUint64: unexpected character: �, error found in #0 byte of ...||..., bigger context ...||...

@taowen
Copy link
Contributor

taowen commented Feb 27, 2018

当前 encoding/json 的行为是

  • 编码:[]byte 始终被编码为 base64,而不是直接做为utf8字符串输出。因为不是所有的byte都是合法的utf8字符。
  • 解码:同时支持 [1,2,3] 这样的输入,和base64的输入。会自动根据json输入来适配。

json-iterator 的默认行为必须和 encoding/json 保持兼容。但是我等会添加一个扩展,支持把 []byte 解读为 string 来处理,对非 utf8 用 \x 转义。

@taowen taowen closed this as completed in 2a93f90 Feb 27, 2018
@taowen
Copy link
Contributor

taowen commented Feb 27, 2018

@liuke20008
Copy link

当前 encoding/json 的行为是

  • 编码:[]byte 始终被编码为 base64,而不是直接做为utf8字符串输出。因为不是所有的byte都是合法的utf8字符。
  • 解码:同时支持 [1,2,3] 这样的输入,和base64的输入。会自动根据json输入来适配。

json-iterator 的默认行为必须和 encoding/json 保持兼容。但是我等会添加一个扩展,支持把 []byte 解读为 string 来处理,对非 utf8 用 \x 转义。

请问是对“非utf8”用\x转义,还是对“utf8”用\x转义。但从实验结果上来看,正常的utf8编码也被\x转义了。
代码:
func main() {
jsoniter.RegisterExtension(&extra.BinaryAsStringExtension{})
a, _ := jsoniter.Marshal(map[string][]byte{
"name": []byte{0xe6, 0x9d, 0xa8}, // 汉字 "杨"
})
fmt.Printf("%s", string(a))
}
输出:
{"name":"\xe6\x9d\xa8"}

是我的理解有问题吗?

zhenzou pushed a commit to zhenzou/jsoniter that referenced this issue Feb 2, 2022
…ty, while the output is valid json, but it can not be decoded by other json codec, as \x01 is decoded as \x01 by them, which is not original input
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants