cannot deserialize zign

[yuzhichang]

Environment

# copypaste this script into your shell and replace it with the output
date
r2 -v
uname -ms

Sat May  6 18:44:13 CST 2023
radare2 5.8.6 30523 @ linux-x86-64
birth: git.5.8.6 2023-05-06__09:46:18
commit: 3c9ad151adf52658d814f56b9bb80bc9231da36b
options: gpl -O? checks=2
Linux x86_64

Description

zo failed to load some signatures(unfortunately they are just what I'm interested in).

Test

Generate signature file:

zhichyu@ck98:~/grpc_whl/notstripped$ ls -l 1.44.0-cygrpc.cpython-310-x86_64-linux-gnu.so
-rwxr-xr-x 1 zhichyu eoi 200942120 Mar 19 11:36 1.44.0-cygrpc.cpython-310-x86_64-linux-gnu.so
zhichyu@ck98:~/grpc_whl/notstripped$ md5sum 1.44.0-cygrpc.cpython-310-x86_64-linux-gnu.so
78427e7551ab6b78c549e5ee3cd8efc6  1.44.0-cygrpc.cpython-310-x86_64-linux-gnu.so

zhichyu@ck98:~/grpc_whl/notstripped$ r2 1.44.0-cygrpc.cpython-310-x86_64-linux-gnu.so
WARN: run r2 with -e bin.cache=true to fix relocations in disassembly
 -- Use /m to carve for known magic headers. speedup with search.
[0x0007f2c0]> e zign.mangled=true;e anal.hasnext=true;afr;aac
[0x0007f2c0]> zg
[0x0007f2c0]> zos /data01/zhichyu/grpc_whl/notstripped/1.44.0-cygrpc.cpython-36m-x86_64-linux-gnu.so.sdb

Load signature file:

zhichyu@ck98:~/grpc_whl/notstripped$ r2 1.44.0-cygrpc.cpython-310-x86_64-linux-gnu.so
[0x0007f2c0]> zo /data01/zhichyu/grpc_whl/notstripped/1.44.0-cygrpc.cpython-36m-x86_64-linux-gnu.so.sdb
WARN: Skipping signature with invalid key ()                                                                                                                                                                      
ERROR: cannot deserialize zign                                                                                                                                                                                    
WARN: Skipping signature with invalid key (nager__HttpFilter____const__grpc_core__XdsRouteConfigR ()|N:_ZNKSt8_Rb_treeISsSt4pairIKSsN9grpc_core17XdsHttpFi)                                                       
ERROR: cannot deserialize zign                                                                                                                                                                                    
WARN: Skipping signature with invalid key (01e884aadeff498b45204889c248d1ea4c39f277cd418b4500488b0c)                                                                                                              
ERROR: cannot deserialize zign                                                                                                                                                                                    
WARN: Skipping signature with invalid key (igned_long__std::allocator_char__const_,sym.std::_Rb_tree_std::string__std::pair_std::string_co)                                                                       
ERROR: cannot deserialize zign                                                                                                                                                                                    
WARN: Skipping signature with invalid key (84489e05b5d415c415d415e415fc30f1f00e843b5f3ff4989c4)                                                                                                                   
ERROR: cannot deserialize zign                                                                                                                                                                                    ```

I've sent the the sample file `1.44.0-cygrpc.cpython-310-x86_64-linux-gnu.so` to [pancake@nopcode.org](mailto:pancake@nopcode.org) with title `sample file for radare2 #21718`.

[yuzhichang]

Following patch print unexpected klen and vlen and advance s->pos accordingly:

diff --git a/shlr/sdb/src/sdb.c b/shlr/sdb/src/sdb.c
index f7e744b694..1e2144cdcb 100644
--- a/shlr/sdb/src/sdb.c
+++ b/shlr/sdb/src/sdb.c
@@ -982,6 +982,7 @@ SDB_API bool sdb_dump_dupnext(Sdb* s, char *key, char **value, int *_vlen) {
        if (!cdb_getkvlen (&s->db, &klen, &vlen, s->pos)) {
                return false;
        }
+       uint32_t sign_off = s->pos;
        s->pos += 4;
        if (klen < 1 || vlen < 1) {
                return false;
@@ -996,6 +997,9 @@ SDB_API bool sdb_dump_dupnext(Sdb* s, char *key, char **value, int *_vlen) {
                                return 0;
                        }
                        key[klen] = 0;
+               }else{
+                       printf("Got unexpected klen %d at offset %x\n", klen, sign_off);
+                       s->pos += klen;
                }
        }
        if (value) {
@@ -1011,6 +1015,9 @@ SDB_API bool sdb_dump_dupnext(Sdb* s, char *key, char **value, int *_vlen) {
                                return false;
                        }
                        (*value)[vlen] = 0;
+               }else{
+                       printf("Got unexpected vlen %d at offset %x\n", vlen, sign_off);
+                       s->pos += vlen;
                }
        }
        return true;

With the above patch, zo catches unexpected klen and vlen, and continue loading signatures after that failed one.

[0x0007f2c0]> zo /data01/zhichyu/grpc_whl/notstripped/1.44.0-cygrpc.cpython-310-x86_64-linux-gnu.so.sdb
Got unexpected klen 255 at offset d19c1  ********HERE!********
WARN: Skipping signature with invalid key ()
ERROR: cannot deserialize zign
ERROR: Invalid types: ```void __cxa_throw (void *thrown_exception, struct std::type_info *tinfo, void *dest)``` in signatuer for zign|*|imp.__cxa_throw
ERROR: cannot deserialize zign

The four bytes at offset 0xd19c1 of the sdb file is FF 0E 18 00:

@radare Looks like radare cannot handle symbols longer than 254 bytes. However mangled long function name is common.

[trufae]

Thanks for the patch. Now we entered into the abi breaking season. Can you submit a pr for that? The size limit will be removed too

[yuzhichang]

Is it appropriate to extend the size limit of symbol from 255 to 65535?
Optimizing zos from O(n*n) to O(n) would be more important than this issue.

[trufae]

More than defining a larger limit is about to use char* insyead of a fixed size. Its abi breaking season now so its time to do those changes :) the abidiff job is disabled. And yes other optimizations and the need to support mangled name storage will be done too. Agree perf is important here

[trufae]

Ping?

[trufae]

Moving fed because enotime to chk as usual. Would be good if the person who filled the ticket could verify it because i think it should work

[yuzhichang]

@trufae The latest r2 still has the issue.