DEVEL commit 212ce80 introduces incorrect Ids/group jids

[jlguardi]

@tgalal Due to 212ce80 some IDs and group jids replace - by A.
I have reverted all coder changes in jlguardi/yowsup/master

This was reported in my repo but considered as new protocol:
jlguardi#9
jlguardi#30

And introduced lot of bugs.
Please be careful and check this commit.

[Brammmm]

I don't think reverting commit 212ce80 is the fix nor the source of the problem..
The original code in readInt24 - which you put back when reverting the commit - looks broken and using the wrong bytes..

When I compare the decoder with what is in ChatAPI I spot 3 differences:

• readInt24:
  • yowsup: for the first byte only 4 bits are used
  • Chat-API the entire first byte is used
• readInt31:
  • yowsup: first byte is discarded, second byte is used twice
  • Chat-API: all bytes used only once
• readPacked8:
  • yowsup: only one condition to set 'remove = 1', inner loop uses the original size/length
  • Chat-API: extra condition when seting 'remove = 1' (n = 251) + loop uses the size/length of the hexdata

A diff to let yowsup use the same as ChatAPI:

diff --git a/yowsup/layers/coder/decoder.py b/yowsup/layers/coder/decoder.py
index afd6bd2..ac5e530 100644
--- a/yowsup/layers/coder/decoder.py
+++ b/yowsup/layers/coder/decoder.py
@@ -70,9 +70,9 @@ class ReadDecoder:
     def readPacked8(self, n, data):
         size = self.readInt8(data)
         remove = 0
-        if (size & 0x80) != 0:
+        if (size & 0x80) != 0 and n == 251:
             remove = 1
-            size = size & 0x7F
+        size = size & 0x7F
         text = bytearray(self.readArray(size, data))
         hexData = binascii.hexlify(text).upper()
         out = []
@@ -80,7 +80,7 @@ class ReadDecoder:
             for i in range(0, len(hexData)):
                 char = chr(hexData[i]) if type(hexData[i]) is int else hexData[i] #python2/3 compat
                 val = ord(binascii.unhexlify("0%s" % char))
-                if i == (size - 1) and val > 11 and n != 251: continue
+                if i == (len(hexData) - 1) and val > 11 and n != 251: continue
                 out.append(self.unpackByte(n, val))
         else:
             out =  map(ord, list(hexData[0: -remove])) if sys.version_info < (3,0) else list(hexData[0: -remove])
@@ -136,7 +136,8 @@ class ReadDecoder:
          int1 = data.pop(0)
          int2 = data.pop(0)
          int3 = data.pop(0)
-         return ((int1 & 0xF) << 16) + (int2 << 8) + (int3 << 0)
+         # return ((int1 & 0xF) << 16) + (int2 << 8) + (int3 << 0)
+         return (int1 << 16) + (int2 << 8) + (int3 << 0)
 
     def readInt24(self, data):
         int1 = data.pop(0)
@@ -146,11 +147,11 @@ class ReadDecoder:
         return value
 
     def readInt31(self, data):
-        data.pop(0)
         int1 = data.pop(0)
         int2 = data.pop(0)
         int3 = data.pop(0)
-        return (int1 << 24) | (int1 << 16) | int2 << 8 | int3
+        int4 = data.pop(0)
+        return (int1 << 24) | (int2 << 16) | int3 << 8 | int4
 
     def readListSize(self,token, data):
         size = 0

My suspicion is that:

• changes to readInt24 are not required;
• changes to readInt31 are required
• changes to readPacked8 are required

When running with only the readInt24 and readInt31 changes I still saw 'A' instead of '-' in ids (it appears, but I did not fully confirm that, that it works fine up till '-9' and then becomes 'A10')..
With the readPacked8 I have not yet seen the 'A' in ids.. (and currently up to '-48' in the id)

But I'm a bit clueless which of the changes are required.. All I did was compare it with the ChatAPI code..

[Brammmm]

With the changes made to readInt24, readInt31 and readPacked8 the client is connected for over 10 hours, ids are now up to -768

[tgalal]

This was due to a problem with readPacked8 function. It should be ok now