1.17.1 (756) protocol

[iceiix]

1.7.1 (756) vs 1.16.4 (754) https://wiki.vg/index.php?title=Protocol&type=revision&diff=16918&oldid=16681
1.16.4 (754, currently supported): https://wiki.vg/index.php?title=Protocol&oldid=16681#Chunk_Data
1.7.1 (756) https://wiki.vg/index.php?title=Protocol&oldid=16918#Chunk_Data

• Clientbound packet IDs
• Tags variant
• TeleportPlayer variant
• New packets: SculkVibrationSignal, ClearTitles, WindowPing, CombatEvent*, ActionBar, WorldBorder*, Title*
• WindowItems variant
• WindowSetSlot variant
• Explosion variant
• WorldBorderInit
• ChunkData variant recognized
• ChunkData_Biomes3D_Bitmasks packet handled
• UpdateLight variant
• ResourcePackSend variant
• SpawnPosition variant
• EntityProperties variant
• ClientSettings variant
• Serverbound packet IDs
• ClickWindow variant
• EditBook variant
• New serverbound packet: WindowPong

[iceiix]

Current status: compiles, but not functional. The server kicks the client: Internal Exception: io.netty.handler.codec.DecoderException: java.lang.IndexOutOfBoundsException: index: 4, length: 105 (expected: range(0, 24))

There are many changed packets on https://wiki.vg/index.php?title=Protocol&type=revision&diff=16918&oldid=16681

[iceiix]

Current status: now able to connect to a 1.17.1 server, sometimes, but it doesn't get far:

still many changed packets to go through one by one (no to mention chunk data). They added "state ids" in many places.

[iceiix]

Getting closer:

[model/mod.rs:196][ERROR] Error loading model minecraft:carved_pumpkin
thread '<unnamed>' panicked at 'unknown pose data: 7', protocol/src/types/metadata.rs:704:18
stack backtrace:
   0: rust_begin_unwind
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/panicking.rs:515:5
   1: std::panicking::begin_panic_fmt
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/panicking.rs:457:5
   2: <steven_protocol::types::metadata::Metadata as steven_protocol::protocol::Serializable>::read_from
   3: steven_protocol::protocol::packet::packet_by_id
   4: steven_protocol::protocol::Conn::read_packet

but https://wiki.vg/Entity_metadata#Entity_Metadata_Format only says: 18 | Pose | A VarInt enum: 0: STANDING, 1: FALL_FLYING, 2: SLEEPING, 3: SWIMMING, 4: SPIN_ATTACK, 5: SNEAKING, 6: DYING - is pose 7 valid not yet documented?

---

https://github.com/aadnk/ProtocolLib/blob/master/src/main/java/com/comphenix/protocol/wrappers/EnumWrappers.java#L373-L387 defines only these poses:

	public enum EntityPose {
		STANDING, 
		FALL_FLYING, 
		SLEEPING, 
		SWIMMING, 
		SPIN_ATTACK, 
		CROUCHING,
		DYING;

https://hub.spigotmc.org/javadocs/spigot/org/bukkit/entity/Pose.html defines more:

STANDING Entity is standing normally.
FALL_FLYING Entity is gliding.
SLEEPING Entity is sleeping.
SWIMMING Entity is swimming.
SPIN_ATTACK Entity is riptiding with a trident.
SNEAKING Entity is sneaking.
DYING Entity is dead.
LONG_JUMPING Entity is long jumping.

updated https://wiki.vg/Entity_metadata#Entity_Metadata_Format

[iceiix]

https://wiki.vg/index.php?title=Protocol&oldid=16918#Chunk_Data
https://wiki.vg/index.php?title=Protocol&oldid=16681#Chunk_Data

The bitmask used to be a VarInt:

Bitmask with bits set to 1 for every 16×16×16 chunk section whose data is included in Data. The least significant bit represents the chunk section at the bottom of the chunk column (from y=0 to y=15).

but now its a VarInt-prefixed array of Long:

BitSet with bits (world height in blocks / 16) set to 1 for every 16×16×16 chunk section whose data is included in Data. The least significant bit represents the chunk section at the bottom of the chunk column (from the lowest y to 15 blocks above).

to get it to compile I've passed the 0th element of this array:

                new,                                // TODO: what should this be?
                chunk_data.bitmasks.data[0] as u16, // TODO: get all bitmasks

which gives the above "partially loaded chunk" image, but will need to take into account all the bits. And what to do with new? From looking at the new packet, it appears new is now always true - which means "full chunk", it was used to control whether the optional biomes array was included, but now it always is.

Logging chunk_data.bitmasks.data.len(), connecting to a 1.17.1, test server, always 1 is observed - bugs elsewhere? Truncation from i64 -> i16? The mask is used in src/world/mod.rs load_chunk19_or_115:

            for i in 0..16 {
                if chunk.sections[i].is_none() {
                    let mut fill_sky = chunk.sections.iter().skip(i).all(|v| v.is_none());
                    fill_sky &= (mask & !((1 << i) | ((1 << i) - 1))) == 0;
                    if !fill_sky || mask & (1 << i) != 0 {
                        chunk.sections[i] = Some(Section::new(i as u8, fill_sky));
                    }
                }
                if mask & (1 << i) == 0 {
                    continue;
                }

[iceiix]

Still wrapping my head around the chunk changes. PrisimarineJS is another useful project for reference how they handled it, in their packet_map_chunk:

https://github.com/PrismarineJS/minecraft-data/blob/master/data/pc/1.16.2/protocol.json#L1982
https://github.com/PrismarineJS/minecraft-data/blob/master/data/pc/1.17.1/protocol.json#L2204

changing from:

            {
              "name": "bitMap",
              "type": "varint"

to:

            {
              "name": "bitMap",
              "type": [
                "array",
                {
                  "countType": "varint",
                  "type": "i64"
                }
              ]
            },

The data is handled in prismarine-chunk, here's their changes: PrismarineJS/prismarine-chunk#133
There are a lot of changes in that PR, but its essentially a rewrite to support both. Would be nice to generalize our code too, avoiding duplication, enhancing what is now load_chunk19_or_115 to allow wider bitmasks, though in the worse case, could create a new function instead. From PrismarineJS:

Now the only API inconsistency is the bitmap Number -> Array of protodef Longs and .loadLight which I don't think is reducible complexity

https://github.com/PrismarineJS/prismarine-chunk/blob/master/src/pc/1.17/ChunkColumn.js#L262
https://github.com/PrismarineJS/prismarine-chunk/blob/master/src/pc/1.16/ChunkColumn.js#L219

[iceiix]

Protocol parsing is now working well, or at least it does not crash as often, but chunk data handling still needs a lot of work. Testing with a level-type=FLAT world, with the player on the ground or in the sky, it cannot see any chunk data:

compared to with a normal world, where it saw some chunk data. The mask array is only one element, a 64-bit integer but only observed up to 6 bits set, yet still not all the chunk data is loading, have to look into this further..

Dumping the chunk data: Vec<u8> in load_chunk19_to_117:

00000000  04 00 04 04 00 21 0a 09  80 02 11 11 11 11 11 11  |.....!..........|
00000010  11 11 11 11 11 11 11 11  11 11 11 11 11 11 11 11  |................|
*
00000080  11 11 11 11 11 11 11 11  11 11 22 22 22 22 22 22  |..........""""""|
00000090  22 22 22 22 22 22 22 22  22 22 22 22 22 22 22 22  |""""""""""""""""|
*
00000180  22 22 22 22 22 22 22 22  22 22 33 33 33 33 33 33  |""""""""""333333|
00000190  33 33 33 33 33 33 33 33  33 33 33 33 33 33 33 33  |3333333333333333|
*
00000200  33 33 33 33 33 33 33 33  33 33 00 00 00 00 00 00  |3333333333......|
00000210  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000800  00 00 00 00 00 00 00 00  00 00                    |..........|
0000080a

There appears to be some kind of scaling or offset issue. If I squint I can see the other player far away on a pillar (and he does move correctly, for example when jumping, I can see the jump) - even though I'm directly beneath him:

https://wiki.vg/Chunk_Format#Data_structure

The palette size is almost always 4, and appears to be read correctly:

section 0 palette size = 4
	palette entry 0 = 0 = Air
	palette entry 1 = 33 = Bedrock
	palette entry 2 = 10 = Dirt { snowy: false, variant: Normal }
	palette entry 3 = 9 = Grass { snowy: false }

but sometimes is:

section 0 palette size = 5
	palette entry 0 = 0 = Air
	palette entry 1 = 33 = Bedrock
	palette entry 2 = 10 = Dirt { snowy: false, variant: Normal }
	palette entry 3 = 9 = Grass { snowy: false }
	palette entry 4 = 1 = Stone { variant: Normal }

or:

section 1 palette size = 2
	palette entry 0 = 0 = Air
	palette entry 1 = 1 = Stone { variant: Normal }
section 1 palette size = 2
	palette entry 0 = 0 = Air
	palette entry 1 = 10 = Dirt { snowy: false, variant: Normal }

this is about what I expect, adding a bunch of debugging in src/world/mod.rs: debug-chunk.patch.zip

                     let count = VarInt::read_from(&mut data)?.0;
+                    println!("section {} palette size = {}", i, count);
                     for i in 0..count {
                         let id = VarInt::read_from(&mut data)?.0;
                         let bl = self
                             .id_map
                             .by_vanilla_id(id as usize, &self.modded_block_ids);
+                        println!("\tpalette entry {} = {} = {:?}", i, id, bl);
                         mappings.insert(i as usize, bl);
                     }
                 }

and printing the block arrays, no problems found yet, maybe the problem lies elsewhere? I did notice when I login, another player immediately observes the player dies lying on the ground, this could be explained by a position update error.

[iceiix]

It was indeed a positioning update error, handling the TeleportPlayer packet variant now shows the world (though lighting and head angles are messed up):

[iceiix]

This does not include the block additions, so the blocks are wrong but the world is at least recognizable, it's a start.

Now for the CI failure… https://github.com/iceiix/stevenarella/pull/605/checks?check_run_id=3518938444 it doesn't seem to be related to anything I changed here:

Run jetli/wasm-pack-action@v0.3.0
Searching the latest version of wasm-pack ...
Installing wasm-pack v0.10.1 ...
Downloading wasm-pack from https://github.com/rustwasm/wasm-pack/releases/download/v0.10.1/wasm-pack-v0.10.1-x86_64-unknown-linux-musl.tar.gz ...
Extracting wasm-pack to /tmp/setup-wasm-pack ...
/usr/bin/tar xz --warning=no-unknown-keyword -C /tmp/setup-wasm-pack -f /home/runner/work/_temp/0d295110-54c4-4869-99e9-38c7b03b7ade
Error: ENOENT: no such file or directory, rename '/tmp/setup-wasm-pack/wasm-pack-v0.10.1-x86_64-unknown-linux-musl/wasm-pack' -> '/home/runner/.cargo/bin/wasm-pack'
Done

.github/workflows/build.yaml

      - name: Install wasm-pack
        uses: jetli/wasm-pack-action@v0.3.0
        with:
          version: 'latest'
      - name: Build binary
        run: |
          rustup target add wasm32-unknown-unknown
          cargo install wasm-bindgen-cli
          wasm-pack build --dev

https://github.com/jetli/wasm-pack-action last changed on Mar 30, 2020 and has 8 open pull requests - unmaintained? Documentation links to an alternative: https://rustwasm.github.io/docs/wasm-bindgen/wasm-bindgen-test/continuous-integration.html#github-actions

      - name: Install
        run: curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh

      - run: cargo test
      - run: wasm-pack test --headless --chrome
      - run: wasm-pack test --headless --firefox

There was a wasm-pack 0.10.1 release within the past hour: rustwasm/wasm-pack@b1e67d3 - reported: rustwasm/wasm-pack#1057