Bug: Handshake may fail when using specific ClientHelloID

[gaukas]

Original Issue

The TLS Handshake below will fail with errMsg local error: tls: unexpected message

tlsUConn := UClient(tcpConn, &Config{ServerName: hostname}, ch)
tlsUConn.Handshake()

when hostname is served by CloudFlare* and ch is set to HelloChrome_Auto.

* Including their site www.cloudflare.com, their DoH servers 1.1.1.1, 1.0.0.1, cloudflare-dns.com, and all my sites with CloudFlare.

Rough investigation

The error was thrown in *Conn.readHandshake() here by

return nil, c.in.setErrorLocked(c.sendAlert(alertUnexpectedMessage))

In other words, an unknown handshake type as data[0] has been received. The unknown byte is 0x19(DEC: 25) which is undefined.

After a few more attempts, all three HelloChrome_70, HelloChrome_72, HelloChrome_83 will be triggering exact the same error.

All other ClientHelloID (except for HelloCustom) work great when handshake with CloudFlare served hosts.

Test Sample

cloudflare_test.go

package tls

import (
	"net"
	"testing"
)

var (
	tlsClientHelloIDs = map[string]ClientHelloID{
		// "golang":  HelloGolang,
		// "random":  HelloRandomized,
		// "firefox":        HelloFirefox_Auto,
		// "HelloChrome_58":        HelloChrome_58,
		// "HelloChrome_62":        HelloChrome_62,
		// "HelloChrome_70":        HelloChrome_70,
		// "HelloChrome_72":        HelloChrome_72,
		// "HelloChrome_83":        HelloChrome_83,
		// "HelloFirefox_55":       HelloFirefox_55,
		// "HelloFirefox_56":       HelloFirefox_56,
		// "HelloFirefox_63":       HelloFirefox_63,
		// "HelloFirefox_65":       HelloFirefox_65,
		// "HelloIOS_11_1":         HelloIOS_11_1,
		// "HelloIOS_12_1":         HelloIOS_12_1,
		// "HelloRandomized":       HelloRandomized,
		// "HelloRandomizedALPN":   HelloRandomizedALPN,
		// "HelloRandomizedNoALPN": HelloRandomizedNoALPN,
		// "HelloCustom": HelloCustom,
		// "ios":            HelloIOS_Auto,
	}

	tlsHostnames = []string{
		// "cloudflare-dns.com",
		// "8.8.8.8",
		// "9.9.9.9",
		// "1.0.0.1",
		"www.cloudflare.com",
		// "caddyserver.com",
		// "github.com",
		// "pkg.go.dev",
	}
)

func TestCloudflareHandshake(t *testing.T) {
	for name, ch := range tlsClientHelloIDs {
		t.Run(name, func(t *testing.T) {
			for _, hostname := range tlsHostnames {
				t.Run(hostname, func(t *testing.T) {
					tcpConn, tcpErr := net.Dial("tcp", hostname+":443")
					if tcpErr != nil {
						t.Fatal("net.Dial():", tcpErr)
					}
					defer tcpConn.Close()
					tlsUConn := UClient(tcpConn, &Config{
						ServerName: hostname,
					}, ch)
					handshakeErr := tlsUConn.Handshake()
					if handshakeErr != nil {
						t.Fatal("tls.Handshake():", handshakeErr)
					}
				})
			}
		})
	}
}

Works better if you print the error byte in conn.go:1061

[gaukas]

So far I didn't find a single host will trigger this error that is not on CloudFlare. Will be interesting if someone can get a working browser with HelloChrome_58's ClientHello.

On a side note, HelloChrome_58 has the same fingerprint as HelloChrome_62: 0a4a74aeebd1bb66

[lyashm]

Similarly. I am observing the same problem with the site on Cloudflare.

remote error: tls: error decoding message

[gaukas]

Hi @lyashm,

Thanks for the feedback.

What error message will you get if you run my code in cloudflare_test.go?

Also, would you like to provide a code snippet that throws remote error: tls: error decoding message? Or if possible, please try to locate on which line the specific error is thrown.

I think these 2 may or may not be related, further investigation is needed. I haven't been able to look into this issue in detail yet.

[gaukas]

This issue is caused by certificate compression and is fixed in #95.

[gaukas]

Fixed since tagged version v1.1.1.