proposal: crypto/x509: Allow to pass flags to syscall.CertGetCertificateChain, when using TLS Transport under Windows

[jkroepke]

Before writing the proposal here, I ask the community. However we found no solution for this issue.

Ref: https://stackoverflow.com/questions/77098977/disable-certificate-revocation-check-for-specific-https-connections-on-windows

---

I'm working on a internet restricted environment. Since our internal services we are using Let's Encrypt certificates to avoid the overhead with an internal PKI.

I'm writing a go program which collects informations from all systems and sent the data to a central system. While this works fine for Linux systems, have I have trouble on Windows system.

It seems like that go is using the os native libraries to establish a secured connection for HTTPS requests.

With using the program below

package main

import (
    "fmt"
    "io"
    "log"
    "net/http"
    "os"
)

func main() {
    resp, err := http.Get(os.Args[1])
    if err != nil {
        log.Fatalln(err)
    }

    defer resp.Body.Close()
    htmlData, err := io.ReadAll(resp.Body)
    if err != nil {
        log.Fatalln(err)
        return
    }
    fmt.Printf("%v\n", resp.Status)
    fmt.Printf(string(htmlData))
}

I got this

2023/09/13 16:19:03 Get "https://google.de": net/http: TLS handshake timeout

It seems like the native libraries tries to contact the CRLs (which are not allowed) and the connection fails here. I inspect the traffic with Wireshark and see HTTP connection to r3.c.lencr.org with User-Agent CryptoAPI.

I also can verify such behavior, if I'm using curl

curl https://google.de
curl: (35) schannel: next InitializeSecurityContext failed: Unknown error (0x80092012) - The revocation function was unable to check revocation for the certificate.

However curl has the option --ssl-no-revoke to skip the specific revoke check. Running curl with the option works fine.

Looking at the source code of CURL

https://github.com/curl/curl/blob/8e74c0729d0cace00a202fc6c33c1b35703e220a/lib/vtls/schannel.c#L474-L492

they are passing flags like

• SCH_CRED_IGNORE_REVOCATION_OFFLINE (When checking for revoked certificates, ignore CRYPT_E_REVOCATION_OFFLINE errors. For additional restrictions, see Remarks.)

Source: https://learn.microsoft.com/en-us/windows/win32/api/schannel/ns-schannel-schannel_cred

I would like have an option the set the flag on go, too.

Also RUST has the support to define such os native flags https://github.com/rust-lang/cargo/blob/3ea3c3a27f49f4926ff32befe48f8b652cd755b2/src/cargo/sources/git/oxide.rs#L291

[ianlancetaylor]

CC @golang/security

[jkroepke]

@ianlancetaylor Are more information required here?

[qmuntal]

AFAIK, neither crypto/tls nor crypto/x509 use of SCHANNEL on Windows, so the proposed solution won't fix the issue. This needs more investigation.

[jkroepke]

@qmuntal Sure, I will dump all information that I had:

How I'm able to reproduce the Issue on

• Windows Server

• Check for publisher’s certificate revocation is enabled

  Details

  - Click the Menu Start -> Run, type 'Control Panel', and click 'OK'
  - Double click 'Internet Options'
  - Navigate to the tab 'Advanced'
  - Check the "Check for publisher’s certificate revocation" under the 'Security' section
  

• HTTP Port 80 blocked (simulating a company firewall here)

• Run the Go Code below:

package main

import (
   "crypto/tls"
   "fmt"
   "io"
   "log"
   "net/http"
   "time"
)

func main() {
   client := http.Client{
      Transport: &http.Transport{
         Proxy:               http.ProxyFromEnvironment,
         TLSHandshakeTimeout: 10 * time.Second,
         TLSClientConfig: &tls.Config{
            InsecureSkipVerify: false,
         },
      },
   }

   resp, err := client.Get("https://grafana.oracle.jkroepke.de")
   if err != nil {
      log.Fatalln(err)
   }

   defer resp.Body.Close()
   htmlData, err := io.ReadAll(resp.Body)
   if err != nil {
      log.Fatalln(err)
      return
   }
   fmt.Printf("%v\n", resp.Status)
   fmt.Printf(string(htmlData))
}

Output:

2023/11/13 00:04:31 Get "https://grafana.oracle.jkroepke.de": net/http: TLS handshake timeout

Dump of all goroutes while go is trying to etablish the TLS connection:

Goroutine 1
  runtime.gopark (C:/Users/jok/sdk/go1.21.3/src/runtime/proc.go:399)
  runtime.selectgo (C:/Users/jok/sdk/go1.21.3/src/runtime/select.go:327)
  net/http.(*Transport).getConn (C:/Users/jok/sdk/go1.21.3/src/net/http/transport.go:1393)
  net/http.(*Transport).roundTrip (C:/Users/jok/sdk/go1.21.3/src/net/http/transport.go:591)
  net/http.(*Transport).RoundTrip (C:/Users/jok/sdk/go1.21.3/src/net/http/roundtrip.go:17)
  net/http.send (C:/Users/jok/sdk/go1.21.3/src/net/http/client.go:260)
  net/http.(*Client).send (C:/Users/jok/sdk/go1.21.3/src/net/http/client.go:181)
  net/http.(*Client).do (C:/Users/jok/sdk/go1.21.3/src/net/http/client.go:724)
  net/http.(*Client).Do (C:/Users/jok/sdk/go1.21.3/src/net/http/client.go:590)
  net/http.(*Client).Get (C:/Users/jok/sdk/go1.21.3/src/net/http/client.go:488)
  main.main (C:/Users/jok/GolandProjects/schannel/main.go:23)
  runtime.main (C:/Users/jok/sdk/go1.21.3/src/runtime/proc.go:267)
  runtime.goexit (C:/Users/jok/sdk/go1.21.3/src/runtime/asm_amd64.s:1650)
  created at: runtime.newproc (<autogenerated>:1)

Goroutine 2
  runtime.gopark (C:/Users/jok/sdk/go1.21.3/src/runtime/proc.go:399)
  runtime.goparkunlock (C:/Users/jok/sdk/go1.21.3/src/runtime/proc.go:404)
  runtime.forcegchelper (C:/Users/jok/sdk/go1.21.3/src/runtime/proc.go:322)
  runtime.goexit (C:/Users/jok/sdk/go1.21.3/src/runtime/asm_amd64.s:1650)
  created at: runtime.init.6 (C:/Users/jok/sdk/go1.21.3/src/runtime/proc.go:310)

Goroutine 3
  runtime.gopark (C:/Users/jok/sdk/go1.21.3/src/runtime/proc.go:399)
  runtime.goparkunlock (C:/Users/jok/sdk/go1.21.3/src/runtime/proc.go:404)
  runtime.bgsweep (C:/Users/jok/sdk/go1.21.3/src/runtime/mgcsweep.go:280)
  runtime.gcenable.func1 (C:/Users/jok/sdk/go1.21.3/src/runtime/mgc.go:200)
  runtime.goexit (C:/Users/jok/sdk/go1.21.3/src/runtime/asm_amd64.s:1650)
  created at: runtime.gcenable (C:/Users/jok/sdk/go1.21.3/src/runtime/mgc.go:200)

Goroutine 4
  runtime.gopark (C:/Users/jok/sdk/go1.21.3/src/runtime/proc.go:399)
  runtime.goparkunlock (C:/Users/jok/sdk/go1.21.3/src/runtime/proc.go:404)
  runtime.(*scavengerState).park (C:/Users/jok/sdk/go1.21.3/src/runtime/mgcscavenge.go:425)
  runtime.bgscavenge (C:/Users/jok/sdk/go1.21.3/src/runtime/mgcscavenge.go:653)
  runtime.gcenable.func2 (C:/Users/jok/sdk/go1.21.3/src/runtime/mgc.go:201)
  runtime.goexit (C:/Users/jok/sdk/go1.21.3/src/runtime/asm_amd64.s:1650)
  created at: runtime.gcenable (C:/Users/jok/sdk/go1.21.3/src/runtime/mgc.go:201)

Goroutine 5
  runtime.gopark (C:/Users/jok/sdk/go1.21.3/src/runtime/proc.go:399)
  runtime.runfinq (C:/Users/jok/sdk/go1.21.3/src/runtime/mfinal.go:193)
  runtime.goexit (C:/Users/jok/sdk/go1.21.3/src/runtime/asm_amd64.s:1650)
  created at: runtime.createfing (C:/Users/jok/sdk/go1.21.3/src/runtime/mfinal.go:163)

Goroutine 6
  runtime.gopark (C:/Users/jok/sdk/go1.21.3/src/runtime/proc.go:399)
  runtime.chanrecv (C:/Users/jok/sdk/go1.21.3/src/runtime/chan.go:583)
  runtime.chanrecv1 (C:/Users/jok/sdk/go1.21.3/src/runtime/chan.go:442)
  net/http.(*persistConn).addTLS (C:/Users/jok/sdk/go1.21.3/src/net/http/transport.go:1561)
  net/http.(*Transport).dialConn (C:/Users/jok/sdk/go1.21.3/src/net/http/transport.go:1635)
  net/http.(*Transport).dialConnFor (C:/Users/jok/sdk/go1.21.3/src/net/http/transport.go:1467)
  net/http.(*Transport).queueForDial.func1 (C:/Users/jok/sdk/go1.21.3/src/net/http/transport.go:1436)
  runtime.goexit (C:/Users/jok/sdk/go1.21.3/src/runtime/asm_amd64.s:1650)
  created at: net/http.(*Transport).queueForDial (C:/Users/jok/sdk/go1.21.3/src/net/http/transport.go:1436)

Goroutine 21
  <unknown>
  runtime.systemstack_switch (C:/Users/jok/sdk/go1.21.3/src/runtime/asm_amd64.s:474)
  runtime.cgocall (C:/Users/jok/sdk/go1.21.3/src/runtime/cgocall.go:175)
  syscall.SyscallN (C:/Users/jok/sdk/go1.21.3/src/runtime/syscall_windows.go:544)
  syscall.Syscall9 (C:/Users/jok/sdk/go1.21.3/src/runtime/syscall_windows.go:494)
  syscall.CertGetCertificateChain (C:/Users/jok/sdk/go1.21.3/src/syscall/zsyscall_windows.go:388)
  crypto/x509.(*Certificate).systemVerify (C:/Users/jok/sdk/go1.21.3/src/crypto/x509/root_windows.go:250)
  crypto/x509.(*Certificate).Verify (C:/Users/jok/sdk/go1.21.3/src/crypto/x509/verify.go:770)
  crypto/tls.(*Conn).verifyServerCertificate (C:/Users/jok/sdk/go1.21.3/src/crypto/tls/handshake_client.go:993)
  crypto/tls.(*clientHandshakeStateTLS13).readServerCertificate (C:/Users/jok/sdk/go1.21.3/src/crypto/tls/handshake_client_tls13.go:531)
  crypto/tls.(*clientHandshakeStateTLS13).handshake (C:/Users/jok/sdk/go1.21.3/src/crypto/tls/handshake_client_tls13.go:96)
  crypto/tls.(*Conn).clientHandshake (C:/Users/jok/sdk/go1.21.3/src/crypto/tls/handshake_client.go:263)
  crypto/tls.(*Conn).clientHandshake-fm (<autogenerated>:1)
  crypto/tls.(*Conn).handshakeContext (C:/Users/jok/sdk/go1.21.3/src/crypto/tls/conn.go:1552)
  crypto/tls.(*Conn).HandshakeContext (C:/Users/jok/sdk/go1.21.3/src/crypto/tls/conn.go:1492)
  net/http.(*persistConn).addTLS.func2 (C:/Users/jok/sdk/go1.21.3/src/net/http/transport.go:1555)
  runtime.goexit (C:/Users/jok/sdk/go1.21.3/src/runtime/asm_amd64.s:1650)
  created at: net/http.(*persistConn).addTLS (C:/Users/jok/sdk/go1.21.3/src/net/http/transport.go:1551)

Port 443 is open and confirmed by curl:

Details

C:\Users\jok>curl -v --ssl-no-revoke -I https://grafana.oracle.jkroepke.de
*   Trying 152.70.167.19:443...
* Connected to grafana.oracle.jkroepke.de (152.70.167.19) port 443 (#0)
* schannel: disabled automatic use of client certificate
* ALPN: offers http/1.1
* ALPN: server accepted http/1.1
* using HTTP/1.1
> HEAD / HTTP/1.1
> Host: grafana.oracle.jkroepke.de
> User-Agent: curl/8.0.1
> Accept: */*
>
* schannel: remote party requests renegotiation
* schannel: renegotiating SSL/TLS connection
* schannel: SSL/TLS connection renegotiated
*
HTTP/1.1 302 Found
Cache-Control: no-store
Content-Type: text/html; charset=utf-8
Date: Sun, 12 Nov 2023 23:10:42 GMT
Location: /login
X-Content-Type-Options: nosniff
X-Frame-Options: deny
X-Xss-Protection: 1; mode=block

---

After open Port 80, the following HTTP request can be captured via Wireshark

GET /MFMwUTBPME0wSzAJBgUrDgMCGgUABBRI2smg%2ByvTLU%2Fw3mjS9We3NfmzxAQUFC6zF7dYVsuuUAlA5h%2BvnYsUwsYCEgT%2FX5ZrtLDDaw%2FQvLTVAR3uXw%3D%3D HTTP/1.1\r\n
Connection: Keep-Alive
Accept: */*
User-Agent: Microsoft-CryptoAPI/10.0
Host: r3.o.lencr.org

[Full request URI: http://r3.o.lencr.org/MFMwUTBPME0wSzAJBgUrDgMCGgUABBRI2smg%2ByvTLU%2Fw3mjS9We3NfmzxAQUFC6zF7dYVsuuUAlA5h%2BvnYsUwsYCEgT%2FX5ZrtLDDaw%2FQvLTVAR3uXw%3D%3D]
[HTTP request 1/1]
[Response in frame: 41131]

Note: After open Port 80 and execute the go code above, Windows is able to validate the revocation status and cache the infomation.

[jkroepke]

Based on the goroutine dump, I think I'm looking for a way to define flags (CERT_CHAIN_REVOCATION_CHECK_CACHE_ONLY) on the CertGetCertificateChain syscall

https://learn.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-certgetcertificatechain

Code:

go/src/crypto/x509/root_windows.go

Line 250 in 8da6405

err = syscall.CertGetCertificateChain(syscall.Handle(0), storeCtx, verifyTime, storeCtx.Store, para, CERT_CHAIN_RETURN_LOWER_QUALITY_CONTEXTS, 0, &topCtx)