Skip to content

[OCSP Stapling] Cached bad response of Server-Side OCSP stabling in .NET7 Linux  #89907

Closed
@cdlliuy

Description

@cdlliuy

Description

In .NET7 Linux, we noticed that the Server Side OCSP Stapling is enabled by default, in which the response from OCSP responder is cached to speed up the cert status check

However, if OCSP responder send back a wrong response ...it will be cached as well in TLS server side. Then ,the client of TLS server will always receive the bad extension.status_request , then failed to setup the SSL connection due to parse failure.

From .NET runtime code, I can see the logic to do OCSP downloading , then cache for about 1 day if the response content is not null
https://github.com/dotnet/runtime/blob/main/src/libraries/System.Net.Security/src/System/Net/Security/SslStreamCertificateContext.Linux.cs#L226C1-L249
Then, inside the download code, I can see the bad http status code is not handled.
https://github.com/dotnet/runtime/blob/main/src/libraries/Common/src/System/Net/Http/X509ResourceClient.cs#L182C24-L243

What happens in our TLS server is, it tries to connection a OCSP website, and it should get a 5xx response. The OCSP website configures "an error html page" when it fails, so the response content is not empty, and our TLS server cached the error html page as the OCSP response.

Then, when client side made a https connection to our TLS server, from the ssl debug info, we can see the error html page inside the handshake payload .. It said "Our services aren't available right now

We're working to restore all services as soon as possible. Please check back soon"

08F0: CE 3C 21 44 4F 43 54 59   50 45 20 68 74 6D 6C 20  .<!DOCTYPE html 
  0900: 50 55 42 4C 49 43 20 27   2D 2F 2F 57 33 43 2F 2F  PUBLIC '-//W3C//
  0910: 44 54 44 20 58 48 54 4D   4C 20 31 2E 30 20 54 72  DTD XHTML 1.0 Tr
  0920: 61 6E 73 69 74 69 6F 6E   61 6C 2F 2F 45 4E 27 20  ansitional//EN' 
  0930: 27 68 74 74 70 3A 2F 2F   77 77 77 2E 77 33 2E 6F  'http://www.w3.o
  0940: 72 67 2F 54 52 2F 78 68   74 6D 6C 31 2F 44 54 44  rg/TR/xhtml1/DTD
  0950: 2F 78 68 74 6D 6C 31 2D   74 72 61 6E 73 69 74 69  /xhtml1-transiti
  0960: 6F 6E 61 6C 2E 64 74 64   27 3E 3C 68 74 6D 6C 20  onal.dtd'><html 
  0970: 78 6D 6C 6E 73 3D 27 68   74 74 70 3A 2F 2F 77 77  xmlns='http://ww
  0980: 77 2E 77 33 2E 6F 72 67   2F 31 39 39 39 2F 78 68  w.w3.org/1999/xh
  0990: 74 6D 6C 27 3E 3C 68 65   61 64 3E 3C 6D 65 74 61  tml'><head><meta
  09A0: 20 63 6F 6E 74 65 6E 74   3D 27 74 65 78 74 2F 68   content='text/h
  09B0: 74 6D 6C 3B 20 63 68 61   72 73 65 74 3D 75 74 66  tml; charset=utf
  09C0: 2D 38 27 20 68 74 74 70   2D 65 71 75 69 76 3D 27  -8' http-equiv='
  09D0: 63 6F 6E 74 65 6E 74 2D   74 79 70 65 27 2F 3E 3C  content-type'/><
  09E0: 73 74 79 6C 65 20 74 79   70 65 3D 27 74 65 78 74  style type='text
  09F0: 2F 63 73 73 27 3E 62 6F   64 79 20 7B 66 6F 6E 74  /css'>body .font
  0A00: 2D 66 61 6D 69 6C 79 3A   41 72 69 61 6C 3B 20 6D  -family:Arial; m
  0A10: 61 72 67 69 6E 2D 6C 65   66 74 3A 34 30 70 78 3B  argin-left:40px;
  0A20: 20 7D 69 6D 67 20 20 7B   20 62 6F 72 64 65 72 3A   .img  . border:
  0A30: 30 20 6E 6F 6E 65 3B 20   7D 23 63 6F 6E 74 65 6E  0 none; .#conten
  0A40: 74 20 7B 20 6D 61 72 67   69 6E 2D 6C 65 66 74 3A  t . margin-left:
  0A50: 20 61 75 74 6F 3B 20 6D   61 72 67 69 6E 2D 72 69   auto; margin-ri
  0A60: 67 68 74 3A 20 61 75 74   6F 20 7D 23 6D 65 73 73  ght: auto .#mess
  0A70: 61 67 65 20 68 32 20 7B   20 66 6F 6E 74 2D 73 69  age h2 . font-si
  0A80: 7A 65 3A 20 32 30 70 78   3B 20 66 6F 6E 74 2D 77  ze: 20px; font-w
  0A90: 65 69 67 68 74 3A 20 6E   6F 72 6D 61 6C 3B 20 63  eight: normal; c
  0AA0: 6F 6C 6F 72 3A 20 23 30   30 30 30 30 30 3B 20 6D  olor: #000000; m
  0AB0: 61 72 67 69 6E 3A 20 33   34 70 78 20 30 70 78 20  argin: 34px 0px 
  0AC0: 30 70 78 20 30 70 78 20   7D 23 6D 65 73 73 61 67  0px 0px .#messag
  0AD0: 65 20 70 20 20 7B 20 66   6F 6E 74 2D 73 69 7A 65  e p  . font-size
  0AE0: 3A 20 31 33 70 78 3B 20   63 6F 6C 6F 72 3A 20 23  : 13px; color: #
  0AF0: 30 30 30 30 30 30 3B 20   6D 61 72 67 69 6E 3A 20  000000; margin: 
  0B00: 37 70 78 20 30 70 78 20   30 70 78 30 70 78 7D 23  7px 0px 0px0px.#
  0B10: 65 72 72 6F 72 72 65 66   20 7B 20 66 6F 6E 74 2D  errorref . font-
  0B20: 73 69 7A 65 3A 20 31 31   70 78 3B 20 63 6F 6C 6F  size: 11px; colo
  0B30: 72 3A 20 23 37 33 37 33   37 33 3B 20 6D 61 72 67  r: #737373; marg
  0B40: 69 6E 2D 74 6F 70 3A 20   34 31 70 78 20 7D 3C 2F  in-top: 41px .</
  0B50: 73 74 79 6C 65 3E 3C 74   69 74 6C 65 3E 43 45 53  style><title>CES
  0B60: 50 4B 49 3C 2F 74 69 74   6C 65 3E 3C 2F 68 65 61  PKI</title></hea
  0B70: 64 3E 3C 62 6F 64 79 3E   3C 64 69 76 20 69 64 3D  d><body><div id=
  0B80: 27 63 6F 6E 74 65 6E 74   27 3E 3C 64 69 76 20 69  'content'><div i
  0B90: 64 3D 27 6D 65 73 73 61   67 65 27 3E 3C 68 32 3E  d='message'><h2>
  0BA0: 4F 75 72 20 73 65 72 76   69 63 65 73 20 61 72 65  Our services are
  0BB0: 6E 27 74 20 61 76 61 69   6C 61 62 6C 65 20 72 69  n't available ri
  0BC0: 67 68 74 20 6E 6F 77 3C   2F 68 32 3E 3C 70 3E 57  ght now</h2><p>W
  0BD0: 65 27 72 65 20 77 6F 72   6B 69 6E 67 20 74 6F 20  e're working to 
  0BE0: 72 65 73 74 6F 72 65 20   61 6C 6C 20 73 65 72 76  restore all serv
  0BF0: 69 63 65 73 20 61 73 20   73 6F 6F 6E 20 61 73 20  ices as soon as 
  0C00: 70 6F 73 73 69 62 6C 65   2E 20 50 6C 65 61 73 65  possible. Please
  0C10: 20 63 68 65 63 6B 20 62   61 63 6B 20 73 6F 6F 6E   check back soon
  0C20: 2E 3C 2F 70 3E 3C 2F 64   69 76 3E 3C 64 69 76 20  .</p></div><div 
  0C30: 69 64 3D 27 65 72 72 6F   72 72 65 66 27 3E 3C 73  id='errorref'><s
  0C40: 70 61 6E 3E 52 65 66 20   41 3A 20 34 33 30 46 37  pan>Ref A: 430F7
  0C50: 35 37 35 34 30 35 44 34   30 36 32 38 32 45 36 44  575405D406282E6D
  0C60: 38 41 31 32 38 33 42 43   45 35 44 20 52 65 66 20  8A1283BCE5D Ref 
  0C70: 42 3A 20 43 48 31 41 41   32 30 34 30 39 30 34 30  B: CH1AA20409040
  0C80: 32 35 20 52 65 66 20 43   3A 20 32 30 32 33 2D 30  25 Ref C: 2023-0
  0C90: 37 2D 32 35 54 31 38 3A   34 35 3A 32 31 5A 3C 2F  7-25T18:45:21Z</
  0CA0: 73 70 61 6E 3E 3C 2F 64   69 76 3E 3C 2F 64 69 76  span></div></div
  0CB0: 3E 3C 2F 62 6F 64 79 3E   3C 2F 68 74 6D 6C 3E 00  ></body></html>.

Then with decrypted, client side saw

    "extensions": {
      "status_request (5)": {
        extra data at the end
      }
    }

and failed with

 Caused by: java.io.IOException: extra data at the end
  	at java.base/sun.security.util.DerValue.<init>(DerValue.java:428)

The issue can be resolved when we restarted our TLS server which will get a refresh ocsp response.

However, I wonder whether it is possible to check the status code of ocsp response, and avoid the caching when the response is not correct?

Reproduction Steps

intermittent issue due to ocsp responder availablity.
maybe setup a bad ocsp responder can reproduce it stably.

Expected behavior

expect the retry on ocsp download or avoid cache the ocsp bad result

Actual behavior

the bad ocsp response is cached.

Regression?

No response

Known Workarounds

restart TLS server to fetch a new OCSP response

Configuration

.NET7 Kershel webserver on Linux

Other information

No response

Drafted a possible fix #89908 for this issue. Please help to review

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions