Add fallback decoding for asgi headers

[rocky-ken]

Description

Add fallback decoding for asgi headers if utf-8 decoding throws an error.

Duplicate of PR 1713 to hopefully get it over the finish line

Fixes #1814
Fixes #1478

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

Test it in asgi

Does This PR Require a Core Repo Change?

• No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

• Followed the style guidelines of this project
• Changelogs have been updated
• Unit tests have been added
• Documentation has been updated

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: rocky-ken / name: Rocky Ken (118f259, 30e3607, 4d596a2, d685bf5, 1180e58, ba53807, 09befa0, 61dd28c)
• ✅ login: emdneto / name: Emídio Neto (a5bc035, 017f9b6)
• ✅ login: xrmx / name: Riccardo Magliocchetti (9bd0030, e3638a9)

[emdneto]

Question: why do we want a fallback strategy to latin-1 for a wrong encoded value?

[xrmx]

Question: why do we want a fallback strategy to latin-1 for a wrong encoded value?

And if we want it we also want tests

[xrmx]

ASGI docs kinda suggests these should be ASCII but that's not mandatory so being less picky may be a good idea:

headers: These are byte strings of the exact byte sequences sent by the client/to be sent by the server. While modern HTTP standards say that headers should be ASCII, older ones did not and allowed a wider range of characters. Frameworks/applications should decode headers as they deem appropriate.

https://asgi.readthedocs.io/en/latest/specs/www.html#wsgi-encoding-differences

[rocky-ken]

Question: why do we want a fallback strategy to latin-1 for a wrong encoded value?

Was following the previous PR and discussion here. Would a different approach be better? (Ex: try-catch the decode and pass-through)

[xrmx]

@rocky-ken we need some tests for this to get in

[emdneto]

Current tests don't cover the changes in ASGIGetter, right? I think would be nice to have tests for ASGIGetter too.

[rocky-ken]

Current tests don't cover the changes in ASGIGetter, right? I think would be nice to have tests for ASGIGetter too.

Not familiar with the ASGIGetter tests but I added some stuff so let me know if that covers it or not

[emdneto]

Not familiar with the ASGIGetter tests but I added some stuff so let me know if that covers it or not

@rocky-ken they are under test_getter.py