False positive when charset=utf-8\x0d\x0a in content-type header

[chladic]

Hello CRS team,

some actions in my mobile application are triggering this rule and block request. I didnt set in APP charset=utf-8\x0d\x0a,
so maybe its coming from android.
When I exclude this rule SecRuleRemoveById 922110 then its fine, but I want to exclude it for everything
and could not figure out any exception:

I tried this:
tx.allowed_request_content_type_charset=|utf-8| |utf-8\x0d\x0a| |iso-8859-1| |iso-8859-15| |windows-1252|'

and also this:

SecRule REQUEST_HEADERS:Content-Type "text/plain; charset=utf-8\x0d\x0a" \
    "phase:1,nolog,pass,id:6,t:none,ctl:ruleRemoveById=922110"

I checked 922110 rule and it cant match above with regex defined there.
Anyone can help me to understand this issue please ?

Many thanks

ModSecurity: Warning. Matched "Operator `Rx' with parameter `^(?:(?:\*|[^\"(),\/:;<=>?![\x5c\]{}]+)\/(?:\*|[^\"(),\/:;<=>?![\x5c\]{}]+))(?:\s*+;\s*+(?:(?:charset\s*+=\s*+(?:\"?(?:iso-8859-15?|windows-1252|utf-8)\b\"?))|(?:(?:c(?:h(?:a(?:r(?:s(?:e[^t\"(),\/:;<=> (714 characters omitted)' against variable `TX:1' (Value: `text/plain; charset=utf-8\x0d\x0a' ) [file "/usr/local/coreruleset-3.3.4/rules/REQUEST-922-MULTIPART-ATTACK.conf"] [line "51"] [id "922110"] [rev ""] [msg "Illegal MIME Multipart Header content-type: charset parameter"] [data "Matched Data: text/plain; charset=utf-8\x0d\x0a found within Content-Type multipart form"] [severity "2"] [ver "OWASP_CRS/3.3.4"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "OWASP_CRS"] [tag "capec/272/220"] [tag "paranoia-level/1"] [hostname "10.151.0.2"] [uri "/something.json"] [unique_id "167059009818.061175"] [ref "o0,41o14,27v974,41t:lowercaset:lowercase"],

• CRS version: 3.3.4
• Paranoia level setting:1
• ModSecurity version: 3.0.8
• Web Server and version: nginx
• Operating System and version: alpine

[fzipi]

Sadly there is no easy way to have the same allowed_request_content_type_charset in the config and in the rule. The content you see is coming from this include file.

In the end, looks like you are receiving an extra \r\n there, right?

CC @theseion

[chladic]

When APP calls PATCH with some data inside then its fine, but when it POST empty data to endpoint {}, this rule gets triggered and I cant figure out why there are 2 extra characters \r\n
@fzipi thanks, for a moment i was not sure if problem is text/plain what was removed in 3.3.4 from default allowed content-type or those 2 extra characters

[theseion]

It definitely looks like an illegal request to me. \r\n is the separation token for headers, so if it show up in the value then there are either more than there should be or they have been badly encoded. Do you have the ability to inspect the HTTP traffic, e.g. with tcpdump or Wireshark?

[RedXanadu]

+1 to what @theseion said. It would be interesting to compare one of your working PATCH requests to one of the broken POST requests in Wireshark or something similar.

[chladic]

@theseion working on it, hopefully will get data soon

[chladic]

So this is request what pass:

PATCH /custom.json HTTP/1.1
X-Forwarded-For: 1.1.1.1
X-Forwarded-Proto: https
X-Forwarded-Port: 443
Host: example.com
X-Amzn-Trace-Id: Root=1-111111
Content-Length: 233
user-agent: user-agent
accept-encoding: gzip
authorization: Bearer *******
content-type: multipart/form-data; boundary=--dio-boundary-2382587621
custom-version: 1.1.1
custom-locale: en

----dio-boundary-2382587621
content-disposition: form-data; name="data[type]"

user
----dio-boundary-2382587621
content-disposition: form-data; name="data[attributes][frequency]"

DAILY
----dio-boundary-2382587621--

this trigger modsec rule:

PATCH /custom.json HTTP/1.1                                                                                                                                                                                                                                                                                                                          X-Forwarded-For: 1.1.1.1                                                                                                                                                                                                                                                                                                                                 X-Forwarded-Proto: https
X-Forwarded-Port: 443
Host: example.com
X-Amzn-Trace-Id: Root=1-11111
Content-Length: 303
user-agent: user-agent
accept-encoding: gzip
authorization: Bearer ******
content-type: multipart/form-data; boundary=--dio-boundary-1533285540
custom-version: 1.1.1
custom-locale: en

----dio-boundary-1533285540
content-disposition: form-data; name="data[type]"

user
----dio-boundary-1533285540
content-disposition: form-data; name="data[attributes][frequency]"
content-type: text/plain; charset=utf-8
content-transfer-encoding: binary


----dio-boundary-1533285540--

In second PATCH there is no string DAILY but only null as we dont want any value there. It looks like this cause adding 2 extra headers at the end what modsec does not like or it considers as form data ?

[theseion]

Interesting! It looks like this could be a bug in ModSecurity. ModSecurity has a dedicated multipart parser that does some magic to detect / handle parts with null termination (https://github.com/SpiderLabs/ModSecurity/blob/v3/master/src/request_body_processor/multipart.cc). I wasn't able to spot the issue while looking at the code though. @airween, this might be something for you. null is definitely allowed as content of a part as per RFC.

[chladic]

Thank you @theseion. Happy to hear its not our application issue. Would you have some smarter idea how to temporarily exclude it without removing whole rule with SecRuleRemoveById 922110 ?

[theseion]

You could write a new rule that looks for utf-8\x0d\x0a in multipart requests and then dynamically disable the rule with ctl:ruleRemoveById=922110 (https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-(v2.x)#ctl). It's not perfect but should create a huge hole.

[chladic]

@theseion sorry to bother you again, but when I write down this exception it does not work:

SecRule MULTIPART_PART_HEADERS "@rx ^content-type\s*:\s*text.*" \
        "phase:1,nolog,pass,id:6,ctl:ruleRemoveById=922110"

is this what you meant ? Similar condition is used by 922110 Rule, so I dont understand why it does not match in exception

[theseion]

Your new rule must be processed before the rule you want to remove (see linked documentation). Is that the case? That means it has to run in the same phase or an earlier one and it needs to come before the other rule physically (in the same file or in a file that is read before 922-...). The regular expression looks correct.

BTW, to be safe I would match the header without case sensitivity, e.g. (?i)^content-type\s*:\s*text.* (922110 uses t:lowercase).

If you have the option, you can enable debug logging and you will see when and how your new rule is processed.