Reliability of log checks with nginx+libmodsecurity3

[Sebitosh]

Hello,

This is a question I had while working on owasp-modsecurity/MRTS#24 .

While working on a libmodsecurity3 test infrastructure with go-ftw, I noticed that across runs on my local machine (Ubuntu, 6 cores @ 2.375GHz and a NVMe ssd) results could be unreliable due to marker logs arriving before the expected ids because of processing by two different threads, similarly to #473 .

I found two ways to mitigate this: 1) reduce logging to a minimum 2) use go-ftw's rate limiter, but it slows execution.

I was wondering if this is a known issue with nginx when requests arrive too fast/logging is to slow ? And if so, how is it avoided in CRS's CI/CD pipeline ? Is there any particular configuration in the nginx container used in the pipeline aimed at avoiding this ?

[theseion]

Hi @Sebitosh. Unfortunately, this is a known issue. We have an ugly workaround (https://github.com/coreruleset/ftw-tests-schema/blob/e01f8213e656ae63efe04ad28471d67f46f3e5b9/spec/v2.3.0/ftw.md?plain=1#L1418) for some cases.
Luckily, the configuration we use (https://github.com/coreruleset/coreruleset/blob/main/tests/docker-compose.yml) works reliably for testing and CI.

I recently had the experience, that the issue kept appearing in Alpine, while it didn't in Debian. Not sure why though.

One idea might be to reduce the number of workers. I believe there might even be the option to run nginx on a single thread (in debug mode?), I don't quite recall the details.

[Sebitosh]

Thanks for the confirmation, didn't see the retry_once option. I think it will do the trick for us, I'll propose to add it in MRTS's generator script. If it works for CRS's CI, we should be able to make it work for ModSecurity's CI.

Thanks again @theseion !