Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ftw ignore first iteration, CI #34

Merged
merged 10 commits into from
Oct 13, 2022

Conversation

M4tteoP
Copy link
Member

@M4tteoP M4tteoP commented Oct 7, 2022

No description provided.

@M4tteoP M4tteoP marked this pull request as ready for review October 8, 2022 12:48
@M4tteoP
Copy link
Member Author

M4tteoP commented Oct 11, 2022

As agreed, go-ftw inside the CI is now commented out in this PR. cc. @anuraaga @jcchavezs

ftw/docker-compose.yml Outdated Show resolved Hide resolved
@@ -37,7 +37,7 @@ static_resources:
{
"rules": [
{"inline": "Include ftw-config.conf"},
{"inline": "Include coraza.conf-recommended"},
{"inline": "Include coraza-ftw.conf"},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should still include coraza recommended for users right?

Can ftw-config.conf be modified to set the SecResponseBodyMimeType etc? It can control priority by doing the Include coraza.conf-recommended there instead of here, not sure if it would be before or after the include but one of them would work I think

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense! Done!

@@ -72,6 +72,11 @@ jobs:
ENVOY_IMAGE=$image go run mage.go e2e
done

# Currently excluded, go-ftw fails to find the destination after a while. Locally works fine.
Copy link
Member

@jcchavezs jcchavezs Oct 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What destination? The envoy one? This is because you need to run it inside a container, same way we do e2e.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can work out this in a next PR

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Go-ftw error message is the "usual": can't find log marker. Am I reading the correct log?..., showing up after about 30 minutes of passing tests.
Looking at Envoy logs, the problem is about running out-of-memory.
These are a couple of CI runs: this repo, my fork.
Snippets:

[source/extensions/common/wasm/context.cc:1170] wasm log coraza-filter my_vm_id: 2781 finished
[source/extensions/common/wasm/context.cc:1170] wasm log: panic: runtime error: out of memory
[source/extensions/common/wasm/wasm_vm.cc:38] Function: malloc failed: Uncaught RuntimeError: unreachable
2022-10-10T12:46:04.5215919Z Proxy-Wasm plugin in-VM backtrace:
2022-10-10T12:46:04.5216118Z   0:  0xaf037 - runtime.runtimePanic
2022-10-10T12:46:04.5216291Z   1:  0xac03d - runtime.alloc
2022-10-10T12:46:04.5216484Z   2:  0xbb02a - runtime.hashmapSet
2022-10-10T12:46:04.5216686Z   3:  0xbad9d - malloc
2022-10-10T12:46:04.5217008Z   4:  0x1af149 - github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm.getMap
2022-10-10T12:46:04.5217272Z   5:  0x1abffb - (*main.httpContext).OnHttpRequestHeaders
2022-10-10T12:46:04.5217480Z   6:  0x1a974a - proxy_on_request_headers
[source/extensions/common/wasm/context.cc:1170] wasm log coraza-filter my_vm_id: panic: runtime error: out of memory
[source/extensions/common/wasm/wasm_vm.cc:38] Function: proxy_on_request_body failed: Uncaught RuntimeError: unreachable
2022-10-08T13:18:29.7599438Z Proxy-Wasm plugin in-VM backtrace:
2022-10-08T13:18:29.7599634Z   0:  0xaf037 - runtime.runtimePanic
2022-10-08T13:18:29.7599837Z   1:  0xac03d - runtime.alloc
2022-10-08T13:18:29.7600021Z   2:  0xbb02a - runtime.hashmapSet
2022-10-08T13:18:29.7600167Z   3:  0xbad9d - malloc
2022-10-08T13:18:29.7600317Z   4:  0x1db05 - _Znwm
2022-10-08T13:18:29.7601000Z   5:  0x90bd8 - _ZNSt3__212__hash_tableIPN3re23DFA5StateENS2_9StateHashENS2_10StateEqualENS_9allocatorIS4_EEE25__emplace_unique_key_argsIS4_JRKS4_EEENS_4pairINS_15__hash_iteratorIPNS_11__hash_nodeIS4_PvEEEEbEERKT_DpOT0_
2022-10-08T13:18:29.7601218Z   6:  0x905e0 - _ZN3re23DFA11CachedStateEPiij
2022-10-08T13:18:29.7601504Z   7:  0x904a2 - _ZN3re23DFA18WorkqToCachedStateEPNS0_5WorkqES2_j
2022-10-08T13:18:29.7601758Z   8:  0x91aca - _ZN3re23DFA14RunStateOnByteEPNS0_5StateEi
2022-10-08T13:18:29.7602116Z   9:  0x9201b - _ZN3re23DFA17InlinedSearchLoopILb0ELb0ELb0EEEbPNS0_12SearchParamsE

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. I misunderstood the comment.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logs look like a memory leak cc @anuraaga

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure relevant but similar envoyproxy/envoy#14856

@jcchavezs jcchavezs merged commit 2084271 into corazawaf:main Oct 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants