This repository has been archived by the owner on Sep 14, 2020. It is now read-only.
Post k8s-events properly from multiple threads (thread-safe) #148
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 5 commits
July 13, 2019 02:48
🤖 zincr found 0 problems , 1 warningDetails on how to resolve are provided below Large CommitsChecks all commits for large additions to a single file. Large commits should be reviewed more carefully for potential copyright and licensing issues This file contains a substantial change, please review to determine if the change comes from an external source and if there are any copyright or licensing issues to be aware of
|
Codecov Report
@@ Coverage Diff @@
## master #148 +/- ##
==========================================
- Coverage 83.55% 82.98% -0.58%
==========================================
Files 30 30
Lines 1545 1587 +42
Branches 227 232 +5
==========================================
+ Hits 1291 1317 +26
- Misses 209 227 +18
+ Partials 45 43 -2
|
samurang87
reviewed
Jul 16, 2019
kopf/engines/posting.py
Outdated
| else: | ||
| # No event-loop or another event-loop - assume another thread. | ||
| # Use the cross-thread thread-safe methods. Block until enqueued there. | ||
| fut = asyncio.run_coroutine_threadsafe(queue.put(event), loop=loop) |
There was a problem hiding this comment.
Some official docs use that too. E.g. asyncio.
But right, it's better to have normal names. Fixed.
samurang87
previously approved these changes
Jul 16, 2019
2 tasks
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Part of an effort to make Kopf-based operators more stable and resilient.
Description
When k8s-event-posting was moved to a background task (#125), some asyncio-related tasks were made way to complicated (multiple dynamically created loggers) and implemented improperly (non-thread-safe
put_nowait()when used across threads).The non-thread-safeness made some k8s-events not posted as soon as it was possible, but waiting for few seconds until something happens on an asyncio event-loop (because
put_nowait()does not wake up the event-loop). And it was failing of debug mode was used.This has become important since #128 — as not only Kopf's internal routines, but also the sync-handlers (which run in a thread-pool) could log INFO+ messages, which should be posted to k8s.
This PR simplifies the solution to one and only one logger (and therefore one and only one handler), and by posting all k8s-events synchronously but instantly via one and only one function (
enqueue()).In that function (
enqueue()), one of the thread-safe methods is used to put an event to a queue: either the same-threaded method (thread-safe by definition), or cross-threaded method (explicitly thread-safe).Types of Changes