Skip to content

Refactor logging for efficiency, add fifo, context printing in text mode #4133

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 68 commits into
base: master
Choose a base branch
from

Conversation

invisig0th
Copy link
Contributor

No description provided.

Copy link

codecov bot commented Feb 14, 2025

Codecov Report

Attention: Patch coverage is 96.29630% with 21 lines in your changes missing coverage. Please review.

Project coverage is 97.29%. Comparing base (e6d8cef) to head (1432c5c).

Files with missing lines Patch % Lines
synapse/lib/logging.py 94.11% 9 Missing ⚠️
synapse/cortex.py 95.34% 4 Missing ⚠️
synapse/lib/agenda.py 95.31% 3 Missing ⚠️
synapse/lib/coro.py 81.25% 3 Missing ⚠️
synapse/tools/cmdr.py 0.00% 1 Missing ⚠️
synapse/tools/rstorm.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4133      +/-   ##
==========================================
- Coverage   97.35%   97.29%   -0.06%     
==========================================
  Files         261      261              
  Lines       59821    59954     +133     
==========================================
+ Hits        58237    58331      +94     
- Misses       1584     1623      +39     
Flag Coverage Δ
linux 97.29% <96.29%> (+0.01%) ⬆️
linux_replay ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@invisig0th invisig0th changed the title VERY WIP: Refactor logging for efficiency, add fifo, context printing in text mode, and additional link exception context WIP: Refactor logging for efficiency, add fifo, context printing in text mode Feb 18, 2025
MichaelSquires
MichaelSquires previously approved these changes Mar 7, 2025
@invisig0th invisig0th changed the title WIP: Refactor logging for efficiency, add fifo, context printing in text mode Refactor logging for efficiency, add fifo, context printing in text mode Mar 7, 2025
@invisig0th invisig0th requested a review from vEpiphyte March 7, 2025 18:16
Cisphyx
Cisphyx previously approved these changes Mar 7, 2025
@vEpiphyte vEpiphyte dismissed stale reviews from Cisphyx and MichaelSquires via 1432c5c March 11, 2025 18:46

if rcode is False:
self.clear()
self.set_status(500)
self.sendRestErr('StormRuntimeError', f'Extended HTTP API {iden} never set status code.')
self.sendRestErr('StormRuntimeError', 'Extended HTTP API never set status code.')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the iden being removed here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seemed like an unnecessary disclosure of "internals" to a caller who knows what API endpoint they hit. Happy to discuss. 👍


logtodo = []
logbase = None
loglock = asyncio.Lock()
Copy link
Contributor

@vEpiphyte vEpiphyte Mar 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These structures ( loglock / logevnt )will bind to the currently running loop when they are first locked / waited on; meaning we'll end up with isolation issues if we move to isolated ioloops for testing.

s_logging.logbase = await s_base.Base.anit()
s_logging.logbase._fini_at_exit = True

s_logging.logbase.schedCoro(s_logging._feedLogTask())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logbase feels like something we should be able to refcount & teardown on cell fini to prevent the feedtask from running beyond the lifetime of a cell; similar to how we handle other global tasks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a bit of a "not necessarily via a single cell in one python interpreter" issue here that would need to be discussed, but def worth talking through 👍

@@ -6626,7 +6647,9 @@ async def enableCronJob(self, iden):
'''
await self.agenda.enable(iden)
await self.feedBeholder('cron:enable', {'iden': iden}, gates=[iden])
logger.info(f'Enabled cron job {iden}', extra=await self.getLogExtra(iden=iden, status='MODIFY'))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to retain these status keys.

@vEpiphyte vEpiphyte added this to the v3.0.0 milestone Apr 4, 2025
@vEpiphyte vEpiphyte self-assigned this Jun 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants