Commit 6ce2f3e
committed
Fix CPU time going backwards when
When we `daemonize`, we fork the process and cputime metrics get confused
about the per-thread resource usage appearing to go backwards because we're
comparing the resource usage (`rusage`) from the original process to the
forked process.
We now kick off the background tasks (`run_as_background_process`) after we
have forked the process so the `rusage` we record when we `start` is in the
same thread when we `stop`.
Bad log examples from before:
```
synapse.logging.context - ERROR - _schedule_next_expiry-0 - utime went backwards! 0.050467 < 0.886526
synapse.logging.context - ERROR - _schedule_db_events-0 - stime went backwards! 0.009941 < 0.155106
synapse.logging.context - ERROR - wake_destinations_needing_catchup-0 - stime went backwards! 0.010175 < 0.130923
synapse.logging.context - ERROR - resume_sync_partial_state_room-0 - utime went backwards! 0.052898 < 0.886526
```
Testing strategy:
1. Run with `daemonize: true` in your `homeserver.yaml`
1. `poetry run synapse_homeserver --config-path homeserver.yaml`
1. Shutdown the server
1. Look for any bad log entries in your homeserver logs:
- `Expected logging context sentinel but found main`
- `Expected logging context main was lost`
- `utime went backwards!`/`stime went backwards!`daemonize
1 parent 93044f4 commit 6ce2f3e
2 files changed
+7
-7
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
601 | 601 | | |
602 | 602 | | |
603 | 603 | | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
604 | 610 | | |
605 | 611 | | |
606 | 612 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
366 | 366 | | |
367 | 367 | | |
368 | 368 | | |
369 | | - | |
370 | | - | |
371 | | - | |
372 | | - | |
373 | | - | |
374 | | - | |
375 | 369 | | |
376 | 370 | | |
377 | 371 | | |
| |||
410 | 404 | | |
411 | 405 | | |
412 | 406 | | |
413 | | - | |
| 407 | + | |
414 | 408 | | |
415 | 409 | | |
416 | 410 | | |
| |||
0 commit comments