Skip to content

Avoid NOTE_ABSOLUTE timers for UPTIME/MONOTONIC on FreeBSD/OpenBSD.#943

Open
max-potapov wants to merge 1 commit into
swiftlang:mainfrom
max-potapov:freebsd-absolute-time-timers
Open

Avoid NOTE_ABSOLUTE timers for UPTIME/MONOTONIC on FreeBSD/OpenBSD.#943
max-potapov wants to merge 1 commit into
swiftlang:mainfrom
max-potapov:freebsd-absolute-time-timers

Conversation

@max-potapov

@max-potapov max-potapov commented May 22, 2026

Copy link
Copy Markdown

Motivation

libdispatch programs every EVFILT_TIMER with NOTE_ABSOLUTE on every
platform, computing an absolute deadline value via
_dispatch_time_now_cached and passing it to kqueue with
NOTE_NSECONDS | NOTE_ABSOLUTE (= NOTE_ABSTIME on BSD).

On FreeBSD / OpenBSD only the WALL clock works that way: kqueue with
NOTE_NSECONDS | NOTE_ABSTIME interprets the data as a
CLOCK_REALTIME nanosecond absolute deadline (FreeBSD subtracts the
boot offset internally; OpenBSD documents the realtime semantics
directly). That matches the value libdispatch already computes for
DISPATCH_CLOCK_WALL, so wall timers behave correctly and continue
to track wall-clock adjustments.

For DISPATCH_CLOCK_UPTIME / DISPATCH_CLOCK_MONOTONIC, libdispatch
hands kqueue a CLOCK_MONOTONIC nanosecond value that BSD kqueue
interprets as realtime — decades in the past after boot. kqueue
reports the timer as already-expired on every register, so the
dispatch event loop fires the timer block immediately, re-arms the
timer, fires it again, and so on. Observed on a long-running
HummingbirdCore HTTP server using Task.sleep as ~48 000 kevent()/s
with one CPU core pinned to ~100 % even with no real work pending.

PRs #879 and #931 corrected the NOTE_NSECONDS scaling for the
relative-time path on FreeBSD but did not touch the absolute-time
deadline computation this PR addresses.

Modifications

For os(FreeBSD) and os(OpenBSD), split the per-clock flag
selection so that NOTE_ABSOLUTE is kept for DISPATCH_CLOCK_WALL
(which works correctly on BSD) but dropped for
DISPATCH_CLOCK_UPTIME and DISPATCH_CLOCK_MONOTONIC. Introduce
DISPATCH_NOTE_ABSOLUTE_<kind> macros next to the existing
DISPATCH_NOTE_CLOCK_<kind> ones, and have
_dispatch_timer_index_to_fflags use them.

In _dispatch_event_loop_timer_arm, when running on BSD with a
non-WALL clock, convert the absolute deadline back to a relative
delay by subtracting the cached now. Using target -= now rather
than the simpler target = range.delay preserves any leeway already
folded into target by _dispatch_timers_force_max_leeway above
(range.leeway has been zeroed by that point, so an overwrite would
silently drop the addition and break LIBDISPATCH_TIMERS_FORCE_MAX_LEEWAY).

The change is guarded behind #if defined(__FreeBSD__) || defined(__OpenBSD__) and is a no-op on every other platform.

Result

Reproduced on FreeBSD 15.0-RELEASE-p9 with the official Apple Swift
FreeBSD preview toolchain (Swift 6.3-dev) by running the same Swift
binary
(HummingbirdCore HTTP server + a single periodic
Task.sleep(for: .seconds(3600)) loop, no real probes/work) against
both upstream and patched libdispatch.so via LD_LIBRARY_PATH:

=== upstream libdispatch ===
%CPU   RSS COMMAND
37.9 39736 /tmp/netwatch-exporter
syscall                     seconds   calls  errors
_umtx_op                3.999564140      16       0
kevent                  0.716818577  117827       0

=== patched libdispatch ===
%CPU   RSS COMMAND
 1.0 39820 /tmp/netwatch-exporter
syscall                     seconds   calls  errors
_umtx_op                3.999207012      16       0
kevent                  1.999276981      10       0

Same binary, same machine, same Swift toolchain — only
/opt/swift/lib/swift/freebsd/libdispatch.so swapped. kevent rate
drops by ~11 000×; CPU drops from one core pinned to ~idle.

In production (a Hummingbird-based Prometheus exporter on FreeBSD 15)
the same swap took the service from 70-98 % CPU down to 0.0 %.

Checks

  • Existing dispatch_timer* tests (dispatch_timer,
    dispatch_timer_short, dispatch_timer_timeout,
    dispatch_timer_set_time, dispatch_timer_bit31,
    dispatch_timer_bit63) pass on FreeBSD with both upstream
    and patched libdispatch. (They exercise the user-facing
    dispatch_after / DispatchSource APIs which complete before the
    busy loop can settle, so they don't independently detect the
    underlying tight-loop bug.)
  • Wall-clock semantics preserved on BSD:
    dispatch_after(dispatch_walltime(NULL, 2s)) returns at 2.001 s
    on the patched build, identical to upstream.
  • Uptime semantics preserved on BSD:
    dispatch_after(dispatch_time(DISPATCH_TIME_NOW, 2s)) returns
    at 2.001 s on the patched build.
  • All changes guarded behind #if defined(__FreeBSD__) || defined(__OpenBSD__); no behaviour change on Apple/Linux.
  • LIBDISPATCH_TIMERS_FORCE_MAX_LEEWAY semantics preserved —
    the relative-time conversion (target -= now) keeps any leeway
    already folded into target above.

Notes

  • No new failing-then-passing C unit test in this PR: the bug
    requires the Swift Concurrency cooperative executor + a concurrent
    NIO event loop to manifest, which is awkward to express against
    libdispatch's existing test suite without depending on the Swift
    runtime. The two smoke tests above (walltime / uptime
    dispatch_after) document the per-clock split and could be folded
    into tests/ if maintainers want them landed — happy to add as a
    follow-up commit.
  • A "deeper" root-cause fix would probably articulate the BSD kqueue
    NOTE_NSECONDS|NOTE_ABSTIME clock semantics in
    _dispatch_time_now_cached itself (so uptime/monotonic values get
    converted to realtime before being handed to kqueue). This PR
    intentionally takes the narrower route — sidestepping the absolute
    path for the two affected clocks — because it keeps the change
    small, self-contained, and lets users with broken
    Task.sleep-on-FreeBSD ship code today.

### Motivation:

libdispatch programs every EVFILT_TIMER with NOTE_ABSOLUTE on every
platform, computing an absolute deadline value via
`_dispatch_time_now_cached` and passing it to kqueue with
`NOTE_NSECONDS | NOTE_ABSOLUTE` (= `NOTE_ABSTIME` on BSD).

On FreeBSD/OpenBSD only the WALL clock works that way — kqueue
interprets the `NOTE_NSECONDS|NOTE_ABSTIME` value as CLOCK_REALTIME
nanoseconds, which matches what libdispatch computes for
`DISPATCH_CLOCK_WALL`.

For UPTIME / MONOTONIC, libdispatch passes a CLOCK_MONOTONIC
nanosecond value that BSD kqueue still interprets as realtime
(decades in the past after boot). kqueue reports the timer as
already-expired on every register, so the dispatch event loop fires
the timer block immediately, re-arms the timer, fires it again, and
so on. Observed on a long-running HummingbirdCore HTTP server using
`Task.sleep` as ~48 000 kevent()/s with one CPU core pinned to
~100 % even with no real work pending.

PRs swiftlang#879 and swiftlang#931 corrected the NOTE_NSECONDS scaling for the
relative-time path on FreeBSD but did not touch the absolute-time
deadline computation this PR addresses.

### Modifications:

For `os(FreeBSD)` and `os(OpenBSD)`, split the per-clock flag
selection so NOTE_ABSOLUTE is kept for DISPATCH_CLOCK_WALL (which
works correctly on BSD) but dropped for DISPATCH_CLOCK_UPTIME and
DISPATCH_CLOCK_MONOTONIC. Introduce `DISPATCH_NOTE_ABSOLUTE_<kind>`
macros next to the existing `DISPATCH_NOTE_CLOCK_<kind>` ones, and
have `_dispatch_timer_index_to_fflags` use them.

In `_dispatch_event_loop_timer_arm`, when running on BSD with a
non-WALL clock, convert the absolute deadline back to a relative
delay by subtracting the cached `now`. This preserves any leeway
already folded into `target` by `_dispatch_timers_force_max_leeway`
above (a simpler `target = range.delay` would silently drop that
addition because `range.leeway` has been zeroed by that point).

The change is guarded behind `#if defined(__FreeBSD__) ||
defined(__OpenBSD__)` and is a no-op on every other platform.

### Result:

Reproduced on FreeBSD 15.0-RELEASE-p9 with the official Apple Swift
FreeBSD preview toolchain (Swift 6.3-dev) by running the same Swift
binary (HummingbirdCore HTTP server + a single periodic
`Task.sleep(for: .seconds(3600))` loop, no real probes/work) against
both upstream and patched libdispatch.so via LD_LIBRARY_PATH:

```
=== upstream libdispatch ===
%CPU   RSS COMMAND
37.9 39736 /tmp/netwatch-exporter
syscall                     seconds   calls  errors
_umtx_op                3.999564140      16       0
kevent                  0.716818577  117827       0

=== patched libdispatch ===
%CPU   RSS COMMAND
 1.0 39820 /tmp/netwatch-exporter
syscall                     seconds   calls  errors
_umtx_op                3.999207012      16       0
kevent                  1.999276981      10       0
```

Same binary, same machine, same Swift toolchain — only
`/opt/swift/lib/swift/freebsd/libdispatch.so` swapped. kevent rate
drops by ~11 000×; CPU drops from one core pinned to ~idle.

In production (a Hummingbird-based Prometheus exporter on FreeBSD 15)
the same swap took the service from 70-98 % CPU down to 0.0 %.

Existing `dispatch_timer*` tests (`dispatch_timer`,
`dispatch_timer_short`, `dispatch_timer_timeout`,
`dispatch_timer_set_time`, `dispatch_timer_bit31`,
`dispatch_timer_bit63`) pass on FreeBSD with both upstream and
patched libdispatch — they exercise the user-facing dispatch_after /
DispatchSource APIs which complete before the busy loop can settle,
so they do not detect the underlying bug. Two smoke tests
documenting the per-clock split also pass cleanly on the patched
build:

```
$ ./dispatch_timer_walltime
dispatch_after(walltime + 2.0s) returned after 2.001s
PASS

$ ./dispatch_timer_uptime
uptime test: dispatch_after(NOW + 2s) returned after 2.001s
PASS
```

These two C-level reproducers are tiny and could be folded into the
existing tests/ directory if maintainers want them landed — happy to
add as a follow-up commit.
@max-potapov max-potapov force-pushed the freebsd-absolute-time-timers branch from caa5570 to 9c26068 Compare May 22, 2026 23:18
@max-potapov max-potapov changed the title Avoid NOTE_ABSOLUTE timers on FreeBSD/OpenBSD. Avoid NOTE_ABSOLUTE timers for UPTIME/MONOTONIC on FreeBSD/OpenBSD. May 22, 2026
@max-potapov max-potapov marked this pull request as ready for review May 22, 2026 23:24
Comment thread src/event/event_kevent.c
Comment on lines +60 to +71
/* On BSD, kqueue with NOTE_NSECONDS|NOTE_ABSTIME interprets data as a
* CLOCK_REALTIME nanosecond absolute deadline. That matches the value
* libdispatch computes for DISPATCH_CLOCK_WALL, so wall timers stay on
* the absolute-time path and continue to track wall-clock adjustments.
*
* For UPTIME / MONOTONIC libdispatch hands kqueue a CLOCK_MONOTONIC
* nanosecond value which BSD kqueue would interpret as realtime
* (decades in the past after boot), reporting the timer as
* already-expired on every register and producing a kevent() tight
* loop. Drop NOTE_ABSOLUTE for these clocks and let
* _dispatch_event_loop_timer_arm fall through to the relative-time
* path below. */

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this comment reads a little more complicated than it probably needs to, and maybe the one below as well. The root of the problem seems to be that _dispatch_time_now_cached is assumed below to always returrn an absolute time, but our UPTIME/MONOTONIC (which are synonyms, at least on FreeBSD) are relative to boot @ time=0.

Writing that out loud, I'm wondering if this breaks other assumptions of users that consume these clocks through _dispatch_time_now* -- it might be better to have _dispatch_uptime and _dispatch_monotonic_time fetch kern.boottime and add that in to avoid bugs like this in the future.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(That is: I see the note in the PR description, but I'm not sure that's actually a much larger change and it avoids other inconsistencies at the same time)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kevans91 – agreed, I can trim the comments.

I also agree that normalizing the BSD uptime/monotonic path around boottime is probably the cleaner architectural fix if those values are meant to feed the absolute-deadline timer path.

My hesitation is mostly validation, not preference for the narrow patch. The existing dispatch_timer tests pass on FreeBSD with both upstream and this patch, and the simple dispatch_after wall/uptime smoke tests also pass. So they don't currently exercise the long-running re-arm/tight-loop behavior that exposed this in the integration workload.

Before I rework this toward the boottime-based fix, what validation would you consider sufficient here? I can either:

  • keep this PR as the narrow fix and a record of the bug,
  • rework it into the boottime-based approach and validate with the current suite plus the original repro,
  • or first try to add a BSD-specific regression test that actually fails upstream.

I'm fine with whichever path is more useful for the project.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kevans91 – agreed, I can trim the comments.

I also agree that normalizing the BSD uptime/monotonic path around boottime is probably the cleaner architectural fix if those values are meant to feed the absolute-deadline timer path.

My hesitation is mostly validation, not preference for the narrow patch. The existing dispatch_timer tests pass on FreeBSD with both upstream and this patch, and the simple dispatch_after wall/uptime smoke tests also pass. So they don't currently exercise the long-running re-arm/tight-loop behavior that exposed this in the integration workload.

Before I rework this toward the boottime-based fix, what validation would you consider sufficient here? I can either:

  • keep this PR as the narrow fix and a record of the bug,
  • rework it into the boottime-based approach and validate with the current suite plus the original repro,
  • or first try to add a BSD-specific regression test that actually fails upstream.

IMO option 3 is the better option here- presumably you wouldn't really need to call it a BSD-specific test: it may not provide the exact same value on Linux, but it exercises a path through libdispatch that is seemingly not well-exercised today (whether that be kqueue or epoll/etc).

I'm fine with whichever path is more useful for the project.

With a disclaimer that I'm only tangentially involved in libdispatch things, though heavily involved in FreeBSD itself- I'd probably want someone else to stamp the path before you expand too much energy on it since their standards and values will certainly differ.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants