Skip to content

Move trivial data access Mutex's to sync Mutex. - #133

Merged
chrisstaite-menlo merged 1 commit into
TraceMachina:masterfrom
chrisstaite-menlo:data_mutex
Jul 6, 2023
Merged

chrisstaite-menlo merged 1 commit into
TraceMachina:masterfrom
chrisstaite-menlo:data_mutex

Conversation

@chrisstaite-menlo

@chrisstaite-menlo chrisstaite-menlo commented Jul 5, 2023 •

Copy link
Copy Markdown
Collaborator

The Tokio documentation suggests that using an async Mutex for trivial data access (where the data is not required over an await) is more expensive and therefore should be avoided. This is due to the fact that the operations performed while the Mutex is locked are trivial and therefore switching task on the tokio thread is far more inefficient than simply waiting for the Mutex to become available.

See: https://docs.rs/tokio/latest/tokio/sync/struct.Mutex.html and https://tokio.rs/tokio/tutorial/shared-state


This change is Reviewable

@allada allada left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 13 of 14 files at r1, all commit messages.
Reviewable status: 13 of 14 files reviewed, 2 unresolved discussions (waiting on @chrisstaite-menlo)


cas/worker/running_actions_manager.rs line 952 at r1 (raw file):

    }

    fn get_action(&self, action_id: &ActionId) -> Result<Arc<Self::RunningAction>, Error> {

nit: I was following some code paths and found this entire function is never called.

Can we clean it up here?


cas/worker/running_actions_manager.rs line 966 at r1 (raw file):

            running_actions
                .iter()
                .filter_map(|(_action_id, action)| action.upgrade())

fyi: This could have been done without creating a Vec using:

.for_each(Self::kill_action)

cas/worker/running_actions_manager.rs line 970 at r1 (raw file):

        };
        for action in kill_actions {
            Self::kill_action(action)

nit: Lets keep this an await for now because #129 (comment) Needs to be fixed.

@allada allada left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: 13 of 14 files reviewed, 2 unresolved discussions (waiting on @chrisstaite-menlo)

@chrisstaite-menlo chrisstaite-menlo left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 13 of 14 files reviewed, all discussions resolved (waiting on @allada)


cas/worker/running_actions_manager.rs line 966 at r1 (raw file):

Previously, allada (Nathan (Blaise) Bruer) wrote…

fyi: This could have been done without creating a Vec using:

.for_each(Self::kill_action)

The Vec is required to drop the lock. I tried inlining it with the for_each and it looked ugly...

@allada allada left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 4 of 4 files at r2, 1 of 1 files at r3, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @chrisstaite-menlo)

@chrisstaite-menlo
chrisstaite-menlo merged commit 812b94d into TraceMachina:master Jul 6, 2023
@chrisstaite-menlo
chrisstaite-menlo deleted the data_mutex branch July 6, 2023 07:50
rejuvenile added a commit to rejuvenile/nativelink that referenced this pull request Aug 19, 2026
…a#262)

During the 2026-05-05 RSS-leak investigation a Bazel fetch-storm drove
RSS to a 27 GB plateau. Six info! log sites dominated the burst-time
output (cumulatively ~2300 events / 30 s) and compounded with mimalloc
page retention. Each is leftover diagnostic from TraceMachina#212/TraceMachina#218/TraceMachina#252
chunked-streaming work. Same shape as TraceMachina#255 / TraceMachina#186 / TraceMachina#253.

Demoted sites:

- bytestream_server.rs:385 "ByteStream::read: CAS read completed"
  525 events / 30 s. Just bumped to info! today via e59b776
  (LoggingReadStream consolidation). The 17 lines/sec sustained rate
  is too high for info! even though it carries throughput/duration —
  the CLAUDE.md rule "info! for transfer completions with throughput"
  was written for whole-RPC summaries, not per-stream-chunk reads.
- bytestream_server.rs:1007 "ByteStream::inner_read entry"
  525 events / 30 s. Per-call entry log; debug! is the right level.
- bytestream_server.rs:1369 "inner_read get_part_fut resolved
  (still awaiting consume)" 520 events / 30 s. Per-event branch trace.
  Sibling at :1332 (`after consume_err`) left at info! — fires only on
  the rarer error path.
- bytestream_server.rs:2010 "ByteStream::write: skipped, blob already
  exists" 377 events / 30 s. The WorkerProxyStore::has() write-skip
  optimization log; TraceMachina#133 already filed to demote it.
- streaming_blob.rs:325 "streaming blob writer sent eof,
  notify_waiters firing" 293 events / 30 s. Per-blob lifecycle event;
  the wakeup propagates to readers without needing a separate log.
- existence_cache_store.rs:147,154,156 eviction-callback trio
  (~258 events / 30 s each). Per-eviction mechanical follow-ups.
  Wedge-digest gates at :150 left at info! — they're explicitly
  filtered to a single digest under investigation.
- fast_slow_store.rs:1939 "populate sending streaming_writer.send_error
  after head failure" 241 events / 30 s. Per-failed-RPC mechanical
  follow-up. The summary line at :1927 ("populate head_result Err")
  left at info! — one line per failure is the right cadence.

streaming_blob.rs's `info` import removed (now unused). All four
files already had `debug` in scope.

Behavior-only change: no flow-control, concurrency, or storage
semantics modified. Build verified --release --features
quic,pprof,chunked_fast_slow.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants