Skip to content

Add timing instrumentation to debug River contract timeouts #1683

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

sanity
Copy link
Collaborator

@sanity sanity commented Jun 26, 2025

Summary

  • Add timing instrumentation to contract execution and packet processing
  • Log warnings when operations take longer than expected thresholds
  • Help identify blocking operations causing River PUT/GET timeouts

Investigation Context

River PUT/GET operations are timing out after 30 seconds. Hypothesis is that WASM contract execution is blocking the message pipeline, causing connection failures.

This instrumentation will help identify:

  • Contract operations taking >10ms
  • Packet processing taking >50ms
  • Correlation with River timeout issues

Test Plan

  • Deploy to ziggy gateway for ARM testing
  • Monitor logs during River CLI operations
  • Analyze timing data to confirm/refute blocking hypothesis

🤖 Generated with Claude Code

sanity and others added 10 commits June 19, 2025 09:34
- Update freenet-stdlib to 0.1.9 (includes panic fix + NodeQuery APIs)
- Fix compilation error in node.rs for release builds
- Bump freenet and fdev versions to 0.1.14
- Add timing logs for contract PUT/GET execution in contract/mod.rs
- Warn when contract operations take >10ms (blocking message pipeline)
- Add timing for overall packet processing in peer_connection.rs
- This will help identify WASM execution bottlenecks causing channel overflow
- Track channel overflow and dropped packets immediately
- Monitor PUT operation start/end timing
- Log message routing through NetworkBridge
- Track UDP send performance and channel backlogs
- Add queue depth monitoring for outbound packets

This instrumentation will help identify:
1. Channel buffer overflows causing packet drops
2. Message routing failures
3. UDP send performance issues
4. Queue buildup locations
- Track SuccessfulPut message reception and generation
- Log PUT state transitions to understand completion flow
- Add debug info to trace when operations move between states
- Focus on identifying why PUT completes with false status

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant