Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Tokio-Console in Sui for debugging #4438

Merged
merged 4 commits into from
Oct 3, 2022
Merged

Conversation

velvia
Copy link
Contributor

@velvia velvia commented Sep 1, 2022

For more details on Tokio Console:
https://docs.rs/tokio-console/latest/tokio_console/

This will give us visibility into running tasks and resources Tokio uses.

Copy link
Contributor

@huitseeker huitseeker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense to me, for now! Perhaps add a comment 'this is meant to enable Tokio console and we plan to revisit this variable post testnet' on the Tokio experimental env var?

@velvia
Copy link
Contributor Author

velvia commented Sep 1, 2022

Actually I'm not sure this is going to work.
I can connect with Tokio-console to a Sui-node binary compiled by these changes, and with TOKIO_CONSOLE=true env var. However, no tasks are displayed.

I followed all the recommendations in https://docs.rs/console-subscriber/0.1.7/console_subscriber/#enabling-tokio-instrumentation

You have to enable TRACE logs for Tokio and tracing, which just spews an unacceptable amount of logs. And even after that I still am not getting any instrumentation. Granted I had no transactions, but I should be able to see something happening just with consensus and checkpoints, etc.

@velvia
Copy link
Contributor Author

velvia commented Sep 30, 2022

Update: Newer version of Mysten-infra fixed the issue so Tokio Console is working now.

@velvia velvia force-pushed the ec/enable-tokio-console branch from 263f0a4 to 4bdffd8 Compare October 3, 2022 17:08
@velvia velvia enabled auto-merge (squash) October 3, 2022 17:14
@velvia velvia force-pushed the ec/enable-tokio-console branch from 4bdffd8 to f7d46dc Compare October 3, 2022 18:17
@velvia velvia merged commit 4a49f38 into main Oct 3, 2022
@velvia velvia deleted the ec/enable-tokio-console branch October 3, 2022 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants