Skip to content

Always ignore kombu library "heartbeat_tick" debug log messages #4557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Feb 19, 2019
Merged

Always ignore kombu library "heartbeat_tick" debug log messages #4557

merged 4 commits into from
Feb 19, 2019

Conversation

Kami
Copy link
Member

@Kami Kami commented Feb 19, 2019 •
edited
Loading

In the past, I opened a change so we ignore kombu heartbeat_tick debug log messages which are logged every 2 ms by default when either --debug service command line flag is used or when system.debug st2.conf config option is set to True.

While troubleshooting release workflows on st2cicd server, I found an edge case.

If DEBUG log level is set in the service logging config, but --debug flag / system.debug config option is not used, those messages won't be ignored.

This is not desired, because those messages will pollute the logs and make it very hard to find other actual important DEBUG and non DEBUG log messages.

Here is an example from st2workflowengine.log on st2cicd:

...
2019-02-19 08:11:54,608 140494121387664 DEBUG connection [-] heartbeat_tick : for connection 4f67e9462fa147adb9377ffb5f4818fa
2019-02-19 08:11:55,608 140494121387664 DEBUG connection [-] heartbeat_tick : for connection 4f67e9462fa147adb9377ffb5f4818fa
2019-02-19 08:11:56,608 140494121387664 DEBUG connection [-] heartbeat_tick : for connection 4f67e9462fa147adb9377ffb5f4818fa
2019-02-19 08:11:57,609 140494121387664 DEBUG connection [-] heartbeat_tick : for connection 4f67e9462fa147adb9377ffb5f4818fa
2019-02-19 08:11:58,609 140494121387664 DEBUG connection [-] heartbeat_tick : for connection 4f67e9462fa147adb9377ffb5f4818fa
2019-02-19 08:11:59,609 140494121387664 DEBUG connection [-] heartbeat_tick : for connection 4f67e9462fa147adb9377ffb5f4818fa
2019-02-19 08:12:00,610 140494121387664 DEBUG connection [-] heartbeat_tick : for connection 4f67e9462fa147adb9377ffb5f4818fa
...

TODO

  • Cherry pick into v2.10 branch

@Kami
Make sure we also exclude "heartbeat_tick" log messages which are logged
9ba7f9e 
y kombu every 2 ms by default even when --debug flag / system.debug
config option is not used.

This way we avoid unnecessary noise when DEBUG level is used, but debug
flag is not.
@Kami Kami added the logging label Feb 19, 2019
@Kami Kami added this to the 2.10.2 milestone Feb 19, 2019
bigmstone
bigmstone approved these changes Feb 19, 2019
View reviewed changes
Copy link
Contributor

@bigmstone bigmstone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@Kami Kami merged commit 955937f into master Feb 19, 2019
@Kami Kami deleted the heartbeat_tick_ignore branch February 19, 2019 15:51
@Kami Kami mentioned this pull request Feb 19, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bigmstone bigmstone bigmstone approved these changes

Assignees
No one assigned
Labels
logging
Projects
None yet
Milestone
2.10.2
Development

Successfully merging this pull request may close these issues.
2 participants
@Kami @bigmstone