Configure AWS SDK logging at runtime to hide ~/.aws/config WARN messages #88133
Conversation
We used to hide some noisy WARN log messages emitted by the AWS SDK by setting them to ERROR level in log4j2.properties. Since the repository plugins have been converted to modules, the log4j2.properties files are not always updated during if users have modified the log4j2.properties. This pull request configures the logging level of some specific AWS classes in order to hide the noisy messages even in the case the log4j2.properties files have been modified.
tlrx
added
>non-issue
:Distributed Coordination/Snapshot/Restore
elasticmachine
added
the
Team:Distributed
|
Pinging @elastic/es-distributed (Team:Distributed) |
| ).stream().map(LogManager::getLogger).collect(Collectors.toUnmodifiableMap(logger -> logger, Logger::getLevel)); | ||
| try { | ||
| awsConfigLoggers.forEach((logger, level) -> { | ||
| if (level.isLessSpecificThan(Level.ERROR)) { |
There was a problem hiding this comment.
Hmm actually I'm not sure about this bit. This means we can't override this behaviour at runtime even if we wanted to see these messages. Is that the intention? I guess we have to override INFO and WARN since these are the possible defaults, but maybe if the level is DEBUG or TRACE we shouldn't override it?
There was a problem hiding this comment.
Looking at my change and your comment, I think I should address this differently. What I did here hides the noisy warning messages during ClientConfiguration class initialization only. This is still possible that BasicProfileConfigFileLoader et al emit more warning messages after that.
I pushed f193c2f to change a bit this logic: it now detects if the AWS loggers are explicitly configured in log4j2.properties and in case they aren't, their levels are set to ERROR. This way we don't override any existing file configuration (if a user really wants to see debug or warn messages) during the startup and until the cluster settings are applied (in which case a different logging level might override the ERROR level we configured). I also added tests.
Let me know what you think.
| BasicProfileConfigFileLoader.class, | ||
| SdkMBeanRegistrySupport.class, | ||
| UseArnRegionResolver.class | ||
| ).stream().map(LogManager::getLogger).collect(Collectors.toUnmodifiableMap(logger -> logger, Logger::getLevel)); |
There was a problem hiding this comment.
Nit: List.of(...).stream() is just Stream.of(...).
There was a problem hiding this comment.
I think this is a reasonable approach too but it does mean that users have to set the level on these specific loggers, and not their ancestors. For instance, we explicitly set com.amazonaws to level WARN in the config file, but changing that isn't going to change these specific loggers.
I think that's ok for the sake of implementation simplicity but feel it deserves a mention in the docs to spell out exactly which classes are affected by this.
|
@elasticmachine update branch |
We used to hide some noisy WARN log messages emitted by
the AWS SDK by setting them to ERROR level in log4j2.properties.
Since the repository plugins have been converted to modules, the
log4j2.properties files are not always updated during if users
have modified the log4j2.properties.
This pull request configures the logging level of some specific
AWS classes in order to hide the noisy messages even in the case
the log4j2.properties files have been modified.
Relates #56346, #62522, #20313