Replace Log4j2 with Logback

[diega]

PR description

As a continuation of the work made on #3285 this PR completely removes Log4j2 as a project dependency, substituting its behaviour with Logback. Most importantly, removes custom Log4j2 management for dealing with ANSI colors by using conditionals in the Logback config file.

TODO

• configure Splunk in logback.xml properly (cc @atoulme)

Documentation changes

Configure logging wiki page will need to be updated. This doc page explains how to override the default location of the configuration file. And also, the Hyperledger Besu Ansible role refers to a besu_log4j_config_file variable.

Changelog

• I thought about the changelog and included a changelog update if required.

[jflo]

So far this looks great, thanks so much for migrating this!

One thing we need to think about is how to deprecate inherited CLI flags. I'm not sure the best way to communicate that to users, it could be a breaking change that causes startup scripts to fail.

[diega]

Thanks @jflo. I tried to respect every existing CLI flag in this PR. Also I followed the exact same output format for logging messages, the only difference is about the colour palette that is used by Log4j2 vs Logback for the different levels.

With regards of the Sonar checks, those were the same approved for #3285, i.e. setting log level explicitly (this time for Logback) and using System.out for when the logging context wasn't initialized yet.

[ajsutton]

Sorry if I've mentioned this before, but how are we dealing with backwards compatibility for people who are using custom log4j.xml configurations?

[atoulme]

It might still be possible to use log4j2 with slf4j, just need to change the jars in the classpath. There's some options folks can use from there on.
Do you know of anyone using custom log4j2 configs, besides what we use for direct Splunk logging here?

[diega]

@atoulme I think we are not that agnostic of the logging backend [yet]. Features like changing the log level at runtime are implementation specific and we require implementation level dependency. Another option would be to add an abstraction layer like the one that vert.x has, but I don't know if it wouldn't be too much for Besu

[atoulme]

I agree, sounds like too much.

[ajsutton]

Do you know of anyone using custom log4j2 configs, besides what we use for direct Splunk logging here?

It's used on all the Besu nodes ConsenSys runs to output logs as JSON which is then sent processed and aggregated etc. I also use it personally particularly for development because it makes it so easy to change logging config at runtime (we do have a JSON-RPC API to do that but it's so much more painful). I've recommended using a custom log4j setup for anyone doing anything serious for years mainly because of that flexibility to change things at runtime - not sure how many people have actually listened...

I don't think it's a big deal to change the custom config format we'd use, but I think we need to give reasonable notice. I'd be against changing the logging backend if there wasn't a roughly equivalent approach to providing custom config, but the logback config looks extremely similar so I don't think we're losing anything, just changing some syntax.

[sonarqubecloud]

SonarCloud Quality Gate failed.   

0 Bugs
0 Vulnerabilities
3 Security Hotspots
2 Code Smells

74.2% Coverage
0.0% Duplication

[non-fungible-nelson]

Hi all - closed for now in favor of the existing solution in #3759. I think we can look at this again when we address tech debt post-merge. There was a larger lift than we could provide at this time.