Skip to content

Commit

Permalink
feat(logging): create global log4j config (#23631) (#29694)
Browse files Browse the repository at this point in the history 
### Proposed Changes
Issue #23631 

* Add global log4j2 configuration
* default log4j2.configurationFile set in tomcat setenv.sh
* default
Log4jContextSelector=org.apache.logging.log4j.core.async.BasicAsyncLoggerContextSelector
   * This provides a single context and is not based on the classloader
* created lib folder for logging jars and added to tomcat classpath
* moved redis session manager jars into their own folder
* Normalized maven dependencies to help with debugging and versions
centrally specified in bom
* Removed log4j reloading functionality 

As well as using a common configuration for tomcat and webapp logs. This
also resolves an inconsistency when The logging class is referenced in
code before the ContextLifecycleListener is called. Previously the wrong
configuration contained in the resources classpath of the ant-tooling
jar was picked up initially.

Tests

* Test tomcat log messages are output to the same location
* Test any issues with logging from plugins
* Test ability to override log4j config and change log levels for core
tomcat classes, webapp classes and plugin classes. currently can map to
/srv/OVERRIDE/WEB-INF/log4j in container or update CATALINA_OPTS to
include -Dlog4j2.configurationFile={path to file}
* Test session manager functionality is not broken
* Will need a review of the logging behavior from cloud engineering when
this image is applied into our cloud environment including any log-level
changes they may want to have by default


### Further changes
There is follow up work to this PR that improves the way that the config
can be specified and overridden. This change maintains the existing
config file behavior but just applies it across the entire application.



### Checklist
- [ ] Tests
- [ ] Translations
- [ ] Security Implications Contemplated (add notes if applicable)

### Additional Info
This PR resolves #23631 (create global log4j config).


### Screenshots
Original             |  Updated
:-------------------------:|:-------------------------:
** original screenshot **  |  ** updated screenshot **
  • Loading branch information
@spbolton
spbolton authored Aug 22, 2024
1 parent 873c6a7 commit 46b0a3d
Show file tree
Hide file tree
Showing 14 changed files with 190 additions and 128 deletions.
2 changes: 1 addition & 1 deletion .mvn/maven-build-cache-config.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ https://maven.apache.org/extensions/maven-build-cache-extension/maven-build-cach
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/BUILD-CACHE-CONFIG/1.0.0 https://maven.apache.org/xsd/build-cache-config-1.0.0.xsd">
<configuration>
<enabled>true</enabled>
<enabled>false</enabled>
<hashAlgorithm>SHA-256</hashAlgorithm>
<validateXml>true</validateXml>
<local>
Expand Down
7 changes: 7 additions & 0 deletions bom/application/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1433,6 +1433,13 @@

<!-- Apache Tomcat -->

<dependency>
<groupId>com.dotcms</groupId>
<artifactId>tomcat-redis-session-manager</artifactId>
<version>1.3</version>
<scope>provided</scope>
</dependency>

<dependency>
<!-- Jasper is the Tomcat JSP Engine -->
<groupId>org.apache.tomcat</groupId>
Expand Down
30 changes: 21 additions & 9 deletions bom/logging/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
<properties>
<maven.deploy.skip>false</maven.deploy.skip>
<flatten.mode>bom</flatten.mode>
<log4j.version>2.20.0</log4j.version>
<slf4j.version>2.0.7</slf4j.version>
<log4j.version>2.23.1</log4j.version>
<slf4j.version>2.0.9</slf4j.version>
</properties>

<dependencyManagement>
Expand All @@ -27,30 +27,42 @@
Do not include incompatible artifacts e.g. -->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<artifactId>log4j-api</artifactId>
<version>${log4j.version}</version>
<scope>compile</scope>
</dependency>
<dependency>
<!-- Support log4j1.2.x API in Log4j2 -->
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-1.2-api</artifactId>
<artifactId>log4j-iostreams</artifactId>
<version>${log4j.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>${log4j.version}</version>
<scope>compile</scope>
</dependency>
<dependency>
<!-- Support Java Commons Logging API in Log4j2 -->
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-1.2-api</artifactId>
<version>${log4j.version}</version>
<scope>runtime</scope>
</dependency>
<dependency>
<!-- Support Java Commons Logging API in Log4j2 -->
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-jcl</artifactId>
<version>${log4j.version}</version>
<scope>compile</scope>
<scope>runtime</scope>
</dependency>
<dependency>
<!-- Support SLF4j2 API in Log4j2 -->
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j2-impl</artifactId>
<version>${log4j.version}</version>
<scope>compile</scope>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
Expand All @@ -61,16 +73,16 @@
<dependency>
<!-- Servlet container initialization of Log4j2 -->
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-web</artifactId>
<artifactId>log4j-appserver</artifactId>
<version>${log4j.version}</version>
<scope>compile</scope>
<scope>runtime</scope>
</dependency>
<dependency>
<!-- Async support for Log4J 2.x -->
<groupId>com.lmax</groupId>
<artifactId>disruptor</artifactId>
<version>3.3.4</version>
<scope>compile</scope>
<scope>runtime</scope>
</dependency>

</dependencies>
Expand Down
102 changes: 72 additions & 30 deletions dotCMS/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,6 @@
<palantirJavaFormat.version>2.29.0</palantirJavaFormat.version>
<src.dir>${project.basedir}/src/main/java</src.dir>
<test.src.dir>${project.basedir}/src/test/java</test.src.dir>
<version.spotless.plugin>2.36.0</version.spotless.plugin>
<batik.version>1.16</batik.version>
<glowroot.version>0.14.1</glowroot.version>
<version.cargo.plugin>1.10.6</version.cargo.plugin>
<assembly-directory>${basedir}/target/dist</assembly-directory>
Expand All @@ -27,6 +25,8 @@
<buildDocker>true</buildDocker>
<tomcat-dist-folder>dotserver/tomcat-${tomcat.version}</tomcat-dist-folder>
<tomcat-lib-folder>${assembly-directory}/${tomcat-dist-folder}/lib</tomcat-lib-folder>
<tomcat-log4j-lib-folder>${assembly-directory}/${tomcat-dist-folder}/log4j2/lib</tomcat-log4j-lib-folder>
<session-manager-lib-folder>${assembly-directory}/${tomcat-dist-folder}/session-manager/lib</session-manager-lib-folder>
<tomcat9-overrides>${project.basedir}/src/main/resources/container/tomcat9</tomcat9-overrides>
<exploded-webapp-dir>${assembly-directory}/${tomcat-dist-folder}/webapps/ROOT</exploded-webapp-dir>
<war.output.file>${project.build.directory}/${project.build.finalName}-war.war</war.output.file>
Expand Down Expand Up @@ -63,10 +63,6 @@
<test.felix.undeployed.folder>felix/core/undeployed</test.felix.undeployed.folder>
<test.felix.system.undeployed.folder>felix/system/undeployed</test.felix.system.undeployed.folder>
<!-- <docker.platforms>linux/arm64,linux/amd64</docker.platforms>-->
<tomcat.redis-session-manager.version>1.3</tomcat.redis-session-manager.version>
<commons-pool2.version>2.11.1</commons-pool2.version>
<org.slfj4.version>1.7.36</org.slfj4.version>
<jedis.version>4.4.6</jedis.version>
<tomcat.run.data.folder>${project.build.directory}/tomcat-run-data/${context.name}</tomcat.run.data.folder>
<cleanup.data.folder>false</cleanup.data.folder>
<!-- Add enterprise to sonar sources -->
Expand All @@ -88,6 +84,11 @@
<type>zip</type>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.dotcms</groupId>
<artifactId>tomcat-redis-session-manager</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.dotcms.plugins</groupId>
<artifactId>dotcms-system-bundles</artifactId>
Expand Down Expand Up @@ -1351,21 +1352,30 @@
<!-- Support Java Commons Logging API in Log4j2 -->
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-jcl</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<!-- Support SLF4j2 API in Log4j2 -->
<!-- Support Java Commons Logging API in Log4j2 -->
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j2-impl</artifactId>
<artifactId>log4j-jcl</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<!-- Servlet container initialization of Log4j2 -->
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<scope>compile</scope>
</dependency>
<dependency>
<!-- Added to tomcat classpath -->
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-web</artifactId>
<artifactId>log4j-appserver</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<!-- Async support for Log4J 2.x -->
<groupId>com.lmax</groupId>
<artifactId>disruptor</artifactId>
<scope>runtime</scope>
</dependency>

<!-- Graalvm Js Engine -->
Expand Down Expand Up @@ -1402,6 +1412,12 @@
</exclusions>
</dependency>

<dependency>
<groupId>org.apache.tomcat</groupId>
<artifactId>tomcat-catalina</artifactId>
<version>${tomcat.version}</version>
<scope>provided</scope>
</dependency>

</dependencies>
<dependencyManagement>
Expand Down Expand Up @@ -1535,37 +1551,69 @@
<configuration>
<artifactItems>
<artifactItem>
<groupId>com.dotcms</groupId>
<artifactId>tomcat-redis-session-manager</artifactId>
<version>${tomcat.redis-session-manager.version}</version>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<type>jar</type>
<overWrite>true</overWrite>
<outputDirectory>${tomcat-lib-folder}</outputDirectory>
<overWrite>false</overWrite>
<outputDirectory>${tomcat-log4j-lib-folder}</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
<version>${commons-pool2.version}</version>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<type>jar</type>
<overWrite>true</overWrite>
<outputDirectory>${tomcat-lib-folder}</outputDirectory>
<overWrite>false</overWrite>
<outputDirectory>${tomcat-log4j-lib-folder}</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-appserver</artifactId>
<type>jar</type>
<overWrite>false</overWrite>
<outputDirectory>${tomcat-log4j-lib-folder}</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j2-impl</artifactId>
<type>jar</type>
<overWrite>false</overWrite>
<outputDirectory>${tomcat-log4j-lib-folder}</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${org.slfj4.version}</version>
<type>jar</type>
<overWrite>false</overWrite>
<outputDirectory>${tomcat-log4j-lib-folder}</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>com.dotcms</groupId>
<artifactId>tomcat-redis-session-manager</artifactId>
<type>jar</type>
<overWrite>true</overWrite>
<outputDirectory>${tomcat-lib-folder}</outputDirectory>
<outputDirectory>${session-manager-lib-folder}</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
<type>jar</type>
<overWrite>true</overWrite>
<outputDirectory>${session-manager-lib-folder}</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>${jedis.version}</version>
<type>jar</type>
<overWrite>true</overWrite>
<outputDirectory>${tomcat-lib-folder}</outputDirectory>
<outputDirectory>${session-manager-lib-folder}</outputDirectory>
</artifactItem>

<dependency>
<!-- Async support for Log4J 2.x -->
<groupId>com.lmax</groupId>
<artifactId>disruptor</artifactId>
<overWrite>true</overWrite>
<outputDirectory>${tomcat-log4j-lib-folder}</outputDirectory>
</dependency>
</artifactItems>
</configuration>
</execution>
Expand Down Expand Up @@ -1913,12 +1961,6 @@
<file>${tomcat9-overrides}/bin/setenv.sh</file>
<todir>bin</todir>
</configfile>
<!--
<configfile>
<file>${tomcat9-overrides}/conf/catalina.properties</file>
<todir>conf</todir>
</configfile>
-->
<configfile>
<file>${tomcat9-overrides}/conf/server.xml</file>
<todir>conf</todir>
Expand Down
16 changes: 10 additions & 6 deletions dotCMS/src/main/java/com/dotcms/business/bytebuddy/ByteBuddyFactory.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
import com.dotcms.business.WrapInTransaction;
import com.dotcms.util.EnterpriseFeature;
import com.dotcms.util.LogTime;
import com.dotmarketing.util.Logger;
import net.bytebuddy.agent.ByteBuddyAgent;
import net.bytebuddy.agent.builder.AgentBuilder;
import net.bytebuddy.asm.Advice;
Expand All @@ -23,6 +22,8 @@
import java.util.Map;
import java.util.Set;
import java.util.concurrent.atomic.AtomicBoolean;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

import static net.bytebuddy.matcher.ElementMatchers.declaresMethod;
import static net.bytebuddy.matcher.ElementMatchers.isAnnotatedWith;
Expand All @@ -39,6 +40,9 @@
*/
public class ByteBuddyFactory {

// Use base log4j logger here to prevent recursive issues with internal Logger class
private static final Logger LOGGER = LogManager.getLogger(ByteBuddyFactory.class);

private static final AtomicBoolean agentLoaded = new AtomicBoolean(false);
private static final Map<Class<? extends Annotation>, Class<?>> adviceMap = Map.of(
WrapInTransaction.class, WrapInTransactionAdvice.class,
Expand Down Expand Up @@ -69,16 +73,16 @@ public static void init() {
if (!agentLoaded.get()) {
try {
premain(null, ByteBuddyAgent.install());
Logger.info(ByteBuddyFactory.class, "Loaded ByteBuddy Advice");
LOGGER.info("Loaded ByteBuddy Advice");
} catch (Exception e) {
Logger.error(ByteBuddyFactory.class, "Cannot install ByteBuddy Advice");
LOGGER.error("Cannot install ByteBuddy Advice");
}
}
}


public static void premain(final String arg, final Instrumentation inst) throws Exception {
Logger.info(ByteBuddyFactory.class, "Starting ByteBuddy Agent");
LOGGER.info("Starting ByteBuddy Agent");
if (!agentLoaded.compareAndSet(false, true)) {
return;
}
Expand Down Expand Up @@ -122,9 +126,9 @@ public ClassFileLocator classFileLocator(final ClassLoader classLoader, final Ja
})

.installOn(inst);
Logger.info(ByteBuddyFactory.class, "ByteBuddy Initialized");
LOGGER.info("ByteBuddy Initialized");
} catch (Exception e) {
Logger.error(ByteBuddyFactory.class, "Error Initializing ByteBuddy", e);
LOGGER.error("Error Initializing ByteBuddy", e);
}


Expand Down
22 changes: 0 additions & 22 deletions dotCMS/src/main/java/com/dotmarketing/listeners/ContextLifecycleListener.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,15 @@
import com.dotcms.util.AsciiArt;
import com.dotmarketing.business.CacheLocator;
import com.dotmarketing.common.reindex.ReindexThread;
import com.dotmarketing.loggers.Log4jUtil;
import com.dotmarketing.quartz.QuartzUtils;
import com.dotmarketing.util.Config;
import com.dotmarketing.util.ConfigUtils;
import com.dotmarketing.util.Logger;
import io.vavr.control.Try;

import javax.servlet.ServletContext;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import javax.websocket.server.ServerContainer;
import java.io.File;

/**
*
Expand Down Expand Up @@ -62,25 +59,6 @@ public void contextInitialized(ServletContextEvent arg0) {

Config.setMyApp(arg0.getServletContext());


String path = null;
try {

String contextPath = Config.CONTEXT.getRealPath("/");
if ( !contextPath.endsWith( File.separator ) ) {
contextPath += File.separator;
}
File file = new File(contextPath + "WEB-INF" + File.separator + "log4j" + File.separator + "log4j2.xml");
path = file.toURI().toString();

} catch (Exception e) {
Logger.error(this,e.getMessage(),e);
}

//Initialises/reconfigures log4j based on a given log4j configuration file
Log4jUtil.initializeFromPath(path);


installWebSocket(arg0.getServletContext());
}

Expand Down
Loading

0 comments on commit 46b0a3d

Please sign in to comment.