slf4j (logback) logs not getting forwarded to open-telemetry-collector by OpenTelemetryAppender

[TanishqDhussa]

I have created a spring-boot micro-service with micrometer tracing . I have also configured a open-telemetry-collector to forward the logs to datadog. The otel-collector is running in a docker container along with the micro-service.
I have configured "io.opentelemetry.instrumentation.logback.appender.v1_0.OpenTelemetryAppender" in logback.xml file to forward the logs also to otel-collector along with traces. However, in datadog, I can only see traces and not logs. The logs printed are using the same traceId and spanId as that of trace. But due to some unknown reason, otel-collector is not exporting logs like it is exporting traces.

Error :
91.68 Caused by: java.lang.ClassNotFoundException: io.opentelemetry.instrumentation.logback.appender.v1_0.OpenTelemetryAppender

91.68 07:21:56,630 |-ERROR in ch.qos.logback.core.model.processor.AppenderModelHandler - Could not create an Appender of type [io.opentelemetry.instrumentation.logback.appender.v1_0.OpenTelemetryAppender]. ch.qos.logback.core.util.DynamicClassLoadingException: Failed to instantiate type io.opentelemetry.instrumentation.logback.appender.v1_0.OpenTelemetryAppender 91.68 at ch.qos.logback.core.util.DynamicClassLoadingException: Failed to instantiate type io.opentelemetry.instrumentation.logback.appender.v1_0.OpenTelemetryAppender

I am using below mentioned dependency in pom.xml
<dependency> <groupId>io.opentelemetry.instrumentation</groupId> <artifactId>opentelemetry-logback-1.0</artifactId> <version>1.9.2-alpha</version> <scope>runtime</scope> </dependency>

And this is how my logback.xml file looks like :
`

<property name="LOGS" value="./logs" />

<appender name="Console"
    class="ch.qos.logback.core.ConsoleAppender">
    <layout class="ch.qos.logback.classic.PatternLayout">
        <Pattern>
            %black(%d{ISO8601}) %highlight(%-5level) [%blue(%t)] %yellow(%C{1}): trace_id: %X{traceId} span_id: %X{spanId} %msg%n%throwable
        </Pattern>
    </layout>
</appender>

<appender name="RollingFile"
    class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${LOGS}/up-user-service.log</file>
    <encoder
        class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
        <Pattern>%d %p %C{1} [%t] %m%n</Pattern>
    </encoder>

    <rollingPolicy
        class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
        <!-- rollover daily and when the file reaches 10 MegaBytes -->
        <fileNamePattern>${LOGS}/archived/up-user-service-%d{yyyy-MM-dd}.%i.log
        </fileNamePattern>
        <timeBasedFileNamingAndTriggeringPolicy
            class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
            <maxFileSize>10MB</maxFileSize>
        </timeBasedFileNamingAndTriggeringPolicy>
    </rollingPolicy>
</appender>

 <!-- io.opentelemetry.instrumentation.logback.v1_0.OpenTelemetryAppender -->
<appender name="OTEL" class="io.opentelemetry.instrumentation.logback.appender.v1_0.OpenTelemetryAppender"></appender>

<!-- LOG everything at INFO level -->
<root level="info">
    <appender-ref ref="RollingFile" />
    <appender-ref ref="Console" />
</root>

<logger name="com.uplight.upuserservice" level="trace" additivity="false">
    <appender-ref ref="RollingFile" />
    <appender-ref ref="Console" />
    <appender-ref ref="OTEL" />
</logger>

`

I also referred below mentioned issue :
#7723

[laurit]

replace

<dependency>
  <groupId>io.opentelemetry.instrumentation</groupId>
  <artifactId>opentelemetry-logback-1.0</artifactId>
  <version>1.9.2-alpha</version>
</dependency>

with https://central.sonatype.com/artifact/io.opentelemetry.instrumentation/opentelemetry-logback-appender-1.0/1.28.0-alpha. opentelemetry-logback-1.0 does not contain the appender class io.opentelemetry.instrumentation.logback.appender.v1_0.OpenTelemetryAppender it has a different name there.

[TanishqDhussa]

Thanks, Lauri, after replacing the dependency with the above-suggested dependency ,
I am getting this error :

`95.55 Description:
95.55 
95.55 An attempt was made to call a method that does not exist. The attempt was made from the following location:
95.55 
95.55     io.opentelemetry.instrumentation.logback.appender.v1_0.OpenTelemetryAppender.append(OpenTelemetryAppender.java:74)
95.55 
95.55 The following method did not exist:
95.55 
95.55     'io.opentelemetry.api.logs.LoggerProvider io.opentelemetry.api.OpenTelemetry.getLogsBridge()'
95.55 
95.55 The calling method's class, io.opentelemetry.instrumentation.logback.appender.v1_0.OpenTelemetryAppender, was loaded from the following location:
95.55 
95.55     jar:file:/root/.m2/repository/io/opentelemetry/instrumentation/opentelemetry-logback-appender-1.0/1.28.0-alpha/opentelemetry-logback-appender-1.0-1.28.0-alpha.jar!/io/opentelemetry/instrumentation/logback/appender/v1_0/OpenTelemetryAppender.class
95.55 
95.55 The called method's class, io.opentelemetry.api.OpenTelemetry, is available from the following locations:
95.55 
95.55     jar:file:/root/.m2/repository/io/opentelemetry/opentelemetry-api/1.25.0/opentelemetry-api-1.25.0.jar!/io/opentelemetry/api/OpenTelemetry.class
95.55 
95.55 The called method's class hierarchy was loaded from the following locations:
95.55 
95.55     io.opentelemetry.api.OpenTelemetry: file:/root/.m2/repository/io/opentelemetry/opentelemetry-api/1.25.0/opentelemetry-api-1.25.0.jar
95.55 
95.55 
95.55 Action:
95.55 
95.55 Correct the classpath of your application so that it contains compatible versions of the classes io.opentelemetry.instrumentation.logback.appender.v1_0.OpenTelemetryAppender and io.opentelemetry.api.OpenTelemetry

95.55 2023-07-13 09:02:28,760 ERROR [main] o.s.t.c.TestContextManager: trace_id:  span_id:  Caught exception while allowing TestExecutionListener [org.springframework.boot.test.mock.mockito.MockitoTestExecutionListener] to prepare test instance [com.uplight.upuserservice.controller.UserControllerTest@7de843ef]
95.55 java.lang.NoSuchMethodError: 'io.opentelemetry.api.logs.LoggerProvider io.opentelemetry.api.OpenTelemetry.getLogsBridge()'
95.55 	at io.opentelemetry.instrumentation.logback.appender.v1_0.OpenTelemetryAppender.append(OpenTelemetryAppender.java:74)
95.55 	at io.opentelemetry.instrumentation.logback.appender.v1_0.OpenTelemetryAppender.append(OpenTelemetryAppender.java:22)
95.55 	at ch.qos.logback.core.UnsynchronizedAppenderBase.doAppend(UnsynchronizedAppenderBase.java:85)

Adding test class for reference :/**
*
*/

package com.uplight.upuserservice.controller;

import static org.hamcrest.CoreMatchers.is;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.BDDMockito.given;
import static org.mockito.BDDMockito.willDoNothing;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.put;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.delete;
import static org.springframework.test.web.servlet.result.MockMvcResultHandlers.print;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

import java.util.ArrayList;
import java.util.List;
import java.util.Optional;

import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest;
import org.springframework.boot.test.mock.mockito.MockBean;
import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.ResultActions;

import com.fasterxml.jackson.databind.ObjectMapper;
import com.uplight.upuserservice.data.entity.User;
import com.uplight.upuserservice.data.repository.UserRepository;
/**
 * @author lokendra
 * @date 25 May 2023
 * 
 * Test class to cover User Controller CRUD APIs
 *
 */
@WebMvcTest
public class UserControllerTest {

	@Autowired
	private MockMvc mockMvc;

	@MockBean
	private UserRepository userRepository;

	@Autowired
	private ObjectMapper objectMapper;

	/**
	 * Covers getAllUsers API
	 * @throws Exception
	 */
	@Test
	public void test_getAllUsers() throws Exception {
		// given - precondition or setup
		List<User> listOfUsers = new ArrayList<>();
		listOfUsers.add(User.builder().firstName("Steve").lastName("Austin").emailId("steve.austin@gmail.com").build());
		listOfUsers.add(User.builder().firstName("John").lastName("Cena").emailId("john.cena@gmail.com").build());
		given(userRepository.findAll()).willReturn(listOfUsers);

		ResultActions response = mockMvc.perform(get("/api/v1/users/"));

		response.andExpect(status().isOk())
		.andExpect(jsonPath("$.size()",
				is(listOfUsers.size())));

	}

	/**
	 * Covers createUser API
	 * @throws Exception
	 */
	@Test
	public void test_createUser() throws Exception {
		User user = User.builder().firstName("Steve").lastName("Austin").emailId("steve.austin@gmail.com").build();
		given(userRepository.save(any(User.class)))
		.willAnswer((invocation)-> invocation.getArgument(0));

		ResultActions response = mockMvc.perform(post("/api/v1/users/")
				.contentType(MediaType.APPLICATION_JSON)
				.content(objectMapper.writeValueAsString(user)));

		response.andExpect(status().isCreated())
		.andExpect(jsonPath("$.firstName",
				is(user.getFirstName())))
		.andExpect(jsonPath("$.lastName",
				is(user.getLastName())))
		.andExpect(jsonPath("$.emailId",
				is(user.getEmailId())));

	}

	/**
	 * Covers getUserById API
	 * @throws Exception
	 */
	@Test
	public void test_getUserById() throws Exception {
		long userId= 1L;
		User user = User.builder().firstName("Steve").lastName("Austin").emailId("steve.austin@gmail.com").build();
		given(userRepository.findById(userId)).willReturn(Optional.of(user));

		ResultActions response = mockMvc.perform(get("/api/v1/users/{id}", userId));
		response.andExpect(status().isOk())
		.andExpect(jsonPath("$.firstName", is(user.getFirstName())))
		.andExpect(jsonPath("$.lastName", is(user.getLastName())))
		.andExpect(jsonPath("$.emailId", is(user.getEmailId())));

	}

	/**
	 * Covers updateUser API
	 * @throws Exception
	 */
	@Test
	public void test_updateUser() throws Exception {
		long userId = 1L;
		User savedUser = User.builder().firstName("Steve").lastName("Austin").emailId("steve.austin@gmail.com").build();
		User updatedUser = User.builder().firstName("John").lastName("Cena").emailId("john.cena@gmail.com").build();
		given(userRepository.findById(userId)).willReturn(Optional.of(savedUser));
		given(userRepository.save(any(User.class)))
		.willAnswer((invocation)-> invocation.getArgument(0));

		ResultActions response = mockMvc.perform(put("/api/v1/users/{id}", userId)
				.contentType(MediaType.APPLICATION_JSON)
				.content(objectMapper.writeValueAsString(updatedUser)));

		response.andExpect(status().isOk())
		.andExpect(jsonPath("$.firstName", is(updatedUser.getFirstName())))
		.andExpect(jsonPath("$.lastName", is(updatedUser.getLastName())))
		.andExpect(jsonPath("$.emailId", is(updatedUser.getEmailId())));
	}

	/**
	 * Covers deleteUser API
	 * @throws Exception
	 */
	@Test
	public void deleteUser() throws Exception {
		long userId = 1L;
		User existingUser = User.builder().firstName("Steve").lastName("Austin").emailId("steve.austin@gmail.com").build();
		given(userRepository.findById(userId)).willReturn(Optional.of(existingUser));
		willDoNothing().given(userRepository).delete(existingUser);

		ResultActions response = mockMvc.perform(delete("/api/v1/users/{id}", userId));

		response.andExpect(status().isOk());

	}



}
`

[laurit]

You have opentelemetry-logback-appender-1.0-1.28.0-alpha.jar and opentelemetry-api-1.25.0.jar. Use the same version for all the opentelemetry artifacts.

[TanishqDhussa]

Thanks for your help, much Appreciated!

[TanishqDhussa]

I also want to set the service name for OTEL
For now, I am getting
`I service name as unknown_service:java.

`

[mateuszrzeszutek]

Hey @TanishqDhussa ,

You have to set the service name either via the otel.service.name system property or the OTEL_SERVICE_NAME environment variable. See docs for an example.

[github-actions]

This has been automatically marked as stale because it has been marked as needing author feedback and has not had any activity for 7 days. It will be closed if no further activity occurs within 7 days of this comment.