[extension/encoding/jsonlogencoding] add new extension for JSON codec (…

…#27484)

**Description:** Create a new extension for JSON. This will be used in
pulsarreceiver/kafkareceiver to populate the log record's map from the
raw body.

**Link to tracking Issue:**
[6272](https://github.com/open-telemetry/opentelemetry-collector/issues/6272)

---------

Co-authored-by: Dmitrii Anoshin <anoshindx@gmail.com>

.chloggen/add-json-encoding-extension.yaml

@@ -0,0 +1,27 @@
+# Use this changelog template to create an entry for release notes.
+
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: new_component
+
+# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
+component: encoding/jsonlogencodingextension
+
+# A brief description of the change.  Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: Add a new extension to support JSON encoding (only logs)
+
+# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
+issues: [6272]
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext:
+
+# If your change doesn't affect end users or the exported elements of any package,
+# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
+# Optional: The change log or logs in which this entry should be included.
+# e.g. '[user]' or '[user, api]'
+# Include 'user' if the change is relevant to end users.
+# Include 'api' if there is a change to a library API.
+# Default: '[user]'
+change_logs: []

.github/CODEOWNERS

@@ -92,6 +92,7 @@ extension/awsproxy/                                                     @open-te
 extension/basicauthextension/                                           @open-telemetry/collector-contrib-approvers @jpkrohling @svrakitin @frzifus
 extension/bearertokenauthextension/                                     @open-telemetry/collector-contrib-approvers @jpkrohling @pavankrish123 @frzifus
 extension/encoding/jaegerencodingextension/                             @open-telemetry/collector-contrib-approvers @MovieStoreGuy @atoulme
+extension/encoding/jsonlogencodingextension/                            @open-telemetry/collector-contrib-approvers @VihasMakwana @atoulme
 extension/encoding/textencodingextension/                               @open-telemetry/collector-contrib-approvers @MovieStoreGuy @atoulme
 extension/headerssetterextension/                                       @open-telemetry/collector-contrib-approvers @jpkrohling
 extension/healthcheckextension/                                         @open-telemetry/collector-contrib-approvers @jpkrohling

.github/ISSUE_TEMPLATE/bug_report.yaml

@@ -84,6 +84,7 @@ body:
       - extension/basicauth
       - extension/bearertokenauth
       - extension/encoding/jaegerencoding
+      - extension/encoding/jsonlogencoding
       - extension/encoding/textencoding
       - extension/headerssetter
       - extension/healthcheck

.github/ISSUE_TEMPLATE/feature_request.yaml

@@ -78,6 +78,7 @@ body:
       - extension/basicauth
       - extension/bearertokenauth
       - extension/encoding/jaegerencoding
+      - extension/encoding/jsonlogencoding
       - extension/encoding/textencoding
       - extension/headerssetter
       - extension/healthcheck

.github/ISSUE_TEMPLATE/other.yaml

@@ -78,6 +78,7 @@ body:
       - extension/basicauth
       - extension/bearertokenauth
       - extension/encoding/jaegerencoding
+      - extension/encoding/jsonlogencoding
       - extension/encoding/textencoding
       - extension/headerssetter
       - extension/healthcheck

.github/dependabot.yml

@@ -7,6 +7,11 @@ updates:
     schedule:
       interval: "weekly"
       day: "wednesday"
+  - package-ecosystem: "gomod"
+    directory: "/cmd/configschema"
+    schedule:
+      interval: "weekly"
+      day: "wednesday"
   - package-ecosystem: "gomod"
     directory: "/cmd/otelcontribcol"
     schedule:
@@ -267,11 +272,6 @@ updates:
     schedule:
       interval: "weekly"
       day: "wednesday"
-  - package-ecosystem: "gomod"
-    directory: "/exporter/syslogexporter"
-    schedule:
-      interval: "weekly"
-      day: "wednesday"
   - package-ecosystem: "gomod"
     directory: "/exporter/tanzuobservabilityexporter"
     schedule:

extension/encoding/jsonlogencodingextension/Makefile

@@ -0,0 +1 @@
+include ../../../Makefile.Common

extension/encoding/jsonlogencodingextension/README.md

@@ -0,0 +1,14 @@
+# JSON Log encoding extension
+
+<!-- status autogenerated section -->
+| Status        |           |
+| ------------- |-----------|
+| Stability     | [development]  |
+| Distributions | [] |
+| Issues        | [![Open issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aopen%20label%3Aextension%2Fjsonlogencoding%20&label=open&color=orange&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Aextension%2Fjsonlogencoding) [![Closed issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aclosed%20label%3Aextension%2Fjsonlogencoding%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Aextension%2Fjsonlogencoding) |
+| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner)    | [@VihasMakwana](https://www.github.com/VihasMakwana), [@atoulme](https://www.github.com/atoulme) |
+
+[development]: https://github.com/open-telemetry/opentelemetry-collector#development
+<!-- end autogenerated section -->
+
+The `jsonlog` encoding extension is used to marshal/unmarshal JSON log body, ignoring other log fields.

extension/encoding/jsonlogencodingextension/config.go

@@ -0,0 +1,10 @@
+// Copyright The OpenTelemetry Authors
+// SPDX-License-Identifier: Apache-2.0
+
+package jsonlogencodingextension // import "github.com/open-telemetry/opentelemetry-collector-contrib/extension/encoding/jsonlogencodingextension"
+
+type Config struct{}
+
+func (c *Config) Validate() error {
+	return nil
+}

extension/encoding/jsonlogencodingextension/extension.go

@@ -0,0 +1,63 @@
+// Copyright The OpenTelemetry Authors
+// SPDX-License-Identifier: Apache-2.0
+
+package jsonlogencodingextension // import "github.com/open-telemetry/opentelemetry-collector-contrib/extension/encoding/jsonlogencodingextension"
+
+import (
+	"context"
+	"fmt"
+	"time"
+
+	jsoniter "github.com/json-iterator/go"
+	"go.opentelemetry.io/collector/component"
+	"go.opentelemetry.io/collector/pdata/pcommon"
+	"go.opentelemetry.io/collector/pdata/plog"
+)
+
+type jsonLogExtension struct {
+}
+
+func (e *jsonLogExtension) MarshalLogs(ld plog.Logs) ([]byte, error) {
+	logRecord := ld.ResourceLogs().At(0).ScopeLogs().At(0).LogRecords().At(0).Body()
+	var raw map[string]any
+	switch logRecord.Type() {
+	case pcommon.ValueTypeMap:
+		raw = logRecord.Map().AsRaw()
+	default:
+		return nil, fmt.Errorf("Marshal: Expected 'Map' found '%v'", logRecord.Type().String())
+	}
+	buf, err := jsoniter.Marshal(raw)
+	if err != nil {
+		return nil, err
+	}
+	return buf, nil
+
+}
+
+func (e *jsonLogExtension) UnmarshalLogs(buf []byte) (plog.Logs, error) {
+	p := plog.NewLogs()
+
+	// get json logs from the buffer
+	jsonVal := map[string]interface{}{}
+	if err := jsoniter.Unmarshal(buf, &jsonVal); err != nil {
+		return p, err
+	}
+
+	// create a new log record
+	logRecords := p.ResourceLogs().AppendEmpty().ScopeLogs().AppendEmpty().LogRecords().AppendEmpty()
+	logRecords.SetObservedTimestamp(pcommon.NewTimestampFromTime(time.Now()))
+
+	// Set the unmarshaled jsonVal as the body of the log record
+	if err := logRecords.Body().SetEmptyMap().FromRaw(jsonVal); err != nil {
+		return p, err
+	}
+	return p, nil
+}
+
+func (e *jsonLogExtension) Start(_ context.Context, _ component.Host) error {
+	return nil
+}
+
+func (e *jsonLogExtension) Shutdown(_ context.Context) error {
+	return nil
+}

extension/encoding/jsonlogencodingextension/extension_test.go

@@ -0,0 +1,20 @@
+// Copyright The OpenTelemetry Authors
+// SPDX-License-Identifier: Apache-2.0
+
+package jsonlogencodingextension
+
+import (
+	"context"
+	"testing"
+
+	"github.com/stretchr/testify/require"
+	"go.opentelemetry.io/collector/component/componenttest"
+)
+
+func TestExtension_Start_Shutdown(t *testing.T) {
+	j := &jsonLogExtension{}
+	err := j.Start(context.Background(), componenttest.NewNopHost())
+	require.NoError(t, err)
+	err = j.Shutdown(context.Background())
+	require.NoError(t, err)
+}

extension/encoding/jsonlogencodingextension/factory.go

@@ -0,0 +1,30 @@
+// Copyright The OpenTelemetry Authors
+// SPDX-License-Identifier: Apache-2.0
+
+package jsonlogencodingextension // import "github.com/open-telemetry/opentelemetry-collector-contrib/extension/encoding/jsonlogencodingextension"
+
+import (
+	"context"
+
+	"go.opentelemetry.io/collector/component"
+	"go.opentelemetry.io/collector/extension"
+
+	"github.com/open-telemetry/opentelemetry-collector-contrib/extension/encoding/jsonlogencodingextension/internal/metadata"
+)
+
+func NewFactory() extension.Factory {
+	return extension.NewFactory(
+		metadata.Type,
+		createDefaultConfig,
+		createExtension,
+		metadata.ExtensionStability,
+	)
+}
+
+func createExtension(_ context.Context, _ extension.CreateSettings, _ component.Config) (extension.Extension, error) {
+	return &jsonLogExtension{}, nil
+}
+
+func createDefaultConfig() component.Config {
+	return &Config{}
+}

extension/encoding/jsonlogencodingextension/go.mod

@@ -0,0 +1,41 @@
+module github.com/open-telemetry/opentelemetry-collector-contrib/extension/encoding/jsonlogencodingextension
+
+go 1.20
+
+require (
+	github.com/json-iterator/go v1.1.12
+	github.com/stretchr/testify v1.8.4
+	go.opentelemetry.io/collector/component v0.87.1-0.20231017160804-ec0725874313
+	go.opentelemetry.io/collector/extension v0.87.1-0.20231017160804-ec0725874313
+	go.opentelemetry.io/collector/pdata v1.0.0-rcv0016.0.20231017160804-ec0725874313
+)
+
+require (
+	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/gogo/protobuf v1.3.2 // indirect
+	github.com/golang/protobuf v1.5.3 // indirect
+	github.com/knadh/koanf/maps v0.1.1 // indirect
+	github.com/knadh/koanf/providers/confmap v0.1.0 // indirect
+	github.com/knadh/koanf/v2 v2.0.1 // indirect
+	github.com/mitchellh/copystructure v1.2.0 // indirect
+	github.com/mitchellh/mapstructure v1.5.1-0.20220423185008-bf980b35cac4 // indirect
+	github.com/mitchellh/reflectwalk v1.0.2 // indirect
+	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
+	github.com/modern-go/reflect2 v1.0.2 // indirect
+	github.com/pmezard/go-difflib v1.0.0 // indirect
+	go.opentelemetry.io/collector/config/configtelemetry v0.87.1-0.20231017160804-ec0725874313 // indirect
+	go.opentelemetry.io/collector/confmap v0.87.1-0.20231017160804-ec0725874313 // indirect
+	go.opentelemetry.io/collector/featuregate v1.0.0-rcv0016.0.20231017160804-ec0725874313 // indirect
+	go.opentelemetry.io/otel v1.19.0 // indirect
+	go.opentelemetry.io/otel/metric v1.19.0 // indirect
+	go.opentelemetry.io/otel/trace v1.19.0 // indirect
+	go.uber.org/multierr v1.11.0 // indirect
+	go.uber.org/zap v1.26.0 // indirect
+	golang.org/x/net v0.17.0 // indirect
+	golang.org/x/sys v0.13.0 // indirect
+	golang.org/x/text v0.13.0 // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20230822172742-b8732ec3820d // indirect
+	google.golang.org/grpc v1.59.0 // indirect
+	google.golang.org/protobuf v1.31.0 // indirect
+	gopkg.in/yaml.v3 v3.0.1 // indirect
+)