Skip to content

Commit

Permalink
[wip] Add skeleton for the otlpjson connector (#34239)
Browse files Browse the repository at this point in the history
**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
This is the first PR for the new `otlpjson` connector. Includes the
overall structure of the new component.

**Link to tracking Issue:** <Issue number if applicable>
#34208

**Testing:** <Describe what testing was performed and which tests were
added.>

**Documentation:** <Describe the documentation added.>

/cc @djaglowski

Signed-off-by: ChrsMark <chrismarkou92@gmail.com>
  • Loading branch information
ChrsMark authored Jul 24, 2024
1 parent d5c40d3 commit daaaf62
Show file tree
Hide file tree
Showing 25 changed files with 680 additions and 0 deletions.
27 changes: 27 additions & 0 deletions .chloggen/add_otlpjson_connector.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: new_component

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: otlpjsonconnector

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: New component that will allow extracting otlpjson data from incoming Logs.

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [34239, 34208]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:

# If your change doesn't affect end users or the exported elements of any package,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: []
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ connector/datadogconnector/ @open-teleme
connector/exceptionsconnector/ @open-telemetry/collector-contrib-approvers @jpkrohling @marctc
connector/failoverconnector/ @open-telemetry/collector-contrib-approvers @akats7 @djaglowski @fatsheep9146
connector/grafanacloudconnector/ @open-telemetry/collector-contrib-approvers @jpkrohling @rlankfo @jcreixell
connector/otlpjsonconnector/ @open-telemetry/collector-contrib-approvers @djaglowski @ChrsMark
connector/roundrobinconnector/ @open-telemetry/collector-contrib-approvers @bogdandrutu
connector/routingconnector/ @open-telemetry/collector-contrib-approvers @jpkrohling @mwear
connector/servicegraphconnector/ @open-telemetry/collector-contrib-approvers @jpkrohling @mapno @JaredTan95
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ body:
- connector/exceptions
- connector/failover
- connector/grafanacloud
- connector/otlpjson
- connector/roundrobin
- connector/routing
- connector/servicegraph
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/feature_request.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ body:
- connector/exceptions
- connector/failover
- connector/grafanacloud
- connector/otlpjson
- connector/roundrobin
- connector/routing
- connector/servicegraph
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/other.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ body:
- connector/exceptions
- connector/failover
- connector/grafanacloud
- connector/otlpjson
- connector/roundrobin
- connector/routing
- connector/servicegraph
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/unmaintained.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ body:
- connector/exceptions
- connector/failover
- connector/grafanacloud
- connector/otlpjson
- connector/roundrobin
- connector/routing
- connector/servicegraph
Expand Down
1 change: 1 addition & 0 deletions connector/otlpjsonconnector/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include ../../Makefile.Common
68 changes: 68 additions & 0 deletions connector/otlpjsonconnector/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# otlpjson Connector

<!-- status autogenerated section -->
| Status | |
| ------------- |-----------|
| Distributions | [contrib] |
| Issues | [![Open issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aopen%20label%3Aconnector%2Fotlpjson%20&label=open&color=orange&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Aconnector%2Fotlpjson) [![Closed issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aclosed%20label%3Aconnector%2Fotlpjson%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Aconnector%2Fotlpjson) |
| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner) | [@djaglowski](https://www.github.com/djaglowski), [@ChrsMark](https://www.github.com/ChrsMark) |

[development]: https://github.com/open-telemetry/opentelemetry-collector#development
[contrib]: https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-contrib

## Supported Pipeline Types

| [Exporter Pipeline Type] | [Receiver Pipeline Type] | [Stability Level] |
| ------------------------ | ------------------------ | ----------------- |
| logs | metrics | [development] |
| logs | traces | [development] |
| logs | logs | [development] |

[Exporter Pipeline Type]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/connector/README.md#exporter-pipeline-type
[Receiver Pipeline Type]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/connector/README.md#receiver-pipeline-type
[Stability Level]: https://github.com/open-telemetry/opentelemetry-collector#stability-levels
<!-- end autogenerated section -->

Allows to extract otlpjson data from incoming Logs and specifically the `Body` field.
The data is written in
[Protobuf JSON
encoding](https://developers.google.com/protocol-buffers/docs/proto3#json)
using [OpenTelemetry
protocol](https://github.com/open-telemetry/opentelemetry-proto).

## Configuration

#### Configuration Example:

```yaml
receivers:
filelog:
include:
- /var/log/foo.log

exporters:
debug:

connectors:
otlpjson:

service:
pipelines:
logs/raw:
receivers: [filelog]
exporters: [otlpjson]
metrics/otlp:
receivers: [otlpjson]
exporters: [debug]
logs/otlp:
receivers: [otlpjson]
exporters: [debug]
traces/otlp:
receivers: [otlpjson]
exporters: [debug]
```
[Connectors README]:https://github.com/open-telemetry/opentelemetry-collector/blob/main/connector/README.md
[Exporter Pipeline Type]:https://github.com/open-telemetry/opentelemetry-collector/blob/main/connector/README.md#exporter-pipeline-type
[Receiver Pipeline Type]:https://github.com/open-telemetry/opentelemetry-collector/blob/main/connector/README.md#receiver-pipeline-type
[contrib]:https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-contrib
10 changes: 10 additions & 0 deletions connector/otlpjsonconnector/config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package otlpjsonconnector // import "github.com/open-telemetry/opentelemetry-collector-contrib/connector/otlpjsonconnector"

type Config struct{}

func (c *Config) Validate() error {
return nil
}
4 changes: 4 additions & 0 deletions connector/otlpjsonconnector/config_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package otlpjsonconnector
6 changes: 6 additions & 0 deletions connector/otlpjsonconnector/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

//go:generate mdatagen metadata.yaml

package otlpjsonconnector // import "github.com/open-telemetry/opentelemetry-collector-contrib/connector/otlpjsonconnector"
60 changes: 60 additions & 0 deletions connector/otlpjsonconnector/factory.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package otlpjsonconnector // import "github.com/open-telemetry/opentelemetry-collector-contrib/connector/otlpjsonconnector"

import (
"context"

"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/connector"
"go.opentelemetry.io/collector/consumer"

"github.com/open-telemetry/opentelemetry-collector-contrib/connector/otlpjsonconnector/internal/metadata"
)

// NewFactory returns a ConnectorFactory.
func NewFactory() connector.Factory {
return connector.NewFactory(
metadata.Type,
createDefaultConfig,
connector.WithLogsToTraces(createTracesConnector, component.StabilityLevelAlpha),
connector.WithLogsToMetrics(createMetricsConnector, component.StabilityLevelAlpha),
connector.WithLogsToLogs(createLogsConnector, component.StabilityLevelAlpha),
)
}

// createDefaultConfig creates the default configuration.
func createDefaultConfig() component.Config {
return &Config{}
}

// createLogsConnector returns a connector which consume logs and export logs
func createLogsConnector(
_ context.Context,
set connector.Settings,
cfg component.Config,
nextConsumer consumer.Logs,
) (connector.Logs, error) {
return newLogsConnector(set, cfg, nextConsumer), nil
}

// createTracesConnector returns a connector which consume logs and export traces
func createTracesConnector(
_ context.Context,
set connector.Settings,
cfg component.Config,
nextConsumer consumer.Traces,
) (connector.Logs, error) {
return newTracesConnector(set, cfg, nextConsumer), nil
}

// createMetricsConnector returns a connector which consume logs and export metrics
func createMetricsConnector(
_ context.Context,
set connector.Settings,
cfg component.Config,
nextConsumer consumer.Metrics,
) (connector.Logs, error) {
return newMetricsConnector(set, cfg, nextConsumer), nil
}
30 changes: 30 additions & 0 deletions connector/otlpjsonconnector/factory_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package otlpjsonconnector

import (
"context"
"testing"

"github.com/stretchr/testify/assert"
"go.opentelemetry.io/collector/connector/connectortest"
"go.opentelemetry.io/collector/consumer"
"go.opentelemetry.io/collector/pdata/plog"
)

func TestNewFactory(t *testing.T) {
cfg := &Config{}

lc, err := consumer.NewLogs(func(context.Context, plog.Logs) error {
return nil
})
assert.NoError(t, err)

factory := NewFactory()
conn, err := factory.CreateLogsToLogs(context.Background(),
connectortest.NewNopSettings(), cfg, lc)

assert.NoError(t, err)
assert.NotNil(t, conn)
}
87 changes: 87 additions & 0 deletions connector/otlpjsonconnector/generated_component_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 13 additions & 0 deletions connector/otlpjsonconnector/generated_package_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit daaaf62

Please sign in to comment.