Introduce model translation and encoding interfaces (open-telemetry#3200

)

* [wip] Introduce model translation and encodings interfaces

This is a somewhat more limited version of open-telemetry#3044. It decouples
translation of models from encodings.

Models refers to the in-memory representation of a protcol like the zipkin v2
SpanModel. After translating from pdata to the model an encoding is used to
serialize the model to a particular byte representation (protobuf, JSON, etc.).
The reverse also applies in deserializing an encoding of bytes to a model then
translating the model to pdata.

* use encode/decode consistently

* review feedback

* decouple encoding from model

Before the encoder interfaces took pdata and serialized it by calling the
translator. This fully separates the concerns by having model do pure
translation, encoding do pure serialization, and the transcoder doing both.

Without this there was no way to use the encoder if you already had a model.

Only updates traces for feedback purposes.

* add high level interface

* standardized on encoding/decoding terminology
* renamed encodings to bytes to avoid confusion with encode/decode terminology
* added high level interfaces to top level protocols package that goes directly pdata <-> bytes

* cleanup

* Apply suggestions from code review

Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com>

* renamings

* reword error

* cleanup

* return interface instead of out parameter

* review feedback

* serialize -> marshal

* put in internal

* lint

Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com>

internal/model/README.md

@@ -0,0 +1,9 @@
+# Protocols
+
+This package provides common ways for decoding serialized bytes into protocol-specific in-memory data models (e.g. Zipkin Span). These data models can then be translated to internal pdata representations. Similarly, pdata can be translated from a data model which can then be serialized into bytes.
+
+[serializer](serializer): Common interfaces for serializing/deserializing bytes from/to protocol-specific data models.
+
+[translator](translator): Common interfaces for translating protocol-specific data models from/to pdata.
+
+This package provides higher level APIs that do both encoding of bytes and data model if going directly pdata ⇔ bytes.

internal/model/serializer/deserialize.go

@@ -0,0 +1,30 @@
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//       http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package serializer
+
+// MetricsUnmarshaler decodes bytes into protocol-specific data model.
+type MetricsUnmarshaler interface {
+	UnmarshalMetrics(buf []byte) (interface{}, error)
+}
+
+// TracesUnmarshaler decodes bytes into protocol-specific data model.
+type TracesUnmarshaler interface {
+	UnmarshalTraces(buf []byte) (interface{}, error)
+}
+
+// LogsUnmarshaler decodes bytes into protocol-specific data model.
+type LogsUnmarshaler interface {
+	UnmarshalLogs(buf []byte) (interface{}, error)
+}

internal/model/serializer/serialize.go

@@ -0,0 +1,30 @@
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//       http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package serializer
+
+// MetricsMarshaler encodes protocol-specific data model into bytes.
+type MetricsMarshaler interface {
+	MarshalMetrics(model interface{}) ([]byte, error)
+}
+
+// TracesMarshaler encodes protocol-specific data model into bytes.
+type TracesMarshaler interface {
+	MarshalTraces(model interface{}) ([]byte, error)
+}
+
+// LogsMarshaler encodes protocol-specific data model into bytes.
+type LogsMarshaler interface {
+	MarshalLogs(model interface{}) ([]byte, error)
+}

internal/model/serializer/serializer.go

@@ -0,0 +1,43 @@
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//       http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package serializer
+
+import "fmt"
+
+// Encoding is the encoding format that a model is serialized to.
+type Encoding string
+
+const (
+	Protobuf Encoding = "protobuf"
+	JSON     Encoding = "json"
+	Thrift   Encoding = "thrift"
+)
+
+func (e Encoding) String() string {
+	return string(e)
+}
+
+// ErrUnavailableEncoding is returned when the requested encoding is not supported.
+type ErrUnavailableEncoding struct {
+	encoding Encoding
+}
+
+func (e *ErrUnavailableEncoding) Error() string {
+	return fmt.Sprintf("unsupported encoding %q", e.encoding)
+}
+
+func NewErrUnavailableEncoding(encoding Encoding) *ErrUnavailableEncoding {
+	return &ErrUnavailableEncoding{encoding: encoding}
+}

internal/model/serializer/serializer_test.go

@@ -0,0 +1,31 @@
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//       http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package serializer
+
+import (
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+)
+
+func TestNewErrUnavailableEncoding(t *testing.T) {
+	err := NewErrUnavailableEncoding("unknown")
+	assert.IsType(t, &ErrUnavailableEncoding{}, err)
+	assert.EqualError(t, err, `unsupported encoding "unknown"`)
+}
+
+func TestEncoding_String(t *testing.T) {
+	assert.Equal(t, "protobuf", Protobuf.String())
+}

internal/model/translator/decoder.go

@@ -0,0 +1,32 @@
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//       http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package translator
+
+import "go.opentelemetry.io/collector/consumer/pdata"
+
+type MetricsDecoder interface {
+	// ToMetrics converts a protocol-specific data model into pdata.
+	ToMetrics(src interface{}) (pdata.Metrics, error)
+}
+
+type TracesDecoder interface {
+	// ToTraces converts a protocol-specific data model into pdata.
+	ToTraces(src interface{}) (pdata.Traces, error)
+}
+
+type LogsDecoder interface {
+	// ToLogs converts a protocol-specific data model into pdata.
+	ToLogs(src interface{}) (pdata.Logs, error)
+}

internal/model/translator/encoder.go

@@ -0,0 +1,32 @@
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//       http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package translator
+
+import "go.opentelemetry.io/collector/consumer/pdata"
+
+type MetricsEncoder interface {
+	// FromMetrics converts pdata to protocol-specific data model.
+	FromMetrics(md pdata.Metrics) (interface{}, error)
+}
+
+type TracesEncoder interface {
+	// FromTraces converts pdata to protocol-specific data model.
+	FromTraces(td pdata.Traces) (interface{}, error)
+}
+
+type LogsEncoder interface {
+	// FromLogs converts pdata to protocol-specific data model.
+	FromLogs(ld pdata.Logs) (interface{}, error)
+}

internal/model/translator/translator.go

@@ -0,0 +1,24 @@
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//       http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package translator
+
+import (
+	"fmt"
+)
+
+// NewErrIncompatibleType returns errIncompatibleType instance
+func NewErrIncompatibleType(expected, given interface{}) error {
+	return fmt.Errorf("expected model type %T but given %T", expected, given)
+}

internal/model/translator/translator_test.go

@@ -0,0 +1,27 @@
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//       http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package translator
+
+import (
+	"testing"
+
+	zipkinmodel "github.com/openzipkin/zipkin-go/model"
+	"github.com/stretchr/testify/assert"
+)
+
+func TestNewErrIncompatibleType(t *testing.T) {
+	err := NewErrIncompatibleType([]*zipkinmodel.SpanModel{}, "given")
+	assert.EqualError(t, err, "expected model type []*model.SpanModel but given string")
+}