Start Entities Prototype

[tigrannajaryan]

Changes

• Updated mdatagen to allow specifying the generated entity type and identifying attributes, which makes very easy for existing receivers and processors to begin emitting extended resources that include entity information.
• Updated hostmetrics receiver's Memory and Process scrapers to produce "host" entity.
• Updated resourcedetection processor's System detector to produce "host" entity and to enrich the Resource/Producing Entity that the processor sees.
• Updated k8sclustrer receiver to produce pentity.EntityEvent instead of previous experimental log-based entity event structs. Note: the pipeline still uses logs since we don't have entity support in the pipelines.

Results

Example output from hostmetrics receiver, with data enriched by resourcedetection processor with system detector is below.

Note "Entity Type" that is set by hostmetrics receiver and "Entity Id" that is set by resourcedetection processor.

2024-02-21T15:21:03.143-0500	info	ResourceMetrics #0
Resource SchemaURL: https://opentelemetry.io/schemas/1.9.0
Resource attributes:
     -> process.pid: Int(1)
     -> process.parent_pid: Int(0)
     -> process.executable.name: Str(launchd)
     -> process.executable.path: Str(/sbin/launchd)
     -> process.command: Str(/sbin/launchd)
     -> process.command_line: Str(/sbin/launchd)
     -> process.owner: Str(root)
     -> host.name: Str(KYW9LWX6YV.lan)
     -> os.type: Str(darwin)
     -> host.id: Str(4A135D30-F122-5869-9399-30339FDAE584)
Entity Type: host
Entity Id:
     -> host.id: Str(4A135D30-F122-5869-9399-30339FDAE584)
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope otelcol/hostmetricsreceiver/process 0.94.0-dev
Metric #0
Descriptor:
     -> Name: process.cpu.time
     -> Description: Total CPU seconds broken down by different states.
     -> Unit: s
     -> DataType: Sum
     -> IsMonotonic: true
     -> AggregationTemporality: Cumulative
NumberDataPoints #0
Data point attributes:
     -> state: Str(user)
StartTimestamp: 2024-01-23 16:23:24.348 +0000 UTC
Timestamp: 2024-02-21 20:20:59.232973 +0000 UTC
Value: 6214.150000

Another example produced by k8scluster receiver bellow.

Note that the prototype still uses logs to represent the events since we don't have have support for entity data type for components. However, the data is produced as first-class pentity data (and converted to logs to show in the output).

The production version will have entities as fully supported new data type in the Collector and k8scluster receiver will se it instead of logs.

Attributes:
     -> otel.entity.type: Str(k8s.daemonset)
     -> otel.entity.id: Map({"k8s.daemonset.uid":"7816638a-043e-4b00-8771-e257c595a15c"})
     -> otel.entity.event.type: Str(entity_state)
     -> otel.entity.attributes: Map({"daemonset.creation_timestamp":"2024-01-23T15:44:10-05:00","k8s-app":"kube-proxy","k8s.workload.kind":"DaemonSet","k8s.workload.name":"kube-proxy"})

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 14 days.

[github-actions]

Closed as inactive. Feel free to reopen if this PR is still being worked on.