[extensions/observer/cfgardenobserver] Implement cfgardenobserver (#3…

…4513)

**Description:** 
First Component PR: #33727
This is the second PR for adding the cfgardenobserver, with the first
suggested implementation. There are definitely some decisions made that
require feedback, such as adding the CloudFoundry application labels to
the Endpoint labels, and the decision to use the `Container`
EndpointType at all.

**Link to tracking Issue:** #33618 

**Testing:** Unit testing of config and extension

**Documentation:** Updated readme with new configuration and endpoints

---------

Co-authored-by: sam clulow <sam.clulow@springernature.com>
Co-authored-by: sam clulow <m1rp@users.noreply.github.com>
Co-authored-by: José Riguera Lopez <jriguera@gmail.com>

.chloggen/implement-cfgardenobserver.yaml

@@ -0,0 +1,27 @@
+# Use this changelog template to create an entry for release notes.
+
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: new_component
+
+# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
+component: extensions/observer/cfgardenobserver
+
+# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: Implemented the observer, the second PR in the 3 PR process for new components
+
+# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
+issues: [33618]
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext:
+
+# If your change doesn't affect end users or the exported elements of any package,
+# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
+# Optional: The change log or logs in which this entry should be included.
+# e.g. '[user]' or '[user, api]'
+# Include 'user' if the change is relevant to end users.
+# Include 'api' if there is a change to a library API.
+# Default: '[user]'
+change_logs: [user]

extension/observer/cfgardenobserver/README.md

@@ -6,7 +6,7 @@
 | Stability     | [development]  |
 | Distributions | [] |
 | Issues        | [![Open issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aopen%20label%3Aextension%2Fcfgardenobserver%20&label=open&color=orange&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Aextension%2Fcfgardenobserver) [![Closed issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aclosed%20label%3Aextension%2Fcfgardenobserver%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Aextension%2Fcfgardenobserver) |
-| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner)    | [@crobert-1](https://www.github.com/crobert-1), [@cemdk](https://www.github.com/cemdk), [@tomasmota](https://www.github.com/tomasmota), [@m1rp](https://www.github.com/m1rp), [@jriguera](https://www.github.com/jriguera) |
+| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner)    | [@crobert-1](https://www.github.com/crobert-1), [@cemdk](https://www.github.com/cemdk), [@m1rp](https://www.github.com/m1rp), [@jriguera](https://www.github.com/jriguera) |
 
 [development]: https://github.com/open-telemetry/opentelemetry-collector#development
 <!-- end autogenerated section -->
@@ -20,34 +20,57 @@ The `cfgarden_observer` looks at the current host to discover Garden containers.
 ```yaml
 extensions:
   cfgarden_observer:
-    # url of the Garden socket, defaults to unix:///var/vcap/data/garden/garden.sock 
-    endpoint: my/path/to/garden.sock
-    # determines how often to look for changes in endpoints.
     refresh_interval: 30s
+    cache_sync_interval: 10m
+    include_app_labels: true
+    garden:
+      endpoint: my/path/to/garden.sock
+    cloud_foundry:
+      endpoint: https://api.cf.mydomain.com
+      auth:
+        type: client_credentials
+        client_id: myclientid
+        client_secret: myclientsecret
 
 receivers:
   receiver_creator:
     watch_observers: [cfgarden_observer]
     receivers:
       prometheus_simple:
-        rule: type == "container" && name == "myapp"
+        rule: type == "container" && labels["prometheus.io/scrape"] == "true" 
         config:
           metrics_path: /metrics
           endpoint: '`endpoint`'
 ```
 
 ### Configuration
 
-| Name             | Type   | Default                                  | Docs                                                   |
-|------------------|--------|------------------------------------------|--------------------------------------------------------|
-| refresh_interval | string | 60s                                      | Determines how often to look for changes in endpoints. |
-| endpoint         | string | unix:///var/vcap/data/garden/garden.sock | The endpoint to connect to the Garden API.             |
+| Name                             | Type   | Default                                                   | Description                                                        |
+| -------------------------------- | ------ | --------------------------------------------------------- | ------------------------------------------------------------------ |
+| refresh_interval                 | string | 1m                                                        | Determines how often to look for changes in endpoints.             |
+| cache_sync_interval              | string | 5m                                                        | Determines how often app metadata cache is refreshed               |
+| include_app_labels               | bool   | false                                                     | Determines whether or not app labels get added to container labels |
+| garden.endpoint                  | string | /var/vcap/data/garden/garden.sock                         | Path to garden socket.                                             |
+| cloud_foundry.endpoint           | string | none. required when `include_app_labels` is set to `true` | CloudFoundry API endpoint                                          |
+| cloud_foundry.auth.type          | string | none. required when `include_app_labels` is set to `true` | Authentication type, one of: user_pass, client_credentials, token  |
+| cloud_foundry.auth.username      | string | none                                                      | Username (auth.type: user_pass)                                    |
+| cloud_foundry.auth.password      | string | none                                                      | Password (auth.type: user_pass)                                    |
+| cloud_foundry.auth.client_id     | string | none                                                      | Client ID (auth.type: client_credentials)                          |
+| cloud_foundry.auth.client_secret | string | none                                                      | Client Secret (auth.type: client_credentials)                      |
+| cloud_foundry.auth.access_token  | string | none                                                      | Access Token (auth.type: token)                                    |
+| cloud_foundry.auth.refresh_token | string | none                                                      | Refresh Token (auth.type: token)                                   |
+
 
 ### Endpoint Variables
 
 Endpoint variables exposed by this observer are as follows.
 
-| Variable  | Description                                                                                |
-|-----------|--------------------------------------------------------------------------------------------|
-| type      | this value is always `container`                                                                              |
-| name      | name of the Garden container associated to the port                                        |
+| Variable     | Description                                                                       |
+| ------------ | --------------------------------------------------------------------------------- |
+| type         | This value is always `container`                                                  |
+| name         | Name of the Garden container associated to the port                               |
+| labels       | map[string]string with labels set on the log_config tags and application resource |
+| port         | Exposed port of the container                                                     |
+| container_id | ID of the container                                                               |
+| host         | Hostname or IP of the underlying host the container is running on                 |
+| transport    | Transport protocol used by the endpoint (TCP or UDP)                              |

extension/observer/cfgardenobserver/config.go

@@ -4,15 +4,121 @@
 package cfgardenobserver // import "github.com/open-telemetry/opentelemetry-collector-contrib/extension/observer/cfgardenobserver"
 
 import (
+	"errors"
+	"fmt"
 	"time"
 )
 
 // Config defines configuration for CF Garden observer.
 type Config struct {
-	// The URL of the CF Garden api. Default is "unix:///var/vcap/data/garden/garden.sock"
-	Endpoint string `mapstructure:"endpoint"`
+	// CloudFoundry API Configuration
+	CloudFoundry CfConfig `mapstructure:"cloud_foundry"`
+
+	// Garden API Configuration
+	Garden GardenConfig `mapstructure:"garden"`
 
 	// RefreshInterval determines the frequency at which the observer
 	// needs to poll for collecting information about new processes.
+	// Default: "1m"
 	RefreshInterval time.Duration `mapstructure:"refresh_interval"`
+
+	// The time to wait before resyncing app information on cached containers
+	// using the CloudFoundry API.
+	// Default: "5m"
+	CacheSyncInterval time.Duration `mapstructure:"cache_sync_interval"`
+
+	// Determines whether or not Application labels get added to the Endpoint labels.
+	// This requires cloud_foundry to be configured, such that API calls can be made
+	// Default: false
+	IncludeAppLabels bool `mapstructure:"include_app_labels"`
+}
+
+// Validate overrides the embedded noop validation so that load config can trigger
+// our own validation logic.
+func (config *Config) Validate() error {
+	if !config.IncludeAppLabels {
+		return nil
+	}
+
+	c := config.CloudFoundry
+	if c.Endpoint == "" {
+		return errors.New("CloudFoundry.Endpoint must be specified when IncludeAppLabels is set to true")
+	}
+	if c.Auth.Type == "" {
+		return errors.New("CloudFoundry.Auth.Type must be specified when IncludeAppLabels is set to true")
+	}
+
+	switch c.Auth.Type {
+	case authTypeUserPass:
+		if c.Auth.Username == "" {
+			return fieldError(authTypeUserPass, "username")
+		}
+		if c.Auth.Password == "" {
+			return fieldError(authTypeUserPass, "password")
+		}
+	case authTypeClientCredentials:
+		if c.Auth.ClientID == "" {
+			return fieldError(authTypeClientCredentials, "client_id")
+		}
+		if c.Auth.ClientSecret == "" {
+			return fieldError(authTypeClientCredentials, "client_secret")
+		}
+	case authTypeToken:
+		if c.Auth.AccessToken == "" {
+			return fieldError(authTypeToken, "access_token")
+		}
+		if c.Auth.RefreshToken == "" {
+			return fieldError(authTypeToken, "refresh_token")
+		}
+	default:
+		return fmt.Errorf("configuration option `auth_type` must be set to one of the following values: [user_pass, client_credentials, token]. Specified value: %s", c.Auth.Type)
+	}
+
+	return nil
+}
+
+func fieldError(authType authType, param string) error {
+	return fmt.Errorf("%s is required when using auth_type: %s", param, authType)
+}
+
+type GardenConfig struct {
+	// The URL of the CF Garden api. Default is "/var/vcap/data/garden/garden.sock"
+	Endpoint string `mapstructure:"endpoint"`
 }
+
+type CfConfig struct {
+	// The URL of the CloudFoundry API
+	Endpoint string `mapstructure:"endpoint"`
+
+	// Authentication details
+	Auth CfAuth `mapstructure:"auth"`
+}
+
+type CfAuth struct {
+	// Authentication method, there are 3 options
+	Type authType `mapstructure:"type"`
+
+	// Used for user_pass authentication method
+	Username string `mapstructure:"username"`
+	Password string `mapstructure:"password"`
+
+	// Used for token authentication method
+	AccessToken  string `mapstructure:"access_token"`
+	RefreshToken string `mapstructure:"refresh_token"`
+
+	// Used for client_credentials authentication method
+	ClientID     string `mapstructure:"client_id"`
+	ClientSecret string `mapstructure:"client_secret"`
+}
+
+// authType describes the type of authentication to use for the CloudFoundry API
+type authType string
+
+const (
+	// authTypeClientCredentials uses a client ID and client secret to authenticate
+	authTypeClientCredentials authType = "client_credentials"
+	// authTypeUserPass uses username and password to authenticate
+	authTypeUserPass authType = "user_pass"
+	// authTypeToken uses access token and refresh token to authenticate
+	authTypeToken authType = "token"
+)