cribl-control-plane-sdk-go

The Cribl Go SDK for the control plane provides operational control over Cribl resources and helps streamline the process of integrating with Cribl.

In addition to the usage examples in this repository, you can adapt the code examples for common use cases in the Cribl documentation to use Go instead of Python.

Complementary API reference documentation is available at https://docs.cribl.io/cribl-as-code/api-reference. Product documentation is available at https://docs.cribl.io.

Important

Preview Feature The Cribl SDKs are Preview features that are still being developed. We do not recommend using them in a production environment, because the features might not be fully tested or optimized for performance, and related documentation could be incomplete.

Please continue to submit feedback through normal Cribl support channels, but assistance might be limited while the features remain in Preview.

SDK Installation

To add the SDK as a dependency to your project:

go get github.com/criblio/cribl-control-plane-sdk-go

SDK Example Usage

Example

package main

import (
	"context"
	criblcontrolplanesdkgo "github.com/criblio/cribl-control-plane-sdk-go"
	"github.com/criblio/cribl-control-plane-sdk-go/models/components"
	"log"
	"os"
)

func main() {
	ctx := context.Background()

	s := criblcontrolplanesdkgo.New(
		"https://api.example.com",
		criblcontrolplanesdkgo.WithSecurity(components.Security{
			BearerAuth: criblcontrolplanesdkgo.Pointer(os.Getenv("CRIBLCONTROLPLANE_BEARER_AUTH")),
		}),
	)

	res, err := s.LakeDatasets.Create(ctx, "<id>", components.CriblLakeDataset{
		AcceleratedFields: []string{
			"<value 1>",
			"<value 2>",
		},
		BucketName: criblcontrolplanesdkgo.Pointer("<value>"),
		CacheConnection: &components.CacheConnection{
			AcceleratedFields: []string{
				"<value 1>",
				"<value 2>",
			},
			BackfillStatus:          components.CacheConnectionBackfillStatusPending.ToPointer(),
			CacheRef:                "<value>",
			CreatedAt:               7795.06,
			LakehouseConnectionType: components.LakehouseConnectionTypeCache.ToPointer(),
			MigrationQueryID:        criblcontrolplanesdkgo.Pointer("<id>"),
			RetentionInDays:         1466.58,
		},
		DeletionStartedAt: criblcontrolplanesdkgo.Pointer[float64](8310.58),
		Description:       criblcontrolplanesdkgo.Pointer("pleased toothbrush long brush smooth swiftly rightfully phooey chapel"),
		Format:            components.CriblLakeDatasetFormatDdss.ToPointer(),
		HTTPDAUsed:        criblcontrolplanesdkgo.Pointer(true),
		ID:                "<id>",
		Metrics: &components.LakeDatasetMetrics{
			CurrentSizeBytes: 6170.04,
			MetricsDate:      "<value>",
		},
		RetentionPeriodInDays: criblcontrolplanesdkgo.Pointer[float64](456.37),
		SearchConfig: &components.LakeDatasetSearchConfig{
			Datatypes: []string{
				"<value 1>",
			},
			Metadata: &components.DatasetMetadata{
				Earliest:           "<value>",
				EnableAcceleration: true,
				FieldList: []string{
					"<value 1>",
					"<value 2>",
				},
				LatestRunInfo: &components.DatasetMetadataRunInfo{
					EarliestScannedTime: criblcontrolplanesdkgo.Pointer[float64](4334.7),
					FinishedAt:          criblcontrolplanesdkgo.Pointer[float64](6811.22),
					LatestScannedTime:   criblcontrolplanesdkgo.Pointer[float64](5303.3),
					ObjectCount:         criblcontrolplanesdkgo.Pointer[float64](9489.04),
				},
				ScanMode: components.ScanModeDetailed,
			},
		},
		StorageLocationID: criblcontrolplanesdkgo.Pointer("<id>"),
		ViewName:          criblcontrolplanesdkgo.Pointer("<value>"),
	})
	if err != nil {
		log.Fatal(err)
	}
	if res.Object != nil {
		// handle response
	}
}

Authentication

Except for the health.get and auth.tokens.get methods, all Cribl SDK requests require you to authenticate with a Bearer token. You must include a valid Bearer token in the configuration when initializing your SDK client. The Bearer token verifies your identity and ensures secure access to the requested resources. The SDK automatically manages the Authorization header for subsequent requests once properly authenticated.

For information about Bearer token expiration, see Token Management in the Cribl as Code documentation.

Authentication happens once during SDK initialization. After you initialize the SDK client with authentication as shown in the authentication examples, the SDK automatically handles authentication for all subsequent API calls. You do not need to include authentication parameters in individual API requests. The SDK Example Usage section shows how to initialize the SDK and make API calls, but if you've properly initialized your client as shown in the authentication examples, you only need to make the API method calls themselves without re-initializing.

Per-Client Security Schemes

This SDK supports the following security schemes globally:

Name	Type	Scheme	Environment Variable
`BearerAuth`	http	HTTP Bearer	`CRIBLCONTROLPLANE_BEARER_AUTH`
`ClientOauth`	oauth2	OAuth2 token	`CRIBLCONTROLPLANE_CLIENT_OAUTH`

To configure authentication on Cribl.Cloud and in hybrid deployments, use the ClientOauth security scheme. The SDK uses the OAuth credentials that you provide to obtain a Bearer token and refresh the token within its expiration window using the standard OAuth2 flow.

In on-prem deployments, use the BearerAuth security scheme. The SDK uses the username/password credentials that you provide to obtain a Bearer token. Automatically refreshing the Bearer token within its expiration window requires a callback function as shown in the On-Prem Authentication Example.

Set the security scheme when initializing the SDK client instance using one of these optional parameters:

WithSecurity: Use for static security values (OAuth2 credentials that the SDK will manage automatically)
WithSecuritySource: Use for dynamic security with a callback function (automatic Bearer token refresh in on-prem deployments)

The SDK uses the selected scheme by default to authenticate with the API for all operations that support it.

Authentication Examples

The Cribl.Cloud and Hybrid Authentication Example demonstrates how to configure authentication on Cribl.Cloud and in hybrid deployments. To obtain the Client ID and Client Secret you'll need to initialize using the ClientOauth security schema, follow the instructions for creating an API Credential in the Cribl as Code documentation.

The On-Prem Authentication Example demonstrates how to configure authentication in on-prem deployments using your username and password.

Available Resources and Operations

Available methods

Auth.Tokens

Get - Log in and fetch an authentication token

Destinations

List - List all Destinations
Create - Create a Destination
Get - Get a Destination
Update - Update a Destination
Delete - Delete a Destination

Destinations.Pq

Clear - Clear the persistent queue for a Destination
Get - Get information about the latest job to clear the persistent queue for a Destination

Destinations.Samples

Get - Get sample event data for a Destination
Create - Send sample event data to a Destination

Groups

List - List all Worker Groups or Edge Fleets for the specified Cribl product
Create - Create a Worker Group or Edge Fleet for the specified Cribl product
Get - Get a Worker Group or Edge Fleet
Update - Update a Worker Group or Edge Fleet
Delete - Delete a Worker Group or Edge Fleet
Deploy - Deploy commits to a Worker Group or Edge Fleet

Groups.ACL

Get - Get the Access Control List for a Worker Group or Edge Fleet

Groups.ACL.Teams

Get - Get the Access Control List for teams with permissions on a Worker Group or Edge Fleet for the specified Cribl product

Groups.Configs.Versions

Get - Get the configuration version for a Worker Group or Edge Fleet

Health

Get - Retrieve health status of the server

LakeDatasets

Create - Create a Lake Dataset
List - List all Lake Datasets
Delete - Delete a Lake Dataset
Get - Get a Lake Dataset
Update - Update a Lake Dataset

Nodes

List - Get detailed metadata for Worker and Edge Nodes
Count - Get a count of Worker and Edge Nodes

Nodes.Summaries

Get - Get a summary of the Distributed deployment

Packs

Install - Install a Pack
List - List all Packs
Upload - Upload a Pack file
Delete - Uninstall a Pack
Get - Get a Pack
Update - Upgrade a Pack

Pipelines

List - List all Pipelines
Create - Create a Pipeline
Get - Get a Pipeline
Update - Update a Pipeline
Delete - Delete a Pipeline

Routes

List - List all Routes
Get - Get a Routing table
Update - Update a Route
Append - Add a Route to the end of the Routing table

Sources

List - List all Sources
Create - Create a Source
Get - Get a Source
Update - Update a Source
Delete - Delete a Source

Sources.HecTokens

Create - Add an HEC token and optional metadata to a Splunk HEC Source
Update - Update metadata for an HEC token for a Splunk HEC Source

Versions.Branches

List - List all branches in the Git repository used for Cribl configuration
Get - Get the name of the Git branch that the Cribl configuration is checked out to

Versions.Commits

Create - Create a new commit for pending changes to the Cribl configuration
Diff - Get the diff for a commit
List - List the commit history
Push - Push local commits to the remote repository
Revert - Revert a commit in the local repository
Get - Get the diff and log message for a commit
Undo - Discard uncommitted (staged) changes

Versions.Commits.Files

Count - Get a count of files that changed since a commit
List - Get the names and statuses of files that changed since a commit

Versions.Configs

Get - Get the configuration and status for the Git integration

Versions.Statuses

Get - Get the status of the current working tree

Retries

Some of the endpoints in this SDK support retries. If you use the SDK without any configuration, it will fall back to the default retry strategy provided by the API. However, the default retry strategy can be overridden on a per-operation basis, or across the entire SDK.

To change the default retry strategy for a single API call, simply provide a retry.Config object to the call by using the WithRetries option:

package main

import (
	"context"
	criblcontrolplanesdkgo "github.com/criblio/cribl-control-plane-sdk-go"
	"github.com/criblio/cribl-control-plane-sdk-go/models/components"
	"github.com/criblio/cribl-control-plane-sdk-go/retry"
	"log"
	"models/operations"
	"os"
)

func main() {
	ctx := context.Background()

	s := criblcontrolplanesdkgo.New(
		"https://api.example.com",
		criblcontrolplanesdkgo.WithSecurity(components.Security{
			BearerAuth: criblcontrolplanesdkgo.Pointer(os.Getenv("CRIBLCONTROLPLANE_BEARER_AUTH")),
		}),
	)

	res, err := s.LakeDatasets.Create(ctx, "<id>", components.CriblLakeDataset{
		AcceleratedFields: []string{
			"<value 1>",
			"<value 2>",
		},
		BucketName: criblcontrolplanesdkgo.Pointer("<value>"),
		CacheConnection: &components.CacheConnection{
			AcceleratedFields: []string{
				"<value 1>",
				"<value 2>",
			},
			BackfillStatus:          components.CacheConnectionBackfillStatusPending.ToPointer(),
			CacheRef:                "<value>",
			CreatedAt:               7795.06,
			LakehouseConnectionType: components.LakehouseConnectionTypeCache.ToPointer(),
			MigrationQueryID:        criblcontrolplanesdkgo.Pointer("<id>"),
			RetentionInDays:         1466.58,
		},
		DeletionStartedAt: criblcontrolplanesdkgo.Pointer[float64](8310.58),
		Description:       criblcontrolplanesdkgo.Pointer("pleased toothbrush long brush smooth swiftly rightfully phooey chapel"),
		Format:            components.CriblLakeDatasetFormatDdss.ToPointer(),
		HTTPDAUsed:        criblcontrolplanesdkgo.Pointer(true),
		ID:                "<id>",
		Metrics: &components.LakeDatasetMetrics{
			CurrentSizeBytes: 6170.04,
			MetricsDate:      "<value>",
		},
		RetentionPeriodInDays: criblcontrolplanesdkgo.Pointer[float64](456.37),
		SearchConfig: &components.LakeDatasetSearchConfig{
			Datatypes: []string{
				"<value 1>",
			},
			Metadata: &components.DatasetMetadata{
				Earliest:           "<value>",
				EnableAcceleration: true,
				FieldList: []string{
					"<value 1>",
					"<value 2>",
				},
				LatestRunInfo: &components.DatasetMetadataRunInfo{
					EarliestScannedTime: criblcontrolplanesdkgo.Pointer[float64](4334.7),
					FinishedAt:          criblcontrolplanesdkgo.Pointer[float64](6811.22),
					LatestScannedTime:   criblcontrolplanesdkgo.Pointer[float64](5303.3),
					ObjectCount:         criblcontrolplanesdkgo.Pointer[float64](9489.04),
				},
				ScanMode: components.ScanModeDetailed,
			},
		},
		StorageLocationID: criblcontrolplanesdkgo.Pointer("<id>"),
		ViewName:          criblcontrolplanesdkgo.Pointer("<value>"),
	}, operations.WithRetries(
		retry.Config{
			Strategy: "backoff",
			Backoff: &retry.BackoffStrategy{
				InitialInterval: 1,
				MaxInterval:     50,
				Exponent:        1.1,
				MaxElapsedTime:  100,
			},
			RetryConnectionErrors: false,
		}))
	if err != nil {
		log.Fatal(err)
	}
	if res.Object != nil {
		// handle response
	}
}

If you'd like to override the default retry strategy for all operations that support retries, you can use the WithRetryConfig option at SDK initialization:

package main

import (
	"context"
	criblcontrolplanesdkgo "github.com/criblio/cribl-control-plane-sdk-go"
	"github.com/criblio/cribl-control-plane-sdk-go/models/components"
	"github.com/criblio/cribl-control-plane-sdk-go/retry"
	"log"
	"os"
)

func main() {
	ctx := context.Background()

	s := criblcontrolplanesdkgo.New(
		"https://api.example.com",
		criblcontrolplanesdkgo.WithRetryConfig(
			retry.Config{
				Strategy: "backoff",
				Backoff: &retry.BackoffStrategy{
					InitialInterval: 1,
					MaxInterval:     50,
					Exponent:        1.1,
					MaxElapsedTime:  100,
				},
				RetryConnectionErrors: false,
			}),
		criblcontrolplanesdkgo.WithSecurity(components.Security{
			BearerAuth: criblcontrolplanesdkgo.Pointer(os.Getenv("CRIBLCONTROLPLANE_BEARER_AUTH")),
		}),
	)

	res, err := s.LakeDatasets.Create(ctx, "<id>", components.CriblLakeDataset{
		AcceleratedFields: []string{
			"<value 1>",
			"<value 2>",
		},
		BucketName: criblcontrolplanesdkgo.Pointer("<value>"),
		CacheConnection: &components.CacheConnection{
			AcceleratedFields: []string{
				"<value 1>",
				"<value 2>",
			},
			BackfillStatus:          components.CacheConnectionBackfillStatusPending.ToPointer(),
			CacheRef:                "<value>",
			CreatedAt:               7795.06,
			LakehouseConnectionType: components.LakehouseConnectionTypeCache.ToPointer(),
			MigrationQueryID:        criblcontrolplanesdkgo.Pointer("<id>"),
			RetentionInDays:         1466.58,
		},
		DeletionStartedAt: criblcontrolplanesdkgo.Pointer[float64](8310.58),
		Description:       criblcontrolplanesdkgo.Pointer("pleased toothbrush long brush smooth swiftly rightfully phooey chapel"),
		Format:            components.CriblLakeDatasetFormatDdss.ToPointer(),
		HTTPDAUsed:        criblcontrolplanesdkgo.Pointer(true),
		ID:                "<id>",
		Metrics: &components.LakeDatasetMetrics{
			CurrentSizeBytes: 6170.04,
			MetricsDate:      "<value>",
		},
		RetentionPeriodInDays: criblcontrolplanesdkgo.Pointer[float64](456.37),
		SearchConfig: &components.LakeDatasetSearchConfig{
			Datatypes: []string{
				"<value 1>",
			},
			Metadata: &components.DatasetMetadata{
				Earliest:           "<value>",
				EnableAcceleration: true,
				FieldList: []string{
					"<value 1>",
					"<value 2>",
				},
				LatestRunInfo: &components.DatasetMetadataRunInfo{
					EarliestScannedTime: criblcontrolplanesdkgo.Pointer[float64](4334.7),
					FinishedAt:          criblcontrolplanesdkgo.Pointer[float64](6811.22),
					LatestScannedTime:   criblcontrolplanesdkgo.Pointer[float64](5303.3),
					ObjectCount:         criblcontrolplanesdkgo.Pointer[float64](9489.04),
				},
				ScanMode: components.ScanModeDetailed,
			},
		},
		StorageLocationID: criblcontrolplanesdkgo.Pointer("<id>"),
		ViewName:          criblcontrolplanesdkgo.Pointer("<value>"),
	})
	if err != nil {
		log.Fatal(err)
	}
	if res.Object != nil {
		// handle response
	}
}

Error Handling

Handling errors in this SDK should largely match your expectations. All operations return a response object or an error, they will never return both.

By Default, an API error will return apierrors.APIError. When custom error responses are specified for an operation, the SDK may also return their associated error. You can refer to respective Errors tables in SDK docs for more details on possible error types for each operation.

For example, the Create function may return the following errors:

Error Type	Status Code	Content Type
apierrors.Error	500	application/json
apierrors.APIError	4XX, 5XX	/

Example

package main

import (
	"context"
	"errors"
	criblcontrolplanesdkgo "github.com/criblio/cribl-control-plane-sdk-go"
	"github.com/criblio/cribl-control-plane-sdk-go/models/apierrors"
	"github.com/criblio/cribl-control-plane-sdk-go/models/components"
	"log"
	"os"
)

func main() {
	ctx := context.Background()

	s := criblcontrolplanesdkgo.New(
		"https://api.example.com",
		criblcontrolplanesdkgo.WithSecurity(components.Security{
			BearerAuth: criblcontrolplanesdkgo.Pointer(os.Getenv("CRIBLCONTROLPLANE_BEARER_AUTH")),
		}),
	)

	res, err := s.LakeDatasets.Create(ctx, "<id>", components.CriblLakeDataset{
		AcceleratedFields: []string{
			"<value 1>",
			"<value 2>",
		},
		BucketName: criblcontrolplanesdkgo.Pointer("<value>"),
		CacheConnection: &components.CacheConnection{
			AcceleratedFields: []string{
				"<value 1>",
				"<value 2>",
			},
			BackfillStatus:          components.CacheConnectionBackfillStatusPending.ToPointer(),
			CacheRef:                "<value>",
			CreatedAt:               7795.06,
			LakehouseConnectionType: components.LakehouseConnectionTypeCache.ToPointer(),
			MigrationQueryID:        criblcontrolplanesdkgo.Pointer("<id>"),
			RetentionInDays:         1466.58,
		},
		DeletionStartedAt: criblcontrolplanesdkgo.Pointer[float64](8310.58),
		Description:       criblcontrolplanesdkgo.Pointer("pleased toothbrush long brush smooth swiftly rightfully phooey chapel"),
		Format:            components.CriblLakeDatasetFormatDdss.ToPointer(),
		HTTPDAUsed:        criblcontrolplanesdkgo.Pointer(true),
		ID:                "<id>",
		Metrics: &components.LakeDatasetMetrics{
			CurrentSizeBytes: 6170.04,
			MetricsDate:      "<value>",
		},
		RetentionPeriodInDays: criblcontrolplanesdkgo.Pointer[float64](456.37),
		SearchConfig: &components.LakeDatasetSearchConfig{
			Datatypes: []string{
				"<value 1>",
			},
			Metadata: &components.DatasetMetadata{
				Earliest:           "<value>",
				EnableAcceleration: true,
				FieldList: []string{
					"<value 1>",
					"<value 2>",
				},
				LatestRunInfo: &components.DatasetMetadataRunInfo{
					EarliestScannedTime: criblcontrolplanesdkgo.Pointer[float64](4334.7),
					FinishedAt:          criblcontrolplanesdkgo.Pointer[float64](6811.22),
					LatestScannedTime:   criblcontrolplanesdkgo.Pointer[float64](5303.3),
					ObjectCount:         criblcontrolplanesdkgo.Pointer[float64](9489.04),
				},
				ScanMode: components.ScanModeDetailed,
			},
		},
		StorageLocationID: criblcontrolplanesdkgo.Pointer("<id>"),
		ViewName:          criblcontrolplanesdkgo.Pointer("<value>"),
	})
	if err != nil {

		var e *apierrors.Error
		if errors.As(err, &e) {
			// handle error
			log.Fatal(e.Error())
		}

		var e *apierrors.APIError
		if errors.As(err, &e) {
			// handle error
			log.Fatal(e.Error())
		}
	}
}

Custom HTTP Client

The Go SDK makes API calls that wrap an internal HTTP client. The requirements for the HTTP client are very simple. It must match this interface:

type HTTPClient interface {
	Do(req *http.Request) (*http.Response, error)
}

The built-in net/http client satisfies this interface and a default client based on the built-in is provided by default. To replace this default with a client of your own, you can implement this interface yourself or provide your own client configured as desired. Here's a simple example, which adds a client with a 30 second timeout.

import (
	"net/http"
	"time"

	"github.com/criblio/cribl-control-plane-sdk-go"
)

var (
	httpClient = &http.Client{Timeout: 30 * time.Second}
	sdkClient  = criblcontrolplanesdkgo.New(criblcontrolplanesdkgo.WithClient(httpClient))
)

This can be a convenient way to configure timeouts, cookies, proxies, custom headers, and other low-level configuration.

Name		Name	Last commit message	Last commit date
Latest commit History 309 Commits
.devcontainer		.devcontainer
.github		.github
.speakeasy		.speakeasy
docs		docs
examples		examples
internal		internal
models		models
optionalnullable		optionalnullable
retry		retry
scripts		scripts
types		types
.genignore		.genignore
.gitattributes		.gitattributes
.gitignore		.gitignore
CONTRIBUTING.md		CONTRIBUTING.md
README.md		README.md
RELEASES.md		RELEASES.md
USAGE.md		USAGE.md
acl.go		acl.go
auth.go		auth.go
branches.go		branches.go
commits.go		commits.go
configsversions.go		configsversions.go
criblcontrolplane.go		criblcontrolplane.go
destinations.go		destinations.go
files.go		files.go
go.mod		go.mod
go.sum		go.sum
groups.go		groups.go
groupsconfigs.go		groupsconfigs.go
health.go		health.go
hectokens.go		hectokens.go
lakedatasets.go		lakedatasets.go
nodes.go		nodes.go
packs.go		packs.go
pipelines.go		pipelines.go
pq.go		pq.go
routes.go		routes.go
samples.go		samples.go
sources.go		sources.go
statuses.go		statuses.go
summaries.go		summaries.go
teams.go		teams.go
tokens.go		tokens.go
versions.go		versions.go
versionsconfigs.go		versionsconfigs.go

criblio/cribl-control-plane-sdk-go

Folders and files

Latest commit

History

Repository files navigation

cribl-control-plane-sdk-go

Table of Contents

SDK Installation

SDK Example Usage

Example

Authentication

Per-Client Security Schemes

Authentication Examples

Available Resources and Operations

Retries

Error Handling

Example

Custom HTTP Client

About

Resources

Contributing

Uh oh!

Stars

Watchers

Forks

Uh oh!

Languages