Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: disable continuous loading of check config #108

Merged
merged 6 commits into from
Feb 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 15 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,7 @@ startupConfig:
...
loader:
type: http
interval: 30s
http:
url: https://url-to-checks-config.de/api/config%2Eyaml

Expand Down Expand Up @@ -194,6 +195,7 @@ loader:
# defines which loader to use. Options: "file | http"
type: http
# the interval in which sparrow tries to fetch a new configuration
# if this isn't set or set to 0, the loader will only retrieve the configuration once
interval: 30s
# config specific to the http loader
http:
Expand All @@ -210,8 +212,7 @@ loader:
count: 3

# config specific to the file loader
# The file loader is not intended for production use and does
# not refresh the config after reading it the first time
# The file loader is not intended for production use
file:
# location of the file in the local filesystem
path: ./config.yaml
Expand Down Expand Up @@ -259,15 +260,16 @@ Available loaders:
- `http` (default): Retrieves the checks' configuration from a remote endpoint during runtime. Additional configuration
parameters are set in the `loader.http` section.

- `file` (experimental): Intended for development, it loads the configuration once from a local file and does not
refresh after the first load. The target manager is currently not functional in combination with this loader type.
- `file`: Loads the configuration from a local file during runtime. Additional configuration
parameters are set in the `loader.file` section.

If you want to retrieve the checks' configuration only once, you can set `loader.interval` to 0.
The target manager is currently not functional in combination with this configuration.

### Checks

In addition to the technical startup configuration, the `sparrow` checks' configuration can be dynamically loaded from
an HTTP endpoint during runtime.
For local use, you may directly load the configuration using a `file` loader. The `loader` is capable of dynamically
loading and configuring checks.
In addition to the technical startup configuration, the `sparrow` checks' configuration can be dynamically loaded during runtime.
The `loader` is capable of dynamically loading and configuring checks.

For detailed information on available loader configuration options, please refer
to [this documentation](docs/sparrow_run.md).
Expand All @@ -287,7 +289,7 @@ the `targetManager`, it will not be used. When configured, it offers various set
in the startup YAML configuration file as shown in the [example configuration](#example-startup-configuration).

| Type | Description |
|--------------------------------------|----------------------------------------------------------------------------------------------|
| ------------------------------------ | -------------------------------------------------------------------------------------------- |
| `targetManager.checkInterval` | Interval for checking new targets. |
| `targetManager.unhealthyThreshold` | Threshold for marking a target as unhealthy. 0 means no cleanup. |
| `targetManager.registrationInterval` | Interval for registering the current sparrow at the target backend. 0 means no registration. |
Expand All @@ -314,7 +316,7 @@ which is named after the DNS name of the `sparrow`. The state file contains the
Available configuration options:

| Field | Type | Description |
|---------------|-------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|
| ------------- | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `interval` | `duration` | Interval to perform the health check. |
| `timeout` | `duration` | Timeout for the health check. |
| `retry.count` | `integer` | Number of retries for the health check. |
Expand Down Expand Up @@ -347,7 +349,7 @@ health:
Available configuration options:

| Field | Type | Description |
|---------------|-------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|
| ------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `interval` | `duration` | Interval to perform the latency check. |
| `timeout` | `duration` | Timeout for the latency check. |
| `retry.count` | `integer` | Number of retries for the latency check. |
Expand Down Expand Up @@ -390,7 +392,7 @@ latency:
Available configuration options:

| Field | Type | Description |
|---------------|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|
| ------------- | ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `interval` | `duration` | Interval to perform the DNS check. |
| `timeout` | `duration` | Timeout for the DNS check. |
| `retry.count` | `integer` | Number of retries for the DNS check. |
Expand Down Expand Up @@ -463,7 +465,7 @@ The application itself and all end-user facing content will be made available in
The following channels are available for discussions, feedback, and support requests:

| Type | Channel |
|------------|--------------------------------------------------------------------------------------------------------------------------------------------------------|
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Issues** | <a href="/../../issues/new/choose" title="General Discussion"><img src="https://img.shields.io/github/issues/caas-team/sparrow?style=flat-square"></a> |

## How to Contribute
Expand Down
14 changes: 13 additions & 1 deletion pkg/config/file.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,23 @@ func NewFileLoader(cfg *Config, cRuntime chan<- runtime.Config) *FileLoader {

// Run gets the runtime configuration from the local file.
// The config will be loaded periodically defined by the loader interval configuration.
// Returns an error if the loader is shutdown or the context is done.
func (f *FileLoader) Run(ctx context.Context) error {
ctx, cancel := logger.NewContextWithLogger(ctx)
defer cancel()
log := logger.FromContext(ctx)

if f.config.Interval == 0 {
cfg, err := f.getRuntimeConfig(ctx)
if err != nil {
log.Warn("Could not get local runtime configuration", "error", err)
return fmt.Errorf("could not get local runtime configuration: %w", err)
}
lvlcn-t marked this conversation as resolved.
Show resolved Hide resolved

f.cRuntime <- cfg
log.Info("File Loader disabled")
return nil
}

tick := time.NewTicker(f.config.Interval)
defer tick.Stop()

Expand Down
28 changes: 23 additions & 5 deletions pkg/config/file_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -71,31 +71,49 @@ func TestFileLoader_Run(t *testing.T) {
},
wantErr: false,
},
{
name: "Continuous loading disabled",
config: LoaderConfig{
Type: "file",
Interval: 0,
File: FileLoaderConfig{
Path: "test/data/config.yaml",
},
},
want: runtime.Config{
Health: &health.Config{
Targets: []string{"http://localhost:8080/health"},
Interval: 1 * time.Second,
Timeout: 1 * time.Second,
},
},
wantErr: false,
},
}

for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
ctx := context.Background()
result := make(chan runtime.Config, 1)
defer close(result)
f := NewFileLoader(&Config{
Loader: tt.config,
}, result)

go func() {
go func(wantErr bool) {
defer close(result)
err := f.Run(ctx)
if (err != nil) != tt.wantErr {
if (err != nil) != wantErr {
t.Errorf("Run() error %v, want %v", err, tt.wantErr)
}
}()
}(tt.wantErr)
defer f.Shutdown(ctx)

if !tt.wantErr {
config := <-result
if !reflect.DeepEqual(config, tt.want) {
t.Errorf("Expected config to be %v, got %v", tt.want, config)
}
}
f.Shutdown(ctx)
})
}
}
Expand Down
24 changes: 18 additions & 6 deletions pkg/config/http.go
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,25 @@ func (hl *HttpLoader) Run(ctx context.Context) error {
ctx, cancel := logger.NewContextWithLogger(ctx)
defer cancel()
log := logger.FromContext(ctx)

var runtimeCfg *runtime.Config
getConfigRetry := helper.Retry(func(ctx context.Context) (err error) {
runtimeCfg, err = hl.getRuntimeConfig(ctx)
return err
}, hl.cfg.Http.RetryCfg)

if hl.cfg.Interval == 0 {
err := getConfigRetry(ctx)
if err != nil {
log.Warn("Could not get remote runtime configuration", "error", err)
return fmt.Errorf("could not get remote runtime configuration: %w", err)
}
lvlcn-t marked this conversation as resolved.
Show resolved Hide resolved

hl.cRuntime <- *runtimeCfg
log.Info("HTTP Loader disabled")
return nil
}

tick := time.NewTicker(hl.cfg.Interval)
defer tick.Stop()

Expand All @@ -68,12 +86,6 @@ func (hl *HttpLoader) Run(ctx context.Context) error {
case <-ctx.Done():
return ctx.Err()
case <-tick.C:
getConfigRetry := helper.Retry(func(ctx context.Context) error {
var err error
runtimeCfg, err = hl.getRuntimeConfig(ctx)
return err
}, hl.cfg.Http.RetryCfg)

if err := getConfigRetry(ctx); err != nil {
log.Warn("Could not get remote runtime configuration", "error", err)
tick.Reset(hl.cfg.Interval)
Expand Down
58 changes: 38 additions & 20 deletions pkg/config/http_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -160,46 +160,62 @@ func TestHttpLoader_Run(t *testing.T) {

tests := []struct {
name string
interval time.Duration
response *runtime.Config
code int
wantErr bool
}{
{
name: "non-200 response",
interval: 500 * time.Millisecond,
response: nil,
code: 500,
code: http.StatusInternalServerError,
wantErr: false,
},
{
name: "empty checks' configuration",
interval: 500 * time.Millisecond,
response: &runtime.Config{},
code: 200,
code: http.StatusOK,
},
{
name: "config with health check",
name: "config with health check",
interval: 500 * time.Millisecond,
response: &runtime.Config{
Health: &health.Config{
Targets: []string{"http://localhost:8080/health"},
Interval: 1 * time.Second,
},
},
code: 200,
code: http.StatusOK,
},
{
name: "continuous loading disabled",
interval: 0,
response: &runtime.Config{
Health: &health.Config{
Targets: []string{"http://localhost:8080/health"},
Interval: 1 * time.Second,
},
},
code: http.StatusOK,
wantErr: false,
},
}

for _, tt := range tests {
body, err := yaml.Marshal(tt.response)
if err != nil {
t.Fatalf("Failed marshaling response to bytes: %v", err)
}
resp := httpmock.NewBytesResponder(tt.code, body)
httpmock.RegisterResponder(http.MethodGet, "https://api.test.com/test", resp)

t.Run(tt.name, func(t *testing.T) {
body, err := yaml.Marshal(tt.response)
if err != nil {
t.Fatalf("Failed marshaling response to bytes: %v", err)
}
resp := httpmock.NewBytesResponder(tt.code, body)
httpmock.RegisterResponder(http.MethodGet, "https://api.test.com/test", resp)

hl := &HttpLoader{
cfg: LoaderConfig{
Type: "http",
Interval: time.Millisecond * 500,
Interval: tt.interval,
Http: HttpLoaderConfig{
Url: "https://api.test.com/test",
RetryCfg: helper.RetryConfig{
Expand All @@ -219,14 +235,16 @@ func TestHttpLoader_Run(t *testing.T) {
ctx := context.Background()
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
time.Sleep(time.Millisecond * 600)
t.Log("Shutting down the Run method")
hl.Shutdown(ctx)
}()

err := hl.Run(ctx)
if tt.interval > 0 {
go func() {
defer wg.Done()
time.Sleep(time.Millisecond * 600)
t.Log("Shutting down the Run method")
hl.Shutdown(ctx)
}()
}

err = hl.Run(ctx)
if (err != nil) != tt.wantErr {
t.Errorf("HttpLoader.Run() error = %v, wantErr %v", err, tt.wantErr)
}
Expand Down
5 changes: 5 additions & 0 deletions pkg/config/validate.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,11 @@ func (c *Config) Validate(ctx context.Context) error {
log.Error("The name of the sparrow must be DNS compliant")
}

if c.Loader.Interval < 0 {
ok = false
log.Error("The loader interval should be equal or above 0", "interval", c.Loader.Interval)
}

switch c.Loader.Type {
case "http":
if _, err := url.ParseRequestURI(c.Loader.Http.Url); err != nil {
Expand Down
Loading