Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[target-allocator] Introduce "per node" allocation strategy to target allocator #2430

Merged
merged 38 commits into from
Feb 9, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
6defa78
Add node name to Collector struct
matej-g Dec 5, 2023
318fd6c
Add per node allocation algorithm
matej-g Dec 5, 2023
d298b69
More docs
matej-g Dec 7, 2023
886a80c
Add tests
matej-g Dec 7, 2023
23089e3
Update APIs and docs with new strategy
matej-g Dec 7, 2023
7f82f69
Add changelog
matej-g Dec 8, 2023
959c3c5
Fix lint
matej-g Dec 8, 2023
082e678
Add more labels to match node name
matej-g Dec 8, 2023
2d29ae6
Better handling of unassignable targets / jobs
matej-g Dec 18, 2023
c08f560
Adjust webhook validation
matej-g Dec 18, 2023
8c0f9e6
Add E2E test; put TA E2E tests into separate target
matej-g Dec 18, 2023
5ad0079
Merge remote-tracking branch 'origin/main' into per-node-strategy
matej-g Dec 18, 2023
acf7644
Format; fix mistakenly changed test image
matej-g Dec 18, 2023
57d23bd
Use node name instead of collector name for collector map
matej-g Dec 19, 2023
0e09fc8
Add TA E2E tests to GH action
matej-g Dec 19, 2023
0c8239e
Merge branch 'main' into per-node-strategy
matej-g Dec 19, 2023
9c6c6e6
Bump kuttl timeout
matej-g Dec 19, 2023
6d08c78
Merge remote-tracking branch 'origin/main' into per-node-strategy
matej-g Dec 20, 2023
94b336e
Validate mode and strategy
matej-g Jan 4, 2024
023ce4b
Handle cases of missing node name when (re)adding collector
matej-g Jan 4, 2024
113f6bb
Merge remote-tracking branch 'origin/main' into per-node-strategy
matej-g Jan 8, 2024
5976307
Adjust test cases after node name fix
matej-g Jan 8, 2024
fe5e28d
Merge remote-tracking branch 'origin/main' into per-node-strategy
matej-g Jan 9, 2024
3ac78c7
Adjust E2E tests
matej-g Jan 9, 2024
85c66e5
Merge remote-tracking branch 'origin/main' into per-node-strategy
matej-g Jan 9, 2024
0c3fbdc
Merge remote-tracking branch 'origin/main' into per-node-strategy
matej-g Jan 15, 2024
0b3363e
Put TA config validation into separate method
matej-g Jan 15, 2024
02ee921
Adjust logging and add metric for jobs with unassigned targets
matej-g Jan 15, 2024
d2d46c9
Merge branch 'main' into per-node-strategy
matej-g Jan 15, 2024
485d9d6
Merge branch 'main' into per-node-strategy
matej-g Jan 15, 2024
b8befdf
Address feedback
matej-g Jan 18, 2024
f7eeb5e
Merge remote-tracking branch 'origin/main' into per-node-strategy
matej-g Jan 18, 2024
253b340
Merge remote-tracking branch 'origin/main' into per-node-strategy
matej-g Feb 1, 2024
9075a84
Simplify flow according to feedback; correct docs
matej-g Feb 6, 2024
339ed5c
Add more unit tests for edge cases
matej-g Feb 6, 2024
7defcdc
Merge remote-tracking branch 'origin/main' into per-node-strategy
matej-g Feb 6, 2024
5d31f67
Fix test case assert file
matej-g Feb 6, 2024
4f195d5
Merge branch 'main' into per-node-strategy
matej-g Feb 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add tests
Signed-off-by: Matej Gera <matejgera@gmail.com>
  • Loading branch information
matej-g committed Dec 8, 2023
commit 886a80cf8489808ddd0cb65b69d61358d4f2c8ee
1 change: 1 addition & 0 deletions cmd/otel-allocator/allocation/allocatortest.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ func MakeNCollectors(n int, startingIndex int) map[string]*Collector {
toReturn[collector] = &Collector{
Name: collector,
NumTargets: 0,
Node: fmt.Sprintf("node-%d", i),
}
}
return toReturn
Expand Down
67 changes: 66 additions & 1 deletion cmd/otel-allocator/allocation/per_node_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,69 @@

package allocation

// TODO: Add tests
import (
"testing"

"github.com/open-telemetry/opentelemetry-operator/cmd/otel-allocator/target"
"github.com/prometheus/common/model"
"github.com/stretchr/testify/assert"
logf "sigs.k8s.io/controller-runtime/pkg/log"
)

var loggerPerNode = logf.Log.WithName("unit-tests")

// Tests that two targets with the same target url and job name but different label set are both added.
func TestAllocationPerNode(t *testing.T) {
matej-g marked this conversation as resolved.
Show resolved Hide resolved
// prepare allocator with initial targets and collectors
s, _ := New("per-node", loggerPerNode)

cols := MakeNCollectors(3, 0)
s.SetCollectors(cols)
firstLabels := model.LabelSet{
"test": "test1",
podNodeNameLabel: "node-0",
}
secondLabels := model.LabelSet{
"test": "test2",
podNodeNameLabel: "node-1",
}
// no label, should be skipped
thirdLabels := model.LabelSet{
"test": "test3",
}
firstTarget := target.NewItem("sample-name", "0.0.0.0:8000", firstLabels, "")
secondTarget := target.NewItem("sample-name", "0.0.0.0:8000", secondLabels, "")
thirdTarget := target.NewItem("sample-name", "0.0.0.0:8000", thirdLabels, "")

targetList := map[string]*target.Item{
firstTarget.Hash(): firstTarget,
secondTarget.Hash(): secondTarget,
thirdTarget.Hash(): thirdTarget,
}

// test that targets and collectors are added properly
s.SetTargets(targetList)

// verify length
actualItems := s.TargetItems()

// one target should be skipped
expectedTargetLen := len(targetList) - 1
assert.Len(t, actualItems, expectedTargetLen)

// verify allocation to nodes
for targetHash, item := range targetList {
actualItem, found := actualItems[targetHash]
// if third target, should be skipped
if targetHash != thirdTarget.Hash() {
assert.True(t, found, "target with hash %s not found", item.Hash())
} else {
assert.False(t, found, "target with hash %s should not be found", item.Hash())
return
}

itemsForCollector := s.GetTargetsForCollectorAndJob(actualItem.CollectorName, actualItem.JobName)
assert.Len(t, itemsForCollector, 1)
assert.Equal(t, actualItem, itemsForCollector[0])
}
}