Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chore][pkg/stanza] Upgrade expr module to latest #24648

Merged
merged 1 commit into from
Jul 31, 2023
Merged

[chore][pkg/stanza] Upgrade expr module to latest #24648

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion cmd/configschema/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -270,7 +270,7 @@ require (
github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect
github.com/aliyun/aliyun-log-go-sdk v0.1.50 // indirect
github.com/andybalholm/brotli v1.0.5 // indirect
github.com/antonmedv/expr v1.12.5 // indirect
github.com/antonmedv/expr v1.12.7 // indirect
github.com/apache/arrow/go/arrow v0.0.0-20211112161151-bc219186db40 // indirect
github.com/apache/pulsar-client-go v0.8.1 // indirect
github.com/apache/pulsar-client-go/oauth2 v0.0.0-20220120090717-25e59572242e // indirect
Expand Down
4 changes: 2 additions & 2 deletions cmd/configschema/go.sum
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion cmd/otelcontribcol/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -289,7 +289,7 @@ require (
github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect
github.com/aliyun/aliyun-log-go-sdk v0.1.50 // indirect
github.com/andybalholm/brotli v1.0.5 // indirect
github.com/antonmedv/expr v1.12.5 // indirect
github.com/antonmedv/expr v1.12.7 // indirect
github.com/apache/arrow/go/arrow v0.0.0-20211112161151-bc219186db40 // indirect
github.com/apache/pulsar-client-go v0.8.1 // indirect
github.com/apache/pulsar-client-go/oauth2 v0.0.0-20220120090717-25e59572242e // indirect
Expand Down
4 changes: 2 additions & 2 deletions cmd/otelcontribcol/go.sum
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion cmd/oteltestbedcol/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ require (
github.com/Microsoft/go-winio v0.6.0 // indirect
github.com/alecthomas/participle/v2 v2.0.0 // indirect
github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect
github.com/antonmedv/expr v1.12.5 // indirect
github.com/antonmedv/expr v1.12.7 // indirect
github.com/apache/thrift v0.18.1 // indirect
github.com/armon/go-metrics v0.4.1 // indirect
github.com/aws/aws-sdk-go v1.44.301 // indirect
Expand Down
4 changes: 2 additions & 2 deletions cmd/oteltestbedcol/go.sum
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion exporter/datadogexporter/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ require (
github.com/DataDog/zstd v1.5.2 // indirect
github.com/Microsoft/go-winio v0.6.1 // indirect
github.com/Showmax/go-fqdn v1.0.0 // indirect
github.com/antonmedv/expr v1.12.5 // indirect
github.com/antonmedv/expr v1.12.7 // indirect
github.com/armon/go-metrics v0.4.1 // indirect
github.com/benbjohnson/clock v1.3.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect
Expand Down
4 changes: 2 additions & 2 deletions exporter/datadogexporter/go.sum
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -267,7 +267,7 @@ require (
github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect
github.com/aliyun/aliyun-log-go-sdk v0.1.50 // indirect
github.com/andybalholm/brotli v1.0.5 // indirect
github.com/antonmedv/expr v1.12.5 // indirect
github.com/antonmedv/expr v1.12.7 // indirect
github.com/apache/arrow/go/arrow v0.0.0-20211112161151-bc219186db40 // indirect
github.com/apache/pulsar-client-go v0.8.1 // indirect
github.com/apache/pulsar-client-go/oauth2 v0.0.0-20220120090717-25e59572242e // indirect
Expand Down
4 changes: 2 additions & 2 deletions go.sum
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion pkg/stanza/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ module github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza
go 1.19

require (
github.com/antonmedv/expr v1.12.5
github.com/antonmedv/expr v1.12.7
github.com/bmatcuk/doublestar/v4 v4.6.0
github.com/cespare/xxhash/v2 v2.2.0
github.com/influxdata/go-syslog/v3 v3.0.1-0.20210608084020-ac565dc76ba6
Expand Down
4 changes: 2 additions & 2 deletions pkg/stanza/go.sum
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

26 changes: 23 additions & 3 deletions pkg/stanza/operator/helper/expr_string.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import (
"sync"

"github.com/antonmedv/expr"
"github.com/antonmedv/expr/ast"
"github.com/antonmedv/expr/vm"

"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/entry"
Expand Down Expand Up @@ -77,7 +78,7 @@ func (e ExprStringConfig) Build() (*ExprString, error) {

subExprs := make([]*vm.Program, 0, len(subExprStrings))
for _, subExprString := range subExprStrings {
program, err := expr.Compile(subExprString, expr.AllowUndefinedVariables())
program, err := expr.Compile(subExprString, expr.AllowUndefinedVariables(), expr.Patch(&patcher{}))
if err != nil {
return nil, errors.Wrap(err, "compile embedded expression")
}
Expand All @@ -90,6 +91,10 @@ func (e ExprStringConfig) Build() (*ExprString, error) {
}, nil
}

func ExprCompileBool(input string) (*vm.Program, error) {
return expr.Compile(input, expr.AllowUndefinedVariables(), expr.Patch(&patcher{}), expr.AsBool())
}

// An ExprString is made up of a list of string literals
// interleaved with expressions. len(SubStrings) == len(SubExprs) + 1
type ExprString struct {
Expand All @@ -113,14 +118,29 @@ func (e *ExprString) Render(env map[string]interface{}) (string, error) {
b.WriteString(outString)
}
b.WriteString(e.SubStrings[len(e.SubStrings)-1])

return b.String(), nil
}

type patcher struct{}

func (p *patcher) Visit(node *ast.Node) {
n, ok := (*node).(*ast.CallNode)
if !ok {
return
}
c, ok := (n.Callee).(*ast.IdentifierNode)
if !ok {
return
}
if c.Value == "env" {
c.Value = "os_env_func"
}
}

var envPool = sync.Pool{
New: func() interface{} {
return map[string]interface{}{
"env": os.Getenv,
"os_env_func": os.Getenv,
}
},
}
Expand Down
17 changes: 15 additions & 2 deletions pkg/stanza/operator/helper/expr_string_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ import (
)

func TestExprString(t *testing.T) {
t.Setenv("TEST_EXPR_STRING_ENV", "foo")
t.Setenv("TEST_EXPR_STRING_ENV_FOO", "foo")
t.Setenv("TEST_EXPR_STRING_ENV_BAR", "bar")

exampleEntry := func() *entry.Entry {
e := entry.New()
Expand Down Expand Up @@ -74,14 +75,26 @@ func TestExprString(t *testing.T) {
"my EXPR(body.test )",
"my value",
},
{
"my EXPR( body.test)",
"my value",
},
{
"my EXPR(body.test)",
"my value",
},
{
"my EXPR(env('TEST_EXPR_STRING_ENV'))",
"my EXPR(env('TEST_EXPR_STRING_ENV_FOO'))",
"my foo",
},
{
"my EXPR( env('TEST_EXPR_STRING_ENV_FOO') )",
"my foo",
},
{
"my EXPR(env('TEST_EXPR_STRING_ENV_FOO')) EXPR(env('TEST_EXPR_STRING_ENV_BAR'))",
"my foo bar",
},
{
"EXPR( resource.id )",
"value",
Expand Down
3 changes: 1 addition & 2 deletions pkg/stanza/operator/helper/transformer.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ import (
"context"
"fmt"

"github.com/antonmedv/expr"
"github.com/antonmedv/expr/vm"
"go.uber.org/zap"

Expand Down Expand Up @@ -53,7 +52,7 @@ func (c TransformerConfig) Build(logger *zap.SugaredLogger) (TransformerOperator
}

if c.IfExpr != "" {
compiled, err := expr.Compile(c.IfExpr, expr.AsBool(), expr.AllowUndefinedVariables())
compiled, err := ExprCompileBool(c.IfExpr)
if err != nil {
return TransformerOperator{}, fmt.Errorf("failed to compile expression '%s': %w", c.IfExpr, err)
}
Expand Down
3 changes: 1 addition & 2 deletions pkg/stanza/operator/transformer/filter/filter.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ import (
"fmt"
"math/big"

"github.com/antonmedv/expr"
"github.com/antonmedv/expr/vm"
"go.uber.org/zap"

Expand Down Expand Up @@ -56,7 +55,7 @@ func (c Config) Build(logger *zap.SugaredLogger) (operator.Operator, error) {
return nil, err
}

compiledExpression, err := expr.Compile(c.Expression, expr.AsBool(), expr.AllowUndefinedVariables())
compiledExpression, err := helper.ExprCompileBool(c.Expression)
if err != nil {
return nil, fmt.Errorf("failed to compile expression '%s': %w", c.Expression, err)
}
Expand Down
4 changes: 2 additions & 2 deletions pkg/stanza/operator/transformer/recombine/recombine.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,13 +80,13 @@ func (c *Config) Build(logger *zap.SugaredLogger) (operator.Operator, error) {
var prog *vm.Program
if c.IsFirstEntry != "" {
matchesFirst = true
prog, err = expr.Compile(c.IsFirstEntry, expr.AsBool(), expr.AllowUndefinedVariables())
prog, err = helper.ExprCompileBool(c.IsFirstEntry)
if err != nil {
return nil, fmt.Errorf("failed to compile is_first_entry: %w", err)
}
} else {
matchesFirst = false
prog, err = expr.Compile(c.IsLastEntry, expr.AsBool(), expr.AllowUndefinedVariables())
prog, err = helper.ExprCompileBool(c.IsLastEntry)
if err != nil {
return nil, fmt.Errorf("failed to compile is_last_entry: %w", err)
}
Expand Down
3 changes: 1 addition & 2 deletions pkg/stanza/operator/transformer/router/router.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ import (
"context"
"fmt"

"github.com/antonmedv/expr"
"github.com/antonmedv/expr/vm"
"go.uber.org/zap"

Expand Down Expand Up @@ -65,7 +64,7 @@ func (c Config) Build(logger *zap.SugaredLogger) (operator.Operator, error) {

routes := make([]*Route, 0, len(c.Routes))
for _, routeConfig := range c.Routes {
compiled, err := expr.Compile(routeConfig.Expression, expr.AsBool(), expr.AllowUndefinedVariables())
compiled, err := helper.ExprCompileBool(routeConfig.Expression)
if err != nil {
return nil, fmt.Errorf("failed to compile expression '%s': %w", routeConfig.Expression, err)
}
Expand Down
2 changes: 1 addition & 1 deletion processor/logstransformprocessor/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ require (
)

require (
github.com/antonmedv/expr v1.12.5 // indirect
github.com/antonmedv/expr v1.12.7 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
Expand Down
4 changes: 2 additions & 2 deletions processor/logstransformprocessor/go.sum
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion receiver/azureeventhubreceiver/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ require (
github.com/Azure/go-autorest/autorest/validation v0.3.1 // indirect
github.com/Azure/go-autorest/logger v0.2.1 // indirect
github.com/Azure/go-autorest/tracing v0.6.0 // indirect
github.com/antonmedv/expr v1.12.5 // indirect
github.com/antonmedv/expr v1.12.7 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
Expand Down
4 changes: 2 additions & 2 deletions receiver/azureeventhubreceiver/go.sum
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion receiver/filelogreceiver/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ require (
)

require (
github.com/antonmedv/expr v1.12.5 // indirect
github.com/antonmedv/expr v1.12.7 // indirect
github.com/bmatcuk/doublestar/v4 v4.6.0 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
Expand Down
4 changes: 2 additions & 2 deletions receiver/filelogreceiver/go.sum
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion receiver/journaldreceiver/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ require (
)

require (
github.com/antonmedv/expr v1.12.5 // indirect
github.com/antonmedv/expr v1.12.7 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
Expand Down
4 changes: 2 additions & 2 deletions receiver/journaldreceiver/go.sum
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion receiver/mongodbatlasreceiver/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ require (
)

require (
github.com/antonmedv/expr v1.12.5 // indirect
github.com/antonmedv/expr v1.12.7 // indirect
github.com/benbjohnson/clock v1.3.0 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
Expand Down
Loading