feat: New CEL and template functions to ease access to different part…

…s of the request and beyond (#689)

docs/content/docs/configuration/rules/pipeline_mechanisms/overview.adoc

@@ -296,9 +296,22 @@ Values = {
 
 == Templating
 
-Some pipeline mechanisms support templating using https://golang.org/pkg/text/template/[Golang Text Templates]. To ease the usage, all http://masterminds.github.io/sprig/[sprig] functions, except `env` and `expandenv`, as well as an `urlenc` function are available. Latter is handy if you need to generate request body or query parameters e.g. for communication with further systems. Templates can act on all objects described above (link:{{< relref "#_subject" >}}[Subject], link:{{< relref "#_request" >}}[Request], link:{{< relref "#_payload" >}}[Payload] and link:{{< relref "#_values" >}}[Values]). Which exactly are supported is mechanism specific.
+Some pipeline mechanisms support templating using https://golang.org/pkg/text/template/[Golang Text Templates]. Templates can act on all objects described above (link:{{< relref "#_subject" >}}[Subject], link:{{< relref "#_request" >}}[Request], link:{{< relref "#_payload" >}}[Payload] and link:{{< relref "#_values" >}}[Values]). Which exactly are supported is mechanism specific.
 
-.Template, rendering a JSON object
+To ease the usage, all http://masterminds.github.io/sprig/[sprig] functions, except `env` and `expandenv`, as well as the following functions are available:
+
+* `urlenc` - Encodes a given string using url encoding. Is handy if you need to generate request body or query parameters e.g. for communication with further systems.
+
+* `atIndex` - Implements python-like access to arrays and takes as a single argument the index to access the element in the array at. With index being a positive values it works exactly the same way, as with the usage of the build-in index function to access array elements. With negative index value, one can access the array elements from the tail of the array. -1 is the index of the last element, -2 the index of the element before the last one, etc.
++
+Example: `{{ atIndex 2 [1,2,3,4,5] }}` evaluates to `3` (behaves the same way as the `index` function) and `{{ atIndex -2 [1,2,3,4,5] }}` evaluates to `4`.
+
+* `splitList` - Splits a given string using a separator (part of the sprig library, but not documented). The result is a string array.
++
+Example: `{{ splitList "/" "/foo/bar" }}` evaluates to the `["", "foo", "bar"]` array.
+
+
+.Rendering a JSON object
 ====
 Imagine, we have a `POST` request for the URL `\http://foobar.baz/zab?foo=bar`, with a header `X-Foo` set to `bar` value, for which heimdall was able to identify a subject, with `ID=foo` and which `Attributes` contain an entry `email: foo@bar`, then you can generate a JSON object with this information with the following template:
 
@@ -331,11 +344,68 @@ This will result in the following JSON object:
 ----
 ====
 
+.Access the last part of the path
+====
+Imagine, we have a `POST` request to the URL `\http://foobar.baz/zab/1234`, with `1234` being the identifier of a file, which should be updated with the contents sent in the body of the request, and you would like to control access to the aforesaid object using e.g. OpenFGA. This can be achieved with the following authorizer:
+
+[source, yaml]
+----
+id: openfga_authorizer
+type: remote
+config:
+  endpoint:
+    url: https://openfga/stores/files/check
+  payload: |
+    {
+      "user": "user:{{ .Subject.ID }}",
+      "relation": "write",
+      "object": "file:{{ splitList "/" .Request.URL.Path | last }}"
+    }
+  expressions:
+  - expression: |
+      Payload.allowed == true
+----
+
+Please note how the `"object"` is set in the `payload` property above. When the `payload` template is rendered and for the above said request heimdall was able to identify the subject with `ID=foo`, following JSON object will be created:
+
+[source, json]
+----
+{
+  "user": "user:foo",
+  "relation": "write",
+  "object": "file:1234"
+}
+----
+====
+
 You can find further examples as part of mechanism descriptions, supporting templating.
 
 == Expressions
 
-Expressions can be used to execute authorization logic. As of today only https://github.com/google/cel-spec[CEL] is supported as expression language. Which of the link:{{<  relref "#_evaluation_objects" >}}[evaluation objects] are available to the expression depends on the mechanism.
+Expressions can be used to execute authorization logic. As of today only https://github.com/google/cel-spec[CEL] is supported as expression language. All standard, as well as https://pkg.go.dev/github.com/google/cel-go/ext#pkg-functions[extension] functions are available. Which of the link:{{<  relref "#_evaluation_objects" >}}[evaluation objects] are available to the expression depends on the mechanism.
+
+In addition to the build-in, respectively extension methods and functions, as well as the methods available on the evaluation objects, following functions are available as well:
+
+* `split` - this function works on strings and expects a separator as a single argument. The result is a string array.
++
+Example: `"/foo/bar/baz".split("/")` returns `["", "foo", "bar", "baz"]`.
+
+* `regexFind` - this function returns the first (left most) match of a regular expression in the given string.
++
+Example: `"abcd1234".regexFind("[a-zA-Z][1-9]")` returns `"d1"`.
+
+* `regexFindAll` - this function returns an array of all matches of a regular expression in the given string.
++
+Example: `"123456789".regexFindAll("[2,4,6,8]")` returns `["2","4","6","8"]`.
+
+* `at` - this function implements python-like access to arrays and takes as a single argument the index to access the element in the array at. With index being a positive values it works exactly the same way, as with the usage of `[]` to access array elements. With negative index value, one can access the array elements from the tail of the array. -1 is the index of the last element, -2 the index of the element before the last one, etc.
++
+Example: `[1,2,3,4,5].at(2)` returns `3` and `[1,2,3,4,5].at(-2)` returns `4`.
+
+* `last` - this function works on arrays and returns the last element of an array or `nil` if the array is empty.
++
+Example: `[1,2,3,4,5].last()` returns `5`
+
 
 Some examples:
 
@@ -355,20 +425,21 @@ a CEL expression to check the `result` attribute is set to `true`, would look as
 ----
 Payload.result == true
 ----
+====
 
-or even simpler:
-
+.Check whether the user is member of the admin group
+====
 [source, cel]
 ----
-Payload.result
+has(Subject.Attributes.groups) &&
+   Subject.Attributes.groups.exists(g, g == "admin")
 ----
 ====
 
-.Check whether the user is member of the admin group
+.Access the last path part of the matched URL
 ====
 [source, cel]
 ----
-has(Subject.Attributes.groups) &&
-   Subject.Attributes.groups.exists(g, g == "admin")
+Request.URL.Path.split("/").last()
 ----
 ====

go.sum

@@ -176,7 +176,6 @@ github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2
 github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
 github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY=
 github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A=
-github.com/go-logfmt/logfmt v0.5.1/go.mod h1:WYhtIu8zTZfxdn5+rREduYbwxfcBr/Vr6KEVveWlfTs=
 github.com/go-logr/logr v1.2.0/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
 github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
 github.com/go-logr/logr v1.2.4 h1:g01GSCwiDw2xSZfjJ2/T9M+S6pFdcNtFYsp+Y43HYDQ=
@@ -269,7 +268,6 @@ github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hf
 github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
 github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
 github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
-github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
 github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
 github.com/google/s2a-go v0.1.4 h1:1kZ/sQM3srePvKs3tXAvQzo66XfcReoqFpIpIccE7Oc=
 github.com/google/s2a-go v0.1.4/go.mod h1:Ej+mSEMGRnqRzjc7VtF+jdBwYG5fuJfiZ8ELkjEwM0A=
@@ -318,12 +316,10 @@ github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfC
 github.com/johannesboyne/gofakes3 v0.0.0-20230506070712-04da935ef877 h1:O7syWuYGzre3s73s+NkgB8e0ZvsIVhT/zxNU7V1gHK8=
 github.com/johannesboyne/gofakes3 v0.0.0-20230506070712-04da935ef877/go.mod h1:AxgWC4DDX54O2WDoQO1Ceabtn6IbktjU/7bigor+66g=
 github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
-github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4=
 github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
 github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
 github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
 github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
-github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
 github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
 github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
 github.com/klauspost/compress v1.16.3 h1:XuJt9zzcnaz6a16/OU53ZjWp/v7/42WcR5t2a0PcNQY=
@@ -379,7 +375,6 @@ github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJ
 github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
 github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
 github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
-github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
 github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=
 github.com/openzipkin/zipkin-go v0.4.1 h1:kNd/ST2yLLWhaWrkgchya40TJabe8Hioj9udfPcEO5A=
 github.com/openzipkin/zipkin-go v0.4.1/go.mod h1:qY0VqDSN1pOBN94dBc6w2GJlWLiovAyg7Qt6/I9HecM=
@@ -422,8 +417,6 @@ github.com/rs/zerolog v1.29.1/go.mod h1:Le6ESbR7hc+DP6Lt1THiV8CQSdkkNrd3R0XbEgp3
 github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
 github.com/ryszard/goskiplist v0.0.0-20150312221310-2dfbae5fcf46 h1:GHRpF1pTW19a8tTFrMLUcfWwyC0pnifVo2ClaLq+hP8=
 github.com/ryszard/goskiplist v0.0.0-20150312221310-2dfbae5fcf46/go.mod h1:uAQ5PCi+MFsC7HjREoAz1BU+Mq60+05gifQSsHSDG/8=
-github.com/santhosh-tekuri/jsonschema/v5 v5.3.0 h1:uIkTLo0AGRc8l7h5l9r+GcYi9qfVPt6lD4/bhmzfiKo=
-github.com/santhosh-tekuri/jsonschema/v5 v5.3.0/go.mod h1:FKdcjfQW6rpZSnxxUvEA5H/cDPdvJ/SZJQLWWXWGrZ0=
 github.com/santhosh-tekuri/jsonschema/v5 v5.3.1 h1:lZUw3E0/J3roVtGQ+SCrUrg3ON6NgVqpn3+iol9aGu4=
 github.com/santhosh-tekuri/jsonschema/v5 v5.3.1/go.mod h1:uToXkOrWAZ6/Oc07xWQrPOhJotwFIyu2bBVN41fcDUY=
 github.com/shabbyrobe/gocovmerge v0.0.0-20190829150210-3e036491d500 h1:WnNuhiq+FOY3jNj6JXFT+eLN3CQ/oPIsDPRanvwsmbI=
@@ -433,7 +426,6 @@ github.com/shopspring/decimal v1.2.0/go.mod h1:DKyhrW/HYNuLGql+MJL6WCR6knT2jwCFR
 github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
 github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
 github.com/spf13/afero v1.2.1/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk=
-github.com/spf13/afero v1.3.3/go.mod h1:5KUK8ByomD5Ti5Artl0RtHeI5pTF7MIDuXL3yY520V4=
 github.com/spf13/cast v1.3.1/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
 github.com/spf13/cast v1.5.0 h1:rj3WzYc11XZaIZMPKmwP96zkFEnnAmV8s6XbB2aY32w=
 github.com/spf13/cast v1.5.0/go.mod h1:SpXXQ5YoyJw6s3/6cMTQuxvgRl3PCJiyaX9p6b155UU=
@@ -587,7 +579,6 @@ golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHl
 golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=
 golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
 golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
-golang.org/x/lint v0.0.0-20210508222113-6edffad5e616/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
 golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
 golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
 golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=

internal/handler/envoyextauth/grpcv3/request_context_test.go

@@ -26,7 +26,7 @@ func TestNewRequestContext(t *testing.T) {
 		Method:   http.MethodPatch,
 		Scheme:   "https",
 		Host:     "foo.bar:8080",
-		Path:     "/test",
+		Path:     "/test/baz",
 		Query:    "bar=moo",
 		Fragment: "foobar",
 		Body:     "content=heimdall",

internal/rules/mechanisms/authorizers/cel_authorizer_test.go

@@ -271,10 +271,12 @@ expressions:
   - expression: Request.URL.Host == 'localhost'
   - expression: Request.URL.Path == '/test'
   - expression: size(Request.URL.Query()) == 2
+  - expression: Request.URL.Query().foo == ["bar"]
   - expression: Request.Header('X-Custom-Header') == "foobar"
   - expression: Request.ClientIP.exists_one(v, v == '127.0.0.1')
   - expression: Request.Cookie("FooCookie") == "barfoo"
   - expression: Request.URL.String() == "http://localhost/test?foo=bar&baz=zab"
+  - expression: Request.URL.Path.split("/").last() == "test"
 `),
 			configureContextAndSubject: func(t *testing.T, ctx *mocks.ContextMock, sub *subject.Subject) {
 				t.Helper()

internal/rules/mechanisms/authorizers/remote_authorizer_test.go

@@ -151,8 +151,8 @@ expressions:
 			id: "authz",
 			config: []byte(`
 endpoint:
-  url: http://foo.bar
-payload: "{{ .Subject.ID }}"
+  url: http://foo.bar/test
+payload: "{{ .Subject.ID }}: {{ splitList \"/\" .Request.URL.Path | atIndex -1 }}"
 expressions:
   - expression: "Payload.foo == 'bar'"
 forward_response_headers_to_upstream:
@@ -165,15 +165,21 @@ values:
 			assert: func(t *testing.T, err error, auth *remoteAuthorizer) {
 				t.Helper()
 
+				require.NoError(t, err)
+
 				ctx := heimdallmocks.NewContextMock(t)
 				ctx.EXPECT().AppContext().Return(context.Background()).Maybe()
 
-				require.NoError(t, err)
+				rfunc := heimdallmocks.NewRequestFunctionsMock(t)
 
 				require.NotNil(t, auth)
 				require.NotNil(t, auth.payload)
 				val, err := auth.payload.Render(map[string]any{
 					"Subject": &subject.Subject{ID: "bar"},
+					"Request": &heimdall.Request{
+						RequestFunctions: rfunc,
+						URL:              &url.URL{Scheme: "http", Host: "foo.bar", Path: "/foo/bar"},
+					},
 				})
 				require.NoError(t, err)
 				require.NotEmpty(t, auth.expressions)
@@ -182,7 +188,7 @@ values:
 				})
 				assert.NoError(t, err)
 				assert.True(t, ok)
-				assert.Equal(t, "bar", val)
+				assert.Equal(t, "bar: bar", val)
 				assert.Len(t, auth.headersForUpstream, 2)
 				assert.Contains(t, auth.headersForUpstream, "Foo")
 				assert.Contains(t, auth.headersForUpstream, "Bar")

internal/rules/mechanisms/cellib/library.go

@@ -17,84 +17,37 @@
 package cellib
 
 import (
-	"net/url"
 	"reflect"
 
 	"github.com/google/cel-go/cel"
-	"github.com/google/cel-go/common/types"
-	"github.com/google/cel-go/common/types/ref"
-	"github.com/google/cel-go/common/types/traits"
 	"github.com/google/cel-go/ext"
 
-	"github.com/dadrus/heimdall/internal/heimdall"
 	"github.com/dadrus/heimdall/internal/rules/mechanisms/subject"
 )
 
-var (
-	//nolint:gochecknoglobals
-	requestType = cel.ObjectType(reflect.TypeOf(heimdall.Request{}).String(), traits.ReceiverType)
-	//nolint:gochecknoglobals
-	urlType = cel.ObjectType(reflect.TypeOf(url.URL{}).String(), traits.ReceiverType)
-)
-
 type heimdallLibrary struct{}
 
 func (heimdallLibrary) LibraryName() string {
-	return "dadrus.heimdall"
+	return "dadrus.heimdall.main"
 }
 
 func (heimdallLibrary) CompileOptions() []cel.EnvOption {
 	return []cel.EnvOption{
 		cel.DefaultUTCTimeZone(true),
-		ext.NativeTypes(
-			reflect.TypeOf(&subject.Subject{}),
-			reflect.TypeOf(&heimdall.Request{}),
-			reflect.TypeOf(&url.URL{})),
+		cel.StdLib(),
+		ext.Lists(),
+		ext.Encoders(),
+		ext.Math(),
+		ext.Sets(),
+		ext.Strings(),
+		Lists(),
+		Strings(),
+		Urls(),
+		Requests(),
+		ext.NativeTypes(reflect.TypeOf(&subject.Subject{})),
 		cel.Variable("Payload", cel.DynType),
 		cel.Variable("Subject", cel.DynType),
-		cel.Variable("Request", cel.ObjectType(requestType.TypeName())),
-		cel.Function("Header",
-			cel.MemberOverload("Header",
-				[]*cel.Type{cel.ObjectType(requestType.TypeName()), cel.StringType}, cel.StringType,
-				cel.BinaryBinding(func(lhs ref.Val, rhs ref.Val) ref.Val {
-					// nolint: forcetypeassert
-					req := lhs.Value().(*heimdall.Request)
-
-					// nolint: forcetypeassert
-					return types.String(req.Header(rhs.Value().(string)))
-				}),
-			),
-		),
-		cel.Function("Cookie",
-			cel.MemberOverload("Cookie",
-				[]*cel.Type{cel.ObjectType(requestType.TypeName()), cel.StringType}, cel.StringType,
-				cel.BinaryBinding(func(lhs ref.Val, rhs ref.Val) ref.Val {
-					// nolint: forcetypeassert
-					req := lhs.Value().(*heimdall.Request)
-
-					// nolint: forcetypeassert
-					return types.String(req.Cookie(rhs.Value().(string)))
-				}),
-			),
-		),
-		cel.Function("String",
-			cel.MemberOverload("String",
-				[]*cel.Type{cel.ObjectType(urlType.TypeName())}, cel.StringType,
-				cel.UnaryBinding(func(value ref.Val) ref.Val {
-					// nolint: forcetypeassert
-					return types.String(value.Value().(*url.URL).String())
-				}),
-			),
-		),
-		cel.Function("Query",
-			cel.MemberOverload("Query",
-				[]*cel.Type{cel.ObjectType(urlType.TypeName())}, cel.DynType,
-				cel.UnaryBinding(func(value ref.Val) ref.Val {
-					// nolint: forcetypeassert
-					return types.NewDynamicMap(types.DefaultTypeAdapter, value.Value().(*url.URL).Query())
-				}),
-			),
-		),
+		cel.Variable("Request", cel.DynType),
 	}
 }