feat: add disabled validator and API config placeholders for streams #6903
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
A JIRA Issue ID is missing from your branch name, PR title and PR description! 🦄 Your branch: feature/streaming-enable-all Your PR title: feat: add disabled validator and API config placeholders for streams Your PR description: - Add EnableAll config option to disable streams validator, e.g. enable all components - Add support for using API configuration in stream configs, $api_config.key Related IssueMotivation and ContextHow This Has Been TestedScreenshots (if appropriate)Types of changes
Checklist
If this is your first time contributing to this repository - welcome! Please refer to jira-lint to get started. Without the JIRA Issue ID in your branch name you would lose out on automatic updates to JIRA via SCM; some GitHub status checks might fail. Valid sample branch names:‣ feature/shiny-new-feature--mojo-10' |
- Add EnableAll config option to disable streams validator - Add support for placeholders in stream configs - Add tests for API config placeholders and disabled validator
buger
force-pushed
the
feature/streaming-enable-all
branch
from
a415e7b to
52739ae
Compare
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Explore these optional code suggestions:
|
|
API Changes --- prev.txt 2025-02-24 14:59:44.680226934 +0000
+++ current.txt 2025-02-24 14:59:39.546174652 +0000
@@ -884,18 +884,6 @@
Values []string
}
-type LogEventHandlerConf struct {
- // Disabled indicates whether the handler is inactive.
- Disabled bool `bson:"disabled" json:"disabled"`
- // Prefix specifies the prefix used for log events.
- Prefix string `bson:"prefix" json:"prefix"`
-}
- LogEventHandlerConf represents the configuration for a log event handler.
-
-func (l *LogEventHandlerConf) Scan(in any) error
- Scan extracts data from the input into the LogEventHandlerConf struct by
- performing type conversion.
-
type MethodTransformMeta struct {
Disabled bool `bson:"disabled" json:"disabled"`
Path string `bson:"path" json:"path"`
@@ -1971,7 +1959,6 @@
const (
WebhookKind = event.WebhookKind
JSVMKind = event.JSVMKind
- LogKind = event.LogKind
)
WebhookKind is an alias maintained to be used in imports.
@@ -2201,8 +2188,6 @@
Path string `bson:"path" json:"path"`
// RawBodyOnly if set to true, do not fill body in request or response object.
RawBodyOnly bool `bson:"rawBodyOnly,omitempty" json:"rawBodyOnly,omitempty"`
- // RequireSession passes down the session information for plugins after authentication.
- RequireSession bool `bson:"requireSession,omitempty" json:"requireSession,omitempty"`
// IDExtractor configures ID extractor with coprocess custom authentication.
IDExtractor *IDExtractor `bson:"idExtractor,omitempty" json:"idExtractor,omitempty"`
}
@@ -2695,9 +2680,6 @@
// JSVMEvent holds information about JavaScript VM events.
JSVMEvent JSVMEvent `bson:"-" json:"-"`
-
- // LogEvent represents the configuration for logging events tied to an event handler.
- LogEvent LogEvent `bson:"-" json:"-"`
}
EventHandler holds information about individual event to be configured on
the API.
@@ -2706,10 +2688,6 @@
GetJSVMEventHandlerConf generates the JavaScript VM event handler
configuration using the current EventHandler instance.
-func (e *EventHandler) GetLogEventHandlerConf() apidef.LogEventHandlerConf
- GetLogEventHandlerConf creates and returns a LogEventHandlerConf based on
- the current EventHandler configuration.
-
func (e *EventHandler) GetWebhookConf() apidef.WebHookHandlerConf
GetWebhookConf converts EventHandler.WebhookEvent apidef.WebHookHandlerConf.
@@ -2854,18 +2832,6 @@
// RequestSizeLimit contains the configuration related to limiting the global request size.
RequestSizeLimit *GlobalRequestSizeLimit `bson:"requestSizeLimit,omitempty" json:"requestSizeLimit,omitempty"`
-
- // IgnoreCase contains the configuration to treat routes as case-insensitive.
- IgnoreCase *IgnoreCase `bson:"ignoreCase,omitempty" json:"ignoreCase,omitempty"`
-
- // SkipRateLimit determines whether the rate-limiting middleware logic should be skipped. Classic: `disable_rate_limit`.
- SkipRateLimit bool `bson:"skipRateLimit,omitempty" json:"skipRateLimit,omitempty"`
-
- // SkipQuota determines whether quota enforcement should be bypassed. Classic: `disable_quota`.
- SkipQuota bool `bson:"skipQuota,omitempty" json:"skipQuota,omitempty"`
-
- // SkipQuotaReset indicates if quota limits should not be reset when creating or updating quotas for the API. Classic: `dont_set_quota_on_create`.
- SkipQuotaReset bool `bson:"skipQuotaReset,omitempty" json:"skipQuotaReset,omitempty"`
}
Global contains configuration that affects the whole API (all endpoints).
@@ -2876,9 +2842,9 @@
Fill fills *Global from apidef.APIDefinition.
func (g *Global) MarshalJSON() ([]byte, error)
- MarshalJSON is a custom JSON marshaller for the Global struct. It is
+ MarshalJSON is a custom JSON marshaler for the Global struct. It is
implemented to facilitate a smooth migration from deprecated fields that
- were previously used to represent the same data. This custom marshaller
+ were previously used to represent the same data. This custom marshaler
ensures backwards compatibility and proper handling of the deprecated fields
during the migration process.
@@ -3010,19 +2976,6 @@
func (i *IPAccessControl) Fill(api apidef.APIDefinition)
Fill fills *IPAccessControl from apidef.APIDefinition.
-type IgnoreCase struct {
- // Enabled activates case insensitive route matching.
- Enabled bool `json:"enabled" bson:"enabled"`
-}
- IgnoreCase will make route matching be case insensitive. This accepts
- request to `/AAA` or `/aaa` if set to true.
-
-func (p *IgnoreCase) ExtractTo(api *apidef.APIDefinition)
- ExtractTo extracts *IgnoreCase into *apidef.APIDefinition.
-
-func (p *IgnoreCase) Fill(api apidef.APIDefinition)
- Fill fills *IgnoreCase from apidef.APIDefinition.
-
type Info struct {
// ID is the unique identifier of the API within Tyk.
// Tyk classic API definition: `api_id`
@@ -3264,13 +3217,6 @@
LoadBalancingTarget represents a single upstream target for load balancing
with a URL and an associated weight.
-type LogEvent struct {
- // LogPrefix defines the prefix used for log messages in the logging event.
- LogPrefix string `json:"logPrefix" bson:"logPrefix"`
-}
- LogEvent represents the configuration for logging events within an event
- handler.
-
type Middleware struct {
// Global contains configuration for middleware that affects the whole API (all endpoints).
Global *Global `bson:"global,omitempty" json:"global,omitempty"`
@@ -3341,13 +3287,6 @@
OAS holds the upstream OAS definition as well as adds functionality like
custom JSON marshalling.
-func FillOASFromClassicAPIDefinition(api *apidef.APIDefinition, oas *OAS) (*OAS, error)
-
-func NewOAS() *OAS
- NewOAS returns an allocated *OAS.
-
-func NewOASFromClassicAPIDefinition(api *apidef.APIDefinition) (*OAS, error)
-
func (s *OAS) AddServers(apiURLs ...string)
AddServers adds a server into the servers definition if not already present.
@@ -3758,24 +3697,6 @@
func (p *PreserveHostHeader) Fill(api apidef.APIDefinition)
Fill fills *PreserveHostHeader from apidef.APIDefinition.
-type PreserveTrailingSlash struct {
- // Enabled activates preserving the trailing slash when routing requests.
- Enabled bool `json:"enabled" bson:"enabled"` // required
-}
- PreserveTrailingSlash holds the configuration for preserving the trailing
- slash when routed to upstream services.
-
- The default behaviour of Tyk is to strip any trailing slash (/) from
- the target URL when proxying the request upstream. In some use cases the
- upstream might expect the trailing slash - or might consider /users/ to be a
- different endpoint from /users (for example).
-
-func (p *PreserveTrailingSlash) ExtractTo(api *apidef.APIDefinition)
- ExtractTo extracts *PreserveTrailingSlash into *apidef.APIDefinition.
-
-func (p *PreserveTrailingSlash) Fill(api apidef.APIDefinition)
- Fill fills *PreserveTrailingSlash from apidef.APIDefinition.
-
type Provider struct {
// Issuer contains a validation value for the issuer claim, usually a domain name e.g. `accounts.google.com` or similar.
Issuer string `bson:"issuer,omitempty" json:"issuer,omitempty"`
@@ -4489,10 +4410,6 @@
// PreserveHostHeader contains the configuration for preserving the host header.
PreserveHostHeader *PreserveHostHeader `bson:"preserveHostHeader,omitempty" json:"preserveHostHeader,omitempty"`
-
- // PreserveTrailingSlash contains the configuration for preserving the host header.
- PreserveTrailingSlash *PreserveTrailingSlash `bson:"preserveTrailingSlash,omitempty" json:"preserveTrailingSlash,omitempty"`
-
// TLSTransport contains the configuration for TLS transport settings.
// Tyk classic API definition: `proxy.transport`
TLSTransport *TLSTransport `bson:"tlsTransport,omitempty" json:"tlsTransport,omitempty"`
@@ -4829,6 +4746,11 @@
ValidateOASObjectWithBentoConfigValidator validates a Tyk Streams document
against a particular OAS version and takes an optional ConfigValidator
+func ValidateOASObjectWithConfig(documentBody []byte, oasVersion string, disableValidator bool) error
+ ValidateOASObjectWithConfig validates a Tyk Streams document against a
+ particular OAS version, using the provided configuration to determine if
+ validation should be disabled.
+
func ValidateOASTemplate(documentBody []byte, oasVersion string) error
ValidateOASTemplate checks a Tyk Streams OAS API template for necessary
fields, acknowledging that some standard Tyk OAS API fields are optional in
@@ -4845,6 +4767,7 @@
const (
DefaultBentoConfigSchemaName string = "bento-v1.2.0-supported-schema.json"
DefaultValidator ValidatorKind = "default-validator"
+ DisabledValidator ValidatorKind = "disabled-validator"
)
TYPES
@@ -4861,6 +4784,15 @@
func (v *DefaultConfigValidator) Validate(document []byte) error
+type DisabledConfigValidator struct{}
+ DisabledConfigValidator is a validator that skips all validation
+
+func NewDisabledConfigValidator() *DisabledConfigValidator
+ NewDisabledConfigValidator creates a new validator that skips all validation
+
+func (v *DisabledConfigValidator) Validate(document []byte) error
+ Validate always returns nil, effectively disabling validation
+
type ValidatorKind string
# Package: ./certs
@@ -6220,11 +6152,6 @@
// none was found, it's the path to the default config file that
// was written.
OriginalPath string `json:"-"`
- // EdgeOriginalAPIKeyPath is the original path to the API key in the configuration file.
- // This is only used when the gateway is running as an edge gateway (slave mode) in an MDCB setup
- // to modify the external KV store in case of API Key Reset.
- // This is set automatically in afterConfSetup()
- EdgeOriginalAPIKeyPath string `json:"-"`
}
Private contains configurations which are private, adding it to be part of
config without exposing to customers.
@@ -6448,6 +6375,8 @@
type StreamingConfig struct {
Enabled bool `json:"enabled"`
AllowUnsafe []string `json:"allow_unsafe"`
+ // EnableAll enables all Bento plugins (except unsafe ones) by disabling the streams validator
+ EnableAll bool `json:"enable_all"`
}
StreamingConfig is for configuring tyk streaming
@@ -9918,7 +9847,6 @@
OAuthPurgeLapsedTokens NotificationCommand = "OAuthPurgeLapsedTokens"
// NoticeDeleteAPICache is the command with which event is emitted from dashboard to invalidate cache for an API.
NoticeDeleteAPICache NotificationCommand = "DeleteAPICache"
- NoticeUserKeyReset NotificationCommand = "UserKeyReset"
)
func (n NotificationCommand) String() string
@@ -10234,7 +10162,7 @@
those keys are considered changes in the keyspace in the management layer,
they could be: regular keys (hashed, unhashed), revoke oauth client, revoke
single oauth token, certificates (added, removed), oauth client (added,
- updated, removed), user key events (reset)
+ updated, removed)
func (r *RPCStorageHandler) Publish(channel, message string) error
@@ -12436,14 +12364,10 @@
func (c *Consul) Get(key string) (string, error)
-func (c *Consul) Put(key string, value string) error
-
func (c *Consul) Store() *consulapi.KV
type Store interface {
Get(key string) (string, error)
- // Put writes a value to the KV store
- Put(key string, value string) error
}
Store is a standard interface to a KV backend
@@ -12462,8 +12386,6 @@
func (v *Vault) Get(key string) (string, error)
-func (v *Vault) Put(key string, value string) error
-
# Package: ./storage/mock
package mock // import "github.com/TykTechnologies/tyk/storage/mock" |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
User description
Related Issue
Motivation and Context
How This Has Been Tested
Screenshots (if appropriate)
Types of changes
Checklist
PR Type
Enhancement, Tests, Configuration changes
Description
Introduced
EnableAllvalidator to bypass stream validation.Added support for
$api_configplaceholders in stream configurations.Enhanced test coverage for new validator and API config placeholders.
Updated dependencies and configuration structures for streaming.
Changes walkthrough 📝
4 files
Added `EnableAll` validator to bypass validation.Integrated `EnableAll` validator into stream validation logic.Integrated `EnableAll` validator into middleware validation.Added support for `$api_config` variable replacement.2 files
Added tests for `EnableAll` validator and API config placeholders.Enhanced streaming tests with new configurations and scenarios.1 files
Added `EnableAll` flag to streaming configuration.2 files
Updated Bento components import for streaming.Updated dependencies for streaming and validation enhancements.1 files