Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: remove lorry and built-in action handler #7964

Merged
merged 39 commits into from
Aug 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
d34b03f
update makefile and dockerfile, rename kb_agent to kbagent
leon-inf Jul 16, 2024
eb1d3e8
refine kb-agent
leon-inf Jul 16, 2024
97fbba4
chore: integrate kb-agent with controller
leon-inf Jul 16, 2024
303a6a0
Merge branch 'main' into support/integrate-with-kbagent
leon-inf Jul 16, 2024
e232282
update
leon-inf Jul 17, 2024
bba66bb
update
leon-inf Jul 17, 2024
415b793
update
leon-inf Jul 17, 2024
a47a6e2
Merge branch 'main' into support/integrate-with-kbagent
leon-inf Jul 18, 2024
f2e400d
update
leon-inf Jul 18, 2024
5aea7a1
Merge branch 'main' into support/integrate-with-kbagent
leon-inf Jul 19, 2024
9437fe0
update
leon-inf Jul 22, 2024
b0843e9
Merge branch 'main' into support/integrate-with-kbagent
leon-inf Jul 24, 2024
04fae71
Merge branch 'main' into support/integrate-with-kbagent
leon-inf Jul 29, 2024
581f876
chore: add envs for lifecycle action
Ursasi Aug 6, 2024
56f7cbc
chore: add timeout error and fix runCommandNonBlocking
Ursasi Aug 7, 2024
5d337bd
chore: add dynamic container port setting
Ursasi Aug 8, 2024
c4fd3af
Merge branch 'main' into support/integrate-with-kbagent
leon-inf Aug 9, 2024
d86c156
chore: add fail and internal error
Ursasi Aug 9, 2024
d8dedae
update
leon-inf Aug 9, 2024
4911053
Merge remote-tracking branch 'origin/support/integrate-with-kbagent' …
leon-inf Aug 9, 2024
e277f31
update
leon-inf Aug 12, 2024
6c97d91
Merge branch 'main' into support/integrate-with-kbagent
leon-inf Aug 12, 2024
fcf2a1d
don't build lorry binary
leon-inf Aug 12, 2024
37954c8
share target container volume mounts
leon-inf Aug 12, 2024
b0aecce
update
leon-inf Aug 12, 2024
07a7f3c
Merge branch 'main' into support/integrate-with-kbagent
leon-inf Aug 13, 2024
bba84eb
remove built-in action handler
leon-inf Aug 13, 2024
9580c64
remove lorry
leon-inf Aug 13, 2024
65c81db
host-network for kb-agent
leon-inf Aug 13, 2024
6eb1b66
Merge branch 'support/integrate-with-kbagent' into support/remove-bui…
leon-inf Aug 13, 2024
b41211a
remove lorry
leon-inf Aug 13, 2024
8868a9c
remove useless action parameters
leon-inf Aug 13, 2024
f95ac3e
update
leon-inf Aug 13, 2024
68ddb3a
Merge branch 'main' into support/remove-builtin-action-handler
leon-inf Aug 14, 2024
588055e
Merge branch 'main' into support/integrate-with-kbagent
leon-inf Aug 14, 2024
e769519
fix check
leon-inf Aug 14, 2024
b547ac4
Merge branch 'support/integrate-with-kbagent' into support/remove-bui…
leon-inf Aug 14, 2024
8767408
Merge branch 'main' into support/remove-builtin-action-handler
leon-inf Aug 14, 2024
92bb075
fix test
leon-inf Aug 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .github/utils/utils.sh
Original file line number Diff line number Diff line change
Expand Up @@ -514,9 +514,6 @@ get_trigger_mode() {
apis/*)
add_trigger_mode "[apis][test]"
;;
pkg/lorry/ctl/*)
add_trigger_mode "[lorry][test]"
;;
*)
add_trigger_mode "[test]"
;;
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/cicd-push.yml
Original file line number Diff line number Diff line change
Expand Up @@ -270,7 +270,7 @@ jobs:
apis-doc:
needs: [ trigger-mode ]
runs-on: ubuntu-latest
if: ${{ contains(needs.trigger-mode.outputs.trigger-mode, '[apis]') || contains(needs.trigger-mode.outputs.trigger-mode, '[lorry]') }}
if: ${{ contains(needs.trigger-mode.outputs.trigger-mode, '[apis]') }}
steps:
- uses: actions/checkout@v4
- name: install lib
Expand Down
5 changes: 0 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -255,11 +255,6 @@ endif
goimports: goimportstool ## Run goimports against code.
$(GOIMPORTS) -local github.com/apecloud/kubeblocks -w $$(git ls-files|grep "\.go$$" | grep -v $(GENERATED_CLIENT_PKG) | grep -v $(GENERATED_DEEP_COPY_FILE))


.PHONY: lorryctl-doc
lorryctl-doc: generate test-go-generate ## generate CLI command reference manual.
$(GO) run ./hack/docgen/lorryctl/main.go ./docs/user_docs/lorryctl

.PHONY: api-doc
api-doc: ## generate API reference manual.
$(GO) run ./hack/docgen/api/main.go -api-dir github.com/apecloud/kubeblocks/apis -config ./hack/docgen/api/gen-api-doc-config.json -template-dir ./hack/docgen/api/template -out-dir ./docs/developer_docs/api-reference/
Expand Down
38 changes: 0 additions & 38 deletions apis/apps/v1alpha1/clusterdefinition_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -224,44 +224,6 @@ type LogConfig struct {
FilePathPattern string `json:"filePathPattern"`
}

// TODO(v1.0): remove this after lorry

// VolumeProtectionSpec is deprecated since v0.9, replaced with ComponentVolume.HighWatermark.
type VolumeProtectionSpec struct {
// The high watermark threshold for volume space usage.
// If there is any specified volumes who's space usage is over the threshold, the pre-defined "LOCK" action
// will be triggered to degrade the service to protect volume from space exhaustion, such as to set the instance
// as read-only. And after that, if all volumes' space usage drops under the threshold later, the pre-defined
// "UNLOCK" action will be performed to recover the service normally.
//
// +kubebuilder:validation:Maximum=100
// +kubebuilder:validation:Minimum=0
// +kubebuilder:default=90
// +optional
HighWatermark int `json:"highWatermark,omitempty"`

// The Volumes to be protected.
//
// +optional
Volumes []ProtectedVolume `json:"volumes,omitempty"`
}

// ProtectedVolume is deprecated since v0.9, replaced with ComponentVolume.HighWatermark.
type ProtectedVolume struct {
// The Name of the volume to protect.
//
// +optional
Name string `json:"name,omitempty"`

// Defines the high watermark threshold for the volume, it will override the component level threshold.
// If the value is invalid, it will be ignored and the component level threshold will be used.
//
// +kubebuilder:validation:Maximum=100
// +kubebuilder:validation:Minimum=0
// +optional
HighWatermark *int `json:"highWatermark,omitempty"`
}

// ServiceRefDeclaration represents a reference to a service that can be either provided by a KubeBlocks Cluster
// or an external service.
// It acts as a placeholder for the actual service reference, which is determined later when a Cluster is created.
Expand Down
121 changes: 11 additions & 110 deletions apis/apps/v1alpha1/componentdefinition_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -697,8 +697,6 @@ const (
)

// ExecAction describes an Action that executes a command inside a container.
// Which may run as a K8s job or be executed inside the Lorry sidecar container, depending on the implementation.
// Future implementations will standardize execution within Lorry.
type ExecAction struct {
// Specifies the container image to be used for running the Action.
//
Expand Down Expand Up @@ -826,8 +824,6 @@ const (
// Actions can be executed in different ways:
//
// - ExecAction: Executes a command inside a container.
// which may run as a K8s job or be executed inside the Lorry sidecar container, depending on the implementation.
// Future implementations will standardize execution within Lorry.
// A set of predefined environment variables are available and can be leveraged within the `exec.command`
// to access context information such as details about pods, components, the overall cluster state,
// or database connection credentials.
Expand Down Expand Up @@ -898,11 +894,6 @@ type Action struct {
type Probe struct {
Action `json:",inline"`

// TODO: remove this later.
//
// +optional
BuiltinHandler *BuiltinActionHandlerType `json:"builtinHandler,omitempty"`

// Specifies the number of seconds to wait after the container has started before the RoleProbe
// begins to detect the container's role.
//
Expand All @@ -928,96 +919,6 @@ type Probe struct {
FailureThreshold int32 `json:"failureThreshold,omitempty"`
}

// BuiltinActionHandlerType defines build-in action handlers provided by Lorry, including:
//
// - `mysql`
// - `wesql`
// - `oceanbase`
// - `redis`
// - `mongodb`
// - `etcd`
// - `postgresql`
// - `official-postgresql`
// - `apecloud-postgresql`
// - `polardbx`
// - `custom`
// - `unknown`
type BuiltinActionHandlerType string

const (
MySQLBuiltinActionHandler BuiltinActionHandlerType = "mysql"
WeSQLBuiltinActionHandler BuiltinActionHandlerType = "wesql"
OceanbaseBuiltinActionHandler BuiltinActionHandlerType = "oceanbase"
RedisBuiltinActionHandler BuiltinActionHandlerType = "redis"
MongoDBBuiltinActionHandler BuiltinActionHandlerType = "mongodb"
ETCDBuiltinActionHandler BuiltinActionHandlerType = "etcd"
PostgresqlBuiltinActionHandler BuiltinActionHandlerType = "postgresql"
OfficialPostgresqlBuiltinActionHandler BuiltinActionHandlerType = "official-postgresql"
ApeCloudPostgresqlBuiltinActionHandler BuiltinActionHandlerType = "apecloud-postgresql"
PolarDBXBuiltinActionHandler BuiltinActionHandlerType = "polardbx"
CustomActionHandler BuiltinActionHandlerType = "custom"
UnknownBuiltinActionHandler BuiltinActionHandlerType = "unknown"
)

// LifecycleActionHandler describes the implementation of a specific lifecycle action.
//
// Each action is deemed successful if it returns an exit code of 0 for command executions,
// or an HTTP 200 status for HTTP(s) actions.
// Any other exit code or HTTP status is considered an indication of failure.
type LifecycleActionHandler struct {
// Specifies the name of the predefined action handler to be invoked for lifecycle actions.
//
// Lorry, as a sidecar agent co-located with the database container in the same Pod,
// includes a suite of built-in action implementations that are tailored to different database engines.
// These are known as "builtin" handlers, includes: `mysql`, `redis`, `mongodb`, `etcd`,
// `postgresql`, `official-postgresql`, `apecloud-postgresql`, `wesql`, `oceanbase`, `polardbx`.
//
// If the `builtinHandler` field is specified, it instructs Lorry to utilize its internal built-in action handler
// to execute the specified lifecycle actions.
//
// The `builtinHandler` field is of type `BuiltinActionHandlerType`,
// which represents the name of the built-in handler.
// The `builtinHandler` specified within the same `ComponentLifecycleActions` should be consistent across all
// actions.
// This means that if you specify a built-in handler for one action, you should use the same handler
// for all other actions throughout the entire `ComponentLifecycleActions` collection.
//
// If you need to define lifecycle actions for database engines not covered by the existing built-in support,
// or when the pre-existing built-in handlers do not meet your specific needs,
// you can use the `customHandler` field to define your own action implementation.
//
// Deprecation Notice:
//
// - In the future, the `builtinHandler` field will be deprecated in favor of using the `customHandler` field
// for configuring all lifecycle actions.
// - Instead of using a name to indicate the built-in action implementations in Lorry,
// the recommended approach will be to explicitly invoke the desired action implementation through
// a gRPC interface exposed by the sidecar agent.
// - Developers will have the flexibility to either use the built-in action implementations provided by Lorry
// or develop their own sidecar agent to implement custom actions and expose them via gRPC interfaces.
// - This change will allow for greater customization and extensibility of lifecycle actions,
// as developers can create their own "builtin" implementations tailored to their specific requirements.
//
// +optional
BuiltinHandler *BuiltinActionHandlerType `json:"builtinHandler,omitempty"`

// Specifies a user-defined hook or procedure that is called to perform the specific lifecycle action.
// It offers a flexible and expandable approach for customizing the behavior of a Component by leveraging
// tailored actions.
//
// An Action can be implemented as either an ExecAction or an HTTPAction, with future versions planning
// to support GRPCAction,
// thereby accommodating unique logic for different database systems within the Action's framework.
//
// In future iterations, all built-in handlers are expected to transition to GRPCAction.
// This change means that Lorry or other sidecar agents will expose the implementation of actions
// through a GRPC interface for external invocation.
// Then the controller will interact with these actions via GRPCAction calls.
//
// +optional
CustomHandler *Action `json:"customHandler,omitempty"`
}

// ComponentLifecycleActions defines a collection of Actions for customizing the behavior of a Component.
type ComponentLifecycleActions struct {
// Specifies the hook to be executed after a component's creation.
Expand Down Expand Up @@ -1055,7 +956,7 @@ type ComponentLifecycleActions struct {
// Note: This field is immutable once it has been set.
//
// +optional
PostProvision *LifecycleActionHandler `json:"postProvision,omitempty"`
PostProvision *Action `json:"postProvision,omitempty"`

// Specifies the hook to be executed prior to terminating a component.
//
Expand Down Expand Up @@ -1098,11 +999,11 @@ type ComponentLifecycleActions struct {
// Note: This field is immutable once it has been set.
//
// +optional
PreTerminate *LifecycleActionHandler `json:"preTerminate,omitempty"`
PreTerminate *Action `json:"preTerminate,omitempty"`

// Defines the procedure which is invoked regularly to assess the role of replicas.
//
// This action is periodically triggered by Lorry at the specified interval to determine the role of each replica.
// This action is periodically triggered at the specified interval to determine the role of each replica.
// Upon successful execution, the action's output designates the role of the replica,
// which should match one of the predefined role names within `componentDefinition.spec.roles`.
// The output is then compared with the previous successful execution result.
Expand Down Expand Up @@ -1176,7 +1077,7 @@ type ComponentLifecycleActions struct {
// Note: This field is immutable once it has been set.
//
// +optional
MemberJoin *LifecycleActionHandler `json:"memberJoin,omitempty"`
MemberJoin *Action `json:"memberJoin,omitempty"`

// Defines the procedure to remove a replica from the replication group.
//
Expand Down Expand Up @@ -1209,7 +1110,7 @@ type ComponentLifecycleActions struct {
// Note: This field is immutable once it has been set.
//
// +optional
MemberLeave *LifecycleActionHandler `json:"memberLeave,omitempty"`
MemberLeave *Action `json:"memberLeave,omitempty"`

// Defines the procedure to switch a replica into the read-only state.
//
Expand All @@ -1226,7 +1127,7 @@ type ComponentLifecycleActions struct {
// Note: This field is immutable once it has been set.
//
// +optional
Readonly *LifecycleActionHandler `json:"readonly,omitempty"`
Readonly *Action `json:"readonly,omitempty"`

// Defines the procedure to transition a replica from the read-only state back to the read-write state.
//
Expand All @@ -1245,7 +1146,7 @@ type ComponentLifecycleActions struct {
// Note: This field is immutable once it has been set.
//
// +optional
Readwrite *LifecycleActionHandler `json:"readwrite,omitempty"`
Readwrite *Action `json:"readwrite,omitempty"`

// Defines the procedure for exporting the data from a replica.
//
Expand All @@ -1264,7 +1165,7 @@ type ComponentLifecycleActions struct {
// Note: This field is immutable once it has been set.
//
// +optional
DataDump *LifecycleActionHandler `json:"dataDump,omitempty"`
DataDump *Action `json:"dataDump,omitempty"`

// Defines the procedure for importing data into a replica.
//
Expand All @@ -1282,7 +1183,7 @@ type ComponentLifecycleActions struct {
// Note: This field is immutable once it has been set.
//
// +optional
DataLoad *LifecycleActionHandler `json:"dataLoad,omitempty"`
DataLoad *Action `json:"dataLoad,omitempty"`

// Defines the procedure that update a replica with new configuration.
//
Expand All @@ -1291,7 +1192,7 @@ type ComponentLifecycleActions struct {
// This Action is reserved for future versions.
//
// +optional
Reconfigure *LifecycleActionHandler `json:"reconfigure,omitempty"`
Reconfigure *Action `json:"reconfigure,omitempty"`

// Defines the procedure to generate a new database account.
//
Expand All @@ -1302,5 +1203,5 @@ type ComponentLifecycleActions struct {
// Note: This field is immutable once it has been set.
//
// +optional
AccountProvision *LifecycleActionHandler `json:"accountProvision,omitempty"`
AccountProvision *Action `json:"accountProvision,omitempty"`
}
Loading
Loading