generated from cloudwego/.github
-
Notifications
You must be signed in to change notification settings - Fork 5
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* feat: add zookeeper && server * update go mod * use gofumpt * fix: atomic.Int64 can't be used under go 1.19 * fix: unreasonable names * fix: update the processing of removed keys and empty data * style: add cancelFuncHolder type * style: update code comments * refactor: optimized loop variable * feat: add client * feat: update client * fix: add missing code snippet * feat: add README * style: use gofumpt * chore(doc): optimize readme * docs: update README * feat: add listen goroutine recover * fix: correct error logging format in project --------- Co-authored-by: kinggo <lilong.21@bytedance.com>
- Loading branch information
Showing
23 changed files
with
4,688 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,320 @@ | ||
| # .github | ||
| # config-zookeeper(*This is a community driven project*) | ||
|
|
||
| [中文](https://github.com/kitex-contrib/config-zookeeper/blob/main/README_CN.md) | ||
|
|
||
| **zookeeper** as config center for service governance. | ||
|
|
||
| ## Usage | ||
|
|
||
| ### Basic | ||
|
|
||
| #### Server | ||
|
|
||
| ```go | ||
| package main | ||
|
|
||
| import ( | ||
| "context" | ||
| "log" | ||
|
|
||
| "github.com/cloudwego/kitex-examples/kitex_gen/api" | ||
| "github.com/cloudwego/kitex-examples/kitex_gen/api/echo" | ||
| "github.com/cloudwego/kitex/pkg/klog" | ||
| "github.com/cloudwego/kitex/pkg/rpcinfo" | ||
| "github.com/cloudwego/kitex/server" | ||
| zookeeperServer "github.com/kitex-contrib/config-zookeeper/server" | ||
| "github.com/kitex-contrib/config-zookeeper/zookeeper" | ||
| ) | ||
|
|
||
| var _ api.Echo = &EchoImpl{} | ||
|
|
||
| // EchoImpl implements the last service interface defined in the IDL. | ||
| type EchoImpl struct{} | ||
|
|
||
| // Echo implements the Echo interface. | ||
| func (s *EchoImpl) Echo(ctx context.Context, req *api.Request) (resp *api.Response, err error) { | ||
| klog.Info("echo called") | ||
| return &api.Response{Message: req.Message}, nil | ||
| } | ||
|
|
||
| func main() { | ||
| zookeeperClient, err := zookeeper.NewClient(zookeeper.Options{}) | ||
| if err != nil { | ||
| panic(err) | ||
| } | ||
| serviceName := "ServiceName" // your server-side service name | ||
| svr := echo.NewServer( | ||
| new(EchoImpl), | ||
| server.WithServerBasicInfo(&rpcinfo.EndpointBasicInfo{ServiceName: serviceName}), | ||
| server.WithSuite(zookeeperServer.NewSuite(serviceName, zookeeperClient)), | ||
| ) | ||
| if err := svr.Run(); err != nil { | ||
| log.Println("server stopped with error:", err) | ||
| } else { | ||
| log.Println("server stopped") | ||
| } | ||
| } | ||
|
|
||
| ``` | ||
|
|
||
| #### Client | ||
|
|
||
| ```go | ||
| package main | ||
|
|
||
| import ( | ||
| "context" | ||
| "log" | ||
| "time" | ||
|
|
||
| "github.com/cloudwego/kitex-examples/kitex_gen/api" | ||
| "github.com/cloudwego/kitex-examples/kitex_gen/api/echo" | ||
| "github.com/cloudwego/kitex/client" | ||
| "github.com/cloudwego/kitex/pkg/klog" | ||
| zookeeperclient "github.com/kitex-contrib/config-zookeeper/client" | ||
| "github.com/kitex-contrib/config-zookeeper/utils" | ||
| "github.com/kitex-contrib/config-zookeeper/zookeeper" | ||
| ) | ||
|
|
||
| type configLog struct{} | ||
|
|
||
| func (cl *configLog) Apply(opt *utils.Options) { | ||
| fn := func(k *zookeeper.ConfigParam) { | ||
| klog.Infof("zookeeper config %v", k) | ||
| } | ||
| opt.ZookeeperCustomFunctions = append(opt.ZookeeperCustomFunctions, fn) | ||
| } | ||
|
|
||
| func main() { | ||
| zookeeperClient, err := zookeeper.NewClient(zookeeper.Options{}) | ||
| if err != nil { | ||
| panic(err) | ||
| } | ||
|
|
||
| cl := configLog{} | ||
|
|
||
| serviceName := "ServiceName" // your server-side service name | ||
| clientName := "ClientName" // your client-side service name | ||
| client, err := echo.NewClient( | ||
| serviceName, | ||
| client.WithHostPorts("0.0.0.0:8888"), | ||
| client.WithSuite(zookeeperclient.NewSuite(serviceName, clientName, zookeeperClient, &cl)), | ||
| ) | ||
| if err != nil { | ||
| log.Fatal(err) | ||
| } | ||
| for { | ||
| req := &api.Request{Message: "my request"} | ||
| resp, err := client.Echo(context.Background(), req) | ||
| if err != nil { | ||
| klog.Errorf("take request error: %v", err) | ||
| } else { | ||
| klog.Infof("receive response %v", resp) | ||
| } | ||
| time.Sleep(time.Second * 10) | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### Zookeeper Configuration | ||
|
|
||
| The client is initialized according to the parameters of `Options` and connects to the zookeeper server. After the connection is established, the suite will listen to the corresponding node according to `Path` and `ServerPathFormat` or`ClientPathFormat`to updates its own policy dynamically. See the `Options` variables below for specific parameters. | ||
|
|
||
| #### CustomFunction | ||
|
|
||
| Provide the mechanism to custom the zookeeper parameter `Path`. | ||
|
|
||
| ```go | ||
| type ConfigParam struct { | ||
| Prefix string | ||
| Path string | ||
| } | ||
| ``` | ||
|
|
||
| The final Path is `param.Prefix + "/" + param.Path` | ||
|
|
||
| #### Options Variable | ||
|
|
||
| | Variable Name | Default Value | Introduction | | ||
| | ---------------- | ----------------------------------------------------------- | ------------------------------------------------------------ | | ||
| | Servers | 127.0.0.1:2181 | Zookeeper server nodes | | ||
| | Prefix | /KitexConfig | The prefix of Zookeeper | | ||
| | ClientPathFormat | {{.ClientServiceName}}/{{.ServerServiceName}}/{{.Category}} | Use go [template](https://pkg.go.dev/text/template) syntax rendering to generate the appropriate ID, and use `ClientServiceName` `ServiceName` `Category` three metadata that can be customised | | ||
| | ServerPathFormat | {{.Category}} | Use go [template](https://pkg.go.dev/text/template) syntax rendering to generate the appropriate ID, and use `ServiceName` `Category` two metadatas that can be customised | | ||
| | ConfigParser | defaultConfigParser | The default is the parser that parses json | | ||
|
|
||
| #### Governance Policy | ||
|
|
||
| > The configPath and configPrefix in the following example use default values, the service name is `ServiceName` and the client name is `ClientName`. | ||
| ##### Rate Limit Category=limit | ||
|
|
||
| > Currently, current limiting only supports the server side, so ClientServiceName is empty. | ||
| [JSON Schema](https://github.com/cloudwego/kitex/blob/develop/pkg/limiter/item_limiter.go#L33) | ||
|
|
||
| | Variable | Introduction | | ||
| | ---------------- | ---------------------------------- | | ||
| | connection_limit | Maximum concurrent connections | | ||
| | qps_limit | Maximum request number every 100ms | | ||
|
|
||
| Example: | ||
|
|
||
| > configPath: /KitexConfig/ServiceName/limit | ||
| ``` | ||
| { | ||
| "connection_limit": 100, | ||
| "qps_limit": 2000 | ||
| } | ||
| ``` | ||
|
|
||
|
|
||
|
|
||
| Note: | ||
|
|
||
| - The granularity of the current limit configuration is server global, regardless of client or method. | ||
| - Not configured or value is 0 means not enabled. | ||
| - connection_limit and qps_limit can be configured independently, e.g. connection_limit = 100, qps_limit = 0 | ||
|
|
||
| ##### Retry Policy Category=retry | ||
|
|
||
| [JSON Schema](https://github.com/cloudwego/kitex/blob/develop/pkg/retry/policy.go#L63) | ||
|
|
||
| | Variable | Introduction | | ||
| | ----------------------------- | ---------------------------------------------- | | ||
| | type | 0: failure_policy 1: backup_policy | | ||
| | failure_policy.backoff_policy | Can only be set one of `fixed` `none` `random` | | ||
|
|
||
| Example： | ||
|
|
||
| > configPath: /KitexConfig/ClientName/ServiceName/retry | ||
| ``` | ||
| { | ||
| "*": { | ||
| "enable": true, | ||
| "type": 0, | ||
| "failure_policy": { | ||
| "stop_policy": { | ||
| "max_retry_times": 3, | ||
| "max_duration_ms": 2000, | ||
| "cb_policy": { | ||
| "error_rate": 0.3 | ||
| } | ||
| }, | ||
| "backoff_policy": { | ||
| "backoff_type": "fixed", | ||
| "cfg_items": { | ||
| "fix_ms": 50 | ||
| } | ||
| }, | ||
| "retry_same_node": false | ||
| } | ||
| }, | ||
| "echo": { | ||
| "enable": true, | ||
| "type": 1, | ||
| "backup_policy": { | ||
| "retry_delay_ms": 100, | ||
| "retry_same_node": false, | ||
| "stop_policy": { | ||
| "max_retry_times": 2, | ||
| "max_duration_ms": 300, | ||
| "cb_policy": { | ||
| "error_rate": 0.2 | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
|
|
||
|
|
||
| Note: retry.Container has built-in support for specifying the default configuration using the `*` wildcard (see the [getRetryer](https://github.com/cloudwego/kitex/blob/v0.5.1/pkg/retry/retryer.go#L240) method for details). | ||
|
|
||
| ##### RPC Timeout Category=rpc_timeout | ||
|
|
||
| [JSON Schema](https://github.com/cloudwego/kitex/blob/develop/pkg/rpctimeout/item_rpc_timeout.go#L42) | ||
|
|
||
| Example： | ||
|
|
||
| > configPath: /KitexConfig/ClientName/ServiceName/rpc_timeout | ||
| ``` | ||
| { | ||
| "*": { | ||
| "conn_timeout_ms": 100, | ||
| "rpc_timeout_ms": 3000 | ||
| }, | ||
| "echo": { | ||
| "conn_timeout_ms": 50, | ||
| "rpc_timeout_ms": 1000 | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
|
|
||
|
|
||
| Note: The circuit breaker implementation of kitex does not currently support changing the global default configuration (see [initServiceCB](https://github.com/cloudwego/kitex/blob/v0.5.1/pkg/circuitbreak/cbsuite.go#L195) for details). | ||
|
|
||
| ##### Circuit Break: Category=circuit_break | ||
|
|
||
| [JSON Schema](https://github.com/cloudwego/kitex/blob/develop/pkg/circuitbreak/item_circuit_breaker.go#L30) | ||
|
|
||
| | Variable | Introduction | | ||
| | ---------- | --------------------------------- | | ||
| | min_sample | Minimum statistical sample number | | ||
|
|
||
| Example： | ||
|
|
||
| The echo method uses the following configuration (0.3, 100) and other methods use the global default configuration (0.5, 200) | ||
|
|
||
| > configPath: /KitexConfig/ClientName/ServiceName/circuit_break | ||
| ``` | ||
| { | ||
| "echo": { | ||
| "enable": true, | ||
| "err_rate": 0.3, | ||
| "min_sample": 100 | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ##### Degradation: Category=degradation | ||
|
|
||
| [JSON Schema](https://github.com/cloudwego/kitex/blob/develop/pkg/circuitbreak/item_circuit_breaker.go#L30) | ||
|
|
||
| | Variable | Introduction | | ||
| |------------|------------------------------------| | ||
| | enable | Whether to enable degradation | | ||
| | percentage | The percentage of dropped requests | | ||
|
|
||
| Example： | ||
|
|
||
| > configPath: /KitexConfig/ClientName/ServiceName/degradation | ||
| ```json | ||
| { | ||
| "enable": true, | ||
| "percentage": 30 | ||
| } | ||
| ``` | ||
| Note: Degradation is not enabled by default. | ||
|
|
||
| ### More Info | ||
|
|
||
| Refer to [example](https://github.com/kitex-contrib/config-zookeeper/tree/main/example) for more usage. | ||
|
|
||
| ### Note | ||
|
|
||
| In ZooKeeper, when a node is created, its parent must exist. | ||
|
|
||
| ## Compatibility | ||
|
|
||
| This Package use https://github.com/go-zookeeper/zk | ||
|
|
||
| maintained by: [jiuxia211](https://github.com/jiuxia211) | ||
|
|
Oops, something went wrong.