feat: add zookeeper && server (#1)

* feat: add zookeeper && server * update go mod * use gofumpt * fix: atomic.Int64 can't be used under go 1.19 * fix: unreasonable names * fix: update the processing of removed keys and empty data * style: add cancelFuncHolder type * style: update code comments * refactor: optimized loop variable * feat: add client * feat: update client * fix: add missing code snippet * feat: add README * style: use gofumpt * chore(doc): optimize readme * docs: update README * feat: add listen goroutine recover * fix: correct error logging format in project --------- Co-authored-by: kinggo <lilong.21@bytedance.com>
kitex-contrib · Dec 9, 2023 · a20b754 · a20b754
1 parent 0d9795b
commit a20b754
Show file tree

Hide file tree

Showing 20 changed files with 4,512 additions and 1 deletion.
diff --git a/README.md b/README.md
@@ -1 +1,299 @@
-# .github
+# config-zookeeper(*This is a community driven project*)
+
+[中文](https://github.com/kitex-contrib/config-zookeeper/blob/main/README_CN.md)
+
+ **zookeeper** as config center for service governance.
+
+## Usage
+
+### Basic
+
+#### Server
+
+```go
+package main
+
+import (
+	"context"
+	"log"
+
+	"github.com/cloudwego/kitex-examples/kitex_gen/api"
+	"github.com/cloudwego/kitex-examples/kitex_gen/api/echo"
+	"github.com/cloudwego/kitex/pkg/klog"
+	"github.com/cloudwego/kitex/pkg/rpcinfo"
+	"github.com/cloudwego/kitex/server"
+	zookeeperServer "github.com/kitex-contrib/config-zookeeper/server"
+	"github.com/kitex-contrib/config-zookeeper/zookeeper"
+)
+
+var _ api.Echo = &EchoImpl{}
+
+// EchoImpl implements the last service interface defined in the IDL.
+type EchoImpl struct{}
+
+// Echo implements the Echo interface.
+func (s *EchoImpl) Echo(ctx context.Context, req *api.Request) (resp *api.Response, err error) {
+	klog.Info("echo called")
+	return &api.Response{Message: req.Message}, nil
+}
+
+func main() {
+	zookeeperClient, err := zookeeper.NewClient(zookeeper.Options{})
+	if err != nil {
+		panic(err)
+	}
+	serviceName := "ServiceName" // your server-side service name
+	svr := echo.NewServer(
+		new(EchoImpl),
+		server.WithServerBasicInfo(&rpcinfo.EndpointBasicInfo{ServiceName: serviceName}),
+		server.WithSuite(zookeeperServer.NewSuite(serviceName, zookeeperClient)),
+	)
+	if err := svr.Run(); err != nil {
+		log.Println("server stopped with error:", err)
+	} else {
+		log.Println("server stopped")
+	}
+}
+
+```
+
+#### Client
+
+```go
+package main
+
+import (
+	"context"
+	"log"
+	"time"
+
+	"github.com/cloudwego/kitex-examples/kitex_gen/api"
+	"github.com/cloudwego/kitex-examples/kitex_gen/api/echo"
+	"github.com/cloudwego/kitex/client"
+	"github.com/cloudwego/kitex/pkg/klog"
+	zookeeperclient "github.com/kitex-contrib/config-zookeeper/client"
+	"github.com/kitex-contrib/config-zookeeper/utils"
+	"github.com/kitex-contrib/config-zookeeper/zookeeper"
+)
+
+type configLog struct{}
+
+func (cl *configLog) Apply(opt *utils.Options) {
+	fn := func(k *zookeeper.ConfigParam) {
+		klog.Infof("zookeeper config %v", k)
+	}
+	opt.ZookeeperCustomFunctions = append(opt.ZookeeperCustomFunctions, fn)
+}
+
+func main() {
+	zookeeperClient, err := zookeeper.NewClient(zookeeper.Options{})
+	if err != nil {
+		panic(err)
+	}
+
+	cl := configLog{}
+
+	serviceName := "ServiceName" // your server-side service name
+	clientName := "ClientName"   // your client-side service name
+	client, err := echo.NewClient(
+		serviceName,
+		client.WithHostPorts("0.0.0.0:8888"),
+		client.WithSuite(zookeeperclient.NewSuite(serviceName, clientName, zookeeperClient, &cl)),
+	)
+	if err != nil {
+		log.Fatal(err)
+	}
+	for {
+		req := &api.Request{Message: "my request"}
+		resp, err := client.Echo(context.Background(), req)
+		if err != nil {
+			klog.Errorf("take request error: %v", err)
+		} else {
+			klog.Infof("receive response %v", resp)
+		}
+		time.Sleep(time.Second * 10)
+	}
+}
+```
+
+### Zookeeper Configuration
+
+The client is initialized according to the parameters of `Options` and connects to the zookeeper server. After the connection is established, the suite will listen to the corresponding node according to `Path` and `ServerPathFormat` or`ClientPathFormat`to updates its own policy dynamically. See the `Options` variables below for specific parameters.
+
+#### CustomFunction
+
+Provide the mechanism to custom the zookeeper parameter `Path`.
+
+```go
+type ConfigParam struct {
+	Prefix string
+	Path   string
+}
+```
+
+The final Path is `param.Prefix + "/" + param.Path`
+
+#### Options Variable
+
+| Variable Name    | Default Value                                               | Introduction                                                 |
+| ---------------- | ----------------------------------------------------------- | ------------------------------------------------------------ |
+| Servers          | 127.0.0.1:2181                                              | Zookeeper server nodes                                       |
+| Prefix           | /KitexConfig                                                | The prefix of Zookeeper                                      |
+| ClientPathFormat | {{.ClientServiceName}}/{{.ServerServiceName}}/{{.Category}} | Use go [template](https://pkg.go.dev/text/template) syntax rendering to generate the appropriate ID, and use `ClientServiceName` `ServiceName` `Category` three metadata that can be customised |
+| ServerPathFormat | {{.Category}}                                               | Use go [template](https://pkg.go.dev/text/template) syntax rendering to generate the appropriate ID, and use `ServiceName` `Category` two metadatas that can be customised |
+| ConfigParser     | defaultConfigParser                                         | The default is the parser that parses json                   |
+
+#### Governance Policy
+
+> The configPath and configPrefix in the following example use default values, the service name is `ServiceName` and the client name is `ClientName`.
+
+##### Rate Limit Category=limit
+
+> Currently, current limiting only supports the server side, so ClientServiceName is empty.
+
+[JSON Schema](https://github.com/cloudwego/kitex/blob/develop/pkg/limiter/item_limiter.go#L33)
+
+| Variable         | Introduction                       |
+| ---------------- | ---------------------------------- |
+| connection_limit | Maximum concurrent connections     |
+| qps_limit        | Maximum request number every 100ms |
+
+Example:
+
+> configPath: /KitexConfig/ServiceName/limit
+
+```
+{
+  "connection_limit": 100,
+  "qps_limit": 2000
+}
+```
+
+
+
+Note:
+
+- The granularity of the current limit configuration is server global, regardless of client or method.
+- Not configured or value is 0 means not enabled.
+- connection_limit and qps_limit can be configured independently, e.g. connection_limit = 100, qps_limit = 0
+
+##### Retry Policy Category=retry
+
+[JSON Schema](https://github.com/cloudwego/kitex/blob/develop/pkg/retry/policy.go#L63)
+
+| Variable                      | Introduction                                   |
+| ----------------------------- | ---------------------------------------------- |
+| type                          | 0: failure_policy 1: backup_policy             |
+| failure_policy.backoff_policy | Can only be set one of `fixed` `none` `random` |
+
+Example：
+
+> configPath: /KitexConfig/ClientName/ServiceName/retry
+
+```
+{
+    "*": {
+        "enable": true,
+        "type": 0,
+        "failure_policy": {
+            "stop_policy": {
+                "max_retry_times": 3,
+                "max_duration_ms": 2000,
+                "cb_policy": {
+                    "error_rate": 0.3
+                }
+            },
+            "backoff_policy": {
+                "backoff_type": "fixed",
+                "cfg_items": {
+                    "fix_ms": 50
+                }
+            },
+            "retry_same_node": false
+        }
+    },
+    "echo": {
+        "enable": true,
+        "type": 1,
+        "backup_policy": {
+            "retry_delay_ms": 100,
+            "retry_same_node": false,
+            "stop_policy": {
+                "max_retry_times": 2,
+                "max_duration_ms": 300,
+                "cb_policy": {
+                    "error_rate": 0.2
+                }
+            }
+        }
+    }
+}
+```
+
+
+
+Note: retry.Container has built-in support for specifying the default configuration using the `*` wildcard (see the [getRetryer](https://github.com/cloudwego/kitex/blob/v0.5.1/pkg/retry/retryer.go#L240) method for details).
+
+##### RPC Timeout Category=rpc_timeout
+
+[JSON Schema](https://github.com/cloudwego/kitex/blob/develop/pkg/rpctimeout/item_rpc_timeout.go#L42)
+
+Example：
+
+> configPath: /KitexConfig/ClientName/ServiceName/rpc_timeout
+
+```
+{
+  "*": {
+    "conn_timeout_ms": 100,
+    "rpc_timeout_ms": 3000
+  },
+  "echo": {
+    "conn_timeout_ms": 50,
+    "rpc_timeout_ms": 1000
+  }
+}
+```
+
+
+
+Note: The circuit breaker implementation of kitex does not currently support changing the global default configuration (see [initServiceCB](https://github.com/cloudwego/kitex/blob/v0.5.1/pkg/circuitbreak/cbsuite.go#L195) for details).
+
+##### Circuit Break: Category=circuit_break
+
+[JSON Schema](https://github.com/cloudwego/kitex/blob/develop/pkg/circuitbreak/item_circuit_breaker.go#L30)
+
+| Variable   | Introduction                      |
+| ---------- | --------------------------------- |
+| min_sample | Minimum statistical sample number |
+
+Example：
+
+The echo method uses the following configuration (0.3, 100) and other methods use the global default configuration (0.5, 200)
+
+> configPath: /KitexConfig/ClientName/ServiceName/circuit_break
+
+```
+{
+  "echo": {
+    "enable": true,
+    "err_rate": 0.3,
+    "min_sample": 100
+  }
+}
+```
+
+### More Info
+
+Refer to [example](https://github.com/kitex-contrib/config-zookeeper/tree/main/example) for more usage.
+
+### Note
+
+In ZooKeeper, when a node is created, its parent must exist.
+
+## Compatibility
+
+This Package use https://github.com/go-zookeeper/zk
+
+maintained by:  [jiuxia211](https://github.com/jiuxia211)
+