Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add zookeeper && server #1

Merged
merged 19 commits into from
Dec 9, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
300 changes: 299 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,299 @@
# .github
# config-zookeeper(*This is a community driven project*)

[中文](https://github.com/kitex-contrib/config-zookeeper/blob/main/README_CN.md)

**zookeeper** as config center for service governance.

## Usage

### Basic

#### Server

```go
package main

import (
"context"
"log"

"github.com/cloudwego/kitex-examples/kitex_gen/api"
"github.com/cloudwego/kitex-examples/kitex_gen/api/echo"
"github.com/cloudwego/kitex/pkg/klog"
"github.com/cloudwego/kitex/pkg/rpcinfo"
"github.com/cloudwego/kitex/server"
zookeeperServer "github.com/kitex-contrib/config-zookeeper/server"
"github.com/kitex-contrib/config-zookeeper/zookeeper"
)

var _ api.Echo = &EchoImpl{}

// EchoImpl implements the last service interface defined in the IDL.
type EchoImpl struct{}

// Echo implements the Echo interface.
func (s *EchoImpl) Echo(ctx context.Context, req *api.Request) (resp *api.Response, err error) {
klog.Info("echo called")
return &api.Response{Message: req.Message}, nil
}

func main() {
zookeeperClient, err := zookeeper.NewClient(zookeeper.Options{})
if err != nil {
panic(err)
}
serviceName := "ServiceName" // your server-side service name
svr := echo.NewServer(
new(EchoImpl),
server.WithServerBasicInfo(&rpcinfo.EndpointBasicInfo{ServiceName: serviceName}),
server.WithSuite(zookeeperServer.NewSuite(serviceName, zookeeperClient)),
)
if err := svr.Run(); err != nil {
log.Println("server stopped with error:", err)
} else {
log.Println("server stopped")
}
}

```

#### Client

```go
package main

import (
"context"
"log"
"time"

"github.com/cloudwego/kitex-examples/kitex_gen/api"
"github.com/cloudwego/kitex-examples/kitex_gen/api/echo"
"github.com/cloudwego/kitex/client"
"github.com/cloudwego/kitex/pkg/klog"
zookeeperclient "github.com/kitex-contrib/config-zookeeper/client"
"github.com/kitex-contrib/config-zookeeper/utils"
"github.com/kitex-contrib/config-zookeeper/zookeeper"
)

type configLog struct{}

func (cl *configLog) Apply(opt *utils.Options) {
fn := func(k *zookeeper.ConfigParam) {
klog.Infof("zookeeper config %v", k)
}
opt.ZookeeperCustomFunctions = append(opt.ZookeeperCustomFunctions, fn)
}

func main() {
zookeeperClient, err := zookeeper.NewClient(zookeeper.Options{})
if err != nil {
panic(err)
}

cl := configLog{}

serviceName := "ServiceName" // your server-side service name
clientName := "ClientName" // your client-side service name
client, err := echo.NewClient(
serviceName,
client.WithHostPorts("0.0.0.0:8888"),
client.WithSuite(zookeeperclient.NewSuite(serviceName, clientName, zookeeperClient, &cl)),
)
if err != nil {
log.Fatal(err)
}
for {
req := &api.Request{Message: "my request"}
resp, err := client.Echo(context.Background(), req)
if err != nil {
klog.Errorf("take request error: %v", err)
} else {
klog.Infof("receive response %v", resp)
}
time.Sleep(time.Second * 10)
}
}
```

### Zookeeper Configuration

The client is initialized according to the parameters of `Options` and connects to the zookeeper server. After the connection is established, the suite will listen to the corresponding node according to `Path` and `ServerPathFormat` or`ClientPathFormat`to updates its own policy dynamically. See the `Options` variables below for specific parameters.

#### CustomFunction

Provide the mechanism to custom the zookeeper parameter `Path`.

```go
type ConfigParam struct {
Prefix string
Path string
}
```

The final Path is `param.Prefix + "/" + param.Path`

#### Options Variable

| Variable Name | Default Value | Introduction |
| ---------------- | ----------------------------------------------------------- | ------------------------------------------------------------ |
| Servers | 127.0.0.1:2181 | Zookeeper server nodes |
| Prefix | /KitexConfig | The prefix of Zookeeper |
| ClientPathFormat | {{.ClientServiceName}}/{{.ServerServiceName}}/{{.Category}} | Use go [template](https://pkg.go.dev/text/template) syntax rendering to generate the appropriate ID, and use `ClientServiceName` `ServiceName` `Category` three metadata that can be customised |
| ServerPathFormat | {{.Category}} | Use go [template](https://pkg.go.dev/text/template) syntax rendering to generate the appropriate ID, and use `ServiceName` `Category` two metadatas that can be customised |
| ConfigParser | defaultConfigParser | The default is the parser that parses json |

#### Governance Policy

> The configPath and configPrefix in the following example use default values, the service name is `ServiceName` and the client name is `ClientName`.
##### Rate Limit Category=limit

> Currently, current limiting only supports the server side, so ClientServiceName is empty.
[JSON Schema](https://github.com/cloudwego/kitex/blob/develop/pkg/limiter/item_limiter.go#L33)

| Variable | Introduction |
| ---------------- | ---------------------------------- |
| connection_limit | Maximum concurrent connections |
| qps_limit | Maximum request number every 100ms |

Example:

> configPath: /KitexConfig/ServiceName/limit
```
{
"connection_limit": 100,
"qps_limit": 2000
}
```



Note:

- The granularity of the current limit configuration is server global, regardless of client or method.
- Not configured or value is 0 means not enabled.
- connection_limit and qps_limit can be configured independently, e.g. connection_limit = 100, qps_limit = 0

##### Retry Policy Category=retry

[JSON Schema](https://github.com/cloudwego/kitex/blob/develop/pkg/retry/policy.go#L63)

| Variable | Introduction |
| ----------------------------- | ---------------------------------------------- |
| type | 0: failure_policy 1: backup_policy |
| failure_policy.backoff_policy | Can only be set one of `fixed` `none` `random` |

Example:

> configPath: /KitexConfig/ClientName/ServiceName/retry
```
{
"*": {
"enable": true,
"type": 0,
"failure_policy": {
"stop_policy": {
"max_retry_times": 3,
"max_duration_ms": 2000,
"cb_policy": {
"error_rate": 0.3
}
},
"backoff_policy": {
"backoff_type": "fixed",
"cfg_items": {
"fix_ms": 50
}
},
"retry_same_node": false
}
},
"echo": {
"enable": true,
"type": 1,
"backup_policy": {
"retry_delay_ms": 100,
"retry_same_node": false,
"stop_policy": {
"max_retry_times": 2,
"max_duration_ms": 300,
"cb_policy": {
"error_rate": 0.2
}
}
}
}
}
```



Note: retry.Container has built-in support for specifying the default configuration using the `*` wildcard (see the [getRetryer](https://github.com/cloudwego/kitex/blob/v0.5.1/pkg/retry/retryer.go#L240) method for details).

##### RPC Timeout Category=rpc_timeout

[JSON Schema](https://github.com/cloudwego/kitex/blob/develop/pkg/rpctimeout/item_rpc_timeout.go#L42)

Example:

> configPath: /KitexConfig/ClientName/ServiceName/rpc_timeout
```
{
"*": {
"conn_timeout_ms": 100,
"rpc_timeout_ms": 3000
},
"echo": {
"conn_timeout_ms": 50,
"rpc_timeout_ms": 1000
}
}
```



Note: The circuit breaker implementation of kitex does not currently support changing the global default configuration (see [initServiceCB](https://github.com/cloudwego/kitex/blob/v0.5.1/pkg/circuitbreak/cbsuite.go#L195) for details).

##### Circuit Break: Category=circuit_break

[JSON Schema](https://github.com/cloudwego/kitex/blob/develop/pkg/circuitbreak/item_circuit_breaker.go#L30)

| Variable | Introduction |
| ---------- | --------------------------------- |
| min_sample | Minimum statistical sample number |

Example:

The echo method uses the following configuration (0.3, 100) and other methods use the global default configuration (0.5, 200)

> configPath: /KitexConfig/ClientName/ServiceName/circuit_break
```
{
"echo": {
"enable": true,
"err_rate": 0.3,
"min_sample": 100
}
}
```

### More Info

Refer to [example](https://github.com/kitex-contrib/config-zookeeper/tree/main/example) for more usage.

### Note

In ZooKeeper, when a node is created, its parent must exist.

## Compatibility

This Package use https://github.com/go-zookeeper/zk

maintained by: [jiuxia211](https://github.com/jiuxia211)

Loading
Loading