Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Istio/XDS support #1804

Merged
merged 21 commits into from
Apr 6, 2022
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/github-actions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
# If you want to matrix build , you can append the following list.
matrix:
go_version:
- 1.15
- 1.17
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我们要升级这个版本吗

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我认为升级吧。
1.15 版本不支持一些基础函数,比如os.ReadFile
开发者本地如果使用高版本,ci的时候会报错
@zhaoyunxing92

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

嗯嗯

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we discussed yesterday night, pls recover the go version.

os:
- ubuntu-latest

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/golangci-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,11 @@ jobs:
strategy:
matrix:
golang:
- 1.15
- 1.17
steps:
- uses: actions/setup-go@v2
with:
go-version: 1.15
go-version: 1.17
- uses: actions/checkout@v3
- name: golangci-lint
uses: golangci/golangci-lint-action@v3.1.0
Expand Down
4 changes: 2 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -345,7 +345,7 @@ Milestone: [https://github.com/apache/dubbo-go/milestone/5](https://github.com/a
- [Fix go client quit abnormally when it connects java server](https://github.com/apache/dubbo-go/pull/820) [@wenxuwan](https://github.com/wenxuwan)
- [Fix sentinel windows issue](https://github.com/apache/dubbo-go/pull/821) [@louyuting](https://github.com/louyuting)
- [Fix metadata default port](https://github.com/apache/dubbo-go/pull/821) [@sanxun0325](https://github.com/sanxun0325)
- [Fix consul can not destory](https://github.com/apache/dubbo-go/pull/788) [@LaurenceLiZhixin](https://github.com/LaurenceLiZhixin)
- [Fix consul can not destroy](https://github.com/apache/dubbo-go/pull/788) [@LaurenceLiZhixin](https://github.com/LaurenceLiZhixin)

Milestone: [https://github.com/apache/dubbo-go/milestone/6](https://github.com/apache/dubbo-go/milestone/6?closed=1)

Expand Down Expand Up @@ -390,7 +390,7 @@ Project: [https://github.com/apache/dubbo-go/projects/10](https://github.com/apa
- [Fix go client quit abnormally when it connects java server](https://github.com/apache/dubbo-go/pull/820) [@wenxuwan](https://github.com/wenxuwan)
- [Fix sentinel windows issue](https://github.com/apache/dubbo-go/pull/821) [@louyuting](https://github.com/louyuting)
- [Fix metadata default port](https://github.com/apache/dubbo-go/pull/821) [@sanxun0325](https://github.com/sanxun0325)
- [Fix consul can not destory](https://github.com/apache/dubbo-go/pull/788) [@LaurenceLiZhixin](https://github.com/LaurenceLiZhixin)
- [Fix consul can not destroy](https://github.com/apache/dubbo-go/pull/788) [@LaurenceLiZhixin](https://github.com/LaurenceLiZhixin)

Milestone: [https://github.com/apache/dubbo-go/milestone/6](https://github.com/apache/dubbo-go/milestone/6?closed=1)

Expand Down
41 changes: 41 additions & 0 deletions cluster/router/meshrouter/factory.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package meshrouter

import (
"dubbo.apache.org/dubbo-go/v3/cluster/router"
"dubbo.apache.org/dubbo-go/v3/common/constant"
"dubbo.apache.org/dubbo-go/v3/common/extension"
)

func init() {
extension.SetRouterFactory(constant.MeshRouterFactoryKey, NewMeshRouterFactory)
}

// UniformRouteFactory is uniform router's factory
type MeshRouterFactory struct{}

// NewMeshRouterFactory constructs a new PriorityRouterFactory
func NewMeshRouterFactory() router.PriorityRouterFactory {
return &MeshRouterFactory{}
}

// NewPriorityRouter construct a new UniformRouteFactory as PriorityRouter
func (f *MeshRouterFactory) NewPriorityRouter() (router.PriorityRouter, error) {
return NewMeshRouter()
}
185 changes: 185 additions & 0 deletions cluster/router/meshrouter/meshrouter.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package meshrouter

import (
"bytes"
"math/rand"
"strings"
)

import (
"dubbo.apache.org/dubbo-go/v3/cluster/router"
"dubbo.apache.org/dubbo-go/v3/common"
"dubbo.apache.org/dubbo-go/v3/common/constant"
"dubbo.apache.org/dubbo-go/v3/common/logger"
"dubbo.apache.org/dubbo-go/v3/config_center"
"dubbo.apache.org/dubbo-go/v3/protocol"
"dubbo.apache.org/dubbo-go/v3/remoting/xds"
"dubbo.apache.org/dubbo-go/v3/xds/client/resource"
"dubbo.apache.org/dubbo-go/v3/xds/utils/resolver"
)

const (
name = "mesh-router"
)

// MeshRouter have
type MeshRouter struct {
client *xds.WrappedClientImpl
}

// NewMeshRouter construct an NewConnCheckRouter via url
func NewMeshRouter() (router.PriorityRouter, error) {
xdsWrappedClient := xds.GetXDSWrappedClient()
if xdsWrappedClient == nil {
logger.Debugf("[Mesh Router] xds wrapped client is not created.")
}
return &MeshRouter{
client: xdsWrappedClient,
}, nil
}

// Route gets a list of routed invoker
func (r *MeshRouter) Route(invokers []protocol.Invoker, url *common.URL, invocation protocol.Invocation) []protocol.Invoker {
if r.client == nil {
return invokers
}
hostAddr, err := r.client.GetHostAddrByServiceUniqueKey(getSubscribeName(url))
if err != nil {
// todo deal with error
return nil
}
rconf := r.client.GetRouterConfig(hostAddr)

clusterInvokerMap := make(map[string][]protocol.Invoker)
for _, v := range invokers {
meshClusterId := v.GetURL().GetParam(constant.MeshClusterIDKey, "")
if _, ok := clusterInvokerMap[meshClusterId]; !ok {
clusterInvokerMap[meshClusterId] = make([]protocol.Invoker, 0)
}
clusterInvokerMap[meshClusterId] = append(clusterInvokerMap[meshClusterId], v)
}

if len(rconf.VirtualHosts) != 0 {
// try to route to sub virtual host
for _, vh := range rconf.VirtualHosts {
// 1. match domain
//vh.Domains == ["*"]

// 2. match http route
for _, r := range vh.Routes {
//route.
ctx := invocation.ToContext()
matcher, err := resource.RouteToMatcher(r)
if err != nil {
panic(err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

panic 不能随便用,这里确实有必要使用 panic 吗?

}
if matcher.Match(resolver.RPCInfo{
Context: ctx,
Method: "/" + invocation.MethodName(),
}) {
// Loop through routes in order and select first match.
if r == nil || r.WeightedClusters == nil {
panic("cluster notfound")
}
invokersWeightPairs := make(invokerWeightPairs, 0)

for clusterId, weight := range r.WeightedClusters {
// cluster -> invokers
targetInvokers := clusterInvokerMap[clusterId]
invokersWeightPairs = append(invokersWeightPairs, invokerWeightPair{
invokers: targetInvokers,
weight: weight.Weight,
})
}
return invokersWeightPairs.GetInvokers()
}
}
}
}
return invokers
}

// Process there is no process needs for uniform Router, as it upper struct RouterChain has done it
func (r *MeshRouter) Process(event *config_center.ConfigChangeEvent) {
}

// Name get name of ConnCheckerRouter
func (r *MeshRouter) Name() string {
return name
}

// Priority get Router priority level
func (r *MeshRouter) Priority() int64 {
return 0
}

// URL Return URL in router
func (r *MeshRouter) URL() *common.URL {
return nil
}

// Notify the router the invoker list
func (r *MeshRouter) Notify(invokers []protocol.Invoker) {
}

type invokerWeightPair struct {
invokers []protocol.Invoker
weight uint32
}

type invokerWeightPairs []invokerWeightPair

func (i *invokerWeightPairs) GetInvokers() []protocol.Invoker {
if len(*i) == 0 {
return nil
}
totalWeight := uint32(0)
tempWeight := uint32(0)
for _, v := range *i {
totalWeight += v.weight
}
randFloat := rand.Float64()
for _, v := range *i {
tempWeight += v.weight
tempPercent := float64(tempWeight) / float64(totalWeight)
if tempPercent >= randFloat {
return v.invokers
}
}
return (*i)[0].invokers
}

func getSubscribeName(url *common.URL) string {
var buffer bytes.Buffer

buffer.Write([]byte(common.DubboNodes[common.PROVIDER]))
appendParam(&buffer, url, constant.InterfaceKey)
appendParam(&buffer, url, constant.VersionKey)
appendParam(&buffer, url, constant.GroupKey)
return buffer.String()
}

func appendParam(target *bytes.Buffer, url *common.URL, key string) {
value := url.GetParam(key, "")
target.Write([]byte(constant.NacosServiceNameSeparator))
if strings.TrimSpace(value) != "" {
target.Write([]byte(value))
}
}
6 changes: 4 additions & 2 deletions common/constant/env.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ package constant

// nolint
const (
ConfigFileEnvKey = "DUBBO_GO_CONFIG_PATH" // key of environment variable dubbogo configure file path
AppLogConfFile = "AppLogConfFile"
ConfigFileEnvKey = "DUBBO_GO_CONFIG_PATH" // key of environment variable dubbogo configure file path
AppLogConfFile = "AppLogConfFile"
PodNameEnvKey = "POD_NAME"
PodNamespaceEnvKey = "POD_NAMESPACE"
)
10 changes: 8 additions & 2 deletions common/constant/key.go
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ const (
TokenFilterKey = "token"
TpsLimitFilterKey = "tps"
TracingFilterKey = "tracing"
XdsCircuitBreakerKey = "xds_circuit_reaker"
)

const (
Expand Down Expand Up @@ -274,6 +275,10 @@ const (
ZookeeperKey = "zookeeper"
)

const (
XDSRegistryKey = "xds"
)

const (
EtcdV3Key = "etcdv3"
)
Expand All @@ -290,14 +295,15 @@ const (

// Use for router module
const (
TagRouterRuleSuffix = ".tag-router" // Specify tag router suffix
TagRouterRuleSuffix = ".tag-router"
ConditionRouterRuleSuffix = ".condition-router" // Specify condition router suffix
MeshRouteSuffix = ".MESHAPPRULE" // Specify mesh router suffix
ForceUseTag = "dubbo.force.tag" // the tag in attachment
Tagkey = "dubbo.tag" // key of tag
AttachmentKey = DubboCtxKey("attachment") // key in context in invoker
TagRouterFactoryKey = "tag"
V3RouterFactoryKey = "mesh"
V3RouterFactoryKey = "v3router"
MeshRouterFactoryKey = "mesh"
)

// Auth filter
Expand Down
36 changes: 36 additions & 0 deletions common/constant/xds.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package constant

const (
MeshClusterIDKey = "meshClusterId"
MeshHostAddrKey = "meshHostAddr"
MeshSubsetKey = "meshSubset"

MeshDeleteClusterPrefix = "-"
MeshAnyAddrMatcher = "*"
)

const (
XDSMetadataClusterIDKey = "CLUSTER_ID"
XDSMetadataLabelsKey = "LABELS"

XDSMetadataDefaultDomainName = "Kubernetes"

XDSMetadataDubboGoMapperKey = "DUBBO_GO"
)
13 changes: 11 additions & 2 deletions common/logger/logger.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ type DubboLogger struct {
type Config struct {
LumberjackConfig *lumberjack.Logger `yaml:"lumberjack-config"`
ZapConfig *zap.Config `yaml:"zap-config"`
CallerSkip int
}

// Logger is the interface for Logger types
Expand Down Expand Up @@ -89,8 +90,16 @@ func InitLogger(conf *Config) {
config.ZapConfig = conf.ZapConfig
}

if conf != nil {
config.CallerSkip = conf.CallerSkip
}

if config.CallerSkip == 0 {
config.CallerSkip = 1
}

if conf == nil || conf.LumberjackConfig == nil {
zapLogger, _ = config.ZapConfig.Build(zap.AddCaller(), zap.AddCallerSkip(1))
zapLogger, _ = config.ZapConfig.Build(zap.AddCaller(), zap.AddCallerSkip(config.CallerSkip))
} else {
config.LumberjackConfig = conf.LumberjackConfig
zapLogger = initZapLoggerWithSyncer(config)
Expand Down Expand Up @@ -144,7 +153,7 @@ func initZapLoggerWithSyncer(conf *Config) *zap.Logger {
zap.NewAtomicLevelAt(conf.ZapConfig.Level.Level()),
)

return zap.New(core, zap.AddCaller(), zap.AddCallerSkip(1))
return zap.New(core, zap.AddCaller(), zap.AddCallerSkip(conf.CallerSkip))
}

// getEncoder get encoder by config, zapcore support json and console encoder
Expand Down
Loading