Merge remote-tracking branch 'upstream/develop' into gorestful-filter

CHANGE.md

@@ -1,6 +1,34 @@
 # Release Notes
 ---
 
+## 1.4.0
+### New Features
+
+- [Condition router](https://github.com/apache/dubbo-go/pull/294)
+- [Context support](https://github.com/apache/dubbo-go/pull/330)
+- [Opentracing & transfer context end to end for jsonrpc protocol](https://github.com/apache/dubbo-go/pull/335)
+- [Opentracing & transfer context end to end for dubbo protocol](https://github.com/apache/dubbo-go/pull/344)
+- [Nacos config center](https://github.com/apache/dubbo-go/pull/357)
+- [Prometheus support](https://github.com/apache/dubbo-go/pull/342)
+- [Support sign and auth for request](https://github.com/apache/dubbo-go/pull/323)
+- [Healthy instance first router](https://github.com/apache/dubbo-go/pull/389)
+- [User can add attachments for dubbo protocol](https://github.com/apache/dubbo-go/pull/398)
+- [K8s as registry](https://github.com/apache/dubbo-go/pull/400)
+- [Rest protocol](https://github.com/apache/dubbo-go/pull/352)
+
+### Enhancement
+
+- [Reduce the scope of lock in zk listener](https://github.com/apache/dubbo-go/pull/346)
+- [Trace error of getGettyRpcClient](https://github.com/apache/dubbo-go/pull/384)
+- [Refactor to add base_registry](https://github.com/apache/dubbo-go/pull/348)
+- [Do not listen to directory event if zkPath ends with providers/ or consumers/](https://github.com/apache/dubbo-go/pull/359)
+
+### Bugfixes
+
+- [Handle the panic when invoker was destroyed](https://github.com/apache/dubbo-go/pull/358)
+- [HessianCodec failed to check package header length](https://github.com/apache/dubbo-go/pull/381)
+
+
 ## 1.3.0
 
 ### New Features

README.md

@@ -3,6 +3,8 @@
 [![Build Status](https://travis-ci.org/apache/dubbo-go.svg?branch=master)](https://travis-ci.org/apache/dubbo-go)
 [![codecov](https://codecov.io/gh/apache/dubbo-go/branch/master/graph/badge.svg)](https://codecov.io/gh/apache/dubbo-go)
 [![go.dev reference](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go&logoColor=white&style=flat-square)](https://pkg.go.dev/github.com/apache/dubbo-go?tab=doc)
+[![Go Report Card](https://goreportcard.com/badge/github.com/apache/dubbo-go)](https://goreportcard.com/report/github.com/apache/dubbo-go)
+![license](https://img.shields.io/badge/license-Apache--2.0-green.svg)
 
 ---
 Apache Dubbo Go Implementation.
@@ -50,16 +52,22 @@ Finished List:
     * Dubbo
     * Jsonrpc2.0
     * [gRPC](https://github.com/apache/dubbo-go/pull/311)
+    * [RESTful](https://github.com/apache/dubbo-go/pull/352)
+
+- Router
+    * [Condition router](https://github.com/apache/dubbo-go/pull/294)
 
 - Registry
     * ZooKeeper
     * [etcd v3](https://github.com/apache/dubbo-go/pull/148)
     * [nacos](https://github.com/apache/dubbo-go/pull/151)
     * [consul](https://github.com/apache/dubbo-go/pull/121)
+    * [k8s](https://github.com/apache/dubbo-go/pull/400)
 
 - Dynamic Configure Center & Service Management Configurator
     * Zookeeper
     * [apollo](https://github.com/apache/dubbo-go/pull/250)
+    * [nacos](https://github.com/apache/dubbo-go/pull/357)
 
 - Cluster Strategy
     * Failover
@@ -86,6 +94,10 @@ Finished List:
 
 - Invoke
     * [generic invoke](https://github.com/apache/dubbo-go/pull/122)
+
+- Monitor
+    * Opentracing API
+    * Prometheus
 
 - Others:
     * start check
@@ -97,13 +109,12 @@ Finished List:
 
 Working List:
 
-- Registry: k8s
 - Metadata Center (dubbo v2.7.x)
-- Metrics: Opentracing/Promethus(dubbo v2.7.x)
+- Service Discovery (dubbo v2.7.x)
 
 You can know more about dubbo-go by its [roadmap](https://github.com/apache/dubbo-go/wiki/Roadmap).
 
-![feature](https://raw.githubusercontent.com/wiki/apache/dubbo-go/dubbo-go-arch.png)
+![feature](./doc/pic/arch/dubbo-go-arch.png)
 
 ## Document
 

README_CN.md

@@ -3,6 +3,8 @@
 [![Build Status](https://travis-ci.org/apache/dubbo-go.svg?branch=master)](https://travis-ci.org/apache/dubbo-go)
 [![codecov](https://codecov.io/gh/apache/dubbo-go/branch/master/graph/badge.svg)](https://codecov.io/gh/apache/dubbo-go)
 [![go.dev reference](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go&logoColor=white&style=flat-square)](https://pkg.go.dev/github.com/apache/dubbo-go?tab=doc)
+[![Go Report Card](https://goreportcard.com/badge/github.com/apache/dubbo-go)](https://goreportcard.com/report/github.com/apache/dubbo-go)
+![license](https://img.shields.io/badge/license-Apache--2.0-green.svg)
 
 ---
 Apache Dubbo Go 语言实现
@@ -49,16 +51,22 @@ Apache License, Version 2.0
     * Dubbo
     * Jsonrpc2.0
     * [gRPC](https://github.com/apache/dubbo-go/pull/311)
-
+    * [RESTful](https://github.com/apache/dubbo-go/pull/352)
+
+- 路由器
+    * [Condition router](https://github.com/apache/dubbo-go/pull/294)
+
 - 注册中心
     * ZooKeeper
     * [etcd v3](https://github.com/apache/dubbo-go/pull/148)
     * [nacos](https://github.com/apache/dubbo-go/pull/151)
     * [consul](https://github.com/apache/dubbo-go/pull/121)
+    * [k8s](https://github.com/apache/dubbo-go/pull/400)
 
 - 动态配置中心与服务治理配置器
     * Zookeeper
     * [apollo](https://github.com/apache/dubbo-go/pull/250)
+    * [nacos](https://github.com/apache/dubbo-go/pull/357)
 
 - 集群策略
     * Failover
@@ -84,6 +92,10 @@ Apache License, Version 2.0
 
 - 调用
     * [泛化调用](https://github.com/apache/dubbo-go/pull/122)
+
+- 监控
+    * Opentracing API
+    * Prometheus
 
 - 其他功能支持:
     * 启动时检查
@@ -95,13 +107,12 @@ Apache License, Version 2.0
 
 开发中列表:
 
-- 注册中心: k8s
 - 元数据中心 (dubbo v2.7.x)
-- Metrics: Opentracing/Promethus(dubbo v2.7.x)
+- 服务发现 (dubbo v2.7.x)
 
 你可以通过访问 [roadmap](https://github.com/apache/dubbo-go/wiki/Roadmap) 知道更多关于 dubbo-go 的信息。
 
-![feature](https://raw.githubusercontent.com/wiki/apache/dubbo-go/dubbo-go-arch.png)
+![feature](./doc/pic/arch/dubbo-go-arch.png)
 
 ## 文档
 

before_ut.bat

@@ -14,10 +14,10 @@
 ::  See the License for the specific language governing permissions and
 ::  limitations under the License.
 
-set zkJarName="zookeeper-3.4.9-fatjar.jar"
+set zkJarName=zookeeper-3.4.9-fatjar.jar
 set remoteJarUrl="https://github.com/dubbogo/resources/raw/master/zookeeper-4unitest/contrib/fatjar/%zkJarName%"
-set zkJarPath="remoting/zookeeper/zookeeper-4unittest/contrib/fatjar"
-set zkJar="%zkJarPath%/%zkJarName%"
+set zkJarPath=remoting/zookeeper/zookeeper-4unittest/contrib/fatjar
+set zkJar=%zkJarPath%/%zkJarName%
 
 if not exist "%zkJar%" (
    md %zkJarPath%

common/extension/rest_config_reader.go → common/extension/config_reader.go

@@ -18,24 +18,33 @@
 package extension
 
 import (
-	"github.com/apache/dubbo-go/config/rest/config_reader"
+	"github.com/apache/dubbo-go/config/interfaces"
 )
 
 var (
-	restConfigReaders = make(map[string]func() config_reader.RestConfigReader)
+	configReaders = make(map[string]func() interfaces.ConfigReader)
+	defaults      = make(map[string]string)
 )
 
-func SetRestConfigReader(name string, fun func() config_reader.RestConfigReader) {
-	restConfigReaders[name] = fun
+// SetConfigReaders set a creator of config reader.
+func SetConfigReaders(name string, v func() interfaces.ConfigReader) {
+	configReaders[name] = v
 }
 
-func GetSingletonRestConfigReader(name string) config_reader.RestConfigReader {
-	if name == "" {
-		name = "default"
+// GetConfigReaders get a config reader by name.
+func GetConfigReaders(name string) interfaces.ConfigReader {
+	if configReaders[name] == nil {
+		panic("config reader for " + name + " is not existing, make sure you have imported the package.")
 	}
-	if restConfigReaders[name] == nil {
-		panic("restConfigReaders for " + name + " is not existing, make sure you have import the package.")
-	}
-	return restConfigReaders[name]()
+	return configReaders[name]()
+}
+
+// SetDefaultConfigReader set {name} to default config reader for {module}
+func SetDefaultConfigReader(module, name string) {
+	defaults[module] = name
+}
 
+// GetDefaultConfigReader
+func GetDefaultConfigReader() map[string]string {
+	return defaults
 }

common/proxy/proxy.go

@@ -140,6 +140,14 @@ func (p *Proxy) Implement(v common.RPCService) {
 				inv.SetAttachments(k, value)
 			}
 
+			// add user setAttachment
+			atm := invCtx.Value("attachment")
+			if m, ok := atm.(map[string]string); ok {
+				for k, value := range m {
+					inv.SetAttachments(k, value)
+				}
+			}
+
 			result := p.invoke.Invoke(invCtx, inv)
 
 			err = result.Error()

common/yaml/yaml.go

@@ -41,10 +41,10 @@ func LoadYMLConfig(confProFile string) ([]byte, error) {
 }
 
 // unmarshalYMLConfig Load yml config byte from file , then unmarshal to object
-func UnmarshalYMLConfig(confProFile string, out interface{}) error {
+func UnmarshalYMLConfig(confProFile string, out interface{}) ([]byte, error) {
 	confFileStream, err := LoadYMLConfig(confProFile)
 	if err != nil {
-		return perrors.Errorf("ioutil.ReadFile(file:%s) = error:%v", confProFile, perrors.WithStack(err))
+		return confFileStream, perrors.Errorf("ioutil.ReadFile(file:%s) = error:%v", confProFile, perrors.WithStack(err))
 	}
-	return yaml.Unmarshal(confFileStream, out)
+	return confFileStream, yaml.Unmarshal(confFileStream, out)
 }

common/yaml/yaml_test.go

@@ -30,7 +30,8 @@ func TestUnmarshalYMLConfig(t *testing.T) {
 	conPath, err := filepath.Abs("./testdata/config.yml")
 	assert.NoError(t, err)
 	c := &Config{}
-	assert.NoError(t, UnmarshalYMLConfig(conPath, c))
+	_, err = UnmarshalYMLConfig(conPath, c)
+	assert.NoError(t, err)
 	assert.Equal(t, "strTest", c.StrTest)
 	assert.Equal(t, 11, c.IntTest)
 	assert.Equal(t, false, c.BooleanTest)
@@ -39,8 +40,10 @@ func TestUnmarshalYMLConfig(t *testing.T) {
 
 func TestUnmarshalYMLConfig_Error(t *testing.T) {
 	c := &Config{}
-	assert.Error(t, UnmarshalYMLConfig("./testdata/config", c))
-	assert.Error(t, UnmarshalYMLConfig("", c))
+	_, err := UnmarshalYMLConfig("./testdata/config", c)
+	assert.Error(t, err)
+	_, err = UnmarshalYMLConfig("", c)
+	assert.Error(t, err)
 }
 
 type Config struct {

config/base_config.go

@@ -18,6 +18,7 @@
 package config
 
 import (
+	"bytes"
 	"reflect"
 	"strconv"
 	"strings"
@@ -47,6 +48,8 @@ type BaseConfig struct {
 	fatherConfig       interface{}
 
 	MetricConfig *MetricConfig `yaml:"metrics" json:"metrics,omitempty"`
+
+	fileStream *bytes.Buffer
 }
 
 // startConfigCenter will start the config center.
@@ -361,5 +364,4 @@ func initializeStruct(t reflect.Type, v reflect.Value) {
 
 		}
 	}
-
 }

config/config_loader.go

@@ -24,9 +24,14 @@ import (
 	"time"
 )
 
+import (
+	perrors "github.com/pkg/errors"
+)
+
 import (
 	"github.com/apache/dubbo-go/common"
 	"github.com/apache/dubbo-go/common/constant"
+	"github.com/apache/dubbo-go/common/extension"
 	"github.com/apache/dubbo-go/common/logger"
 )
 
@@ -90,10 +95,19 @@ func Load() {
 	if consumerConfig == nil {
 		logger.Warnf("consumerConfig is nil!")
 	} else {
-		// init rest consumer config
-		if err := ConsumerRestConfigInit(consumerConfig.RestConfigType); err != nil {
-			log.Printf("[initConsumerRestConfig] %#v", err)
+		// init other consumer config
+		conConfigType := consumerConfig.ConfigType
+		for key, value := range extension.GetDefaultConfigReader() {
+			if conConfigType == nil {
+				if v, ok := conConfigType[key]; ok {
+					value = v
+				}
+			}
+			if err := extension.GetConfigReaders(value).ReadConsumerConfig(consumerConfig.fileStream); err != nil {
+				logger.Errorf("ReadConsumerConfig error: %#v for %s", perrors.WithStack(err), value)
+			}
 		}
+
 		metricConfig = consumerConfig.MetricConfig
 		applicationConfig = consumerConfig.ApplicationConfig
 
@@ -154,10 +168,19 @@ func Load() {
 	if providerConfig == nil {
 		logger.Warnf("providerConfig is nil!")
 	} else {
-		// init rest provider config
-		if err := ProviderRestConfigInit(providerConfig.RestConfigType); err != nil {
-			log.Printf("[initProviderRestConfig] %#v", err)
+		// init other provider config
+		proConfigType := providerConfig.ConfigType
+		for key, value := range extension.GetDefaultConfigReader() {
+			if proConfigType != nil {
+				if v, ok := proConfigType[key]; ok {
+					value = v
+				}
+			}
+			if err := extension.GetConfigReaders(value).ReadProviderConfig(providerConfig.fileStream); err != nil {
+				logger.Errorf("ReadProviderConfig error: %#v for %s", perrors.WithStack(err), value)
+			}
 		}
+
 		// so, you should know that the consumer's config will be override
 		metricConfig = providerConfig.MetricConfig
 		applicationConfig = providerConfig.ApplicationConfig

config/consumer_config.go

@@ -18,6 +18,7 @@
 package config
 
 import (
+	"bytes"
 	"time"
 )
 
@@ -58,7 +59,7 @@ type ConsumerConfig struct {
 	ProtocolConf   interface{}                 `yaml:"protocol_conf" json:"protocol_conf,omitempty" property:"protocol_conf"`
 	FilterConf     interface{}                 `yaml:"filter_conf" json:"filter_conf,omitempty" property:"filter_conf" `
 	ShutdownConfig *ShutdownConfig             `yaml:"shutdown_conf" json:"shutdown_conf,omitempty" property:"shutdown_conf" `
-	RestConfigType string                      `default:"default" yaml:"rest_config_type" json:"rest_config_type,omitempty" property:"rest_config_type"`
+	ConfigType     map[string]string           `yaml:"config_type" json:"config_type,omitempty" property:"config_type"`
 }
 
 // UnmarshalYAML ...
@@ -89,10 +90,11 @@ func ConsumerInit(confConFile string) error {
 		return perrors.Errorf("application configure(consumer) file name is nil")
 	}
 	consumerConfig = &ConsumerConfig{}
-	err := yaml.UnmarshalYMLConfig(confConFile, consumerConfig)
+	fileStream, err := yaml.UnmarshalYMLConfig(confConFile, consumerConfig)
 	if err != nil {
 		return perrors.Errorf("unmarshalYmlConfig error %v", perrors.WithStack(err))
 	}
+	consumerConfig.fileStream = bytes.NewBuffer(fileStream)
 	//set method interfaceId & interfaceName
 	for k, v := range consumerConfig.References {
 		//set id for reference
@@ -116,6 +118,7 @@ func ConsumerInit(confConFile string) error {
 		}
 	}
 	logger.Debugf("consumer config{%#v}\n", consumerConfig)
+
 	return nil
 }
 
@@ -139,5 +142,6 @@ func configCenterRefreshConsumer() error {
 			return perrors.WithMessagef(err, "time.ParseDuration(Connect_Timeout{%#v})", consumerConfig.Connect_Timeout)
 		}
 	}
-	return err
+
+	return nil
 }