Skip to content

Commit

Permalink
Authenticated Websocket support (thrasher-corp#315)
Browse files Browse the repository at this point in the history 
* Improves subscribing by not allowing duplicates. Adds bitmex auth support

* Adds coinbase pro support. Partial BTCC support. Adds WebsocketAuthenticatedEndpointsSupported websocket feature. Adds GateIO support

* Adds Coinut support

* Moves Coinut WS types to file. Implements Gemini's secure WS endpoint

* Adds HitBTC ws authenticated support. Fixes var names

* Adds huobi and hadax authenticated websocket support

* Adds auth to okgroup (okex, okcoin). Fixes some linting

* Adds Poloniex support

* Adds ZB support

* Adds proper bitmex support

* Improves bitfinex support, improves websocket functionality definitions

* Fixes coinbasepro auth

* Tests all endpoints

* go formatting, importing, linting run

* Adds wrapper supports

* General clean up. Data race destruction

* Improves testing on all exchanges except ZB

* Fixes ZB hashing, parsing and tests

* minor nits before someone else sees them <_<

* Fixes some nits pertaining to variable usage, comments, typos and rate limiting

* Addresses nits regarding types and test responses where applicable

* fmt import

* Fixes linting issues

* No longer returns an error on failure to authenticate, just logs. Adds new AuthenticatedWebsocketAPISupport config value to allow a user to seperate auth from REST and WS. Prevents WS auth if AuthenticatedWebsocketAPISupport is false, adds additional login check 'CanUseAuthenticatedEndpoints' for when login only occurs once (not per request). Removes unnecessary time.Sleeps from code. Moves WS auth error logic to auth function so that wrappers can get involved in all the auth fun. New-fandangled shared test package, used exclusively in testing, will be the store of all the constant boilerplate things like timeout values. Moves WS test setup function to only run once when there are multiple WS endpoint tests. Cleans up some struct types

* Increases test coverage with tests for config.areAuthenticatedCredentialsValid config.CheckExchangeConfigValues, exchange.SetAPIKeys, exchange.GetAuthenticatedAPISupport, exchange_websocket.CanUseAuthenticatedEndpoitns and exchange_websocket.SetCanUseAuthenticatedEndpoints. Adds b.Websocket.SetCanUseAuthenticatedEndpoints(false) when bitfinex fails to authenticate
Fixes a typo. gofmt and goimport

* Trim Test Typos

* Reformats various websocket types. Adds more specific error messaging to config.areAuthenticatedCredentialsValid
  • Loading branch information
@gloriousCode @thrasher-
gloriousCode authored and thrasher- committed Jun 19, 2019
1 parent 67a58a1 commit 3a66e99
Show file tree
Hide file tree
Showing 94 changed files with 5,770 additions and 1,034 deletions.
115 changes: 70 additions & 45 deletions config/config.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -150,34 +150,35 @@ type NTPClientConfig struct {

// ExchangeConfig holds all the information needed for each enabled Exchange.
type ExchangeConfig struct {
Name string `json:"name"`
Enabled bool `json:"enabled"`
Verbose bool `json:"verbose"`
Websocket bool `json:"websocket"`
UseSandbox bool `json:"useSandbox"`
RESTPollingDelay time.Duration `json:"restPollingDelay"`
HTTPTimeout time.Duration `json:"httpTimeout"`
HTTPUserAgent string `json:"httpUserAgent"`
HTTPDebugging bool `json:"httpDebugging"`
AuthenticatedAPISupport bool `json:"authenticatedApiSupport"`
APIKey string `json:"apiKey"`
APISecret string `json:"apiSecret"`
APIAuthPEMKeySupport bool `json:"apiAuthPemKeySupport,omitempty"`
APIAuthPEMKey string `json:"apiAuthPemKey,omitempty"`
APIURL string `json:"apiUrl"`
APIURLSecondary string `json:"apiUrlSecondary"`
ProxyAddress string `json:"proxyAddress"`
WebsocketURL string `json:"websocketUrl"`
ClientID string `json:"clientId,omitempty"`
AvailablePairs currency.Pairs `json:"availablePairs"`
EnabledPairs currency.Pairs `json:"enabledPairs"`
BaseCurrencies currency.Currencies `json:"baseCurrencies"`
AssetTypes string `json:"assetTypes"`
SupportsAutoPairUpdates bool `json:"supportsAutoPairUpdates"`
PairsLastUpdated int64 `json:"pairsLastUpdated,omitempty"`
ConfigCurrencyPairFormat *CurrencyPairFormatConfig `json:"configCurrencyPairFormat"`
RequestCurrencyPairFormat *CurrencyPairFormatConfig `json:"requestCurrencyPairFormat"`
BankAccounts []BankAccount `json:"bankAccounts"`
Name string `json:"name"`
Enabled bool `json:"enabled"`
Verbose bool `json:"verbose"`
Websocket bool `json:"websocket"`
UseSandbox bool `json:"useSandbox"`
RESTPollingDelay time.Duration `json:"restPollingDelay"`
HTTPTimeout time.Duration `json:"httpTimeout"`
HTTPUserAgent string `json:"httpUserAgent"`
HTTPDebugging bool `json:"httpDebugging"`
AuthenticatedAPISupport bool `json:"authenticatedApiSupport"`
AuthenticatedWebsocketAPISupport bool `json:"authenticatedWebsocketApiSupport"`
APIKey string `json:"apiKey"`
APISecret string `json:"apiSecret"`
APIAuthPEMKeySupport bool `json:"apiAuthPemKeySupport,omitempty"`
APIAuthPEMKey string `json:"apiAuthPemKey,omitempty"`
APIURL string `json:"apiUrl"`
APIURLSecondary string `json:"apiUrlSecondary"`
ProxyAddress string `json:"proxyAddress"`
WebsocketURL string `json:"websocketUrl"`
ClientID string `json:"clientId,omitempty"`
AvailablePairs currency.Pairs `json:"availablePairs"`
EnabledPairs currency.Pairs `json:"enabledPairs"`
BaseCurrencies currency.Currencies `json:"baseCurrencies"`
AssetTypes string `json:"assetTypes"`
SupportsAutoPairUpdates bool `json:"supportsAutoPairUpdates"`
PairsLastUpdated int64 `json:"pairsLastUpdated,omitempty"`
ConfigCurrencyPairFormat *CurrencyPairFormatConfig `json:"configCurrencyPairFormat"`
RequestCurrencyPairFormat *CurrencyPairFormatConfig `json:"requestCurrencyPairFormat"`
BankAccounts []BankAccount `json:"bankAccounts"`
}

// BankAccount holds differing bank account details by supported funding
Expand Down Expand Up @@ -798,25 +799,18 @@ func (c *Config) CheckExchangeConfigValues() error {
if len(c.Exchanges[i].BaseCurrencies) == 0 {
return fmt.Errorf(ErrExchangeBaseCurrenciesEmpty, c.Exchanges[i].Name)
}
if c.Exchanges[i].AuthenticatedAPISupport { // non-fatal error
if c.Exchanges[i].APIKey == "" || c.Exchanges[i].APIKey == DefaultUnsetAPIKey {
c.Exchanges[i].AuthenticatedAPISupport = false
}

if (c.Exchanges[i].APISecret == "" || c.Exchanges[i].APISecret == DefaultUnsetAPISecret) &&
c.Exchanges[i].Name != "COINUT" {
c.Exchanges[i].AuthenticatedAPISupport = false
}

if (c.Exchanges[i].ClientID == "ClientID" || c.Exchanges[i].ClientID == "") &&
(c.Exchanges[i].Name == "ITBIT" || c.Exchanges[i].Name == "Bitstamp" || c.Exchanges[i].Name == "COINUT" || c.Exchanges[i].Name == "CoinbasePro") {
c.Exchanges[i].AuthenticatedAPISupport = false
}

if !c.Exchanges[i].AuthenticatedAPISupport {
log.Warnf(WarningExchangeAuthAPIDefaultOrEmptyValues, c.Exchanges[i].Name)
}
var areAuthenticatedCredentialsValid bool
if c.Exchanges[i].AuthenticatedWebsocketAPISupport || c.Exchanges[i].AuthenticatedAPISupport {
areAuthenticatedCredentialsValid = c.areAuthenticatedCredentialsValid(i)
}
if c.Exchanges[i].AuthenticatedWebsocketAPISupport {
c.Exchanges[i].AuthenticatedWebsocketAPISupport = areAuthenticatedCredentialsValid
}
if c.Exchanges[i].AuthenticatedAPISupport {
c.Exchanges[i].AuthenticatedAPISupport = areAuthenticatedCredentialsValid
}

if !c.Exchanges[i].SupportsAutoPairUpdates {
lastUpdated := common.UnixTimestampToTime(c.Exchanges[i].PairsLastUpdated)
lastUpdated = lastUpdated.AddDate(0, 0, configPairsLastUpdatedWarningThreshold)
Expand Down Expand Up @@ -877,6 +871,37 @@ func (c *Config) CheckExchangeConfigValues() error {
return nil
}

func (c *Config) areAuthenticatedCredentialsValid(i int) bool {
if c.Exchanges == nil {
log.Error("Config: Failed to check exchange authenticated credentials due to c.Exchanges not setup")
return false
}
if i < 0 || c.Exchanges == nil || len(c.Exchanges) < i {
log.Error("Config: Failed to check exchange authenticated credentials due to invalid index")
return false
}

resp := true
if c.Exchanges[i].APIKey == "" || c.Exchanges[i].APIKey == DefaultUnsetAPIKey {
resp = false
}

if (c.Exchanges[i].APISecret == "" || c.Exchanges[i].APISecret == DefaultUnsetAPISecret) &&
c.Exchanges[i].Name != "COINUT" {
resp = false
}

if (c.Exchanges[i].ClientID == "ClientID" || c.Exchanges[i].ClientID == "") &&
(c.Exchanges[i].Name == "ITBIT" || c.Exchanges[i].Name == "Bitstamp" || c.Exchanges[i].Name == "COINUT" || c.Exchanges[i].Name == "CoinbasePro") {
resp = false
}
// non-fatal error
if !resp {
log.Warnf(WarningExchangeAuthAPIDefaultOrEmptyValues, c.Exchanges[i].Name)
}
return resp
}

// CheckWebserverConfigValues checks information before webserver starts and
// returns an error if values are incorrect.
func (c *Config) CheckWebserverConfigValues() error {
Expand Down
92 changes: 92 additions & 0 deletions config/config_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -647,6 +647,7 @@ func TestUpdateExchangeConfig(t *testing.T) {
}
}

// TestCheckExchangeConfigValues logic test
func TestCheckExchangeConfigValues(t *testing.T) {
checkExchangeConfigValues := Config{}

Expand All @@ -673,12 +674,16 @@ func TestCheckExchangeConfigValues(t *testing.T) {
checkExchangeConfigValues.Exchanges[0].APIKey = "Key"
checkExchangeConfigValues.Exchanges[0].APISecret = "Secret"
checkExchangeConfigValues.Exchanges[0].AuthenticatedAPISupport = true
checkExchangeConfigValues.Exchanges[0].AuthenticatedWebsocketAPISupport = true
err = checkExchangeConfigValues.CheckExchangeConfigValues()
if err != nil {
t.Errorf(
"Test failed. checkExchangeConfigValues.CheckExchangeConfigValues Error",
)
}
if checkExchangeConfigValues.Exchanges[0].AuthenticatedWebsocketAPISupport {
t.Error("Expected AuthenticatedWebsocketAPISupport to be false from invalid API keys")
}

checkExchangeConfigValues.Exchanges[0].AuthenticatedAPISupport = true
checkExchangeConfigValues.Exchanges[0].APIKey = "TESTYTEST"
Expand All @@ -690,6 +695,24 @@ func TestCheckExchangeConfigValues(t *testing.T) {
"Test failed. checkExchangeConfigValues.CheckExchangeConfigValues Error",
)
}
if checkExchangeConfigValues.Exchanges[0].AuthenticatedAPISupport {
t.Error("Expected AuthenticatedAPISupport to be true from valid API keys")
}

checkExchangeConfigValues.Exchanges[0].AuthenticatedAPISupport = true
checkExchangeConfigValues.Exchanges[0].AuthenticatedWebsocketAPISupport = true
checkExchangeConfigValues.Exchanges[0].APIKey = ""
checkExchangeConfigValues.Exchanges[0].APISecret = ""
checkExchangeConfigValues.Exchanges[0].Name = "ITBIT"
err = checkExchangeConfigValues.CheckExchangeConfigValues()
if err != nil {
t.Errorf(
"Test failed. checkExchangeConfigValues.CheckExchangeConfigValues Error",
)
}
if checkExchangeConfigValues.Exchanges[0].AuthenticatedAPISupport || checkExchangeConfigValues.Exchanges[0].AuthenticatedWebsocketAPISupport {
t.Error("Expected AuthenticatedAPISupport and AuthenticatedWebsocketAPISupport to be false from invalid API keys")
}

checkExchangeConfigValues.Exchanges[0].BaseCurrencies = currency.NewCurrenciesFromStringArray([]string{""})
err = checkExchangeConfigValues.CheckExchangeConfigValues()
Expand Down Expand Up @@ -1027,3 +1050,72 @@ func TestCheckNTPConfig(t *testing.T) {
t.Error("ntpclient with nil allowednegativedifference should default to sane value")
}
}

// TestAreAuthenticatedCredentialsValid logic test
func TestAreAuthenticatedCredentialsValid(t *testing.T) {
var c Config
resp := c.areAuthenticatedCredentialsValid(0)
if resp {
t.Error("Expecting false with no exchanges loaded")
}
resp = c.areAuthenticatedCredentialsValid(-1)
if resp {
t.Error("Expecting false with an invalid index")
}

c.Exchanges = []ExchangeConfig{
{
APIKey: "",
APISecret: "",
ClientID: "",
Name: "",
},
}
resp = c.areAuthenticatedCredentialsValid(0)
if resp {
t.Error("Expecting false with no credentials set")
}

c.Exchanges[0].APIKey = DefaultUnsetAPIKey
c.Exchanges[0].APISecret = DefaultUnsetAPISecret
resp = c.areAuthenticatedCredentialsValid(0)
if resp {
t.Error("Expecting false with default credentials set")
}

c.Exchanges[0].Name = "COINUT"
resp = c.areAuthenticatedCredentialsValid(0)
if resp {
t.Error("Expecting false with COINUT and no APIKEY set")
}
c.Exchanges[0].APIKey = "Im a key!"
c.Exchanges[0].ClientID = "Im a Client!"
resp = c.areAuthenticatedCredentialsValid(0)
if !resp {
t.Error("Expecting true with COINUT api credentials set")
}

c.Exchanges[0].Name = "Bitstamp"
resp = c.areAuthenticatedCredentialsValid(0)
if resp {
t.Error("Expecting false with Bitstamp and no APISecret set")
}
c.Exchanges[0].APISecret = "Im a Secret!"
resp = c.areAuthenticatedCredentialsValid(0)
if !resp {
t.Error("Expecting true with Bitstamp api credentials set")
}

c.Exchanges[0].Name = "ANX"
c.Exchanges[0].APIKey = DefaultUnsetAPIKey
c.Exchanges[0].APISecret = "Im a Secret!"
resp = c.areAuthenticatedCredentialsValid(0)
if resp {
t.Error("Expecting false with ANX and no APIKey set")
}

resp = c.areAuthenticatedCredentialsValid(1337)
if resp {
t.Error("Expecting false with an invalid index")
}
}
11 changes: 11 additions & 0 deletions config_example.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,7 @@
"httpUserAgent": "",
"httpDebugging": false,
"authenticatedApiSupport": false,
"authenticatedWebsocketApiSupport": false,
"apiKey": "Key",
"apiSecret": "Secret",
"apiUrl": "NON_DEFAULT_HTTP_LINK_TO_EXCHANGE_API",
Expand Down Expand Up @@ -379,6 +380,7 @@
"httpUserAgent": "",
"httpDebugging": false,
"authenticatedApiSupport": false,
"authenticatedWebsocketApiSupport": false,
"apiKey": "Key",
"apiSecret": "Secret",
"apiUrl": "NON_DEFAULT_HTTP_LINK_TO_EXCHANGE_API",
Expand Down Expand Up @@ -669,6 +671,7 @@
"httpUserAgent": "",
"httpDebugging": false,
"authenticatedApiSupport": false,
"authenticatedWebsocketApiSupport": false,
"apiKey": "Key",
"apiSecret": "Secret",
"apiUrl": "NON_DEFAULT_HTTP_LINK_TO_EXCHANGE_API",
Expand Down Expand Up @@ -711,6 +714,7 @@
"httpUserAgent": "",
"httpDebugging": false,
"authenticatedApiSupport": false,
"authenticatedWebsocketApiSupport": false,
"apiKey": "Key",
"apiSecret": "Secret",
"apiUrl": "NON_DEFAULT_HTTP_LINK_TO_EXCHANGE_API",
Expand Down Expand Up @@ -753,6 +757,7 @@
"httpUserAgent": "",
"httpDebugging": false,
"authenticatedApiSupport": false,
"authenticatedWebsocketApiSupport": false,
"apiKey": "Key",
"apiSecret": "Secret",
"apiUrl": "NON_DEFAULT_HTTP_LINK_TO_EXCHANGE_API",
Expand Down Expand Up @@ -793,6 +798,7 @@
"httpUserAgent": "",
"httpDebugging": false,
"authenticatedApiSupport": false,
"authenticatedWebsocketApiSupport": false,
"apiKey": "Key",
"apiSecret": "Secret",
"apiUrl": "NON_DEFAULT_HTTP_LINK_TO_EXCHANGE_API",
Expand Down Expand Up @@ -834,6 +840,7 @@
"httpUserAgent": "",
"httpDebugging": false,
"authenticatedApiSupport": false,
"authenticatedWebsocketApiSupport": false,
"apiKey": "Key",
"apiSecret": "Secret",
"apiAuthPemKey": "-----BEGIN EC PRIVATE KEY-----\nJUSTADUMMY\n-----END EC PRIVATE KEY-----\n",
Expand Down Expand Up @@ -876,6 +883,7 @@
"httpUserAgent": "",
"httpDebugging": false,
"authenticatedApiSupport": false,
"authenticatedWebsocketApiSupport": false,
"apiKey": "Key",
"apiSecret": "Secret",
"apiAuthPemKey": "-----BEGIN EC PRIVATE KEY-----\nJUSTADUMMY\n-----END EC PRIVATE KEY-----\n",
Expand Down Expand Up @@ -1082,6 +1090,7 @@
"httpUserAgent": "",
"httpDebugging": false,
"authenticatedApiSupport": false,
"authenticatedWebsocketApiSupport": false,
"apiKey": "Key",
"apiSecret": "Secret",
"apiUrl": "NON_DEFAULT_HTTP_LINK_TO_EXCHANGE_API",
Expand Down Expand Up @@ -1124,6 +1133,7 @@
"httpUserAgent": "",
"httpDebugging": false,
"authenticatedApiSupport": false,
"authenticatedWebsocketApiSupport": false,
"apiKey": "Key",
"apiSecret": "Secret",
"apiUrl": "NON_DEFAULT_HTTP_LINK_TO_EXCHANGE_API",
Expand Down Expand Up @@ -1166,6 +1176,7 @@
"httpUserAgent": "",
"httpDebugging": false,
"authenticatedApiSupport": false,
"authenticatedWebsocketApiSupport": false,
"apiKey": "Key",
"apiSecret": "Secret",
"apiUrl": "NON_DEFAULT_HTTP_LINK_TO_EXCHANGE_API",
Expand Down
10 changes: 10 additions & 0 deletions exchanges/alphapoint/alphapoint_wrapper.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -329,3 +329,13 @@ func (a *Alphapoint) SubscribeToWebsocketChannels(channels []exchange.WebsocketC
func (a *Alphapoint) UnsubscribeToWebsocketChannels(channels []exchange.WebsocketChannelSubscription) error {
return common.ErrFunctionNotSupported
}

// GetSubscriptions returns a copied list of subscriptions
func (a *Alphapoint) GetSubscriptions() ([]exchange.WebsocketChannelSubscription, error) {
return nil, common.ErrFunctionNotSupported
}

// AuthenticateWebsocket sends an authentication message to the websocket
func (a *Alphapoint) AuthenticateWebsocket() error {
return common.ErrFunctionNotSupported
}
10 changes: 10 additions & 0 deletions exchanges/anx/anx_wrapper.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -457,3 +457,13 @@ func (a *ANX) SubscribeToWebsocketChannels(channels []exchange.WebsocketChannelS
func (a *ANX) UnsubscribeToWebsocketChannels(channels []exchange.WebsocketChannelSubscription) error {
return common.ErrFunctionNotSupported
}

// GetSubscriptions returns a copied list of subscriptions
func (a *ANX) GetSubscriptions() ([]exchange.WebsocketChannelSubscription, error) {
return nil, common.ErrFunctionNotSupported
}

// AuthenticateWebsocket sends an authentication message to the websocket
func (a *ANX) AuthenticateWebsocket() error {
return common.ErrFunctionNotSupported
}
10 changes: 10 additions & 0 deletions exchanges/binance/binance_wrapper.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -425,3 +425,13 @@ func (b *Binance) SubscribeToWebsocketChannels(channels []exchange.WebsocketChan
func (b *Binance) UnsubscribeToWebsocketChannels(channels []exchange.WebsocketChannelSubscription) error {
return common.ErrFunctionNotSupported
}

// GetSubscriptions returns a copied list of subscriptions
func (b *Binance) GetSubscriptions() ([]exchange.WebsocketChannelSubscription, error) {
return b.Websocket.GetSubscriptions(), nil
}

// AuthenticateWebsocket sends an authentication message to the websocket
func (b *Binance) AuthenticateWebsocket() error {
return common.ErrFunctionNotSupported
}
Loading

0 comments on commit 3a66e99

Please sign in to comment.