Merge pull request Jigsaw-Code#78 from Jigsaw-Code/bemasc-blocksalt

Prevent replays of server data

README.md

@@ -10,7 +10,7 @@ The Outline Shadowsocks service allows for:
 - Whitebox monitoring of the service using [prometheus.io](https://prometheus.io)
   - Includes traffic measurements and other health indicators.
 - Live updates via config change + SIGHUP
-- Experimental: optional replay defense (--replay_history).
+- Replay defense (add `--replay_history 10000`).  See [PROBES](shadowsocks/PROBES.md) for details.
 
 ![Graphana Dashboard](https://user-images.githubusercontent.com/113565/44177062-419d7700-a0ba-11e8-9621-db519692ff6c.png "Graphana Dashboard")
 

server.go

@@ -139,7 +139,7 @@ func (s *ssServer) loadConfig(filename string) error {
 		if !ok {
 			return fmt.Errorf("Only AEAD ciphers are supported. Found %v", keyConfig.Cipher)
 		}
-		cipherList.PushBack(&shadowsocks.CipherEntry{ID: keyConfig.ID, Cipher: aead})
+		cipherList.PushBack(shadowsocks.MakeCipherEntry(keyConfig.ID, aead, keyConfig.Secret))
 	}
 	for port := range s.ports {
 		portChanges[port] = portChanges[port] - 1

shadowsocks/PROBES.md

@@ -0,0 +1,35 @@
+# Outline Shadowsocks Probing and Replay Defenses
+
+## Attacks
+
+To ensure that proxied connections have not been modified in transit, the Outline implementation of Shadowsocks only supports modern [AEAD cipher suites](https://shadowsocks.org/en/spec/AEAD-Ciphers.html).  This protects users from a wide range of potential attacks.  However, even with [AEAD's authenticity guarantees](https://en.wikipedia.org/wiki/Authenticated_encryption), there are still ways for an attacker to abuse the Shadowsocks protocol.
+
+One category of attacks are "probing" attacks, in which the adversary sends test data to the proxy in order to confirm that it is actually a Shadowsocks proxy.  This is a violation of the Shadowsocks security design, which is intended to ensure that only an authenticated user can identify the proxy.  For example, one [probing attack against Shadowsocks](https://scholar.google.com/scholar?cluster=8542824533765048218) sends different numbers of random bytes to a target server, and identifies how many bytes the server reads before detecting an error and closing the connection.  This number can be distinctive, identifying the server software.
+
+Another [reported](https://gfw.report/blog/gfw_shadowsocks/) category of attacks are "replay" attacks, in which an adversary records a conversation between a Shadowsocks client and server, then replays the contents of that connection.  The contents are valid Shadowsocks AEAD data, so the proxy will forward the connection to the specified destination, as usual.  In some cases, this can cause a duplicated action (e.g. uploading a file twice with HTTP POST).  However, modern secure protocols such as HTTPS are not replayable, so this will normally have no ill effect.
+
+A greater concern for Outline is the use of replays in probing attacks to identify Shadowsocks proxies.  By sending modified and unmodified replays, an attacker might be able to confirm that a server is in fact a Shadowsocks proxy, by observing distinctive behaviors.
+
+## Outline's defenses
+
+Outline contains several defenses against probing and replay attacks.
+
+### Invalid probe data
+
+If Outline detects that the initial data is invalid, it will continue to read data (exactly as if it were valid), but will not reply, and will not close the connection until a timeout.  This leaves the attacker with minimal information about the server.
+
+### Client replays
+
+When client replay protection is enabled, every incoming valid handshake is reduced to a 32-bit checksum and stored in a hash table.  When the table is full, it is archived and replaced with a fresh one, ensuring that the recent history is always in memory.  Using 32-bit checksums results in a false-positive detection rate of 1 in 4 billion for each entry in the history.  At the maximum history size (two sets of 20,000 checksums each), that results in a false-positive failure rate of 1 in 100,000 sockets ... still far lower than the error rate expected from network unreliability.
+
+This feature is on by default in Outline.  Admins who are using outline-ss-server directly can enable this feature by adding "--replay_history 10000" to their outline-ss-server invocation.  This costs approximately 20 bytes of memory per checksum.
+
+### Server replays
+
+Shadowsocks uses the same Key Derivation Function for both upstream and downstream flows, so in principle an attacker could record data sent from the server to the client, and use it in a "reflected replay" attack as simulated client->server data.  The data would appear to be valid and authenticated to the server, but the connection would most likely fail when attempting to parse the destination address header, perhaps leading to a distinctive failure behavior.
+
+To avoid this class of attacks, outline-ss-server uses an [HMAC](https://en.wikipedia.org/wiki/HMAC) with a 32-bit tag to mark all server handshakes, and checks for the presence of this tag in all incoming handshakes.  If the tag is present, the connection is a reflected replay, with a false positive probability of 1 in 4 billion.
+
+## Metrics
+
+Outline provides server operators with metrics on a variety of aspects of server activity, including any detected attacks.  To observe attacks detected by your server, look at the `tcp_probes` histogram vector in Prometheus.  The `status` field will be `"ERR_CIPHER"` (indicating invalid probe data), `"ERR_REPLAY_CLIENT"`, or `"ERR_REPLAY_SERVER"`, depending on the kind of attack your server observed.  You can also see what country each probe appeared to originate from, and approximately how many bytes were sent before giving up.

shadowsocks/cipher_list.go

@@ -27,12 +27,34 @@ import (
 // All ciphers must have a nonce size this big or smaller.
 const maxNonceSize = 12
 
+// Don't add a tag if it would reduce the salt entropy below this amount.
+const minSaltEntropy = 16
+
 // CipherEntry holds a Cipher with an identifier.
-// The public fields are constant, but lastAddress is mutable under cipherList.mu.
+// The public fields are constant, but lastClientIP is mutable under cipherList.mu.
 type CipherEntry struct {
-	ID           string
-	Cipher       shadowaead.Cipher
-	lastClientIP net.IP
+	ID            string
+	Cipher        shadowaead.Cipher
+	SaltGenerator ServerSaltGenerator
+	lastClientIP  net.IP
+}
+
+// MakeCipherEntry constructs a CipherEntry.
+func MakeCipherEntry(id string, cipher shadowaead.Cipher, secret string) CipherEntry {
+	var saltGenerator ServerSaltGenerator
+	if cipher.SaltSize()-ServerSaltMarkLen >= minSaltEntropy {
+		// Mark salts with a tag for reverse replay protection.
+		saltGenerator = NewServerSaltGenerator(secret)
+	} else {
+		// Adding a tag would leave too little randomness to protect
+		// against accidental salt reuse, so don't mark the salts.
+		saltGenerator = RandomSaltGenerator
+	}
+	return CipherEntry{
+		ID:            id,
+		Cipher:        cipher,
+		SaltGenerator: saltGenerator,
+	}
 }
 
 // CipherList is a thread-safe collection of CipherEntry elements that allows for

shadowsocks/cipher_testing.go

@@ -43,7 +43,8 @@ func MakeTestCiphers(secrets []string) (CipherList, error) {
 		if err != nil {
 			return nil, fmt.Errorf("Failed to create cipher %v: %v", i, err)
 		}
-		l.PushBack(&CipherEntry{ID: cipherID, Cipher: cipher.(shadowaead.Cipher)})
+		entry := MakeCipherEntry(cipherID, cipher.(shadowaead.Cipher), secrets[i])
+		l.PushBack(&entry)
 	}
 	cipherList := NewCipherList()
 	cipherList.Update(l)

shadowsocks/salt_generator.go

@@ -0,0 +1,125 @@
+// Copyright 2020 Jigsaw Operations LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package shadowsocks
+
+import (
+	"bytes"
+	"crypto"
+	"crypto/hmac"
+	"crypto/rand"
+	"fmt"
+	"io"
+
+	"golang.org/x/crypto/hkdf"
+)
+
+// SaltGenerator generates unique salts to use in Shadowsocks connections.
+type SaltGenerator interface {
+	// Returns a new salt
+	GetSalt(salt []byte) error
+}
+
+// ServerSaltGenerator offers the ability to check if a salt was marked as
+// server-originated.
+type ServerSaltGenerator interface {
+	SaltGenerator
+	// IsServerSalt returns true if the salt was created by this generator
+	// and is marked as server-originated.
+	IsServerSalt(salt []byte) bool
+}
+
+// randomSaltGenerator generates a new random salt.
+type randomSaltGenerator struct{}
+
+// GetSalt outputs a random salt.
+func (randomSaltGenerator) GetSalt(salt []byte) error {
+	_, err := rand.Read(salt)
+	return err
+}
+
+func (randomSaltGenerator) IsServerSalt(salt []byte) bool {
+	return false
+}
+
+// RandomSaltGenerator is a basic SaltGenerator.
+var RandomSaltGenerator ServerSaltGenerator = randomSaltGenerator{}
+
+// serverSaltGenerator generates unique salts that are secretly marked.
+type serverSaltGenerator struct {
+	key []byte
+}
+
+// ServerSaltMarkLen is the number of bytes of salt to use as a marker.
+// Increasing this value reduces the false positive rate, but increases
+// the likelihood of salt collisions.
+const ServerSaltMarkLen = 4 // Must be less than or equal to SHA1.Size()
+
+// Constant to identify this marking scheme.
+var serverSaltLabel = []byte("outline-server-salt")
+
+// NewServerSaltGenerator returns a SaltGenerator whose output is apparently
+// random, but is secretly marked as being issued by the server.
+// This is useful to prevent the server from accepting its own output in a
+// reflection attack.
+func NewServerSaltGenerator(secret string) ServerSaltGenerator {
+	// Shadowsocks already uses HKDF-SHA1 to derive the AEAD key, so we use
+	// the same derivation with a different "info" to generate our HMAC key.
+	keySource := hkdf.New(crypto.SHA1.New, []byte(secret), nil, serverSaltLabel)
+	// The key can be any size, but matching the block size is most efficient.
+	key := make([]byte, crypto.SHA1.Size())
+	io.ReadFull(keySource, key)
+	return serverSaltGenerator{key}
+}
+
+func (sg serverSaltGenerator) splitSalt(salt []byte) (prefix, mark []byte, err error) {
+	prefixLen := len(salt) - ServerSaltMarkLen
+	if prefixLen < 0 {
+		return nil, nil, fmt.Errorf("Salt is too short: %d < %d", len(salt), ServerSaltMarkLen)
+	}
+	return salt[:prefixLen], salt[prefixLen:], nil
+}
+
+// getTag takes in a salt prefix and returns the tag.
+func (sg serverSaltGenerator) getTag(prefix []byte) []byte {
+	// Use HMAC-SHA1, even though SHA1 is broken, because HMAC-SHA1 is still
+	// secure, and we're already using HKDF-SHA1.
+	hmac := hmac.New(crypto.SHA1.New, sg.key)
+	hmac.Write(prefix) // Hash.Write never returns an error.
+	return hmac.Sum(nil)
+}
+
+// GetSalt returns an apparently random salt that can be identified
+// as server-originated by anyone who knows the Shadowsocks key.
+func (sg serverSaltGenerator) GetSalt(salt []byte) error {
+	prefix, mark, err := sg.splitSalt(salt)
+	if err != nil {
+		return err
+	}
+	if _, err := rand.Read(prefix); err != nil {
+		return err
+	}
+	tag := sg.getTag(prefix)
+	copy(mark, tag)
+	return nil
+}
+
+func (sg serverSaltGenerator) IsServerSalt(salt []byte) bool {
+	prefix, mark, err := sg.splitSalt(salt)
+	if err != nil {
+		return false
+	}
+	tag := sg.getTag(prefix)
+	return bytes.Equal(tag[:ServerSaltMarkLen], mark)
+}