Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Middleware for determining the real ip of the client #682

Merged
merged 5 commits into from
Dec 13, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 95 additions & 0 deletions interceptors/realip/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
// Copyright (c) The go-grpc-middleware Authors.
// Licensed under the Apache License 2.0.

/*
Package realip is a middleware that extracts the real IP of requests based on
header values.

The real IP is subsequently placed inside the context of each request and can
be retrieved using the [FromContext] function.

The middleware is designed to work with gRPC servers serving clients over
TCP/IP connections. If no headers are found, the middleware will return the
remote peer address as the real IP. If remote peer address is not a TCP/IP
address, the middleware will return nil as the real IP.

Headers provided by clients in the request will be searched for in the order
of the list provided to the middleware. The first header that contains a valid
IP address will be used as the real IP.

Comma separated headers are supported. The last, rightmost, IP address in the
header will be used as the real IP.

# Security

There are 2 main security concerns when deriving the real IP from request
headers:

1. Risk of spoofing the real IP by setting a header value.
2. Risk of injecting a header value that causes a denial of service.

To mitigate the risk of spoofing, the middleware introduces the concept of
"trusted peers". Trusted peers are defined as a list of IP networks that are
verified by the gRPC server operator to be trusted. If the peer address is found
to be within one of the trusted networks, the middleware will attempt to extract
the real IP from the request headers. If the peer address is not found to be
within one of the trusted networks, the peer address will be returned as the
real IP.

"trusted" in this context means that the peer is configured to overwrite the
header value with the real IP. This is typically done by a proxy or load
balancer that is configured to forward the real IP of the client in a header
value. Alternatively, the peer may be configured to append the real IP to the
header value. In this case, the middleware will use the last, rightmost, IP
address in the header as the real IP. Most load balancers, such as NGINX, AWS
ELB, and Google Cloud Load Balancer, are configured to append the real IP to
the header value as their default action.

To mitigate the risk of a denial of service by proxy of a malicious header,
the middleware validates that the header value contains a valid IP address. Only
if a valid IP address is found will the middleware use that value as the real
IP.

# Individual IP addresses as trusted peers

When creating the list of trusted peers, it is possible to specify individual IP
addresses. This is useful when your proxy or load balancer has a set of
well-known addresses.

The following example shows how to specify individual IP addresses as trusted
peers:

trusted := []net.IPNet{
{IP: net.IPv4(192, 168, 0, 1), Mask: net.IPv4Mask(255, 255, 255, 255)},
{IP: net.IPv4(192, 168, 0, 2), Mask: net.IPv4Mask(255, 255, 255, 255)},
}

In the above example, the middleware will only attempt to extract the real IP
from the request headers if the peer address is either 192.168.0.1 or
192.168.0.2.

# Headers

Headers to search for are specified as a list of strings when creating the
middleware. The middleware will search for the headers in the order specified
and use the first header that contains a valid IP address as the real IP.

The following are examples of headers that may contain the real IP:

- X-Forwarded-For: This header is set by proxies and contains a comma
separated list of IP addresses. Each proxy that forwards the request will
append the real IP to the header value.
- X-Real-IP: This header is set by NGINX and contains the real IP as a string
containing a single IP address.
- Forwarded-For: Header defined by RFC7239. This header is set by proxies and
contains the real IP as a string containing a single IP address. Please note
that the obfuscated identifier from section 6.3 of RFC7239, and that the
unknown identifier from section 6.2 of RFC7239 are not supported.
- True-Client-IP: This header is set by Cloudflare and contains the real IP
as a string containing a single IP address.

# Usage

Please see examples for simple examples of use.
*/
package realip
43 changes: 43 additions & 0 deletions interceptors/realip/examples_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
// Copyright (c) The go-grpc-middleware Authors.
// Licensed under the Apache License 2.0.

package realip_test

import (
"net/netip"

"github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors/realip"
"google.golang.org/grpc"
)

// Simple example of a unary server initialization code.
func ExampleUnaryServerInterceptor() {
// Define list of trusted peers from which we accept forwarded-for and
// real-ip headers.
trustedPeers := []netip.Prefix{
netip.MustParsePrefix("127.0.0.1/32"),
}
// Define headers to look for in the incoming request.
headers := []string{realip.X_FORWARDED_FOR, realip.X_REAL_IP}
_ = grpc.NewServer(
grpc.ChainUnaryInterceptor(
realip.UnaryServerInterceptor(trustedPeers, headers),
),
)
}

// Simple example of a streaming server initialization code.
func ExampleStreamServerInterceptor() {
// Define list of trusted peers from which we accept forwarded-for and
// real-ip headers.
trustedPeers := []netip.Prefix{
netip.MustParsePrefix("127.0.0.1/32"),
}
// Define headers to look for in the incoming request.
headers := []string{realip.X_FORWARDED_FOR, realip.X_REAL_IP}
_ = grpc.NewServer(
grpc.ChainStreamInterceptor(
realip.StreamServerInterceptor(trustedPeers, headers),
),
)
}
135 changes: 135 additions & 0 deletions interceptors/realip/realip.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
// Copyright (c) The go-grpc-middleware Authors.
// Licensed under the Apache License 2.0.

package realip

import (
"context"
"net"
"net/netip"
"strings"

"google.golang.org/grpc"
"google.golang.org/grpc/metadata"
"google.golang.org/grpc/peer"
)

// X_REAL_IP, X_FORWARDED_FOR and TRUE_CLIENT_IP are header keys
// used to extract the real client IP from the request. They represent common
// conventions for identifying the originating IP address of a client connecting
// through proxies or load balancers.
const (
X_REAL_IP = "x-real-ip"
X_FORWARDED_FOR = "x-forwarded-for"
TRUE_CLIENT_IP = "true-client-ip"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for defining these, do you mind making these constants PascalCase to make it consistent with the rest of our constant styling? Also I think we should canonicalize these header keys.

Suggested change
X_REAL_IP = "x-real-ip"
X_FORWARDED_FOR = "x-forwarded-for"
TRUE_CLIENT_IP = "true-client-ip"
XRealIp = "X-Real-IP"
XForwardedFor = "X-Forwarded-For"
TrueClientIp = "True-Client-IP"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolutely - oversight on my part.

Copy link
Collaborator

@johanbrandhorst johanbrandhorst Dec 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry to bang on, but I don't see that the case of the headers constants was canonicalized?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @johanbrandhorst - I guess I must have misunderstood the meaning of canonicalising the header keys. Let me go ahead and try to fix them ;)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've changed the casing of the header keys that are the values of the constants - Is that what you meant by canonicalising the header keys?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that is what I meant. I see you had to lowercase the headers in the map, which is fine, you could also canonicalize them using https://pkg.go.dev/net/textproto#CanonicalMIMEHeaderKey, but this will do.

)

var noIP = netip.Addr{}

type realipKey struct{}

// FromContext extracts the real client IP from the context.
// It returns the IP and a boolean indicating if it was present.
func FromContext(ctx context.Context) (netip.Addr, bool) {
ip, ok := ctx.Value(realipKey{}).(netip.Addr)
return ip, ok
}

func remotePeer(ctx context.Context) net.Addr {
pr, ok := peer.FromContext(ctx)
if !ok {
return nil
}
return pr.Addr
}

func ipInNets(ip netip.Addr, nets []netip.Prefix) bool {
for _, n := range nets {
if n.Contains(ip) {
return true
}
}
return false
}

func getHeader(ctx context.Context, key string) string {
md, ok := metadata.FromIncomingContext(ctx)
if !ok {
return ""
}

if md[key] == nil {
return ""
}

return md[key][0]
}

func ipFromHeaders(ctx context.Context, headers []string) netip.Addr {
for _, header := range headers {
a := strings.Split(getHeader(ctx, header), ",")
h := strings.TrimSpace(a[len(a)-1])
ip, err := netip.ParseAddr(h)
if err == nil {
return ip
}
}
return noIP
}

func getRemoteIP(ctx context.Context, trustedPeers []netip.Prefix, headers []string) netip.Addr {
pr := remotePeer(ctx)
if pr == nil {
return noIP
}
ip, err := netip.ParseAddr(strings.Split(pr.String(), ":")[0])
if err != nil {
return noIP
}
johanbrandhorst marked this conversation as resolved.
Show resolved Hide resolved
if len(trustedPeers) == 0 || !ipInNets(ip, trustedPeers) {
return ip
}
if ip := ipFromHeaders(ctx, headers); ip != noIP {
return ip
}
// No ip from the headers, return the peer ip.
return ip
}

type serverStream struct {
grpc.ServerStream
ctx context.Context
}

func (s *serverStream) Context() context.Context {
return s.ctx
}

// UnaryServerInterceptor returns a new unary server interceptor that extracts the real client IP from request headers.
// It checks if the request comes from a trusted peer, and if so, extracts the IP from the configured headers.
// The real IP is added to the request context.
func UnaryServerInterceptor(trustedPeers []netip.Prefix, headers []string) grpc.UnaryServerInterceptor {
return func(ctx context.Context, req any, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (any, error) {
ip := getRemoteIP(ctx, trustedPeers, headers)
if ip != noIP {
ctx = context.WithValue(ctx, realipKey{}, ip)
}
return handler(ctx, req)
}
}

// StreamServerInterceptor returns a new stream server interceptor that extracts the real client IP from request headers.
// It checks if the request comes from a trusted peer, and if so, extracts the IP from the configured headers.
// The real IP is added to the request context.
func StreamServerInterceptor(trustedPeers []netip.Prefix, headers []string) grpc.StreamServerInterceptor {
return func(srv any, stream grpc.ServerStream, info *grpc.StreamServerInfo, handler grpc.StreamHandler) error {
ip := getRemoteIP(stream.Context(), trustedPeers, headers)
if ip != noIP {
return handler(srv, &serverStream{
ServerStream: stream,
ctx: context.WithValue(stream.Context(), realipKey{}, ip),
})
}
return handler(srv, stream)
}
}
Loading