Bug: HTTP API with router event handler fails on whitespace in path parameter

[sthuber90]

Expected Behaviour

Calling GET /pets/{name} or GET /pets/{name}/vets should return a successful response with status code 200 when called with "Chester Mall" or "Chester%20Mall" (without ") as name. API Gateway will already decode encoded whitespaces as you can see in the example events.

GET /pets/Fiffi

  {
      "version": "2.0",
      "routeKey": "GET /pets/{name}",
      "rawPath": "/pets/Fiffi",
      "rawQueryString": "",
      "headers": {
          "accept": "*/*",
          "accept-encoding": "gzip, deflate, br",
          "cache-control": "no-cache",
          "content-length": "0",
          "x-forwarded-port": "443",
          "x-forwarded-proto": "https"
      },
      "requestContext": {
          "http": {
              "method": "GET",
              "path": "/pets/Fiffi",
              "protocol": "HTTP/1.1"
          },
          "requestId": "QQCtFipSIAMEPNQ=",
          "routeKey": "GET /pets/{name}",
          "stage": "$default",
          "time": "08/Apr/2022:07:31:05 +0000",
          "timeEpoch": 1649403065999
      },
      "pathParameters": {
          "name": "Fiffi"
      },
      "isBase64Encoded": false
  }

GET /pets/Fiffi/vets

  {
          "version": "2.0",
          "routeKey": "GET /pets/{name}/vets",
          "rawPath": "/pets/Fiffi/vets",
          "rawQueryString": "",
          "headers": {
              "accept": "*/*",
              "accept-encoding": "gzip, deflate, br",
              "cache-control": "no-cache",
              "content-length": "0",
              "x-forwarded-port": "443",
              "x-forwarded-proto": "https"
          },
          "requestContext": {
              "http": {
                  "method": "GET",
                  "path": "/pets/Fiffi/vets",
                  "protocol": "HTTP/1.1"
              },
              "requestId": "QQGmWiEcoAMEPqA=",
              "routeKey": "GET /pets/{name}/vets",
              "stage": "$default",
              "time": "08/Apr/2022:07:57:41 +0000",
              "timeEpoch": 1649404661221
          },
          "pathParameters": {
              "name": "Fiffi"
          },
          "isBase64Encoded": false
      }

GET /pets/Chester Mall

{
      "version": "2.0",
      "routeKey": "GET /pets/{name}",
      "rawPath": "/pets/Chester Mall",
      "rawQueryString": "",
      "headers": {
          "accept": "*/*",
          "accept-encoding": "gzip, deflate, br",
          "cache-control": "no-cache",
          "content-length": "0",
          "x-forwarded-port": "443",
          "x-forwarded-proto": "https"
      },
      "requestContext": {
          "http": {
              "method": "GET",
              "path": "/pets/Chester Mall",
              "protocol": "HTTP/1.1"
          },
          "requestId": "QQCu1ioiIAMEPlg=",
          "routeKey": "GET /pets/{name}",
          "stage": "$default",
          "time": "08/Apr/2022:07:31:17 +0000",
          "timeEpoch": 1649403077140
      },
      "pathParameters": {
          "name": "Chester Mall"
      },
      "isBase64Encoded": false
  }

GET /pets/Chester Mall/vets

 {
      "version": "2.0",
      "routeKey": "GET /pets/{name}/vets",
      "rawPath": "/pets/Chester Mall/vets",
      "rawQueryString": "",
      "headers": {
          "accept": "*/*",
          "accept-encoding": "gzip, deflate, br",
          "cache-control": "no-cache",
          "content-length": "0"
          "x-forwarded-port": "443",
          "x-forwarded-proto": "https"
      },
      "requestContext": {
          "http": {
              "method": "GET",
              "path": "/pets/Chester Mall/vets",
              "protocol": "HTTP/1.1"
          },
          "requestId": "QQGlpij8oAMEPig=",
          "routeKey": "GET /pets/{name}/vets",
          "stage": "$default",
          "time": "08/Apr/2022:07:57:36 +0000",
          "timeEpoch": 1649404656715
      },
      "pathParameters": {
          "name": "Chester Mall"
      },
      "isBase64Encoded": false
  }

Current Behaviour

Instead of a successful response a 404 "Not Found" is returned, with response body

{
    "statusCode": 404,
    "message": "Not found"
}

Code snippet

import aws_lambda_powertools.event_handler.api_gateway
from aws_lambda_powertools import Logger
from aws_lambda_powertools.event_handler import APIGatewayHttpResolver
from aws_lambda_powertools.logging import correlation_paths

logger = Logger()
app = APIGatewayHttpResolver()

# Add to code to handle whitespaces in path parameters
# aws_lambda_powertools.event_handler.api_gateway._UNSAFE_URI = "%<> \[\]{}|^"
# aws_lambda_powertools.event_handler.api_gateway._NAMED_GROUP_BOUNDARY_PATTERN = rf"(?P\1[{aws_lambda_powertools.event_handler.api_gateway._SAFE_URI}{aws_lambda_powertools.event_handler.api_gateway._UNSAFE_URI}\\w]+)"


@app.get("/pets/<name>")
def get_petname(name):
    return {"name": name}


@app.get("/pets/<name>/vets")
def get_petname(name):
    return {"pet_name": name, "vets": []}


@logger.inject_lambda_context(
    correlation_id_path=correlation_paths.API_GATEWAY_HTTP, log_event=True
)
def lambda_handler(event, context):
    return app.resolve(event, context)

Possible Solution

Change the _UNSAFE_URI constant in aws_lambda_powertools/event_handler/api_gateway from:

_UNSAFE_URI = "%<>\[\]{}|^"

to

# whitespace between <> and \[\]
_UNSAFE_URI = "%<> \[\]{}|^"

Steps to Reproduce

1. Deploy reproducible example (https://github.com/sthuber90/aws-python-http-api-project)
2. Call /pets/{name} and /pets/{name}/vets endpoint with different names

AWS Lambda Powertools for Python version

latest (layer version 16)

AWS Lambda function runtime

{"label"=>"3.9"}

Packaging format used

{"label"=>"Lambda Layers"}

Debugging logs

timestamp,message
1649404657592,"START RequestId: 46aafc05-0539-4035-83cb-cbd4268db189 Version: $LATEST
"
1649404657592,"2022-04-08 07:57:37,592 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Adding route using rule /pets/<name> and methods: GET
"
1649404657593,"2022-04-08 07:57:37,592 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Adding route using rule /pets/<name>/vets and methods: GET
"
1649404657593,"2022-04-08 07:57:37,593 aws_lambda_powertools.logging.logger [DEBUG] Decorator called with parameters
"
1649404657596,"2022-04-08 07:57:37,595 aws_lambda_powertools.logging.logger [DEBUG] Event received
"
1649404657596,"{""level"":""INFO"",""location"":""decorate:352"",""message"":{""version"":""2.0"",""routeKey"":""GET /pets/{name}/vets"",""rawPath"":""/pets/Chester Mall/vets"",""rawQueryString"":"""",""headers"":{""accept"":""*/*"",""accept-encoding"":""gzip, deflate, br"",""cache-control"":""no-cache"",""content-length"":""0"",""host"":""938e36howc.execute-api.us-east-1.amazonaws.com"",""postman-token"":""d44c4268-96ea-4e76-a308-ac9f2b0eb952"",""user-agent"":""PostmanRuntime/7.29.0"",""x-amzn-trace-id"":""Root=1-624feaf0-4b992fda408853251e402d93"",""x-forwarded-for"":""95.208.248.101"",""x-forwarded-port"":""443"",""x-forwarded-proto"":""https""},""requestContext"":{""http"":{""method"":""GET"",""path"":""/pets/Chester Mall/vets"",""protocol"":""HTTP/1.1"",""userAgent"":""PostmanRuntime/7.29.0""},""requestId"":""QQGlpij8oAMEPig="",""routeKey"":""GET /pets/{name}/vets"",""stage"":""$default"",""time"":""08/Apr/2022:07:57:36 +0000"",""timeEpoch"":1649404656715},""pathParameters"":{""name"":""Chester Mall""},""isBase64Encoded"":false},""timestamp"":""2022-04-08 07:57:37,596+0000"",""service"":""service_undefined"",""cold_start"":true,""function_name"":""aws-python-http-api-project-dev-hello"",""function_memory_size"":""1024"",""function_arn"":""arn:aws:lambda:us-east-1:373270804851:function:aws-python-http-api-project-dev-hello"",""function_request_id"":""46aafc05-0539-4035-83cb-cbd4268db189"",""correlation_id"":""QQGlpij8oAMEPig="",""xray_trace_id"":""1-624feaf0-1f7281e9141a9e970179de6f""}
"
1649404657596,"2022-04-08 07:57:37,596 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Converting event to API Gateway HTTP API contract
"
1649404657596,"2022-04-08 07:57:37,596 aws_lambda_powertools.event_handler.api_gateway [DEBUG] No match found for path /pets/Chester Mall/vets and method GET
"
1649404657597,"END RequestId: 46aafc05-0539-4035-83cb-cbd4268db189
"
1649404657597,"REPORT RequestId: 46aafc05-0539-4035-83cb-cbd4268db189	Duration: 2.12 ms	Billed Duration: 3 ms	Memory Size: 1024 MB	Max Memory Used: 57 MB	Init Duration: 531.20 ms	
"
1649404661256,"START RequestId: 23baf67a-158c-4246-a950-4eea00efba7d Version: $LATEST
"
1649404661259,"2022-04-08 07:57:41,259 aws_lambda_powertools.logging.logger [DEBUG] Event received
"
1649404661259,"{""level"":""INFO"",""location"":""decorate:352"",""message"":{""version"":""2.0"",""routeKey"":""GET /pets/{name}/vets"",""rawPath"":""/pets/Fiffi/vets"",""rawQueryString"":"""",""headers"":{""accept"":""*/*"",""accept-encoding"":""gzip, deflate, br"",""cache-control"":""no-cache"",""content-length"":""0"",""host"":""938e36howc.execute-api.us-east-1.amazonaws.com"",""postman-token"":""7d1e6998-1851-44fc-b029-279c7cbce941"",""user-agent"":""PostmanRuntime/7.29.0"",""x-amzn-trace-id"":""Root=1-624feaf5-1cf0e266526a59a700b13d68"",""x-forwarded-for"":""95.208.248.101"",""x-forwarded-port"":""443"",""x-forwarded-proto"":""https""},""requestContext"":{""http"":{""method"":""GET"",""path"":""/pets/Fiffi/vets"",""protocol"":""HTTP/1.1"",""userAgent"":""PostmanRuntime/7.29.0""},""requestId"":""QQGmWiEcoAMEPqA="",""routeKey"":""GET /pets/{name}/vets"",""stage"":""$default"",""time"":""08/Apr/2022:07:57:41 +0000"",""timeEpoch"":1649404661221},""pathParameters"":{""name"":""Fiffi""},""isBase64Encoded"":false},""timestamp"":""2022-04-08 07:57:41,259+0000"",""service"":""service_undefined"",""cold_start"":false,""function_name"":""aws-python-http-api-project-dev-hello"",""function_memory_size"":""1024"",""function_arn"":""arn:aws:lambda:us-east-1:373270804851:function:aws-python-http-api-project-dev-hello"",""function_request_id"":""23baf67a-158c-4246-a950-4eea00efba7d"",""correlation_id"":""QQGmWiEcoAMEPqA="",""xray_trace_id"":""1-624feaf5-44c0297c56144d6e0eb23a2f""}
"
1649404661260,"2022-04-08 07:57:41,260 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Converting event to API Gateway HTTP API contract
"
1649404661260,"2022-04-08 07:57:41,260 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Found a registered route. Calling function
"
1649404661260,"2022-04-08 07:57:41,260 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Simple response detected, serializing return before constructing final response
"
1649404661260,"END RequestId: 23baf67a-158c-4246-a950-4eea00efba7d
"
1649404661260,"REPORT RequestId: 23baf67a-158c-4246-a950-4eea00efba7d	Duration: 1.85 ms	Billed Duration: 2 ms	Memory Size: 1024 MB	Max Memory Used: 57 MB	
"

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.

[sthulb]

I'll look into this today.

[michaelbrewer]

Awesome thanks for the detailed logs and examples. Should be easy enough to make a patch for this.

[sthuber90]

@michaelbrewer PR was already merged by @heitorlessa. I'll leave it up to you to guys to close the issue. Hope that's alright

[michaelbrewer]

haha, @sthuber90 i just woke up and was catching up. Sorry about the bug and thanks for fixing it! When i originally created the handler i missed these test cases.

[sthuber90]

No worries. Thank you for this great project and the awesome support ❤️

[sthulb]

@sthuber90 thanks for the contribution.

@michaelbrewer thanks for trying to shepherd this, but the project maintainers have already validated it and merged it