Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merging to release-4-lts: TT-8934 Fix chunked response analytics (#5495) #5746

Conversation

buger
Copy link
Member

@buger buger commented Nov 7, 2023

TT-8934 Fix chunked response analytics (#5495)

Description

When transfer-encoding is chunked on the upstream response, analytics
records raw responses also contain the chunked characters (example
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding#examples).
This is not expected.

The proposed solution is to delete the transfer-encoding header before
reading and writing the response body to the rawResponse field. Without
this header, Go will process the header as it should.

Related Issue

https://tyktech.atlassian.net/browse/TT-8934

Motivation and Context

https://tyktech.atlassian.net/browse/TT-8934

How This Has Been Tested

Added tests

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing
    functionality to change)
  • Refactoring or add test (improvements in base code or adds test
    coverage to functionality)

Checklist

  • I ensured that the documentation is up to date
  • I explained why this PR updates go.mod in detail with reasoning
    why it's required
  • I would like a code coverage CI quality gate exception and have
    explained why

<!-- Provide a general summary of your changes in the Title above -->

<!-- Describe your changes in detail -->

When transfer-encoding is chunked on the upstream response, analytics
records raw responses also contain the chunked characters (example
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding#examples).
This is not expected.

The proposed solution is to delete the transfer-encoding header before
reading and writing the response body to the rawResponse field. Without
this header, Go will process the header as it should.

<!-- This project only accepts pull requests related to open issues. -->
<!-- If suggesting a new feature or change, please discuss it in an
issue first. -->
<!-- If fixing a bug, there should be an issue describing it with steps
to reproduce. -->
<!-- OSS: Please link to the issue here. Tyk: please create/link the
JIRA ticket. -->
https://tyktech.atlassian.net/browse/TT-8934

<!-- Why is this change required? What problem does it solve? -->
https://tyktech.atlassian.net/browse/TT-8934

<!-- Please describe in detail how you tested your changes -->
<!-- Include details of your testing environment, and the tests -->
<!-- you ran to see how your change affects other areas of the code,
etc. -->
<!-- This information is helpful for reviewers and QA. -->

Added tests

<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->

- [X] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Refactoring or add test (improvements in base code or adds test
coverage to functionality)

<!-- Go over all the following points, and put an `x` in all the boxes
that apply -->
<!-- If there are no documentation updates required, mark the item as
checked. -->
<!-- Raise up any additional concerns not covered by the checklist. -->

- [ ] I ensured that the documentation is up to date
- [ ] I explained why this PR updates go.mod in detail with reasoning
why it's required
- [ ] I would like a code coverage CI quality gate exception and have
explained why

(cherry picked from commit e4b0f9e)
Copy link

sweep-ai bot commented Nov 7, 2023

Apply Sweep Rules to your PR?

  • Apply: All new business logic should have corresponding unit tests.
  • Apply: Refactor large functions to be more modular.

@buger
Copy link
Member Author

buger commented Nov 7, 2023

API tests result: failure 🚫
Branch used: refs/pull/5746/merge
Commit:
Triggered by: pull_request (@buger)
Execution page

@buger
Copy link
Member Author

buger commented Nov 7, 2023

API tests result: success
Branch used: refs/heads/release-4-lts
Commit: 00d326e Merging to release-4-lts: TT-8942 Change default RPC pool size (#5030)

TT-8942 Change default RPC pool size (#5030)

Description

Default RPC pool size is set to 20 and because of the relatively heavy
CPU and Memory footprint of the RPC connections in the MDCB model this
can cause issues where either the gateways or MDCB fail on load cause by
scaling events or component failure at MDCB side.

Related Issue

https://tyktech.atlassian.net/browse/TT-8942

Motivation and Context

https://tyktech.atlassian.net/browse/TT-8942

How This Has Been Tested

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing
    functionality to change)
  • Refactoring or add test (improvements in base code or adds test
    coverage to functionality)

Checklist

  • I ensured that the documentation is up to date
  • I explained why this PR updates go.mod in detail with reasoning
    why it's required
  • I would like a code coverage CI quality gate exception and have
    explained why
    Triggered by: push (@buger)
    Execution page

@titpetric titpetric marked this pull request as ready for review November 7, 2023 07:33
@buger
Copy link
Member Author

buger commented Nov 7, 2023

API tests result: success
Branch used: refs/pull/5746/merge
Commit: 4264c08
Triggered by: pull_request (@titpetric)
Execution page

@titpetric titpetric merged commit 7dd5874 into release-4-lts Nov 7, 2023
11 of 12 checks passed
@titpetric titpetric deleted the merge/release-4-lts/e4b0f9eb55c3f3814c21506fd4eb6039b5312c1d branch November 7, 2023 07:43
@buger
Copy link
Member Author

buger commented Nov 7, 2023

API tests result: success
Branch used: refs/pull/5746/merge
Commit: 69264bc
Triggered by: pull_request (@titpetric)
Execution page

lghiur pushed a commit that referenced this pull request Jan 22, 2024
…) (#5746)

TT-8934 Fix chunked response analytics (#5495)

<!-- Provide a general summary of your changes in the Title above -->

## Description

<!-- Describe your changes in detail -->

When transfer-encoding is chunked on the upstream response, analytics
records raw responses also contain the chunked characters (example

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding#examples).
This is not expected.

The proposed solution is to delete the transfer-encoding header before
reading and writing the response body to the rawResponse field. Without
this header, Go will process the header as it should.


## Related Issue

<!-- This project only accepts pull requests related to open issues. -->
<!-- If suggesting a new feature or change, please discuss it in an
issue first. -->
<!-- If fixing a bug, there should be an issue describing it with steps
to reproduce. -->
<!-- OSS: Please link to the issue here. Tyk: please create/link the
JIRA ticket. -->
https://tyktech.atlassian.net/browse/TT-8934
## Motivation and Context

<!-- Why is this change required? What problem does it solve? -->
https://tyktech.atlassian.net/browse/TT-8934
## How This Has Been Tested

<!-- Please describe in detail how you tested your changes -->
<!-- Include details of your testing environment, and the tests -->
<!-- you ran to see how your change affects other areas of the code,
etc. -->
<!-- This information is helpful for reviewers and QA. -->

Added tests

## Screenshots (if appropriate)

## Types of changes

<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->

- [X] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Refactoring or add test (improvements in base code or adds test
coverage to functionality)

## Checklist

<!-- Go over all the following points, and put an `x` in all the boxes
that apply -->
<!-- If there are no documentation updates required, mark the item as
checked. -->
<!-- Raise up any additional concerns not covered by the checklist. -->

- [ ] I ensured that the documentation is up to date
- [ ] I explained why this PR updates go.mod in detail with reasoning
why it's required
- [ ] I would like a code coverage CI quality gate exception and have
explained why

---------

Co-authored-by: Tomas Buchaillot <tombuchaillot89@gmail.com>
Co-authored-by: Tit Petric <tit@tyk.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants