Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[APPSEC-7946] Update the client IP extraction resolution base on the latest RFC. #2665

Merged
merged 1 commit into from
Mar 7, 2023

Conversation

GustavoCaso
Copy link
Member

@GustavoCaso GustavoCaso commented Mar 6, 2023

What does this PR do?

  • Update the list of HTTP headers to look for IP information
  • Add support to parsing HTTP IP Headers with multiple IP addresses
  • Update specs
  • Change Tracing::ClientIp module methods visibility
  • Create Datadog::Core::Utils::Network module. Main functionality is parsing and extraction IP information for HTTP request headers
  • Update rbs files

I had to vendor the code from IPAddr default gem into the re; Because the ruby version below 2.5 does not have the methods: private?, link_local?, and loopback?

That code should be temporary until we deprecated those old ruby versions

Motivation

Fixing a bug found with the IP extraction resolution logic. Not supporting an HTTP header with multiple IPs MDN

Additional Notes

How to test the change?

CI

@github-actions github-actions bot added appsec Application Security monitoring product integrations Involves tracing integrations tracing labels Mar 6, 2023
@GustavoCaso GustavoCaso changed the title Update the client IP extraction resolution base on the latest RFC. [APPSEC-7946] Update the client IP extraction resolution base on the latest RFC. Mar 6, 2023
@github-actions github-actions bot added the core Involves Datadog core libraries label Mar 6, 2023
@GustavoCaso GustavoCaso marked this pull request as ready for review March 6, 2023 12:42
@GustavoCaso GustavoCaso requested review from a team and lloeki March 6, 2023 12:42
@codecov-commenter
Copy link

Codecov Report

Merging #2665 (f062d9a) into master (989b271) will decrease coverage by 0.01%.
The diff coverage is 96.42%.

@@            Coverage Diff             @@
##           master    #2665      +/-   ##
==========================================
- Coverage   98.07%   98.06%   -0.01%     
==========================================
  Files        1166     1168       +2     
  Lines       63832    63872      +40     
  Branches     2849     2856       +7     
==========================================
+ Hits        62605    62638      +33     
- Misses       1227     1234       +7     
Impacted Files Coverage Δ
lib/datadog/core/utils/network.rb 90.00% <90.00%> (ø)
lib/datadog/appsec/contrib/rack/gateway/request.rb 93.33% <100.00%> (+1.49%) ⬆️
lib/datadog/tracing/client_ip.rb 100.00% <100.00%> (ø)
spec/datadog/core/utils/network_spec.rb 100.00% <100.00%> (ø)
spec/datadog/tracing/client_ip_spec.rb 100.00% <100.00%> (ø)
...dog/profiling/collectors/cpu_and_wall_time_spec.rb 97.58% <0.00%> (-0.41%) ⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

lib/datadog/core/utils/network.rb Outdated Show resolved Hide resolved
def set_client_ip_tag(span, headers: nil, remote_ip: nil)
return unless configuration.enabled

set_client_ip_tag!(span, headers: headers, remote_ip: remote_ip)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to check on something internally for this.

Copy link
Member

@ivoanjo ivoanjo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few notes :)

lib/datadog/core/vendor/ipaddr.rb Outdated Show resolved Hide resolved
lib/datadog/tracing/client_ip.rb Show resolved Hide resolved
lib/datadog/tracing/client_ip.rb Outdated Show resolved Hide resolved
sig/datadog/tracing/client_ip.rbs Outdated Show resolved Hide resolved
lib/datadog/core/utils/network.rb Outdated Show resolved Hide resolved
lib/datadog/core/vendor/ipaddr.rb Outdated Show resolved Hide resolved
spec/datadog/core/utils/network_spec.rb Outdated Show resolved Hide resolved
spec/datadog/tracing/client_ip_spec.rb Outdated Show resolved Hide resolved
Comment on lines +5 to +9
def self.from_hash: (untyped hash) -> HashHeaderCollection
end

class HashHeaderCollection < HeaderCollection
def initialize: (untyped hash) -> void
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: I guess the hash could be ::Hash? Although any duck type of Hash that provides each_pair would work as well I guess :) (I don't know if rbs provides such duck types or not)

Copy link
Contributor

@lloeki lloeki Mar 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any duck type of Hash that provides each_pair would work as well I guess :) (I don't know if rbs provides such duck types or not)

RBS provides interfaces! I have yet to make use of those.

sig/datadog/core/utils/network.rbs Outdated Show resolved Hide resolved
Copy link
Member

@ivoanjo ivoanjo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 LGTM

lib/datadog/core/utils/network.rb Outdated Show resolved Hide resolved
sig/datadog/core/utils/network.rbs Outdated Show resolved Hide resolved
@GustavoCaso GustavoCaso force-pushed the appsec-client-ip-extraction branch 3 times, most recently from f2be200 to fc3877b Compare March 7, 2023 15:21
- Update the list of HTTP headers to look for IP
- Add support to parsing HTTP IP Headers with multiple IP addresses
- Update specs
- Change Tracing::ClientIp module methods visibility
@GustavoCaso GustavoCaso merged commit 5dfea5e into master Mar 7, 2023
@GustavoCaso GustavoCaso deleted the appsec-client-ip-extraction branch March 7, 2023 16:00
@github-actions github-actions bot added this to the 1.11.0 milestone Mar 7, 2023
ivoanjo added a commit that referenced this pull request Mar 7, 2023
…:AddressFamilyError"

**What does this PR do?**:

This PR fixes a bug in the recently-introduced `IPAddr` module.

During review of #2665 the `IPAddr` module was introduced, and while
experimenting with the steep type checker, I noticed there was a
collision between our `IPAddr` module and the `IPAddr` module where
we were looking for execptions.

I'm not sure if this would happen in production, since we always
created an `IPAddr` instance with an IP, never a specific `family`,
but let's fix this anyway.

**Motivation**:

Less bugs!

**Additional Notes**:

(N/A)

**How to test the change?**:

This module is a backport of `IPAddr` to older Rubies. Previously,
we only tested it indirectly.

I added a minimal test to validate this change.
@lloeki lloeki modified the milestones: 1.11.0, 1.10.1 Mar 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
appsec Application Security monitoring product core Involves Datadog core libraries integrations Involves tracing integrations tracing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants