Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OpenSearch Integration #2940

Merged
merged 3 commits into from
Jul 7, 2023
Merged

OpenSearch Integration #2940

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3e80a38
opensearch integration
sarahchen6 Jul 5, 2023
aa29b35
adjusted comment.
sarahchen6 Jul 6, 2023
b5b8dfa
adjusted based on David's comments.
sarahchen6 Jul 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ test_containers:
GRPC_RUBY_BUILD_PROCS: 6
DD_INSTRUMENTATION_TELEMETRY_ENABLED: false
TEST_REDIS_OLD_HOST: redis_old
TEST_OPENSEARCH_HOST: opensearch
TEST_OPENSEARCH_PORT: 9200
DD_AGENT_HOST: testagent
DD_TRACE_AGENT_PORT: 9126
DDTRACE_CI: true
Expand Down Expand Up @@ -69,6 +71,14 @@ test_containers:
- MYSQL_PASSWORD=mysql
- MYSQL_USER=mysql
- &mysql_port 3306
- &opensearch_host opensearch
- &container_opensearch
image: opensearchproject/opensearch:2.8.0
name: *opensearch_host
environment:
- discovery.type=single-node
- DISABLE_SECURITY_PLUGIN=true
- &opensearch_port 9200
- &container_elasticsearch
image: elasticsearch:8.1.3
environment:
Expand Down Expand Up @@ -284,6 +294,7 @@ orbs:
- *container_postgres
- *container_presto
- *container_mysql
- *container_opensearch
- *container_elasticsearch
- *container_redis
- *container_redis_old
Expand All @@ -310,6 +321,9 @@ orbs:
port: *presto_port
- docker-wait:
port: *mysql_port
- docker-wait:
host: *opensearch_host
port: *opensearch_port
- docker-wait:
port: *elasticsearch_port
- docker-wait:
Expand Down
2 changes: 2 additions & 0 deletions .env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ TEST_MYSQL_PASSWORD=mysql
TEST_MYSQL_PORT=3306
TEST_MYSQL_ROOT_PASSWORD=root
TEST_MYSQL_USER=mysql
TEST_OPENSEARCH_HOST=127.0.0.1
TEST_OPENSEARCH_PORT=9201
TEST_POSTGRES_DB=postgres
TEST_POSTGRES_HOST=127.0.0.1
TEST_POSTGRES_PASSWORD=postgres
Expand Down
5 changes: 5 additions & 0 deletions Appraisals
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -679,6 +679,7 @@ elsif ruby_version?('2.4')
gem 'mongo', '>= 2.8.0', '< 2.15.0' # TODO: FIX TEST BREAKAGES ON >= 2.15 https://github.com/DataDog/dd-trace-rb/issues/1596
gem 'minitest', '>= 5.0.0'
gem 'mysql2', '< 0.5'
gem 'opensearch-ruby'
gem 'pg', '>= 0.18.4'
gem 'racecar', '>= 0.3.5'
gem 'rack'
Expand Down Expand Up @@ -948,6 +949,7 @@ elsif ruby_version?('2.5')
gem 'mongo', '>= 2.8.0', '< 2.15.0' # TODO: FIX TEST BREAKAGES ON >= 2.15 https://github.com/DataDog/dd-trace-rb/issues/1596
gem 'mysql2', '< 1', platform: :ruby
gem 'activerecord-jdbcmysql-adapter', '>= 60.2', platform: :jruby
gem 'opensearch-ruby'
gem 'pg', '>= 0.18.4', platform: :ruby
gem 'activerecord-jdbcpostgresql-adapter', '>= 60.2', platform: :jruby
gem 'racecar', '>= 0.3.5'
Expand Down Expand Up @@ -1199,6 +1201,7 @@ elsif ruby_version?('2.6')
gem 'mongo', '>= 2.8.0', '< 2.15.0' # TODO: FIX TEST BREAKAGES ON >= 2.15 https://github.com/DataDog/dd-trace-rb/issues/1596
gem 'mysql2', '< 1', platform: :ruby
gem 'activerecord-jdbcmysql-adapter', platform: :jruby
gem 'opensearch-ruby'
gem 'pg', '>= 0.18.4', platform: :ruby
gem 'activerecord-jdbcpostgresql-adapter', platform: :jruby
gem 'racecar', '>= 0.3.5'
Expand Down Expand Up @@ -1435,6 +1438,7 @@ elsif ruby_version?('2.7')
gem 'minitest', '>= 5.0.0'
gem 'mongo', '>= 2.8.0', '< 2.15.0' # TODO: FIX TEST BREAKAGES ON >= 2.15 https://github.com/DataDog/dd-trace-rb/issues/1596
gem 'mysql2', '< 1', platform: :ruby
gem 'opensearch-ruby'
gem 'pg', '>= 0.18.4', platform: :ruby
gem 'racecar', '>= 0.3.5'
gem 'rack'
Expand Down Expand Up @@ -1572,6 +1576,7 @@ elsif ruby_version?('3.0') || ruby_version?('3.1') || ruby_version?('3.2') || ru
gem 'mongo', '>= 2.8.0', '< 2.15.0' # TODO: FIX TEST BREAKAGES ON >= 2.15 https://github.com/DataDog/dd-trace-rb/issues/1596
gem 'mysql2', '>= 0.5.3', platform: :ruby
gem 'activerecord-jdbcmysql-adapter', platform: :jruby
gem 'opensearch-ruby'
gem 'pg', platform: :ruby
gem 'activerecord-jdbcpostgresql-adapter', platform: :jruby
gem 'racecar', '>= 0.3.5'
Expand Down
2 changes: 2 additions & 0 deletions Rakefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,7 @@ namespace :spec do
:lograge,
:mongodb,
:mysql2,
:opensearch,
:pg,
:presto,
:qless,
Expand Down Expand Up @@ -302,6 +303,7 @@ task :ci do
declare '❌ 2.1 / ✅ 2.2 / ✅ 2.3 / ✅ 2.4 / ✅ 2.5 / ✅ 2.6 / ✅ 2.7 / ✅ 3.0 / ✅ 3.1 / ✅ 3.2 / ✅ 3.3 / ✅ jruby' => 'bundle exec appraisal contrib rake spec:lograge'
declare '✅ 2.1 / ✅ 2.2 / ✅ 2.3 / ✅ 2.4 / ✅ 2.5 / ✅ 2.6 / ✅ 2.7 / ✅ 3.0 / ✅ 3.1 / ✅ 3.2 / ✅ 3.3 / ✅ jruby' => 'bundle exec appraisal contrib rake spec:mongodb'
declare '✅ 2.1 / ✅ 2.2 / ✅ 2.3 / ✅ 2.4 / ✅ 2.5 / ✅ 2.6 / ✅ 2.7 / ✅ 3.0 / ✅ 3.1 / ✅ 3.2 / ✅ 3.3 / ❌ jruby' => 'bundle exec appraisal contrib rake spec:mysql2' # disabled on JRuby, built-in jdbc is used instead
declare '❌ 2.1 / ❌ 2.2 / ❌ 2.3 / ✅ 2.4 / ✅ 2.5 / ✅ 2.6 / ✅ 2.7 / ✅ 3.0 / ✅ 3.1 / ✅ 3.2 / ✅ 3.3 / ✅ jruby' => 'bundle exec appraisal contrib rake spec:opensearch'
declare '✅ 2.1 / ✅ 2.2 / ✅ 2.3 / ✅ 2.4 / ✅ 2.5 / ✅ 2.6 / ✅ 2.7 / ✅ 3.0 / ✅ 3.1 / ✅ 3.2 / ✅ 3.3 / ❌ jruby' => 'bundle exec appraisal contrib rake spec:pg'
declare '✅ 2.1 / ✅ 2.2 / ❌ 2.3 / ❌ 2.4 / ❌ 2.5 / ❌ 2.6 / ❌ 2.7 / ❌ 3.0 / ❌ 3.1 / ❌ 3.2 / ❌ 3.3 / ❌ jruby' => 'bundle exec appraisal contrib rake spec:presto'
declare '❌ 2.1 / ✅ 2.2 / ✅ 2.3 / ✅ 2.4 / ✅ 2.5 / ✅ 2.6 / ✅ 2.7 / ✅ 3.0 / ✅ 3.1 / ✅ 3.2 / ✅ 3.3 / ✅ jruby' => 'bundle exec appraisal contrib rake spec:que'
Expand Down
25 changes: 25 additions & 0 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ services:
- memcached
- mongodb
- mysql
- opensearch
- postgres
- presto
- redis
Expand All @@ -26,6 +27,8 @@ services:
TEST_MEMCACHED_HOST: memcached
TEST_MONGODB_HOST: mongodb
TEST_MYSQL_HOST: mysql
TEST_OPENSEARCH_HOST: opensearch
TEST_OPENSEARCH_PORT: 9200
TEST_POSTGRES_HOST: postgres
TEST_PRESTO_HOST: presto
TEST_PRESTO_PORT: 8080
Expand All @@ -49,6 +52,7 @@ services:
- memcached
- mongodb
- mysql
- opensearch
- postgres
- presto
- redis
Expand All @@ -74,6 +78,7 @@ services:
- memcached
- mongodb
- mysql
- opensearch
- postgres
- presto
- redis
Expand All @@ -99,6 +104,7 @@ services:
- memcached
- mongodb
- mysql
- opensearch
- postgres
- presto
- redis
Expand All @@ -124,6 +130,7 @@ services:
- memcached
- mongodb
- mysql
- opensearch
- postgres
- presto
- redis
Expand All @@ -149,6 +156,7 @@ services:
- memcached
- mongodb
- mysql
- opensearch
- postgres
- presto
- redis
Expand All @@ -174,6 +182,7 @@ services:
- memcached
- mongodb
- mysql
- opensearch
- postgres
- presto
- redis
Expand All @@ -199,6 +208,7 @@ services:
- memcached
- mongodb
- mysql
- opensearch
- postgres
- presto
- redis
Expand All @@ -216,6 +226,8 @@ services:
TEST_MEMCACHED_HOST: memcached
TEST_MONGODB_HOST: mongodb
TEST_MYSQL_HOST: mysql
TEST_OPENSEARCH_HOST: opensearch
TEST_OPENSEARCH_PORT: 9200
TEST_POSTGRES_HOST: postgres
TEST_REDIS_HOST: redis
TEST_REDIS_OLD_HOST: redis_old
Expand All @@ -237,6 +249,7 @@ services:
- memcached
- mongodb
- mysql
- opensearch
- postgres
- presto
- redis
Expand All @@ -262,6 +275,7 @@ services:
- memcached
- mongodb
- mysql
- opensearch
- postgres
- presto
- redis
Expand All @@ -287,6 +301,7 @@ services:
- memcached
- mongodb
- mysql
- opensearch
- postgres
- presto
- redis
Expand All @@ -312,6 +327,7 @@ services:
- memcached
- mongodb
- mysql
- opensearch
- postgres
- presto
- redis
Expand All @@ -336,6 +352,7 @@ services:
- memcached
- mongodb
- mysql
- opensearch
- postgres
- presto
- redis
Expand All @@ -360,6 +377,7 @@ services:
- memcached
- mongodb
- mysql
- opensearch
- postgres
- presto
- redis
Expand Down Expand Up @@ -445,6 +463,13 @@ services:
- "3306"
ports:
- "127.0.0.1:${TEST_MYSQL_PORT}:3306"
opensearch:
image: opensearchproject/opensearch:2.8.0
environment:
- discovery.type=single-node
- DISABLE_SECURITY_PLUGIN=true
ports:
- 9201:9200
postgres:
image: postgres:9.6
environment:
Expand Down
37 changes: 34 additions & 3 deletions docs/GettingStarted.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ To contribute, check out the [contribution guidelines][contribution docs] and [d
- [MongoDB](#mongodb)
- [MySQL2](#mysql2)
- [Net/HTTP](#nethttp)
- [OpenSearch](#opensearch)
- [Postgres](#postgres)
- [Presto](#presto)
- [Qless](#qless)
Expand Down Expand Up @@ -482,7 +483,7 @@ For a list of available integrations, and their configuration options, please re
| Action Mailer | `action_mailer` | `>= 5.0` | `>= 5.0` | *[Link](#action-mailer)* | *[Link](https://github.com/rails/rails/tree/master/actionmailer)* |
| Action Pack | `action_pack` | `>= 3.2` | `>= 3.2` | *[Link](#action-pack)* | *[Link](https://github.com/rails/rails/tree/master/actionpack)* |
| Action View | `action_view` | `>= 3.2` | `>= 3.2` | *[Link](#action-view)* | *[Link](https://github.com/rails/rails/tree/master/actionview)* |
| Active Job | `active_job` | `>= 4.2` | `>= 4.2` | *[Link](#active-job)* | *[Link](https://github.com/rails/rails/tree/master/activejob)* |
| Active Job | `active_job` | `>= 4.2` | `>= 4.2` | *[Link](#active-job)* | *[Link](https://github.com/rails/rails/tree/master/activejob)* |
| Active Model Serializers | `active_model_serializers` | `>= 0.9` | `>= 0.9` | *[Link](#active-model-serializers)* | *[Link](https://github.com/rails-api/active_model_serializers)* |
| Active Record | `active_record` | `>= 3.2` | `>= 3.2` | *[Link](#active-record)* | *[Link](https://github.com/rails/rails/tree/master/activerecord)* |
| Active Support | `active_support` | `>= 3.2` | `>= 3.2` | *[Link](#active-support)* | *[Link](https://github.com/rails/rails/tree/master/activesupport)* |
Expand All @@ -499,14 +500,15 @@ For a list of available integrations, and their configuration options, please re
| gRPC | `grpc` | `>= 1.7` | *gem not available* | *[Link](#grpc)* | *[Link](https://github.com/grpc/grpc/tree/master/src/rubyc)* |
| hanami | `hanami` | `>= 1`, `< 2` | `>= 1`, `< 2` | *[Link](#hanami)* | *[Link](https://github.com/hanami/hanami)* |
| http.rb | `httprb` | `>= 2.0` | `>= 2.0` | *[Link](#httprb)* | *[Link](https://github.com/httprb/http)* |
| httpclient | `httpclient` | `>= 2.2` | `>= 2.2` | *[Link](#httpclient)* | *[Link](https://github.com/nahi/httpclient)* |
| httpclient | `httpclient` | `>= 2.2` | `>= 2.2` | *[Link](#httpclient)* | *[Link](https://github.com/nahi/httpclient)* |
| httpx | `httpx` | `>= 0.11` | `>= 0.11` | *[Link](#httpx)* | *[Link](https://gitlab.com/honeyryderchuck/httpx)* |
| Kafka | `ruby-kafka` | `>= 0.7.10` | `>= 0.7.10` | *[Link](#kafka)* | *[Link](https://github.com/zendesk/ruby-kafka)* |
| Makara (via Active Record) | `makara` | `>= 0.3.5` | `>= 0.3.5` | *[Link](#active-record)* | *[Link](https://github.com/instacart/makara)* |
| MongoDB | `mongo` | `>= 2.1` | `>= 2.1` | *[Link](#mongodb)* | *[Link](https://github.com/mongodb/mongo-ruby-driver)* |
| MySQL2 | `mysql2` | `>= 0.3.21` | *gem not available* | *[Link](#mysql2)* | *[Link](https://github.com/brianmario/mysql2)* |
| Net/HTTP | `http` | *(Any supported Ruby)* | *(Any supported Ruby)* | *[Link](#nethttp)* | *[Link](https://ruby-doc.org/stdlib-2.4.0/libdoc/net/http/rdoc/Net/HTTP.html)* |
| Postgres | `pg` | `>= 0.18.4` | *gem not available* | *[Link](#postgres)* | *[Link](https://github.com/ged/ruby-pg)* |
| OpenSearch | `opensearch-ruby` | `>= 1.0.0` | `>= 1.0.0` | *[Link](#opensearch)* | *[Link](https://github.com/opensearch-project/opensearch-ruby)* |
| Postgres | `pg` | `>= 0.18.4` | *gem not available* | *[Link](#postgres)* | *[Link](https://github.com/ged/ruby-pg)* |
| Presto | `presto` | `>= 0.5.14` | `>= 0.5.14` | *[Link](#presto)* | *[Link](https://github.com/treasure-data/presto-client-ruby)* |
| Qless | `qless` | `>= 0.10.0` | `>= 0.10.0` | *[Link](#qless)* | *[Link](https://github.com/seomoz/qless)* |
| Que | `que` | `>= 1.0.0.beta2` | `>= 1.0.0.beta2` | *[Link](#que)* | *[Link](https://github.com/que-rb/que)* |
Expand Down Expand Up @@ -1433,6 +1435,35 @@ If you wish to configure each connection object individually, you may use the `D
client = Net::HTTP.new(host, port)
Datadog.configure_onto(client, **options)
```

### OpenSearch
sarahchen6 marked this conversation as resolved.
Show resolved Hide resolved

The OpenSearch integration will trace any call to `perform_request` in the `Client` object:

```ruby
require 'opensearch'
require 'ddtrace'

Datadog.configure do |c|
c.tracing.instrument :opensearch, **options
end

# Perform a query to OpenSearch
client = OpenSearch::Client.new(
host: 'https://localhost:9200',
user: 'user',
password: 'password',
)
client.cluster.health
```

`options` are the following keyword arguments:

| Key | Description | Default |
| --- | ----------- | ------- |
| `quantize` | Hash containing options for quantization. May include `:show` with an Array of keys to not quantize (or `:all` to skip quantization), or `:exclude` with Array of keys to exclude entirely. | `{}` |
| `service_name` | Service name used for `opensearch` instrumentation | `'opensearch'` |

### Postgres

The PG integration traces SQL commands sent through the `pg` gem via:
Expand Down
1 change: 1 addition & 0 deletions gemfiles/jruby_9.2.21.0_contrib.gemfile
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions gemfiles/jruby_9.3.9.0_contrib.gemfile
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions gemfiles/jruby_9.4.0.0_contrib.gemfile
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions gemfiles/ruby_2.4.10_contrib.gemfile
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions gemfiles/ruby_2.5.9_contrib.gemfile
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions gemfiles/ruby_2.6.10_contrib.gemfile
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions gemfiles/ruby_2.7.6_contrib.gemfile
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions gemfiles/ruby_3.0.4_contrib.gemfile
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions gemfiles/ruby_3.1.2_contrib.gemfile
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading