Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Azure IoT Edge integration #7465

Merged
merged 65 commits into from
Oct 22, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
96fad1a
Add skeleton
Aug 28, 2020
9fef3f0
Working Docker setup for 1.0.9
Aug 31, 2020
ce9bc5c
Attempt 1.0.10-rc2 setup
Aug 31, 2020
5f25b45
Finalize RC2 dev setup
Sep 1, 2020
6fd6fc2
Fix double-endpoint setup, implement scraping of Prometheus endpoints
Sep 1, 2020
1685edd
Update CI config
Sep 2, 2020
a535b5b
Add config class, add failing integration test
Sep 2, 2020
313fc8f
Successfully collect and test metrics, improve env up/down robustness
Sep 2, 2020
28d33a2
Make tests pass
Sep 2, 2020
03c0fbf
Use local mock server for CI tests
Sep 4, 2020
f8629f1
Add Edge Agent metrics
Sep 4, 2020
5d0c6b0
Update codecov config
Sep 7, 2020
b302b8b
Tweak exclude_labels
Sep 7, 2020
72cb919
Fix invalid manifest
Sep 7, 2020
18ca0ec
Add edgeHub metrics
Sep 7, 2020
b2213c6
Document mock server metrics generation
Sep 7, 2020
6423b81
Fix Python 2 tests compatibility
Sep 7, 2020
c94f10b
Assert E2E tags
Sep 7, 2020
1574b3a
Skip E2E tests if IOT_EDGE_CONNSTR is missing
Sep 7, 2020
2832868
Use Windows-compatible mock server setup
Sep 7, 2020
640032a
Add security daemon health service check
Sep 7, 2020
0fd2abe
Simplify prometheus url config, add config tests
Sep 7, 2020
e35340c
Fix style, fix Windows test compat
Sep 7, 2020
47e9d74
Verify service check in e2e
Sep 7, 2020
d9b5234
Fix check class name case
Sep 7, 2020
8f24b1d
Add config spec
Sep 8, 2020
b15cddf
Add logs to config spec and test env
Sep 8, 2020
08508b9
Use auto-discovery for log collection
Sep 8, 2020
cc1550c
Merge branch 'master' into fm/iot_edge
Sep 9, 2020
777a102
Enable log collection via Docker labels
Sep 9, 2020
ce86835
Merge branch 'master' into fm/iot_edge
Sep 10, 2020
f01f898
Set required properties in config spec
Sep 10, 2020
39d3735
Merge branch 'master' into fm/iot_edge
Sep 11, 2020
b7ccdbb
Reorganize config options order
Sep 11, 2020
6b625a7
Loosen wait conditions
Sep 14, 2020
bf95787
Merge branch 'master' into fm/iot_edge
Sep 16, 2020
34a4ffd
Merge branch 'master' into fm/iot_edge
Sep 17, 2020
8bbfddf
Merge branch 'master' into fm/iot_edge
Sep 18, 2020
e43fef6
Merge branch 'master' into fm/iot_edge
Sep 23, 2020
9e268a3
Update namespace to azure.iot_edge
Sep 23, 2020
c64698b
Merge branch 'master' into fm/iot_edge
Sep 25, 2020
3504ee3
Add version metadata collection
Sep 25, 2020
f80e4ec
Update manifest.json
Sep 25, 2020
6041ead
Check types
Sep 25, 2020
37ace98
Write up metadata.csv
Sep 25, 2020
591e5b2
Merge branch 'master' into fm/iot_edge
Sep 28, 2020
846bcac
Fill in service_checks.json
Sep 28, 2020
13d0000
Add TLS support to E2E environment
Sep 28, 2020
4aa5660
Add code comment about single-instance and composition approaches
Sep 28, 2020
c727097
Drop note about setting certs in config.yaml
Sep 28, 2020
60fca78
Merge branch 'master' into fm/iot_edge
Sep 29, 2020
33cccaf
Write up README
Sep 29, 2020
ec868e8
Lingo: security daemon -> security manager
Sep 29, 2020
7b081e4
Add recommended monitors
Sep 29, 2020
1f95257
Merge branch 'master' into fm/iot_edge
Sep 30, 2020
04b990a
Apply no-brainer suggestions
florimondmanca Oct 1, 2020
a1d86cd
Update version metadata transformer
Oct 2, 2020
38e8b19
Address feedback
Oct 2, 2020
dbb1370
Move instance config to Edge Agent labels
Oct 2, 2020
a25a15a
Merge branch 'master' into fm/iot_edge
Oct 9, 2020
0de144a
Merge branch 'master' into fm/iot_edge
Oct 13, 2020
ab0b18f
Apply suggestions from docs review
florimondmanca Oct 13, 2020
042ab4e
Merge branch 'master' into fm/iot_edge
Oct 20, 2020
926c566
Fix type of renotify_interval in monitors json
Oct 20, 2020
145a931
Merge branch 'master' into fm/iot_edge
Oct 22, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Use auto-discovery for log collection
  • Loading branch information
Florimond Manca committed Sep 8, 2020
commit 08508b9dce38788d690ed85f102da9b3368eaa55
8 changes: 0 additions & 8 deletions azure_iot_edge/assets/configuration/spec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,3 @@ files:
type: string
example: http://localhost:15580
- template: instances/default

- template: logs
example:
# Requires setting DD_CONTAINER_INCLUDE_LOGS="name:<CONTAINER_1> name:<CONTAINER_2> ..."
# See: https://docs.datadoghq.com/agent/guide/autodiscovery-management/?tab=containerizedagent#include-containers
- type: docker
source: azure_iot_edge
service: "<SERVICE_NAME>"
22 changes: 22 additions & 0 deletions azure_iot_edge/datadog_checks/azure_iot_edge/data/auto_conf.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
## @param ad_identifiers - list of strings - required
## A list of container identifiers that are used by Autodiscovery to identify
## which container the check should be run against. For more information, see:
## https://docs.datadoghq.com/agent/guide/ad_identifiers/
#
ad_identifiers:
- azure_iot_edge

## Must be empty.
#
init_config: {}

## Must be empty
#
instances: []

## Logs collection configuration.
#
logs:
- type: docker
source: azure_iot_edge
service: azure_iot_edge
Original file line number Diff line number Diff line change
Expand Up @@ -57,21 +57,3 @@ instances:
## This is useful for cluster-level checks.
#
# empty_default_hostname: false

## Log Section
##
## type - required - Type of log input source (tcp / udp / file / windows_event)
## port / path / channel_path - required - Set port if type is tcp or udp.
## Set path if type is file.
## Set channel_path if type is windows_event.
## source - required - Attribute that defines which Integration sent the logs.
## service - optional - The name of the service that generates the log.
## Overrides any `service` defined in the `init_config` section.
## tags - optional - Add tags to the collected logs.
##
## Discover Datadog log collection: https://docs.datadoghq.com/logs/log_collection/
#
# logs:
# - type: docker
# source: azure_iot_edge
# service: <SERVICE_NAME>
15 changes: 4 additions & 11 deletions azure_iot_edge/tests/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -364,22 +364,15 @@
'iothub:iot-edge-dev-hub.azure-devices.net',
]

E2E_LOGS_CONFIG = [
{
'type': 'docker',
'source': 'azure_iot_edge',
'service': 'azure_iot_edge_dev',
},
]

E2E_CONTAINERS = ['iot-edge-device'] + MODULES

E2E_METADATA = {
'env_vars': {
'DD_LOGS_ENABLED': 'true',
'DD_CONTAINER_INCLUDE_LOGS': ' '.join('name:{}'.format(name) for name in E2E_CONTAINERS),
},
'docker_volumes': [
'/var/run/docker.sock:/var/run/docker.sock',
],
'start_commands': [
# Ensure the Agent has access to the autodiscovery config template.
'cp -r /home/azure_iot_edge/datadog_checks/azure_iot_edge/data/auto_conf.yaml /etc/datadog-agent/conf.d/azure_iot_edge.d/auto_conf.yaml', # noqa: E501
],
}
2 changes: 2 additions & 0 deletions azure_iot_edge/tests/compose/docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ services:
- /var/run/docker.sock:/var/run/docker.sock
networks:
- iot-edge-network
labels:
com.datadoghq.ad.check.id: azure_iot_edge

networks:
iot-edge-network:
Expand Down
8 changes: 1 addition & 7 deletions azure_iot_edge/tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,14 +39,8 @@ def dd_environment(e2e_instance):
up = e2e_utils.IoTEdgeUp(compose_file, network_name=common.E2E_NETWORK)
down = e2e_utils.IoTEdgeDown(compose_file, stop_extra_containers=common.E2E_EXTRA_SPAWNED_CONTAINERS)

e2e_config = {
'init_config': {},
'instances': [e2e_instance],
'logs': common.E2E_LOGS_CONFIG,
}

with docker_run(conditions=conditions, env_vars=env_vars, up=up, down=down):
yield e2e_config, common.E2E_METADATA
yield e2e_instance, common.E2E_METADATA


@pytest.fixture(scope='session')
Expand Down