-
Notifications
You must be signed in to change notification settings - Fork 345
feat: support influxdb #413
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
alexanderankin
merged 13 commits into
testcontainers:main
from
lucsorel:feat/support-influxdb
Mar 20, 2024
Merged
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
ab3bf4f
feat(influxdb): add InfluxDbContainer with methods to retrieve the co…
lucsorel af7dbbf
ci(influxdb): add references to the InfluxDbContainer in the general …
lucsorel e643ac1
feat(influxdb): add client v1 getter for INfluxDB 1.x instances
lucsorel 4e27304
feat(influxdb2): add support for InfluxDB 2 client and unit tests
lucsorel f8d84a4
refactor(influxdb): split the code for InfluxDB 1.x and 2.x in separa…
lucsorel af4c18a
refactor(influxdb): remove unused imports, order imports
lucsorel 4ff505d
tests(influxdb): replace use of dateutil parser by datetime fromisofo…
lucsorel c28bd8a
ci(influxdb): replace Tuple and Dict typing by tuple and dict, apply …
lucsorel 31ecb5d
test(influxdb): fix get_version test with InfluxDB 2.x, which still e…
lucsorel e01621d
docs(influxdb): add title about InfluxDbContainer
lucsorel 155c416
ci(poetry.lock): update poetry lock file
lucsorel 38d96fe
Merge branch 'main' into feat/support-influxdb
lucsorel e595200
ci(poetry.lock): update poetry lock file
lucsorel File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| .. autoclass:: testcontainers.influxdb.InfluxDbContainer | ||
| .. title:: testcontainers.influxdb.InfluxDbContainer | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); you may | ||
| # not use this file except in compliance with the License. You may obtain | ||
| # a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT | ||
| # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the | ||
| # License for the specific language governing permissions and limitations | ||
| # under the License. | ||
|
|
||
| """ | ||
| testcontainers/influxdb provides means to spawn an InfluxDB instance within a Docker container. | ||
|
|
||
| - this influxdb.py module provides the common mechanism to spawn an InfluxDB container. | ||
| You are not likely to use this module directly. | ||
| - import the InfluxDb1Container class from the influxdb1/__init__.py module to spawn | ||
| a container for an InfluxDB 1.x instance | ||
| - import the InfluxDb2Container class from the influxdb2/__init__.py module to spawn | ||
| a container for an InfluxDB 2.x instance | ||
|
|
||
| The 2 containers are separated in different modules for 2 reasons: | ||
| - because the Docker images are not designed to be used in the same way | ||
| - because the InfluxDB clients are different for 1.x and 2.x versions, | ||
| so you won't have to install dependencies that you do not need | ||
| """ | ||
| from typing import Optional | ||
|
|
||
| from requests import get | ||
| from requests.exceptions import ConnectionError, ReadTimeout | ||
|
|
||
| from testcontainers.core.container import DockerContainer | ||
| from testcontainers.core.waiting_utils import wait_container_is_ready | ||
|
|
||
|
|
||
| class InfluxDbContainer(DockerContainer): | ||
| """ | ||
| Abstract class for Docker containers of InfluxDB v1 and v2. | ||
|
|
||
| Concrete implementations for InfluxDB 1.x and 2.x are separated iun different packages | ||
| because their respective clients rely on different Python libraries which we don't want | ||
| to import at the same time. | ||
| """ | ||
|
|
||
| def __init__( | ||
| self, | ||
| # Docker image name | ||
| image: str, | ||
| # in the container, the default port for influxdb is often 8086 and not likely to change | ||
| container_port: int = 8086, | ||
| # specifies the port on the host machine where influxdb is exposed; a random available port otherwise | ||
| host_port: Optional[int] = None, | ||
| **docker_client_kw, | ||
| ): | ||
| super().__init__(image=image, **docker_client_kw) | ||
| self.container_port = container_port | ||
| self.host_port = host_port | ||
| self.with_bind_ports(self.container_port, self.host_port) | ||
|
|
||
| def get_url(self) -> str: | ||
| """ | ||
| Returns the url to interact with the InfluxDB container (health check, REST API, etc.) | ||
| """ | ||
| host = self.get_container_host_ip() | ||
| port = self.get_exposed_port(self.container_port) | ||
|
|
||
| return f"http://{host}:{port}" | ||
|
|
||
| @wait_container_is_ready(ConnectionError, ReadTimeout) | ||
| def _health_check(self) -> dict: | ||
| """ | ||
| Performs a health check on the running InfluxDB container. | ||
| The call is retried until it works thanks to the @wait_container_is_ready decorator. | ||
| See its documentation for the max number of retries or the timeout. | ||
| """ | ||
|
|
||
| url = self.get_url() | ||
| response = get(f"{url}/health", timeout=1) | ||
| response.raise_for_status() | ||
|
|
||
| return response.json() | ||
|
|
||
| def get_influxdb_version(self) -> str: | ||
| """ | ||
| Returns the version of the InfluxDB service, as returned by the healthcheck. | ||
| """ | ||
|
|
||
| return self._health_check().get("version") | ||
|
|
||
| def start(self) -> "InfluxDbContainer": | ||
| """ | ||
| Spawns a container of the InfluxDB Docker image, ready to be used. | ||
| """ | ||
| super().start() | ||
| self._health_check() | ||
|
|
||
| return self |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); you may | ||
| # not use this file except in compliance with the License. You may obtain | ||
| # a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT | ||
| # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the | ||
| # License for the specific language governing permissions and limitations | ||
| # under the License. | ||
|
|
||
| from typing import Optional | ||
|
|
||
| from influxdb import InfluxDBClient | ||
|
|
||
| from testcontainers.influxdb import InfluxDbContainer | ||
|
|
||
|
|
||
| class InfluxDb1Container(InfluxDbContainer): | ||
| """ | ||
| Docker container for InfluxDB 1.x. | ||
| Official Docker images for InfluxDB are hosted at https://hub.docker.com/_/influxdb/. | ||
|
|
||
| Example: | ||
|
|
||
| .. doctest:: | ||
|
|
||
| >>> from testcontainers.influxdb1 import InfluxDbContainer | ||
|
|
||
| >>> with InfluxDbContainer() as influxdb: | ||
| ... version = influxdb.get_version() | ||
| """ | ||
|
|
||
| def __init__( | ||
| self, | ||
| image: str = "influxdb:1.8", | ||
| # in the container, the default port for influxdb is often 8086 and not likely to change | ||
| container_port: int = 8086, | ||
| # specifies the port on the host machine where influxdb is exposed; a random available port otherwise | ||
| host_port: Optional[int] = None, | ||
| **docker_client_kw, | ||
| ): | ||
| super().__init__(image, container_port, host_port, **docker_client_kw) | ||
|
|
||
| def get_client(self, **client_kwargs): | ||
| """ | ||
| Returns an instance of the influxdb client, for InfluxDB 1.x versions. | ||
| Note that this client is not maintained anymore, but it is the only | ||
| official client available for 1.x InfluxDB versions: | ||
| - https://github.com/influxdata/influxdb-python | ||
| - https://pypi.org/project/influxdb/ | ||
|
|
||
| To some extent, you can use the v2 client with InfluxDB v1.8+: | ||
| - https://github.com/influxdata/influxdb-client-python#influxdb-18-api-compatibility | ||
| """ | ||
|
|
||
| return InfluxDBClient(self.get_container_host_ip(), self.get_exposed_port(self.container_port), **client_kwargs) | ||
|
|
||
| def start(self) -> "InfluxDb1Container": | ||
| """ | ||
| Overridden for better typing reason | ||
| """ | ||
| return super().start() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,106 @@ | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); you may | ||
| # not use this file except in compliance with the License. You may obtain | ||
| # a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT | ||
| # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the | ||
| # License for the specific language governing permissions and limitations | ||
| # under the License. | ||
|
|
||
| from os import getenv | ||
| from typing import Optional | ||
|
|
||
| from influxdb_client import InfluxDBClient, Organization | ||
|
|
||
| from testcontainers.influxdb import InfluxDbContainer | ||
|
|
||
|
|
||
| class InfluxDb2Container(InfluxDbContainer): | ||
| """ | ||
| Docker container for InfluxDB 2.x. | ||
| Official Docker images for InfluxDB are hosted at https://hub.docker.com/_/influxdb/. | ||
|
|
||
| Example: | ||
|
|
||
| .. doctest:: | ||
|
|
||
| >>> from testcontainers.influxdb2 import InfluxDb2Container | ||
|
|
||
| >>> with InfluxDb2Container() as influxdb2: | ||
| ... version = influxdb2.get_version() | ||
| """ | ||
|
|
||
| def __init__( | ||
| self, | ||
| image: str = "influxdb:latest", | ||
| # in the container, the default port for influxdb is often 8086 and not likely to change | ||
| container_port: int = 8086, | ||
| # specifies the port on the host machine where influxdb is exposed; a random available port otherwise | ||
| host_port: Optional[int] = None, | ||
| # parameters used by the InfluxDSB 2.x Docker container when spawned in setup mode | ||
| # (which is likely what you want). In setup mode, init_mode should be "setup" and all | ||
| # the other parameters should be set (via this constructor or their respective | ||
| # environment variables); retention does not need to be explicitely set. | ||
| init_mode: Optional[str] = None, | ||
| admin_token: Optional[str] = None, | ||
| username: Optional[str] = None, | ||
| password: Optional[str] = None, | ||
| org_name: Optional[str] = None, | ||
| bucket: Optional[str] = None, | ||
| retention: Optional[str] = None, | ||
| **docker_client_kw, | ||
| ): | ||
| super().__init__(image, container_port, host_port, **docker_client_kw) | ||
|
|
||
| configuration = { | ||
| "DOCKER_INFLUXDB_INIT_MODE": init_mode, | ||
| "DOCKER_INFLUXDB_INIT_ADMIN_TOKEN": admin_token, | ||
| "DOCKER_INFLUXDB_INIT_USERNAME": username, | ||
| "DOCKER_INFLUXDB_INIT_PASSWORD": password, | ||
| "DOCKER_INFLUXDB_INIT_ORG": org_name, | ||
| "DOCKER_INFLUXDB_INIT_BUCKET": bucket, | ||
| "DOCKER_INFLUXDB_INIT_RETENTION": retention, | ||
| } | ||
| for env_key, constructor_param in configuration.items(): | ||
| env_value = constructor_param or getenv(env_key) | ||
| if env_value: | ||
| self.with_env(env_key, env_value) | ||
|
|
||
| def start(self) -> "InfluxDb2Container": | ||
| """ | ||
| Overridden for better typing reason | ||
| """ | ||
| return super().start() | ||
|
|
||
| def get_client( | ||
| self, token: Optional[str] = None, org_name: Optional[str] = None, **influxdb_client_kwargs | ||
| ) -> tuple[InfluxDBClient, Organization]: | ||
| """ | ||
| Returns an instance of the influxdb client with the associated test organization created | ||
| when the container is spawn; for InfluxDB 2.x versions. | ||
| - https://github.com/influxdata/influxdb-client-python | ||
| - https://pypi.org/project/influxdb-client/ | ||
|
|
||
| This InfluxDB client requires to specify the organization when using most of the API's endpoints, | ||
| as an Organisation instance rather than its name or id (deprecated). As a convenience, this | ||
| client getter can also retrieve and return the organization instance along with the client. | ||
| Otherwise, None is returned in place of the organization instance. | ||
|
|
||
| This organization is created when spawning the container in setup mode (which is likely what you | ||
| want) by giving its name to the 'org_name' parameter constructor. | ||
| """ | ||
|
|
||
| influxclient = InfluxDBClient(self.get_url(), token=token, **influxdb_client_kwargs) | ||
|
|
||
| if org_name is None: | ||
| return influxclient, None | ||
|
|
||
| orgs = influxclient.organizations_api().find_organizations(org=org_name) | ||
| if len(orgs) == 0: | ||
| raise ValueError(f"Could not retrieved the Organization corresponding to name '{org_name}'") | ||
|
|
||
| return influxclient, orgs[0] |
Empty file.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.