Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flaky fastapi test #2588

Closed
xrmx opened this issue Jun 7, 2024 · 1 comment · Fixed by #2591
Closed

flaky fastapi test #2588

xrmx opened this issue Jun 7, 2024 · 1 comment · Fixed by #2591
Labels
bug Something isn't working good first issue Good for newcomers

Comments

@xrmx
Copy link
Contributor

xrmx commented Jun 7, 2024

Steps to reproduce

tox -e py38-test-instrumentation-fastapi

What is the expected behavior?

test is not flaky

What is the actual behavior?

_____ TestAutoInstrumentationHooks.test_basic_post_request_metric_success ______

self = <tests.test_fastapi_instrumentation.TestAutoInstrumentationHooks testMethod=test_basic_post_request_metric_success>

    def test_basic_post_request_metric_success(self):
        start = default_timer()
        response = self._client.post(
            "/foobar",
            json={"foo": "bar"},
        )
        duration = max(round((default_timer() - start) * 1000), 0)
        response_size = int(response.headers.get("content-length"))
        request_size = int(response.request.headers.get("content-length"))
        metrics_list = self.memory_metrics_reader.get_metrics_data()
        for metric in (
            metrics_list.resource_metrics[0].scope_metrics[0].metrics
        ):
            for point in list(metric.data.data_points):
                if isinstance(point, HistogramDataPoint):
                    self.assertEqual(point.count, 1)
                    if metric.name == "http.server.duration":
>                       self.assertAlmostEqual(duration, point.sum, delta=30)
E                       AssertionError: 34 != 1 within 30 delta (33 difference)

and

____________ TestAutoInstrumentationHooks.test_basic_metric_success ____________

self = <tests.test_fastapi_instrumentation.TestAutoInstrumentationHooks testMethod=test_basic_metric_success>

    def test_basic_metric_success(self):
        start = default_timer()
        self._client.get("/foobar")
        duration = max(round((default_timer() - start) * 1000), 0)
        expected_duration_attributes = {
            "http.method": "GET",
            "http.host": "testserver:443",
            "http.scheme": "https",
            "http.flavor": "1.1",
            "http.server_name": "testserver",
            "net.host.port": 443,
            "http.status_code": 200,
            "http.target": "/foobar",
        }
        expected_requests_count_attributes = {
            "http.method": "GET",
            "http.host": "testserver:443",
            "http.scheme": "https",
            "http.flavor": "1.1",
            "http.server_name": "testserver",
        }
        metrics_list = self.memory_metrics_reader.get_metrics_data()
        for metric in (
            metrics_list.resource_metrics[0].scope_metrics[0].metrics
        ):
            for point in list(metric.data.data_points):
                if isinstance(point, HistogramDataPoint):
                    self.assertDictEqual(
                        expected_duration_attributes,
                        dict(point.attributes),
                    )
                    self.assertEqual(point.count, 1)
>                   self.assertAlmostEqual(duration, point.sum, delta=30)
E                   AssertionError: 39 != 1 within 30 delta (38 difference)

Additional context

@xrmx xrmx added bug Something isn't working good first issue Good for newcomers labels Jun 7, 2024
@emdneto
Copy link
Member

emdneto commented Jun 8, 2024

I got same error here. I noticed there was a try to fix that in #1344 but test seems still flaky.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants