Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky test with pypy #2410

Closed
xrmx opened this issue Apr 11, 2024 · 4 comments · Fixed by #2809
Closed

Flaky test with pypy #2410

xrmx opened this issue Apr 11, 2024 · 4 comments · Fixed by #2809
Labels
bug Something isn't working good first issue Good for newcomers

Comments

@xrmx
Copy link
Contributor

xrmx commented Apr 11, 2024

Describe your environment

Got the a fastapi test fail on CI

Steps to reproduce

tox -f pypy3-fastapi -- -ra --benchmark-json=pypy3-fastapi-ubuntu-20.04-benchmark.json

What is the expected behavior?

Tests always pass

What is the actual behavior?

=================================== FAILURES ===================================
_____ TestAutoInstrumentationHooks.test_basic_post_request_metric_success ______

self = <tests.test_fastapi_instrumentation.TestAutoInstrumentationHooks testMethod=test_basic_post_request_metric_success>

    def test_basic_post_request_metric_success(self):
        start = default_timer()
        response = self._client.post(
            "/foobar",
            json={"foo": "bar"},
        )
        duration = max(round((default_timer() - start) * 1000), 0)
        response_size = int(response.headers.get("content-length"))
        request_size = int(response.request.headers.get("content-length"))
        metrics_list = self.memory_metrics_reader.get_metrics_data()
        for metric in (
            metrics_list.resource_metrics[0].scope_metrics[0].metrics
        ):
            for point in list(metric.data.data_points):
                if isinstance(point, HistogramDataPoint):
                    self.assertEqual(point.count, 1)
                    if metric.name == "http.server.duration":
>                       self.assertAlmostEqual(duration, point.sum, delta=30)
E                       AssertionError: 36 != 2 within 30 delta (34 difference)

Additional context

https://github.com/open-telemetry/opentelemetry-python-contrib/actions/runs/8647172961/job/23708138454#step:6:490

@xrmx xrmx added the bug Something isn't working label Apr 11, 2024
xrmx added a commit to xrmx/opentelemetry-python-contrib that referenced this issue Apr 12, 2024
Should fix the following failure seen in the CI:

 >  self.assertAlmostEqual(duration, point.sum, delta=30)
E   AssertionError: 36 != 2 within 30 delta (34 difference)

Fixes open-telemetry#2410
xrmx added a commit to xrmx/opentelemetry-python-contrib that referenced this issue May 27, 2024
Should fix the following failure seen in the CI:

 >  self.assertAlmostEqual(duration, point.sum, delta=30)
E   AssertionError: 36 != 2 within 30 delta (34 difference)

Fixes open-telemetry#2410
@xrmx xrmx added the good first issue Good for newcomers label Jun 13, 2024
@gongbenhua
Copy link
Contributor

can you assign this to me?

@emdneto
Copy link
Member

emdneto commented Aug 6, 2024

@gongbenhua please feel free to open a PR

@gongbenhua gongbenhua mentioned this issue Aug 21, 2024
5 tasks
@xrmx
Copy link
Contributor Author

xrmx commented Aug 25, 2024

Delta has been updated in in da78275

@xrmx xrmx closed this as completed Aug 25, 2024
@xrmx xrmx reopened this Aug 25, 2024
@xrmx xrmx closed this as completed Aug 26, 2024
@xrmx xrmx reopened this Aug 26, 2024
@xrmx
Copy link
Contributor Author

xrmx commented Aug 26, 2024

Reopening because we still have issues on pypy/windows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants