Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky flask tests with pypy in CI #2883

Closed
xrmx opened this issue Sep 23, 2024 · 1 comment · Fixed by #2884
Closed

Flaky flask tests with pypy in CI #2883

xrmx opened this issue Sep 23, 2024 · 1 comment · Fixed by #2884
Labels
bug Something isn't working

Comments

@xrmx
Copy link
Contributor

xrmx commented Sep 23, 2024

Describe your environment

OS: (e.g, Ubuntu)
Python version: pypy-3.8
Package version: 6fcfbd6

What happened?

_______________ TestProgrammatic.test_flask_metrics_new_semconv ________________

self = <tests.test_programmatic.TestProgrammatic testMethod=test_flask_metrics_new_semconv>

    def test_flask_metrics_new_semconv(self):
        start = default_timer()
        self.client.get("/hello/123")
        self.client.get("/hello/321")
        self.client.get("/hello/756")
        duration_s = max(default_timer() - start, 0)
        metrics_list = self.memory_metrics_reader.get_metrics_data()
        number_data_point_seen = False
        histogram_data_point_seen = False
        self.assertTrue(len(metrics_list.resource_metrics) != 0)
        for resource_metric in metrics_list.resource_metrics:
            self.assertTrue(len(resource_metric.scope_metrics) != 0)
            for scope_metric in resource_metric.scope_metrics:
                self.assertTrue(len(scope_metric.metrics) != 0)
                for metric in scope_metric.metrics:
                    self.assertIn(metric.name, _expected_metric_names_new)
                    data_points = list(metric.data.data_points)
                    self.assertEqual(len(data_points), 1)
                    for point in data_points:
                        if isinstance(point, HistogramDataPoint):
                            self.assertEqual(point.count, 3)
>                           self.assertAlmostEqual(
                                duration_s, point.sum, places=2
                            )
E                           AssertionError: 0.018902038000021548 != 0.013057114999980968 within 2 places (0.0058449230000405805 difference)

Steps to Reproduce

tox -e pypy3-test-instrumentation-flask-1 -- -ra

Expected Result

pass

Actual Result

fail

Additional context

No response

Would you like to implement a fix?

No

@xrmx xrmx added the bug Something isn't working label Sep 23, 2024
@xrmx
Copy link
Contributor Author

xrmx commented Sep 23, 2024

Another try:

>>> round(0.0012783256999966852, 2)
0.0
>>> round(0.007484300999976767, 2)
0.01

The fix would be to use places=1 or delta I guess

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
1 participant