Open
Description
openedon Nov 1, 2024
We are managing our dashboard using a predetermined TraceQL. For example, when sending a query with the condition { span.http.status_code = 500 }
over a lengthy period of 2 days, we have noticed it takes an excessively long time if no traces meet this condition. Here are the response times we've observed:
30-minute period: 6 seconds
2-day period: 26 seconds
For traces that do exist, we achieve response times between 500ms and 700ms. How can we quickly identify the absence of data and improve the response time in such cases?
For reference, we are continuously monitoring the resources of our Tempo components, and we have observed that executing such queries fully utilizes the CPU of all 18 queriers, each with 8 vCPUs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Metadata
Assignees
Labels
No labels