DecodeError retrieving data from monitor api

[rcampos87]

• Package Name: azure-mgmt-monitor
• Package Version: 6.0.2
• Operating System: Debian bullseye
• Python Version: 3.10

Describe the bug
In the past few days I have been seeing a lot of DecodeError from the sdk when trying to ingest monitor data from VMs.

As far as I know, it happens randomly when the request has a high cost.

JSON is invalid: Unterminated string starting at: line 1 column 23743 (char 23742)
Content: {"cost":968,"timespan":"2025-03-21T00:05:00Z/2025-03-21T04:08:14Z","interval":"PT1M","value":[{"id":"/subscriptions/5b7974bc-b401-4d16-98f8-90ba97ca232d/resourceGroups/USP-APP-RG/providers/Microsoft.Compute/virtualMachines/VR2VDHCP1/providers/Microsoft.Insights/metrics/Percentage CPU","type":"Microsoft.Insights/metrics","name":{"value":"Percentage CPU","localizedValue":"Percentage CPU"},"displayDescription":"The percentage of allocated compute units that are currently in use by the Virtual Machine(s)","unit":"Percent","timeseries":[{"metadatavalues":[],"data":[{"timeStamp":"2025-03-21T00:05:00Z","average":4.325},{"timeStamp":"2025-03-21T00:06:00Z","average":2.805},{"timeStamp":"2025-03-21T00:07:00Z","average":15.38},{"timeStamp":"2025-03-21T00:08:00Z","average":2.905},{"timeStamp":"2025-03-21T00:09:00Z","average":3.6},{"timeStamp":"2025-03-21T00:10:00Z","average":5.1},{"timeStamp":"2025-03-21T00:11:00Z","average":3.26},{"timeStamp":"2025-03-21T00:12:00Z","average":2.51},{"timeStamp":"2025-03-21T00:13:00Z","average":2.675},{"timeStamp":"2025-03-21T00:14:00Z","average":2.38},{"timeStamp":"2025-03-21T00:15:00Z","average":7.205},{"timeStamp":"2025-03-21T00:16:00Z","average":3.89},{"timeStamp":"2025-03-21T00:17:00Z","average":10.82},{"timeStamp":"2025-03-21T00:18:00Z","average":4.015},{"timeStamp":"2025-03-21T00:19:00Z","average":7.37},{"timeStamp":"2025-03-21T00:20:00Z","average":11.765},{"timeStamp":"2025-03-21T00:21:00Z","average":3.125},{"timeStamp":"2025-03-21T00:22:00Z","average":3.055},{"timeStamp":"2025-03-21T00:23:00Z","average":3.025},{"timeStamp":"2025-03-21T00:24:00Z","average":2.115},{"timeStamp":"2025-03-21T00:25:00Z","average":3.775},{"timeStamp":"2025-03-21T00:26:00Z","average":5.24},{"timeStamp":"2025-03-21T00:27:00Z","average":9.745},{"timeStamp":"2025-03-21T00:28:00Z","average":3.11},{"timeStamp":"2025-03-21T00:29:00Z","average":3.075},{"timeStamp":"2025-03-21T00:30:00Z","average":3.825},{"timeStamp":"2025-03

Others json is invalid include:

JSON is invalid: Expecting ':' delimiter: line 1 column 23750 (char 23749)
JSON is invalid: Expecting value: line 1 column 23749 (char 23748)

To Reproduce
It occurs randomly.

Expected behavior
Either return a proper response or none at all.

Additional context
This error has been caught by NewRelic, it happens at

…usr/local/lib/python3.10/site-packages/azure/mgmt/monitor/v2021_05_01/operations/_metrics_operations.py, line 801, in list
                               File /usr/local/lib/python3.10/site-packages/azure/core/pipeline/_base.py, line 240, in run
                               File /usr/local/lib/python3.10/site-packages/azure/core/pipeline/_base.py, line 96, in send
                               File /usr/local/lib/python3.10/site-packages/azure/core/pipeline/_base.py, line 96, in send
                               File /usr/local/lib/python3.10/site-packages/azure/core/pipeline/_base.py, line 96, in send
                               File /usr/local/lib/python3.10/site-packages/azure/core/pipeline/_base.py, line 96, in send
                               File /usr/local/lib/python3.10/site-packages/azure/core/pipeline/_base.py, line 100, in send
                              File /usr/local/lib/python3.10/site-packages/azure/core/pipeline/_tools.py, line 49, in await_result
                 File /usr/local/lib/python3.10/site-packages/azure/core/pipeline/policies/_universal.py, line 714, in on_response
                 File /usr/local/lib/python3.10/site-packages/azure/core/pipeline/policies/_universal.py, line 681, in deserialize_from_http_generics
                 File /usr/local/lib/python3.10/site-packages/azure/core/pipeline/policies/_universal.py, line 611, in deserialize_from_text

[github-actions]

Hello @rcampos87. I'm an AI assistant for the azure-sdk-for-python repository. I have some suggestions that you can try out while the team gets back to you.

• Ensure you’re using the latest version of the library in case this is a bug that’s already been addressed. See the azure-mgmt-monitor README for upgrade instructions.

• If the error occurs when cost is high (resulting in larger payloads), consider adding retry logic or inspecting network conditions—malformed or truncated JSON may indicate issues with the service response.

• Compare this behavior with similar issues in the repository (for example, issues #11271 and #29570) to see if there’s a pattern or if any workarounds have been suggested.

• If the issue persists, consider capturing complete request/response traces to help with debugging or file a new issue with detailed steps and logs.

The team will get back to you shortly, hopefully this helps in the meantime.

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @gulopesd @Haiying-MSFT @jairmyree @joshfree @KarishmaGhiya @KevinBlasko @kurtzeborn @nisha-bhatia @pvaneck @scottaddie @srnagar @ToddKingMSFT.

[msyyc]

@ChenxiJiang333 Please help make an investigation for this issue.

[ChenxiJiang333]

got it

[ChenxiJiang333]

Hi @rcampos87, it seems some json data returned by the service doesn't respect the right schema.

Since the DecodeError happens often, is that possible for you to share part of the log where you meet these errors to help us locate the invalid json data?

You could save the whole output of SDK in a .txt file by copying the following code in your .py file:

import logging, os
logging.basicConfig(level=logging.DEBUG,
                    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
                    filename=f"{os.path.splitext(os.path.basename(__file__))[0]}.log",
                    filemode="w")

And set logging_enable=True when calling operation:

client.operation_group.operation(..., logging_enable=True)

Please also remember to conceal the sensitive info in your log.

[rcampos87]

@ChenxiJiang333 just to confirm, logging_enable=True is passed to client.metrics.list? I enabled it but I dont see any actual logs from azure in the file