Wrong Encoding of the MDE Query

[ererdfdfcvcv]

Hello,

the current Version of
main/msticpy/data/drivers/odata_driver.py
contains a wrong encoding within the request creation.

I tried to request some queries from a Microsoft Defender for Endpoint API, but all failed with HTTP 400 - Bad Request.
The following Code example shows the lines, which caused the error in my case. The query/body of the request is loaded as a string (line 251 of the file/script), but must be JSON encoded.
As a solution, I added the correct encoding of the body with the function json.dumps(),

Could you please take a look at the code and fix the error, if it is one? I am quite new to (semi-) professional coding and still a university student. Consequently, I am not 100% confident, that I am right about this problem.

Best Regards

---------- odata_driver.py code ----------

# Build request based on whether endpoint requires data to be passed in
# request body in or URL
body = None
if kwargs["body"] is True:
req_url = self.request_uri + kwargs["api_end"]
req_url = urllib.parse.quote(req_url, safe="%/:=&?~#+!$,;'@()*[]")
body = {"Query": query}
response = httpx.post(
url=req_url,
headers=self.req_headers,
content=str(body),
timeout=self.get_http_timeout(**kwargs),
)

[ianhelle]

Hey @ererdfdfcvcv thanks for the report.
Can you give me a repo scenario so that I can see how you're using the existing code?
Looking at the code it seems like you're correct but I've been using this for a while without this encoding issue - which seems odd to me.
Are you supplying the query body as parameter to QueryProvider.exec_query()? Or using one of the built-in queries?

I'll check into the code history and see if the str() has been introduced recently. Another possibility is that the API has recently changed and used to support stringified JSON but no longer does.

[ianhelle]

I have a branch with this fix in but need to do some more testing before PRing it.
ianhelle/odata_driver_json_encode-2022-08-05

[ererdfdfcvcv]

Hey @ianhelle,
sry for the delayed reply. Had some busy days.

I got the error for both self-created queries and for every build-in query I tested.
An example is the following query, which I used to test the implementation. Without local changes, it returned the '400 Bad request'.

md_prov = QueryProvider("M365D") 
md_prov.connect(tenant_id=tenant_id, client_id=app_id, client_secret=app_secret)

start_time = md_prov.query_time.start.strftime('%Y-%m-%d %H:%M:%S.%f') 
end_time = md_prov.query_time.end.strftime('%Y-%m-%d %H:%M:%S.%f')

md_prov.MDE.user_logons(start=start_time, end=end_time, account_name='<user>')

I was able to recreate the error with a function from a previous project of mine which accessed the Microsoft Defender for Endpoint API. By removing the json.dumps encoding, the API returned the same error.

headers = {
        'Content-Type': 'application/json',
        'Authorization': "Bearer " + access_token
    }
url = "https://api-eu.securitycenter.windows.com/api/advancedqueries/run"
param = json.dumps({'Query': query}).encode("utf-8")
response = requests.post(url, headers=headers, data=param)

The changes in other project where made in November 2021, so the API seems to not have changed. It's pretty odd, that it is working for your but not for me. There was no other way for me than changing the odata.py script locally.
Sorry I could not provide more information right now.

[ianhelle]

It seems using the data param does require json encoding.
requests and httpx can also take a json parameter, which handles the json encoding. My fix looks like this:

        body = None
        if kwargs["body"] is True:
            req_url = f"{self.request_uri}{kwargs['api_end']}"
            req_url = urllib.parse.quote(req_url, safe="%/:=&?~#+!$,;'@()*[]")
            body = {"Query": query}
            response = httpx.post(
                url=req_url,
                headers=self.req_headers,
                json={"Query": query},
                timeout=self.get_http_timeout(**kwargs),
            )

which should do the same thing.
Just testing and should get into a release in next couple of days.