DPDV-6531: Retry when server does not return JSON

[martin-majlis-s1]

Jira Link: https://sentinelone.atlassian.net/browse/DPDV-6531

🥅 Goal

Retry to send requests even when the server is not sending expected (JSON) response.

🛠️ Solution

Original version was not retrying when it was not possible to connect to the server or when it returned response in different format then expected. Since this situation can happen during some system outages - upstream connect error or disconnect/reset before headers. reset reason: connection termination:

error:            "unable to parse response body: invalid character 'u' looking for beginning of value, url: https://app.scalyr.com/api/addEvents, response: upstream connect error or disconnect/reset before headers. reset reason: connection termination",

We should retry. Hopefully the server recovers meanwhile.

🏫 Testing

I have added test for this scenario.

📓 Note

I have also updated dependabot configuration, so it updates also examples, so we do not have this version changes in the future.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 85.71429% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 75.79%. Comparing base (422e686) to head (9cd8492).

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #81      +/-   ##
==========================================
- Coverage   75.95%   75.79%   -0.17%     
==========================================
  Files          12       12              
  Lines        2096     2094       -2     
==========================================
- Hits         1592     1587       -5     
- Misses        419      421       +2     
- Partials       85       86       +1     

Files	Coverage Δ
pkg/client/client.go	85.08% <100.00%> (-0.05%)	⬇️
pkg/client/add_events.go	81.55% <66.67%> (-0.64%)	⬇️

[martinrataj]

@martin-majlis-s1 I don't know much about this library. But does it make sense to retry if the response has unexpected format? Do you have some examples when a retry could help in this case?

[tomaz-s1]

LGTM, thanks.

[martin-majlis-s1]

@martinrataj : It looks that when there is some outage, it's returning - upstream connect error or disconnect/reset before headers. reset reason: connection termination