Getting com.google.gson.stream.MalformedJsonException on all API calls #120

thonier · 2018-07-23T10:55:16Z

I have a strange problem: on our development machines, all calls to the MangoPay API work very well, but on our test server I get this exception for ALL calls to the API:
com.google.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 25 path $

Would anyone know where that could come from and how to address that?

Thanks a lot!

thonier · 2018-07-23T11:06:38Z

never mind, it was a completely unrelated config issue. No clue why it manifested itself in such a cryptic way

thonier · 2018-08-02T16:30:08Z

The issue is back, in fact it's not a config issue in the end. It appears to be intermittent - which means it is also unrelated to the server itself. If I let the server run long enough, at some point the problem occurs, and then there is no turning back, once this error happens, ALL subsequent API calls fail. The only option is to restart the server.

So any idea/help would indeed be much appreciated.

mickaelpois · 2018-08-24T13:31:42Z

Hello @thonier and sorry for this late answer.

I guess this issue is always not recurring one on your side?
We will try to reproduce and understand what happen on SDK / Server side, and fix what could/should be fixed if relevant.
Could you have some extra info (date and time for example, sandbox or production env.) ?

Thanks for your feedback, we will keep you posted here with any useful updates.

Mickaël

cosmin-margarit · 2018-08-29T09:14:48Z

Hello @thonier,

Is this issue still present, if so can you please provide some extra information?

Thank you,
Cosmin

thonier · 2018-09-03T13:04:30Z

Hi guys,

Just as I was saying we had not seen the issue in a while, it is back... so right now our server is in a state where any call to mangopay via the sdk fails with the error above.

Here is a sample log:
com.google.gson.JsonSyntaxException: com.google.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 25 path $
at com.google.gson.JsonParser.parse(JsonParser.java:65)
at com.google.gson.JsonParser.parse(JsonParser.java:45)
at com.mangopay.core.RestTool.checkResponseCode(RestTool.java:1076)
at com.mangopay.core.RestTool.doRequest(RestTool.java:426)
at com.mangopay.core.RestTool.request(RestTool.java:142)
at com.mangopay.core.RestTool.request(RestTool.java:186)
at com.mangopay.core.APIs.implementation.OAuthApiImpl.createToken(OAuthApiImpl.java:55)
at com.mangopay.core.AuthorizationTokenManager.getToken(AuthorizationTokenManager.java:39)
at com.mangopay.core.AuthenticationHelper.getHttpHeaderStrong(AuthenticationHelper.java:61)
at com.mangopay.core.AuthenticationHelper.getHttpHeaderKey(AuthenticationHelper.java:30)
at com.mangopay.core.RestTool.getHttpHeaders(RestTool.java:1037)
at com.mangopay.core.RestTool.doRequest(RestTool.java:336)
at com.mangopay.core.RestTool.request(RestTool.java:142)
at com.mangopay.core.APIs.ApiBase.createObject(ApiBase.java:242)
at com.mangopay.core.APIs.ApiBase.createObject(ApiBase.java:260)
at com.mangopay.core.APIs.ApiBase.createObject(ApiBase.java:274)
at com.mangopay.core.APIs.implementation.UserApiImpl.create(UserApiImpl.java:50)
at com.mangopay.core.APIs.implementation.UserApiImpl.create(UserApiImpl.java:41)

I wonder if the trigger is not trying to create a new User via the sdk. I will try to explore this clue.

Thanks.

thonier · 2018-09-03T14:31:40Z

So it does seem to happen consistently on our test server when doing a mango User creation - with the stack trace above. However if I attempt to debug it locally, all goes well, so no idea what is going on. Not sure how to get you more info on that.

Any suggestion?

thonier · 2018-09-07T16:53:30Z

Not sure if that helps, but I have compiled from source and added some debug log. It turns out the faulty message you are trying to parse (and which comes back from Mango presumably) is:

<html><head><title>403 Forbidden</title></head><body bgcolor="white"><center><h1>403 Forbidden</h1></center><hr><center>cloudflare</center></body></html>

That makes no sense to me, but maybe it does to you?

Thanks

thonier · 2018-09-08T17:03:34Z

So I figured out what triggered the 403 issue, though I have no idea why.

We had the following piece of code somewhere which was sometimes called. As soon as it was, the MangoPay SDK would instantly stop working.

          Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider());
          TrustManager[] trustAllCerts = new TrustManager[]{
              new X509TrustManager() {
                  public X509Certificate[] getAcceptedIssuers() {
                      return null;
                  }
                  public void checkServerTrusted(X509Certificate[] certs, String authType) throws CertificateException {
                      return;
                  }
                  public void checkClientTrusted(X509Certificate[] certs, String authType) throws CertificateException {
                      return;
                  }
              }
          };
          SSLContext sc = SSLContext.getInstance("SSL");
          sc.init(null, trustAllCerts, new SecureRandom());
          HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
          HostnameVerifier hv = new HostnameVerifier() {
              public boolean verify(String urlHostName, SSLSession session) {
                  if (!urlHostName.equalsIgnoreCase(session.getPeerHost())) {
                      System.out.println("Warning: URL host '" + urlHostName + "' is different to SSLSession host '" + session.getPeerHost() + "'.");
                  }
                  return true;
              }
          };
          HttpsURLConnection.setDefaultHostnameVerifier(hv);

On our end we can remove this "not so safe" code anyways, but you may want to add some protection our your end to prevent possible global side effects like this one.

Thanks.

noguespi · 2019-01-10T16:15:02Z

Got the same issue, I don't know yet where does it come from, using 2.5.0.
Could it come from oauth ? I'm using the MangoPayApi object as a singleton.

noguespi · 2019-01-10T16:47:37Z

With debugging, exact same error as @thonier

[2019-01-10 16:37:55,166] [XNIO-1 task-2] INFO  c.m.c.RestTool - Response ERROR: <html><head><title>403 Forbidden</title></head><body bgcolor="white"><center><h1>403 Forbidden</h1></center><hr><center>cloudflare</center></body></html>

noguespi · 2019-01-10T17:17:10Z

it looks like a cloudflare blacklist for us.

mickaelpois · 2019-01-10T17:20:17Z

Hello @noguespi,
Thanks for your quick analysis on this issue, which seems useful, as some operations has been done on our servers regarding Cloudfare activation.

Are you facing this issue on Sandbox and/or Production env. ?

Mickaël

noguespi · 2019-01-10T17:26:58Z

Hello @mickaelpois I just mailed your support about this issue.

So it only happens on the sandbox API we don't have a production account yet.

It triggers only from our dev server, I didn't faced the issue from localhost.

I can reach you by mail with more technical details (IP, client ID) if you want.

mickaelpois · 2019-01-10T17:30:44Z

@noguespi Feel free to share some technical infos (and feel free to erase all sensitive infos).
Thanks a lot.

Mickaël

noguespi · 2019-01-11T08:35:31Z

Not sure how it is related to cloudflare ban, because when I restart the app, I don't get the 403.
But after a first request to https://api.sandbox.mangopay.com/v2.01/mkpsas-dev/payins/card/web/ I get a 403 error on the second request.

noguespi · 2019-01-14T18:00:29Z

OK so I worked all day on it and I think I found the issue.

I did try to handle and reuse the cookies (cloudflare send a __cfduid= 1 year cookie) but it didn't solve the issue.
I did try to set a desktop user-agent, still 403 after the first request.

Then I refactored RestTool to isolate all the HTTP stuff and used another http client (the new java http client shipped with java 11).

At first it didn't worked, because I was using it the same way HTTPUrlConnection (legacy java http client) works. It works like this :

So it does create a TCP connection to cloudflare server, send a HTTP/1.1 request, close the connection.
Then, the second request it a create a new TCP connection, send a HTTP/1.1 request, and then I'm (randomly) blocked.

There is nothing wrong here, but I suspect it trigger a flag to cloudflare and increase your bad behavior score because an HTTP/1.1 connection is supposed to reuse the previous TCP connection for subsequent requests (it is not mandatory, and there is nothing wrong in not doing it).

I have another big hint cloudflare may be using a mix of TCP layer and HTTP protocol information : when I add the "Connection: close" header, I'm blocked at the second request. See :

package mkp.scratch;

import java.io.ByteArrayOutputStream;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.URL;

class Scratch {

    public static void main(String[] args) throws Exception {
        Scratch scratch = new Scratch();
        scratch.request();
        Thread.sleep(3_000);
        scratch.request();
    }

    String url = "https://api.sandbox.mangopay.com/v2.01/sdk-unit-tests/clients";
//    String url = "https://proxychecker.serphacker.com/";
    String authorizationHeader = "Basic c2RrLXVuaXQtdGVzdHM6Y3FGZkZyV2ZDY2I3VWFkSE54eDJDOUxvNkRqdzhaZHVMaTdKOVVTVG11OGJoeHhwanU=";

    void request() throws Exception {

        URL url = new URL(this.url);

        final HttpURLConnection connection = (HttpURLConnection) url.openConnection();
        connection.addRequestProperty("Authorization", authorizationHeader);
        connection.addRequestProperty("Connection", "close"); // comment this line to stop 403 triggering

        System.out.println(connection.getResponseCode());
        System.out.println(toString(connection.getErrorStream() != null ? connection.getErrorStream() : connection.getInputStream()));
    }

    String toString(InputStream inputStream) throws Exception {
        ByteArrayOutputStream result = new ByteArrayOutputStream();
        byte[] buffer = new byte[1024];
        int length;
        while ((length = inputStream.read(buffer)) != -1) {
            result.write(buffer, 0, length);
        }
        return result.toString("UTF-8");
    }

}

I think it is a cloudflare anti ddos protection which is blocking suspicious behavior. You won't be systematically blocked, it may depends on many factors like your IP reputation, the kind of request you do (POST are more likely to get blocked than GET) and many more things.

Now with the java 11 client I can reuse the TCP connection (just reuse the HttpClient instance) and I'm no more blocked, but for how long ?

I have yet to try an HTTP/1.0 request with a connection: close header which should be totally valid but I have to use apache HTTP client to try that.

A hack would be to use the apache HTTP client and try to looks the more "legit" possible.

The real problem is on your server and cloudflare. The bot mitigation/detection should be disabled, only a layer 3 protection protecting bandwith attack should be activated. Only bots and scripts are consuming the api so using a bot mitigation anti ddos looks like a big issue for me.

noguespi · 2019-01-15T09:30:06Z

I did implement HTTP interface using https://hc.apache.org/ libs with HTTP/1.0 protocol and Connection: close header. I recreate a new TCP connection for each request to API, I have not been blocked so far.

felixhagspiel · 2019-08-27T11:17:03Z

I get the same error when creating a new UboDeclaration with SDK v2.7.0:

12:05:15,548 ERROR [stderr] (default task-10) com.google.gson.JsonSyntaxException: com.google.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 18 path $
12:05:15,555 ERROR [stderr] (default task-10) 	at com.google.gson.JsonParser.parse(JsonParser.java:65)
12:05:15,555 ERROR [stderr] (default task-10) 	at com.google.gson.JsonParser.parse(JsonParser.java:45)
12:05:15,555 ERROR [stderr] (default task-10) 	at com.mangopay.core.RestTool.checkResponseCode(RestTool.java:1076)
12:05:15,555 ERROR [stderr] (default task-10) 	at com.mangopay.core.RestTool.doRequest(RestTool.java:426)
12:05:15,556 ERROR [stderr] (default task-10) 	at com.mangopay.core.RestTool.request(RestTool.java:142)
12:05:15,556 ERROR [stderr] (default task-10) 	at com.mangopay.core.APIs.ApiBase.createObject(ApiBase.java:242)
12:05:15,556 ERROR [stderr] (default task-10) 	at com.mangopay.core.APIs.ApiBase.createObject(ApiBase.java:260)
12:05:15,556 ERROR [stderr] (default task-10) 	at com.mangopay.core.APIs.implementation.UserApiImpl.createUboDeclaration(UserApiImpl.java:243)

All other calls are working fine.
EDIT:
I did some debugging and the JSON that is passed to the JsonParser is actually this HTML:

<html><head> <title>Error Page</title> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <link rel="stylesheet" type="text/css" href="/Content/themes/barnie/css/bootstrap.min.css" /> <link rel="shortcut icon" href="/Content/themes/barnie/img/custom/favicon.png"> <style type="text/css"> body { background: #f7f7f7 url(/Content/themes/barnie/img/main-back.png) repeat; text-align: center; } div { margin-top: 50px; } div.title { font-size: 5em; font-weight: bold; } div.message { font-size: 2em; } </style></head><body> <div class="logo"> <img src="/Content/themes/barnie/img/custom/logo.png"> </div> <div class="title">:-(</div> <div class="message">Oops, that page can not be found.</div></body></html>

mangomaxoasis · 2020-06-19T12:28:05Z

Hi All,
https://github.com/Mangopay/mangopay2-java-sdk/pull/150/files
A big gson was released last autumn
This should resolve a number of these issues, please contact us if problems persist
Max

thonier closed this as completed Jul 23, 2018

thonier reopened this Aug 2, 2018

mickaelpois added the analysis needed label Aug 24, 2018

mickaelpois assigned catacraciun Feb 26, 2019

mangomaxoasis closed this as completed Jun 19, 2020

hikashop-jerome mentioned this issue Mar 17, 2021

Cannot perform authentication from some servers Mangopay/mangopay2-php-sdk#473

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Getting com.google.gson.stream.MalformedJsonException on all API calls #120

Getting com.google.gson.stream.MalformedJsonException on all API calls #120

thonier commented Jul 23, 2018

thonier commented Jul 23, 2018

thonier commented Aug 2, 2018

mickaelpois commented Aug 24, 2018 •

edited

Loading

cosmin-margarit commented Aug 29, 2018

thonier commented Sep 3, 2018 •

edited

Loading

thonier commented Sep 3, 2018

thonier commented Sep 7, 2018

thonier commented Sep 8, 2018

noguespi commented Jan 10, 2019

noguespi commented Jan 10, 2019

noguespi commented Jan 10, 2019

mickaelpois commented Jan 10, 2019

noguespi commented Jan 10, 2019

mickaelpois commented Jan 10, 2019

noguespi commented Jan 11, 2019

noguespi commented Jan 14, 2019

noguespi commented Jan 15, 2019

felixhagspiel commented Aug 27, 2019 •

edited

Loading

mangomaxoasis commented Jun 19, 2020

Getting com.google.gson.stream.MalformedJsonException on all API calls #120

Getting com.google.gson.stream.MalformedJsonException on all API calls #120

Comments

thonier commented Jul 23, 2018

thonier commented Jul 23, 2018

thonier commented Aug 2, 2018

mickaelpois commented Aug 24, 2018 • edited Loading

cosmin-margarit commented Aug 29, 2018

thonier commented Sep 3, 2018 • edited Loading

thonier commented Sep 3, 2018

thonier commented Sep 7, 2018

thonier commented Sep 8, 2018

noguespi commented Jan 10, 2019

noguespi commented Jan 10, 2019

noguespi commented Jan 10, 2019

mickaelpois commented Jan 10, 2019

noguespi commented Jan 10, 2019

mickaelpois commented Jan 10, 2019

noguespi commented Jan 11, 2019

noguespi commented Jan 14, 2019

noguespi commented Jan 15, 2019

felixhagspiel commented Aug 27, 2019 • edited Loading

mangomaxoasis commented Jun 19, 2020

mickaelpois commented Aug 24, 2018 •

edited

Loading

thonier commented Sep 3, 2018 •

edited

Loading

felixhagspiel commented Aug 27, 2019 •

edited

Loading