Some logs are uploaded with corrupted JSON/inline garbage data

[erichoracek]

Describe the bug
We are encountering an issue where the Segment SDK is uploading malformed JSON.

We believe approximately 2.5-5% of uploads are exhibiting this issue, and we see it across a large range of our app versions.

To Reproduce
We haven't been able to reproduce this locally in the debugger, but have many logs and examples from production.

Expected behavior
The SDK should not upload malformed JSON.

Screenshots
N/A

Platform (please complete the following information):

• Library Version in use: 1.4.7 to 1.5.5
• Platform being tested: iOS
• Integrations in use: None

Additional context
Here are some examples:

Batch elements with garbage data

{
    "batch": [
        { "redacted", "redacted" },
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,
        { "redacted", "redacted" },
    ],
    "sentAt": "redacted",
    "writeKey": "redacted"
}

Multiple concatenated batches in a single batch

{
    "batch": [
        { "redacted", "redacted" },
        { "redacted", "redacted" },
        { "redacted", "redacted" },
    ],
    "sentAt": "redacted",
    "writeKey": "redacted"
},
{ "redacted", "redacted" },
{ "redacted", "redacted" },
],
"sentAt": "redacted",
"writeKey": "redacted"
}

Empty batch

],
"sentAt": "redacted",
"writeKey": "redacted"
}

[erichoracek]

We are currently updating our app to use SDK version 1.5.9 and will let you know if this continues to occur after, it appears that 1.5.5 did not contain these changes #304

[bsneed]

Thanks @erichoracek, please keep me posted. I'm hoping that issue is now resolved (along with others). Please reopen this if you're seeing any problems and I'll jump on it.

[erichoracek]

@bsneed We have deployed newer versions of our app with those changes integrated and are unfortunately still seeing malformed JSON being uploaded with SDK versions above 1.5.9. Can we reopen this issue?

[bsneed]

Hi @erichoracek, it might be best to reach out to friends@segment.com or contact support. They'll be able to dig into your sources/workspace and help troubleshoot or identify what's happening. Happy to reopen and work with you here on it as well, but I wouldn't have the visibility that they do.

If you have any additional details or info, lemme know! Thanks Eric!

[erichoracek]

We are using Segment with a custom apiHost and cdnHost, so it doesn't go through the segment.com backend systems.

[bsneed]

Sorry I missed your reply Eric, was digging into this more today and came back. So you're not sending data to segment at all, just using the sdk?

[bsneed]

Made some additional changes in #343 and more stress testing along with file validation during stress test. I don't think this needs to be open any longer.