Skip to content

Envelope Only Transport API #426

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

denrase
Copy link
Collaborator

@denrase denrase commented Apr 20, 2021

📜 Description

  • Change Transport API to only offer send(envelope: SentryEnvelope) method.
  • Implement deserialization form JSON for SentryEvent an associated models.

💡 Motivation and Context

  • We could not construct recreate SentryEvent instances once they were serialized to an envelope, making it difficult to test transport implementations.

💚 How did you test it?

  • Added unit tests for json deserialization of all affected models.

📝 Checklist

  • I reviewed submitted code
  • I added tests to verify changes
  • I updated the docs if needed
  • All tests passing
  • No breaking changes

🔮 Next steps

  • Cleanup

#skip-changelog

@denrase denrase self-assigned this Apr 20, 2021
@ueman ueman mentioned this pull request May 7, 2021
8 tasks
@marandaneto
Copy link
Contributor

@bruno-garcia would you mind doing a final review? quite busy these days.

Copy link
Member

@bruno-garcia bruno-garcia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be beneficial to add some snapshot tests to validate that we're not breaking compatibility with data that's already serialized. The round trip tests are great but have this limitations.

Worth noting this implementation is not keeping track of json data that doesn't map to the type we have and will lose data if we deserialize an envelope written by an SDK on a newer version, with new fields.

Ideally such fields would be kept on a 'bag' and when/if the type gets serialized again, they are written back with the same key as they had.

That said, we attempted to get that on Android when we wrote it but didn't manage, and don't have it with .NET either.

I'm writing this to bring awareness of this situation, not to state that this blocks the PR.
For example, if sentry-native is writing envelopes, and they introduce new fields without communicating with the mobile team, we won't be adding such fields to the protocol on other layers and the data won't get through to Sentry.

The issue with version (say an app rolls back to an older version of the SDK) is always going to be present though.
The SDK would read that envelope from a newer version and submit a broken version of it.

@jan-auer @marandaneto @Swatinem

@marandaneto
Copy link
Contributor

Could be beneficial to add some snapshot tests to validate that we're not breaking compatibility with data that's already serialized. The round trip tests are great but have this limitations.

makes sense, but serialization didn't change much but the endpoint, @denrase take a look at this.
an example is, install the latest SDK version, capture an envelope and parse it using this version of the SDK, something like that.

Worth noting this implementation is not keeping track of json data that doesn't map to the type we have and will lose data if we deserialize an envelope written by an SDK on a newer version, with new fields.

Ideally such fields would be kept on a 'bag' and when/if the type gets serialized again, they are written back with the same key as they had.

That said, we attempted to get that on Android when we wrote it but didn't manage, and don't have it with .NET either.

I'm writing this to bring awareness of this situation, not to state that this blocks the PR.
For example, if sentry-native is writing envelopes, and they introduce new fields without communicating with the mobile team, we won't be adding such fields to the protocol on other layers and the data won't get through to Sentry.

The issue with version (say an app rolls back to an older version of the SDK) is always going to be present though.
The SDK would read that envelope from a newer version and submit a broken version of it.

@jan-auer @marandaneto @Swatinem

raised #469

thanks @bruno-garcia

denrase added 2 commits May 17, 2021 10:53
…lope-only-transport-api

# Conflicts:
#	dart/lib/src/protocol/sentry_device.dart
#	dart/lib/src/protocol/sentry_level.dart
#	dart/test/protocol/device_test.dart
#	flutter/test/mocks.mocks.dart
@denrase
Copy link
Collaborator Author

denrase commented May 17, 2021

makes sense, but serialization didn't change much but the endpoint, @denrase take a look at this.
an example is, install the latest SDK version, capture an envelope and parse it using this version of the SDK, something like that.

Not sure if I follow. 🤔 Is this something we can do as a test or just something to check manually? Which SDKs are we talking about when its "the latest" and "this version of the SDK"? Should I capture an envelope using the latest native SDK, serialize it and try to deserialize it with the new functionality of this PR and add this as a test?

@marandaneto
Copy link
Contributor

makes sense, but serialization didn't change much but the endpoint, @denrase take a look at this.
an example is, install the latest SDK version, capture an envelope and parse it using this version of the SDK, something like that.

Not sure if I follow. 🤔 Is this something we can do as a test or just something to check manually? Which SDKs are we talking about when its "the latest" and "this version of the SDK"? Should I capture an envelope using the latest native SDK, serialize it and try to deserialize it with the new functionality of this PR and add this as a test?

yeah sorry I could be more specific, so here we go:

Use the Android SDK only, no Flutter, on an emulator, disable the internet so you can inspect the file locally, the important bits is here https://github.com/getsentry/sentry-java/blob/main/sentry-samples/sentry-samples-android/src/main/java/io/sentry/samples/android/MainActivity.java#L33-L52

the envelope would contain an event and an attachment (binary data), now you copy this envelope to the flutter tests and assert the deserialization.

now with the deserialized envelope, you capture it again in the Flutter SDK, which writes a file to the disk again, and the Android SDK will send it to sentry, the envelope and image should not be broken, that's how we'd know it works as expected.

I guess the deserialization with binary data can be asserted in unit tests, the serialization too, now if the image didn't get broken on sentry, that you can do it manually, we can have a call about that if its the case.

@denrase
Copy link
Collaborator Author

denrase commented May 18, 2021

@marandaneto Pls take a look at the last commits. Envelopes with arbitrary binary data are now supported like we discussed today. Thank you for your help! 🙇

@denrase denrase requested a review from marandaneto May 21, 2021 13:49
@marandaneto
Copy link
Contributor

@denrase the analyze CI step is broken though

Comment on lines +266 to +271
// We need to replace this when we have the possibility to access envelope deserialization.
guard let sentrySerialization = NSClassFromString("SentrySerialization") as? NSObject.Type,
let unmanaged = sentrySerialization.perform(NSSelectorFromString("envelopeWithData:"), with: data),
let envelope = unmanaged.takeUnretainedValue() as? SentryEnvelope else {
print("Cannot parse the envelope data")
result(FlutterError(code: "3", message: "Cannot parse the envelope data", details: nil))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bruno-garcia @philipphofmann can we expose a way how to deserialize an envelope out of an array of bytes? we're always using a String but now with the possibility of attachments etc this would not work great anymore

@marandaneto marandaneto merged commit f9a4bae into feat/implement-envelope-based-transport May 26, 2021
@marandaneto marandaneto deleted the feat/envelope-only-transport-api branch May 26, 2021 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants