Workflow 2.0

[dcramer]

Our core workflow has begun to suffer over the years - from growth of technology, scale of data, and general lack of attention. Recognizing this, we had the following conversation this week internally, and are setting out to tackle this concern. That core workflow includes several components that are loosely coupled together:

• the release metadata which allows us to identify the version of code an event is present in
• the new alert (and other variations) approach to notifications
• the new deployment notification - which requires a good amount of effort to achieve correctness
• the resolved in release and ignore issue actions and the general triage flow
• the regression notification which is key to understanding if an issue remains unresolved

These all string together to create a workflow that is tightly coupled to how I - and I believe most developers - think about shipping code. We dry run a bunch of changes, pray our tests are accurate enough, ship the code, and inevitably find a problem. The faster Sentry can connect those dots with accurate diagnostics, the better the outcomes for our customers.

The challenge here is that there's a fundamental gap in the workflow in how we notify about issues. We rely on "New Issue" to be timely and contextual. That is, we duct taped a solution that assumed most of the time new issues happened with new code. That's not always true, and technologies like JavaScript have decreased the signal to noise ratio over the years. So let's tackle that problem. There's a few key things we should look at as part of this:

1. What does a timely notification look like connected to the release lifecycle? That should focus on how we help identify truly "new issues caused by code changes".
2. How do those notifications change (or become addititive) with things like code push or feature flags, where we're making behavioral changes but not shipping a new SHA?
3. How can we improve things like "Ignore Issue" to be more functional usable? We shouldn't rely on custom alerts, or custom saved searches for such common concerns.
4. "Resolved in Next Release" can be problematic in many environments, but we now have SemVer support. Can we leverage that better?
5. Where is fingerprint breaking down? Are there platforms that are not effective that we need to resolve our native heuristics?
6. How does this apply to performance? to csp? to other "problems" (ala "issues")? Issues needs to be a platform to enable the workflow.

Most importantly, with all this in mind, how do we resolve the workflow without requiring customers to configure anything? We've - for better, or more likely worse - taken the approach over the years to put this problem into our customers hands by giving them complicated solutions to create alerts, complicated solutions to improve fingerprinting, and we've stopped trying to build a first-class curated solution to the problem. This is our chance to correct that.

For the community, if you have opinions, what are they? What can we do better here? What works well? What is completely terrible? We already have some solid foundations, but if you've got an opinion, let's hear it!

[alexbjorlig]

In my perspective the key issue with Sentry is configuration and setup. I think the solution to this problem is for Sentry to carefully choose a set of technologies and platforms to support; and then really support thoose. A bit like how the adapter architecture with SvelteKit works. You develop a solution, but can essentially ship it anywhere as long as there is an adapter provided.

Thanks for being open about this - cool decision 😎

[chadwhitacre]

carefully choose a set of technologies and platforms to support; and then really support those

@alexbjorlig You mean, like, limit the number of SDKs we support? Sentry has a core value "for every developer" so that goes against the grain, but maybe there's more we can do to support community-supported SDKs? Or better clarify which are "officially" supported and which are community supported? What SDK/platform do you use Sentry with the most?

[alexbjorlig]

carefully choose a set of technologies and platforms to support; and then really support those

@alexbjorlig You mean, like, limit the number of SDKs we support? Sentry has a core value "for every developer" so that goes against the grain, but maybe there's more we can do to support community-supported SDKs? Or better clarify which are "officially" supported and which are community supported? What SDK/platform do you use Sentry with the most?

I think a fundamental change of perspective is needed. SDK's, APIS's and CLI's are nice, but not what really deliver value. It's the working configuration, that actually ships the correct artifacts to Sentry and makes source-maps from typescript work. And currently Sentry has no solution for this - the only answer is to carefully configure SDK's and CLI usage.

In my vision for Sentry, SDK's, API's and CLI's are tools that Sentry's employees (and expert users from the community) use to build adapters. An adapter would for example be:

Sentry for Next on Heroku
Sentry for Next on Vercel
Sentry for Next on Firbase
Sentry for Sveltekit on Heroku
Sentry for Sveltekit on Vercel
Sentry for Sveltekit on Firebase
Sentry for Vue on Heroku
Sentry for Vue on Vercel
Sentry for Vue on Firebase

I completely understand that there are almost infinite combinations. Howerver the benefit of such an adapter setup would be the configuration hassle is now on Sentry developers and not developers having to figure out a ton of things --> out of the box all essential stuff is configured.

I have personally dealt with the setup for source-maps for an Angular app deployed to Heroku, and a Sveltekit app deployed to Heroku. It's really not easy at all. An abnormal amount of effort and pain is needed for just small things to work.

[dcramer]

I think a fundamental change of perspective is needed. SDK's, APIS's and CLI's are nice, but not what really deliver value. It's the working configuration, that actually ships the correct artifacts to Sentry and makes source-maps from typescript work. And currently Sentry has no solution for this - the only answer is to carefully configure SDK's and CLI usage.

FWIW we are aligned. Sometimes its a tricky balance of us trying to do too much, or accepting a not-quite-mature SDK into the main stream. A good example is our challenge with Next.js support recently. We built out a POC, but even after 2 major iterations we still havent quite nailed the UX.

I think your sentiment aligns well with our improvements to Workflow, but we do have quite a few resources on engineering and so its less of a problem of us providing a curated SDK experience, and more that we need to build better community/feedback cycles to make sure we're doing a great job.