2023 and the JavaScript SDK

[AbhiPrasad]

2023 and the JavaScript SDK

As we look toward the next year, we wanted to take some time to write down higher level initiatives we can take a look at:

1. Unplugin (expanding the bundlers) https://github.com/getsentry/hackweek-sentry-unplugin
2. Isolated exceptions/transactions (micro-frontends/browser extensions/3rd party scripts isolation) [request feedback] Micro Frontend support #5217

• Hub/scopes changes
• Parent child relationship with tracing
• Establishing proper context propogation

3. Support different JS runtimes Support different JS runtimes #5611
4. Extensible lifecycle hooks for the SDK SDK lifecycle hooks #7262
5. Node SDK improvements (async hooks Introduce Async Hooks into the SDK #7691, v8 instrumentation)
6. Sessions in the Browser What is a session? sentry-replay#38
7. Sourcemaps improvements
8. Becoming community leaders
9. Dynamic Loading of SDK functionality

Some of this can only be solved with a new major version, but we've started to think about that: https://github.com/getsentry/sentry-javascript/milestone/15

Unplugin

✅ Done with https://github.com/getsentry/sentry-javascript-bundler-plugins/

Unplugin is a unified plugin system for JavaScript bundlers. Using the unplugin library, we can have a single codebase that support 4 bundlers: Webpack, Rollup, Vite and Esbuild.

Right now we only support Webpack though the Sentry Webpack Plugin, and so by adopting Unplugin, we would get 3 extra bundlers out of the box! Supporting more bundlers leads to better out of the box experience for users - they don't have to worry about figuring out sourcemap uploads or setting up releases.

As per the numbers, combining Vite, Esbuild and Rollup gives us 20 million downloads total, 80% of webpack's total downloads, allowing us to improve the Sentry experience for thousands of orgs and users.

Expanded support for JavaScript bundlers also provides us the opportunity to do more build time manipulation. For example, we had a great amount of success in our NextJS SDK automatically generating and injecting code at build-time to wrap functions for performance monitoring. If we supported every major bundler, we could do similar things for frameworks like React, Vue and Angular. Imagine automatically detecting the most high impact components and wrapping them with Sentry instrumentation - no configuration needed.

None of our competitors are thinking about expanding support for bundlers or exploring this problem space - we'll have a real first mover advantage if we invest here.

Exceptions/Scope Data Quality (micro-frontends/browser extensions/3rd party scripts isolation)

#7622

Right now with the current model for the unified API, there are possibilities of data quality issues. These are apparent in two ways.

1. Multiple instances of the SDK on a single page

When you are running multiple instances of the SDK on the same page like with (Micro Frontend with multiple apps) or have a chrome extension that runs Sentry on your Sentry instrumented webpage, often times data is not routed correctly to the correct Sentry instance. All current attempts at making this work are unsuccessful - and it's a source of pain from customers. The web can be a wild west - we need to make sure that our SDKs account for these more interesting scenarios.

2. Incorrect parent-child relationship between spans

Because of how having multiple hubs/scopes/clients on the same page works, it's hard for us to maintain a proper parent-child relationship between executed spans. In performance monitoring, we associate all spans to the root transaction, which makes it difficult for user to understand execution flows. It also reduces our ability to generate more impactful performance issues.

The root cause of both of these issues are our current hub/scopes model. We don't correctly follow the flow of execution to ensure that exceptions are routed to the correct sentry instance, and scope is applied properly to the correct exception/transaction.

Steps:

1. Fix getCurrentHub() in Node (for ex. setTimeout() does not work). We do this by using async_hooks in Node.
2. Monkey-patch all native async calls + Promises in the browser. This makes hub.run() work in the browser.
3. Re-evaluate how global instrumentation works with multiple hubs. How do we route global errors, breadcrumbs to a hub.
4. Decide what breaking changes to make for v8 (getting rid of the hub, removing pushScope and popScope)

(Potentially useful for step 2: https://developer.chrome.com/blog/devtools-modern-web-debugging/#linked-stack-traces)

WinterCG Support

WinterCG is a spec for JavaScript runtimes. It essentially describes a base amount of APIs and standards a JS runtime has to follow. As per what they say:

The Web-interoperable Runtimes Community Group (WinterCG) is a community of people who are interested in using Web Platform APIs outside of browsers, namely on the server (Deno / Node.js) or edge runtimes (Cloudflare Workers / Deno).

We only support NodeJS through @sentry/node, but right now it's not possible to support other things because we use NodeJS APIs everywhere. By making changes to how we structure our packages (which Tim describes here: #5611 (comment)), we can make it easy for SDKs for Deno, Cloudflare Workers, and Vercel Edge Middleware (and whatever else comes next) to be built.

This means we can expand our support to more developers - and help push the server-side JavaScript community.

[Before Major] Part 1: Make changes to get it working for errors.

[v8 - Breaking Changes] Part 2: Update tracing package to make it work with performance.

This is also an opportunity for us to get involved early in this working group and help define standards around errors, performance, profiling and more in this community.

Extensible lifecycle hooks for the SDK

✅ Done with getsentry/rfcs#34 and #7262

Recently Sentry announced some exciting news - we would be adding products for Browser Session Replay and Profiling. These products will become new JavaScript SDKs that we have to start looking to support.

• https://github.com/getsentry/sentry-replay
• https://github.com/getsentry/profiling-node

These will have to be separate packages though, since we don't want them to be bundled. In browser this is especially important, but for profiling this is important because bundle size also matters for serverless.

One thing we noted was that hooking into the SDK was difficult as the entry points into the SDK were not exposed (@JonasBa for Profiling and @billyvg for Replay to comment more about this). In the SDK we only expose the event processors as part of the integrations API, so Profiling and Replay folks ended up having to monkey patch the API to construct new envelope types and send them to Sentry.

To solve this, and future issues, we can adopt a hook system, where users of the SDK can register callbacks that will be executed by the SDK at various parts of the event processing pipeline.

For example:

Sentry.register('on-session-create', () => {});
Sentry.register('on-envelope-create', () => {});

Node SDK improvements

There are a variety of improvements we can make to the Node SDK to improve reliability, performance, and data quality.

1. Adopting Async Hooks. Node Domains are deprecated, and all other APM vendors (including OpenTelemetry) use Async Hooks for instrumentations. Domains are also slow, and can cause unexpected behaviour in complex use cases.

✅ Done with #7691

3. Expanding instrumentation by hooking into v8. The Node Profiling SDK hooks into the Native Layer for Profiling. We can use similar bindings to expand instrumentation for both errors and performance monitoring on the Node SDK.
4. Introducing specific packages for Node frameworks, like @sentry/express and similar.

We can also look at using worker threads.

Sessions in the Browser

Currently sessions in the Browser are page based. Every history change we generate a new session.

With the introduction of a session replay product though, we now have two different semantic meanings of a session. Sessions in the Session Replay product persist between navigations, and even tab switches, since they use SessionStorage to track session state.

Having two definitions about will be very confusing to users (since it means different numbers will show up on the Release Health screen vs. what they are being billed for in Replay) - we need to make sure we align here.

A part of this is also exploring instrumenting user Interactions in the Browser #5750

Sourcemaps Improvements

✅ Done with https://www.notion.so/sentry/Debug-IDs-for-Source-Maps-Brainstorming-66601f1cd5f646a399f9b46fa0dfa3a5

By adopting unplugin, we can look at further improvements around sourcemaps. For example, we could inject metadata at the module level in the browser bundle so that we automatically can associate sourcemaps to errors without needing to rely on a configured release or proper filename.

TODO more ideas

Becoming Community Leaders

At Sentry we're exposed to JavaScript at all levels building our SDKs. We support SDKs for Browser, Node, Electron, React Native, and more, and deal with low/high level APIs in all these platforms. With that domain knowledge, we should work toward getting more involved with the JavaScript community, notably with TC39 and W3C, so we can help push forward our opinions on error tracking and performance monitoring.

Dynamic Loading of SDK functionality

• User Feedback Integration
• React.lazy()
• dynamic imports of integrations?
• dynamic imports of client processing pipeline?

[AbhiPrasad]

Roadmap

We put together a rough roadmap for the 3 top items listed above. Internal Sentry Notion link.

Unplugin

1. [PRE] Update unplugin to add necessary build hooks [Medium Risk, Medium Reward]
   1. We need this for sourcemaps upload
   2. Option to fork unplugin if we can’t get our changes merged in
2. [PRE] Feature parity with Webpack Plugin [Low Risk, High Reward]
   1. All the webpack plugin options supported
   2. PRE because we only release for Rollup, Vite, Esbuild
      1. Webpack Plugin is blocked on how we decide to approach sentry-cli configuration (.sentryclirc + sentry.properties + environmental variables)
   3. Demo: create-vite-app + Vite unplugin just works™️
3. [BREAKING] Release Webpack Plugin with unplugin [Medium Risk, Medium Reward]
   1. Need to figure out sentry-cli configuration
   2. Remove dependency on sentry-cli binary
4. [PRE] Use Unplugin Webpack Plugin in NextJS SDK [Medium Risk, Low Reward]
5. Deprecate old Webpack Plugin Repo [HAS TO HAPPEN]
   1. 🫡

https://github.com/getsentry/hackweek-sentry-unplugin

WinterCG + Multiple JS Runtimes

1. [PRE] Make it work for the Errors product [Low Risk, High Reward] Support different JS runtimes #5611
   1. Demo: Have Sentry run on NextJS Middleware and send an error
2. [PRE] Have someone from the team join the WinterCG working group [Medium Risk, Medium Reward]
   1. Demo: a nice #shipped email
3. [BREAKING] Make it work for the Performance product [Medium Risk, High Reward]
   1. Remove @sentry/tracing for Node and Browser (no separate package, no extra installation complications, no versioning problems)
   2. Open up opportunity for 1st party WinterCG SDK (NextJS Middleware, Deno)
   3. Demo: SDK works with Deno

TODO: Think about #5886

Exceptions/Scope Data Quality (micro-frontends/browser extensions/3rd party scripts isolation)

1. [PRE] Move toward async_hooks instead of domains in the Node SDK [Low Risk, Low Reward]
   1. Possible performance improvements, better parent-child reference for spans, removal of deprecated API that people complain about (domains)
   2. We learn about how hub propagation/scope propagation APIs should work, figure out edge cases, do more research about async context mechanisms
   3. Demo: Fastify (framework we don’t support) works without explicitly using Domains
   4. Demo: Compare performance before vs. after
2. [PRE] Wrap Browser built-ins (native async methods + Promises) to automatically propagate hub/scope [High Risk, High Reward]
   1. Same behaviour as using async hooks in Node for the Browser: better parent-child reference for spans, correct hub/scope being used during getCurrentHub call.
   2. Possible bundle size impact, have to explore edge cases/unknowns
   3. Have to think about if this is opt-in or not (can users configure what behaviour they want?)
   4. Goal: Make hub.run work
   5. Demo: Chrome extension can initialize Sentry without interfering with website Sentry
   6. Demo: Webpack Module Federation Micro Frontend sends errors to correct Sentry instance
3. [PRE] Out of the box Webpack Module Federation Support [Medium Risk, Medium Reward]
   1. We automatically wrap modules with hub.run to isolate them using the Webpack Plugin
   2. Demo: Webpack Module Federation Micro Frontend sends errors to correct Sentry instance without any extra config
4. [BREAKING] Rethink Integrations Behaviour + API [High Risk, High Reward]
   1. How do integrations get registered?
   2. How do breadcrumbs/spans get routed to a hub? What hub do they go to?
   3. Managing integrations that monkeypatch vs. integrations that apply event processors?
5. [BREAKING] Restructure the hub/scope API [High Risk, Low/Medium Reward]
   1. Re-evaluate hub.run vs. withScope
   2. Re-evaluate pushScope and popScope
   3. Re-evaluate bindClient
   4. Do we need both Clients and Hubs?
   5. No demo, just better API for users :)

[nwalters512]

Very excited to see that you're planning a move away from Domains in Node! That should help fix complex issues like #7031.

The "Node SDK improvements" mentions using Async hooks, but the Node docs explicitly recommend migrating away from it. AsyncLocalStorage is almost certainly what you want instead.

[AbhiPrasad]

Done with https://github.com/getsentry/sentry-javascript/releases/tag/7.48.0!

[cstavitsky]

Making a comment for folks to express interest in official Sentry micro-frontend/microapp support.

Give this a 👍 if you want micro-frontend support to be prioritized!

Per the top-level post:

Isolated exceptions/transactions (micro-frontends/browser extensions/3rd party scripts isolation) #5217
Hub/scopes changes
Parent child relationship with tracing
Establishing proper context propogation

[AbhiPrasad]

I wrote down #7622 as a preliminary step we should take to make this experience better.

[schankam]

Really looking forward for better scope support for http servers. Currently running Apollo and transactions are not relevant / reliable in the current implementation.

Do you guys have an approximative ETA for when this will be released ?

[AbhiPrasad]

@schankam for scope isolation, you can use domains for now. This is not well documented, but see #4947 (comment) for more details.

[AbhiPrasad]

We've also started work on better scope isolation in #7691 - we'll have something you can test with in the next 1-2 weeks!

[AbhiPrasad]

Done with https://github.com/getsentry/sentry-javascript/releases/tag/7.48.0 - see our docs on https://docs.sentry.io/platforms/node/configuration/async-context/.