fix(nextjs): Widen scope for client file upload

[lobsterkatie]

When nextjs builds an app, compiled versions of each page end up in .next/static/chunks/pages. In some circumstances, however, user-generated code ends up outside of pages/, and in those cases sourcemaps are broken because the relevant files aren't being uploaded.

This fixes that by optionally widening the scope of the upload, while excluding files known to contain only nextjs and webpack code. (Every build will generate other files which should be excluded, but there's no way to tell in advance what they're going to be called.) In order to reduce the number of irrelevant files being uploaded to Sentry, this behavior is off by default, but can be turned on with a new option widenClientFileUpload:

const { withSentryConfig } = require("@sentry/nextjs");

const moduleExports = {
  sentry: {
    widenClientFileUpload: true,
  },
};

const sentryWebpackPluginOptions = {
  // ...
};

module.exports = withSentryConfig(moduleExports, sentryWebpackPluginOptions);

This change is documented in getsentry/sentry-docs#4827.

Fixes #3896

Ref: https://getsentry.atlassian.net/browse/WEB-254

[github-actions]

size-limit report

Path	Base Size (ba14cc5)	Current Size	Change
@sentry/browser - ES5 CDN Bundle (gzipped + minified)	20.43 KB	20.43 KB	+0.01% 🔺
@sentry/browser - ES5 CDN Bundle (minified)	65.2 KB	65.2 KB	0%
@sentry/browser - ES6 CDN Bundle (gzipped + minified)	19.01 KB	19.01 KB	+0.02% 🔺
@sentry/browser - ES6 CDN Bundle (minified)	58.22 KB	58.22 KB	0%
@sentry/browser - Webpack (gzipped + minified)	22.3 KB	22.3 KB	0%
@sentry/browser - Webpack (minified)	76.58 KB	76.58 KB	0%
@sentry/react - Webpack (gzipped + minified)	22.33 KB	22.33 KB	0%
@sentry/nextjs Client - Webpack (gzipped + minified)	46.56 KB	46.56 KB	0%
@sentry/browser + @sentry/tracing - ES5 CDN Bundle (gzipped + minified)	27.25 KB	27.25 KB	+0.01% 🔺
@sentry/browser + @sentry/tracing - ES6 CDN Bundle (gzipped + minified)	25.62 KB	25.62 KB	+0.01% 🔺

[lobsterkatie]

@iker-barriocanal - since Abhi is out, mind taking a quick look at this?

[iker-barriocanal]

The code looks good. A nit: use more descriptive test names, "...has the correct value..." doesn't tell much about the test when it fails; for example, "excludes next's core files" is more descriptive.

My first question by reading the PR description is whether we want the SDK to behave that way - small scope by default and optionally increase it vs big scope by default and optionally increase it? In terms of UX this means enabling an option for an improvement vs enabling it to do what's expected from the SDK(?). Not familiar with other trade-offs, so your call.

[AbhiPrasad]

What do you think about allowing config to users by allowing them to provide regex/string array/glob instead? This allows them to more finely grained allowlist files, instead of just a generic widen vs. our specific allowlist.

[iker-barriocanal]

Allowing users to provide an allowlist would be ideal.

My initial concern is that the SDK should work out of the box, and then provide alternatives for users who want specific use cases - it's easy and straightforward to get started and everything works, and you have these options that improve your use case. If you think about new issues, they should be closer to "I have X use case and want to improve it so that I get the benefit Y" than "I just installed the SDK and X doesn't work".

[lobsterkatie]

My initial concern is that the SDK should work out of the box, and then provide alternatives for users who want specific use cases

Indeed. The standard, straightforward case is that all user code ends up in /pages, and that presumably is working for most folks, or this would have been a much-more-widely-reported issue ever since #3845 was merged. (If you recall, that was the one which vercel was really pushing to get done.) That said, it is possible for user code to end up outside of /pages, which is why people are having the issue at all.

What do you think about allowing config to users by allowing them to provide regex/string array/glob instead?

Would love to, but here is what the output from a typical nextjs build (a.k.a., the files we have to decide whether or not to upload) looks like:

You can see the problem. The named files at the bottom are always generated, and are only ever nextjs or third-party code, which is why they're excluded, but the rest...

My first question by reading the PR description is whether we want the SDK to behave that way - small scope by default and optionally increase it vs big scope by default and optionally increase it?

The reason not to go big -> small is that the majority of what will get uploaded under the "big" scenario contains no user code and therefore really shouldn't be uploaded at all (which is where the push for that PR I linked above came from in the first place). Because there's no (easy) way to tell which files should get uploaded and which files shouldn't, the only way to get the good ones is to take them all, and we don't want to do that if we don't have to - it's a waste of build time for them and a waste of storage for us.

(The less easy, doesn't-fit-into-the-roadmap-right-now, not-even-dead-sure-we'd-have-access-to-the-right-files-at-the-right-times way is to load all of the sourcemaps, checkout out their respective "sources" entries, and ditch all files which only draw from nextjs or node_modules, or perhaps only include files which draw from user-specified sources.)

I actually would love to try implementing the less easy way at some point (in which case I'd be much more open to switching the default from "small" to "big"), but for now I'm going to merge this because at least it is an improvement from the current state.