Skip to content

Conversation

@lobsterkatie
Copy link
Member

@lobsterkatie lobsterkatie commented Jul 27, 2021

In the @sentry/cli code, the Releases. uploadSourceMaps() method iterates over the include array in options and calls a separate upload command for each entry (see here).

This means that it's possible (at least in theory) to have different options for each include entry. This PR brings that theory to fruition.

The motivation here is to make it easier to include multiple sets of files in the upload.

Suppose you have the following file structure:

animals
  aardvarks
    facts
    photos
    research
      google.txt
      wikipedia.txt
      ...
  baboons
    ...
  coatis
    ...
  ...
  zebras
    gifs
      dramatic_zebra.gif
      stampede.gif
      ...
    sightings
    sounds

and you want to upload only the files in animals/aardvarks/research and animals/zebras/gifs. Further suppose that you want to prefix all of your files with ~/zoo.

Under the current system, you might try the following options:

include: ['animals/aardvarks/research', 'animals/zebras/gifs'],
urlPrefix: '~/zoo`

but you'd end up with files with names like ~/zoo/google.txt and ~/zoo/dramatic_zebra.gif when what you need is ~/zoo/animals/aardvarks/research/google.txt and ~/zoo/animals/zebras/gifs/dramatic_zebra.gif. No one urlPrefix works for both, and you'd similarly run into trouble if you tried to use an ignore value which applied to one and not the other, or different ext values for each.

To solve the prefix problem, you could do

include: ['animals'],
ignore: ['animals/baboons/*', 'animals/coatis/*', 'animals/donkeys/*', ... , 'animals/yaks/*'],
urlPrefix: '~/zoo/animals`

but that involves writing out all of the things you don't want (brittle and also a giant pain if there are a lot of them), and it still doesn't solve the "different ext values for each entry" problem (or any of its friends related to other options like stripPrefix, rewrite, and the like).

This solves that by allowing the user to specify options per include value, like this:

include: [ 
  { paths: ['animals/aardvarks/research'], urlPrefix: '~/zoo/animals/aardvarks/research '},
  { paths: ['animals/zebras/gifs'], urlPrefix: '~/zoo/animals/zebras/gifs '},
],

Any include entry which doesn't have its own options specified continues to use the globally-specified ones, as is the case today.

NB: Since this new option is primarily here in order to unblock @sentry/nextjs's use of SentryWebpackPlugin, I have chosen not to document it anywhere, since to my knowledge no one has actually asked for it. If I were to do so, the question would then become whether to do it in the README here, in getsentry/sentry-webpack-plugin's README, in some other place, or any combination of the three. Open to opinions on this score.

@lobsterkatie lobsterkatie changed the title feat(js): Add ability for include in sourcemap upload options to be an object in JS feat(js): Add ability for include in JS sourcemap upload options to be an object Jul 27, 2021
@kamilogorek
Copy link
Contributor

kamilogorek commented Jul 28, 2021

I wonder if we actually need a new option path here. If it's solely for the purpose of unblocking next.js, we could do this recursively and reuse include here tbh.

include: [ 
  { include: 'animals/aardvarks/research', urlPrefix: '~/zoo/animals/aardvarks/research '},
  { include: 'animals/zebras/gifs', urlPrefix: '~/zoo/animals/zebras/gifs '},
]

This would be mapped to Promise.all(Promise.all()) kind of thing, which should work just fine.

Eventually, we could move to webpack-like entry syntax - https://webpack.js.org/concepts/entry-points/ which is kinda what we have with this PR already.

@lobsterkatie
Copy link
Member Author

I wonder if we actually need a new option path here.

we could do this recursively and reuse include

I specifically chose not to do that because I don't think we want to allow nesting, as it's not clear what that'd mean. path really should be just a string (or, I suppose, an array of strings, though that's not currently implemented in this PR), so I named it differently so that it's clear that the options for it are different for the options for include.

As for webpack entry syntax, they actually do this, too. The overall property is called entry, but if you give it an object instead of a string/array of strings, the path goes into import, rather than a nested entry property.

WDYT?

@kamilogorek
Copy link
Contributor

Got it, makes sense, we can go with it. Can we however extract the 3rd possible type and give it some meaningful name? Array<Omit<OtherType is hard to decypher 😅

@lobsterkatie
Copy link
Member Author

lobsterkatie commented Jul 30, 2021

Can we however extract the 3rd possible type and give it some meaningful name? Array<Omit<OtherType is hard to decypher 😅

Done.

I also made two more changes:

  1. The type for include included a string option, even though we don't actually allow that (we've just been calling .map on the value assuming - but not actually checking - that it was an array). I removed that option from the type and added an array check to the existence check at the beginning of the function.

  2. Before this PR, you could have multiple paths in your include array and have all of the other options apply to each one. To make this new type work the same way, I changed path in the descriptor object to paths and made it an array also.

@kamilogorek kamilogorek merged commit fb8a60b into master Jul 30, 2021
@kamilogorek kamilogorek deleted the kmclb-let-sourcemap-upload-include-be-an-object-in-js branch July 30, 2021 09:47
lobsterkatie added a commit to getsentry/sentry-webpack-plugin that referenced this pull request Aug 3, 2021
This mirrors the change in getsentry/sentry-cli#1001, which allows for path-specific options to be passed as part of the `include` option value. Specifically, this:

- Fixes normalization of the `include` option to account for the new possibility of it being an object, and of the `ignore` option within that object
- Adds tests for the normalization
- Updates the README to reflect the new possibility
- Fixes a bug in the normalization function `toArray()` so that falsy (but defined) values aren't ignored. We don't actually ever use it in a way which would make this an issue, but it seemed silly to leave the bug in place when the fix was so simple.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants