Unable to build or serve application that imports @aws/sdk packages

[iamskok]

Describe the bug

I'm trying to migrate React application from CRA to Vite, but experience errors serving and building application. I've checked similar issues with AWS SDK 1, 2, but none of the suggestion seems to work. Not sure if it's related to Vite 3 (most of suggestion were posted for Vite 2) or something else.

Reproduction

malex37/marathon#15

System Info

System:
    OS: macOS 12.4
    CPU: (12) x64 Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
    Memory: 80.32 MB / 16.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 18.1.0 - ~/.nvm/versions/node/v18.1.0/bin/node
    npm: 8.8.0 - ~/.nvm/versions/node/v18.1.0/bin/npm
  Browsers:
    Chrome: 103.0.5060.134
    Firefox: 91.11.0
    Safari: 15.5
  npmPackages:
    @vitejs/plugin-react: ^2.0.0 => 2.0.0
    vite: ^3.0.0 => 3.0.4

Used Package Manager

npm

Logs

vite build --debug

> tsc && vite build --debug

  vite:config bundled config file loaded in 165.95ms +0ms
  vite:config using resolved config: {
  vite:config   plugins: [
  vite:config     'vite:build-metadata',
  vite:config     'vite:pre-alias',
  vite:config     'alias',
  vite:config     'vite:react-babel',
  vite:config     'vite:react-refresh',
  vite:config     'vite:react-jsx',
  vite:config     'vite:modulepreload-polyfill',
  vite:config     'vite:resolve',
  vite:config     'vite:html-inline-proxy',
  vite:config     'vite:css',
  vite:config     'vite:esbuild',
  vite:config     'vite:json',
  vite:config     'vite:wasm-helper',
  vite:config     'vite:worker',
  vite:config     'vite:asset',
  vite:config     'vite:wasm-fallback',
  vite:config     'vite:define',
  vite:config     'vite:css-post',
  vite:config     'vite:build-html',
  vite:config     'vite:worker-import-meta-url',
  vite:config     'vite:watch-package-data',
  vite:config     'commonjs',
  vite:config     'vite:data-uri',
  vite:config     'vite:asset-import-meta-url',
  vite:config     'node-polyfills',
  vite:config     'vite:dynamic-import-vars',
  vite:config     'vite:import-glob',
  vite:config     'vite:build-import-analysis',
  vite:config     'vite:esbuild-transpile',
  vite:config     'vite:terser',
  vite:config     'vite:reporter',
  vite:config     'vite:load-fallback'
  vite:config   ],
  vite:config   build: {
  vite:config     target: [ 'es2020', 'edge88', 'firefox78', 'chrome87', 'safari13' ],
  vite:config     polyfillModulePreload: true,
  vite:config     outDir: 'dist',
  vite:config     assetsDir: 'assets',
  vite:config     assetsInlineLimit: 4096,
  vite:config     cssCodeSplit: true,
  vite:config     cssTarget: [ 'es2020', 'edge88', 'firefox78', 'chrome87', 'safari13' ],
  vite:config     sourcemap: false,
  vite:config     rollupOptions: { plugins: [Array] },
  vite:config     minify: 'esbuild',
  vite:config     terserOptions: {},
  vite:config     write: true,
  vite:config     emptyOutDir: null,
  vite:config     manifest: false,
  vite:config     lib: false,
  vite:config     ssr: false,
  vite:config     ssrManifest: false,
  vite:config     reportCompressedSize: true,
  vite:config     chunkSizeWarningLimit: 500,
  vite:config     watch: null,
  vite:config     commonjsOptions: { include: [Array], extensions: [Array] },
  vite:config     dynamicImportVarsOptions: { warnOnError: true, exclude: [Array] }
  vite:config   },
  vite:config   optimizeDeps: {
  vite:config     disabled: 'build',
  vite:config     force: undefined,
  vite:config     include: [ 'react/jsx-runtime', 'react/jsx-dev-runtime' ],
  vite:config     esbuildOptions: { preserveSymlinks: undefined }
  vite:config   },
  vite:config   resolve: { dedupe: [ 'react', 'react-dom' ], alias: [ [Object], [Object] ] },
  vite:config   configFile: '/Users/vladsk/dev/marathon/vite.config.ts',
  vite:config   configFileDependencies: [ '/Users/vladsk/dev/marathon/vite.config.ts' ],
  vite:config   inlineConfig: {
  vite:config     root: undefined,
  vite:config     base: undefined,
  vite:config     mode: undefined,
  vite:config     configFile: undefined,
  vite:config     logLevel: undefined,
  vite:config     clearScreen: undefined,
  vite:config     optimizeDeps: { force: undefined },
  vite:config     build: {}
  vite:config   },
  vite:config   root: '/Users/vladsk/dev/marathon',
  vite:config   base: '/',
  vite:config   publicDir: '/Users/vladsk/dev/marathon/public',
  vite:config   cacheDir: '/Users/vladsk/dev/marathon/node_modules/.vite',
  vite:config   command: 'build',
  vite:config   mode: 'production',
  vite:config   ssr: {
  vite:config     format: 'esm',
  vite:config     target: 'node',
  vite:config     optimizeDeps: { disabled: true, esbuildOptions: [Object] }
  vite:config   },
  vite:config   isWorker: false,
  vite:config   mainConfig: null,
  vite:config   isProduction: true,
  vite:config   server: {
  vite:config     preTransformRequests: true,
  vite:config     middlewareMode: false,
  vite:config     fs: { strict: true, allow: [Array], deny: [Array] }
  vite:config   },
  vite:config   preview: {
  vite:config     port: undefined,
  vite:config     strictPort: undefined,
  vite:config     host: undefined,
  vite:config     https: undefined,
  vite:config     open: undefined,
  vite:config     proxy: undefined,
  vite:config     cors: undefined,
  vite:config     headers: undefined
  vite:config   },
  vite:config   env: { BASE_URL: '/', MODE: 'production', DEV: false, PROD: true },
  vite:config   assetsInclude: [Function: assetsInclude],
  vite:config   logger: {
  vite:config     hasWarned: false,
  vite:config     info: [Function: info],
  vite:config     warn: [Function: warn],
  vite:config     warnOnce: [Function: warnOnce],
  vite:config     error: [Function: error],
  vite:config     clearScreen: [Function: clearScreen],
  vite:config     hasErrorLogged: [Function: hasErrorLogged]
  vite:config   },
  vite:config   packageCache: Map(0) { set: [Function (anonymous)] },
  vite:config   createResolver: [Function: createResolver],
  vite:config   worker: {
  vite:config     format: 'iife',
  vite:config     plugins: [
  vite:config       [Object], [Object], [Object],
  vite:config       [Object], [Object], [Object],
  vite:config       [Object], [Object], [Object],
  vite:config       [Object], [Object], [Object],
  vite:config       [Object], [Object], [Object],
  vite:config       [Object], [Object], [Object],
  vite:config       [Object], [Object], [Object],
  vite:config       [Object], [Object], [Object],
  vite:config       [Object], [Object], [Object],
  vite:config       [Object], [Object]
  vite:config     ],
  vite:config     rollupOptions: {}
  vite:config   },
  vite:config   appType: 'spa',
  vite:config   experimental: { importGlobRestoreExtension: false, hmrPartialAccept: false }
  vite:config } +9ms
vite v3.0.4 building for production...
transforming (685) node_modules/entities/lib/maps/decode.jsonError when using sourcemap for reporting an error: Can't resolve original location of error.
✓ 699 modules transformed.
'Buffer' is not exported by __vite-browser-external, imported by node_modules/@aws-crypto/util/build/convertToBuffer.js
file: /Users/vladsk/dev/marathon/node_modules/@aws-crypto/util/build/convertToBuffer.js:1:9
1: import { Buffer as Buffer } from 'buffer';
            ^
2:
3: import * as commonjsHelpers from "commonjsHelpers.js";
error during build:
Error: 'Buffer' is not exported by __vite-browser-external, imported by node_modules/@aws-crypto/util/build/convertToBuffer.js
    at error (file:///Users/vladsk/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:1858:30)
    at Module.error (file:///Users/vladsk/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:12412:16)
    at Module.traceVariable (file:///Users/vladsk/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:12771:29)
    at ModuleScope.findVariable (file:///Users/vladsk/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:11423:39)
    at Identifier.bind (file:///Users/vladsk/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:7422:40)
    at UnaryExpression.bind (file:///Users/vladsk/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:5252:23)
    at BinaryExpression.bind (file:///Users/vladsk/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:5252:23)
    at LogicalExpression.bind (file:///Users/vladsk/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:5252:23)
    at ConditionalExpression.bind (file:///Users/vladsk/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:5252:23)
    at VariableDeclarator.bind (file:///Users/vladsk/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:5252:23)

vite --debug

> vite


  VITE v3.0.4  ready in 332 ms

  ➜  Local:   http://localhost:5173/
  ➜  Network: use --host to expose
✘ [ERROR] No matching export in "node_modules/@aws-sdk/credential-provider-web-identity/dist-es/index.js" for import "fromTokenFile"

    node_modules/@aws-sdk/credential-provider-ini/dist-es/resolveWebIdentityCredentials.js:2:9:
      2 │ import { fromTokenFile } from "@aws-sdk/credential-provider-web-identity";
        ╵          ~~~~~~~~~~~~~

✘ [ERROR] No matching export in "node_modules/@aws-sdk/credential-provider-web-identity/dist-es/index.js" for import "fromTokenFile"

    node_modules/@aws-sdk/credential-provider-node/dist-es/defaultProvider.js:6:9:
      6 │ import { fromTokenFile } from "@aws-sdk/credential-provider-web-identity";
        ╵          ~~~~~~~~~~~~~

✘ [ERROR] No matching export in "node_modules/@aws-sdk/credential-provider-web-identity/dist-es/index.js" for import "fromTokenFile"

    node_modules/@aws-sdk/credential-providers/dist-es/fromTokenFile.js:3:9:
      3 │ import { fromTokenFile as _fromTokenFile, } from "@aws-sdk/credential-provider-web-identity";
        ╵          ~~~~~~~~~~~~~

/Users/vladsk/dev/marathon/node_modules/esbuild/lib/main.js:1621
  let error = new Error(`${text}${summary}`);
              ^

Error: Build failed with 3 errors:
node_modules/@aws-sdk/credential-provider-ini/dist-es/resolveWebIdentityCredentials.js:2:9: ERROR: No matching export in "node_modules/@aws-sdk/credential-provider-web-identity/dist-es/index.js" for import "fromTokenFile"
node_modules/@aws-sdk/credential-provider-node/dist-es/defaultProvider.js:6:9: ERROR: No matching export in "node_modules/@aws-sdk/credential-provider-web-identity/dist-es/index.js" for import "fromTokenFile"
node_modules/@aws-sdk/credential-providers/dist-es/fromTokenFile.js:3:9: ERROR: No matching export in "node_modules/@aws-sdk/credential-provider-web-identity/dist-es/index.js" for import "fromTokenFile"
    at failureErrorWithLog (/Users/vladsk/dev/marathon/node_modules/esbuild/lib/main.js:1621:15)
    at /Users/vladsk/dev/marathon/node_modules/esbuild/lib/main.js:1263:28
    at runOnEndCallbacks (/Users/vladsk/dev/marathon/node_modules/esbuild/lib/main.js:1043:63)
    at buildResponseToResult (/Users/vladsk/dev/marathon/node_modules/esbuild/lib/main.js:1261:7)
    at /Users/vladsk/dev/marathon/node_modules/esbuild/lib/main.js:1374:14
    at /Users/vladsk/dev/marathon/node_modules/esbuild/lib/main.js:675:9
    at handleIncomingPacket (/Users/vladsk/dev/marathon/node_modules/esbuild/lib/main.js:772:9)
    at Socket.readFromStdout (/Users/vladsk/dev/marathon/node_modules/esbuild/lib/main.js:641:7)
    at Socket.emit (node:events:527:28)
    at addChunk (node:internal/streams/readable:324:12) {
  errors: [
    {
      detail: undefined,
      id: '',
      location: {
        column: 9,
        file: 'node_modules/@aws-sdk/credential-provider-ini/dist-es/resolveWebIdentityCredentials.js',
        length: 13,
        line: 2,
        lineText: 'import { fromTokenFile } from "@aws-sdk/credential-provider-web-identity";',
        namespace: '',
        suggestion: ''
      },
      notes: [],
      pluginName: '',
      text: 'No matching export in "node_modules/@aws-sdk/credential-provider-web-identity/dist-es/index.js" for import "fromTokenFile"'
    },
    {
      detail: undefined,
      id: '',
      location: {
        column: 9,
        file: 'node_modules/@aws-sdk/credential-provider-node/dist-es/defaultProvider.js',
        length: 13,
        line: 6,
        lineText: 'import { fromTokenFile } from "@aws-sdk/credential-provider-web-identity";',
        namespace: '',
        suggestion: ''
      },
      notes: [],
      pluginName: '',
      text: 'No matching export in "node_modules/@aws-sdk/credential-provider-web-identity/dist-es/index.js" for import "fromTokenFile"'
    },
    {
      detail: undefined,
      id: '',
      location: {
        column: 9,
        file: 'node_modules/@aws-sdk/credential-providers/dist-es/fromTokenFile.js',
        length: 13,
        line: 3,
        lineText: 'import { fromTokenFile as _fromTokenFile, } from "@aws-sdk/credential-provider-web-identity";',
        namespace: '',
        suggestion: ''
      },
      notes: [],
      pluginName: '',
      text: 'No matching export in "node_modules/@aws-sdk/credential-provider-web-identity/dist-es/index.js" for import "fromTokenFile"'
    }
  ],
  warnings: []
}

Node.js v18.1.0

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Make sure this is a Vite issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to vuejs/core instead.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[iamskok]

Did some more digging. This StackOverflow answer partially fixed the vite build, but it's based on outdated rollup-plugin-node-polyfills package that misses certain polyfills (e.g., homedir from os). rollup-plugin-polyfill-node is an updated, maintained fork of rollup-plugin-node-polyfills though I have some problems making it work specifically; I have issues with extname import from path.

'extname' is not exported by node_modules/rollup-plugin-polyfill-node/dist/es/index.js, imported by node_modules/@rollup/pluginutils/dist/es/index.js
file: /Users/skok/dev/marathon/node_modules/@rollup/pluginutils/dist/es/index.js:1:9
1: import { extname, sep, resolve, posix } from 'path';
            ^
2: import pm from 'picomatch';
error during build:
Error: 'extname' is not exported by node_modules/rollup-plugin-polyfill-node/dist/es/index.js, imported by node_modules/@rollup/pluginutils/dist/es/index.js
    at error (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:1858:30)
    at Module.error (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:12412:16)
    at Module.traceVariable (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:12771:29)
    at ModuleScope.findVariable (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:11423:39)
    at FunctionScope.findVariable (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:6355:38)
    at ChildScope.findVariable (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:6355:38)
    at Identifier.bind (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:7422:40)
    at CallExpression.bind (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:5252:23)
    at CallExpression.bind (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:8918:15)
    at UnaryExpression.bind (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:5252:23)

Keeping all polyfills from rollup-plugin-polyfill-node and replacing only path with rollup-plugin-node-polyfills throws another error:

'posix' is not exported by node_modules/rollup-plugin-node-polyfills/polyfills/path.js, imported by node_modules/@rollup/pluginutils/dist/es/index.js
file: /Users/skok/dev/marathon/node_modules/@rollup/pluginutils/dist/es/index.js:1:32
1: import { extname, sep, resolve, posix } from 'path';
                                   ^
2: import pm from 'picomatch';
error during build:
Error: 'posix' is not exported by node_modules/rollup-plugin-node-polyfills/polyfills/path.js, imported by node_modules/@rollup/pluginutils/dist/es/index.js
    at error (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:1858:30)
    at Module.error (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:12412:16)
    at Module.traceVariable (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:12771:29)
    at ModuleScope.findVariable (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:11423:39)
    at FunctionScope.findVariable (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:6355:38)
    at ChildScope.findVariable (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:6355:38)
    at MemberExpression.bind (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:8584:49)
    at CallExpression.bind (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:5252:23)
    at CallExpression.bind (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:8918:15)
    at ReturnStatement.bind (file:///Users/skok/dev/marathon/node_modules/rollup/dist/es/shared/rollup.js:5252:23)

I'm not able to say why exactly path polyfill was working when all aliases were from rollup-plugin-node-polyfills but stopped working when started mixing both (rollup-plugin-node-polyfills and rollup-plugin-polyfill-node).

[luxaritas]

Assuming this is the same issue as I was running into, my solution (which I found in some other post) was to tell Vite to externalize node's builtins. Ie, import { builtinModules } from 'module'; And then add builtinModules to build.rollupOptions.external

[XioDone]

You can try this polyfill vite-plugin-ngmi-polyfill
@aws/sdk it work

[ChrisTowles]

Workaround

Repo to reproduce solution to : Vite Issue 9445

Looking into this it appears to be with aws-sdk. It is not just related to vite. With some changes i was able to reproduce the issue and then solve it with configuration changes.

Demonstrated loading AWS-SDK@2.1188.0 and building with vite@3.0.4.

Edit on StackBlitz ⚡️

Found solutions at Solutions

Examples of the solution for AWS-SDK:

• Cannot use amplify-js in browser environment (breaking vite / snowpack / esbuild) aws-amplify/amplify-js#9639 (comment)
• Vite apps will not build or run while using Amplify libraries with AWS-SDK aws/aws-sdk-js#3673
• Unable to use with vite aws/aws-sdk-js-v3#2478

Required Changes

Change in vite.config.js

export default defineConfig({
 resolve: {
    alias: {
      // added for AWS-SDK with vite
      "./runtimeConfig": "./runtimeConfig.browser"
    },
  },
})

Change to index.html

<script>
    
      // Add above `main.js` script for for AWS SDK.
      window.global = window;
    
</script>

[ffxsam]

Still an issue in Vite 3.0.7:

ERROR 'request' is not exported by __vite-browser-external, imported by nuxt/node_modules/@aws-sdk/credential-provider-node/node_modules/@aws-sdk/credential-provider-imds/dist/es/remoteProvider/httpRequest.js

@sapphi-red

[patak-dev]

@ffxsam would you test 3.0.8?

[sapphi-red]

@ffxsam I've confirmed that the original reproduction works with 3.0.8. Please create a new issue with reproduction if it still happens.