@sls-next/serverless-components 3.4.x break deployment with Prisma ORM (schema.prisma not found)

[baronnoraz]

Issue Summary

Prior to the 3.4.x releases of @sls-next/serverless-components, I had a working serverless build that included Prisma ORM.

The working build required me to use the postBuildCommands property to copy the schema.prisma file and query engine (query-engine-rhel-*) to the appropriate folders to make the deployment work on AWS.

Now, when deploying the app, I get an error stating that .next/serverless/chunks/schema.prisma can't be found.

Error: ENOENT: no such file or directory, open '/Users/nbuckner/git/nextjs-prisma-monorepo-serverless-deployment-issue/apps/client/.next/serverless/chunks/schema.prisma'

I also get a warning, but I don't think it's actually causing problems.

../../node_modules/.bin/next build
(node:29546) [DEP_WEBPACK_CHUNK_HAS_ENTRY_MODULE] DeprecationWarning: Chunk.hasEntryModule: Use new ChunkGraph API

The postBuildCommands that made the build work before were...

postBuildCommands:
  - PDIR=../../node_modules/.prisma/client/;
    LDIR=.serverless_nextjs/api-lambda/;
    TDIR="$LDIR"chunks/;
    if [ "$(ls -A $LDIR)" ]; then
    mkdir -p $TDIR;
    cp "$PDIR"query-engine-rhel-* $TDIR;
    cp "$PDIR"schema.prisma $TDIR;
    fi;
  - PDIR=../../node_modules/.prisma/client/;
    LDIR=.serverless_nextjs/default-lambda/;
    TDIR="$LDIR"chunks/;
    if [ "$(ls -A $LDIR)" ]; then
    mkdir -p $TDIR;
    cp "$PDIR"query-engine-rhel-* $TDIR;
    cp "$PDIR"schema.prisma $TDIR;
    fi;

Actual behavior

When I run the following command, I get an error stating that schema.prisma cannot be found.

npx -w apps/client serverless

Expected behavior

When I run the following command, the app should be deployed to AWS as before.

npx -w apps/client serverless

Steps to reproduce

Use NPM v7 for workspace support.

# ~/git/example
node -v
v14.17.4

npm -v
7.20.5

Checkout the code from

git clone git@github.com:baronnoraz/nextjs-prisma-monorepo-serverless-deployment-issue.git example

A more detailed README.md is located in the root of the project, but these are the minimal steps.

Configure AWS credentials in example/apps/client/.env.

Optionally (if you want to run the application), create an AWS RDS instance and configure the DATABASE_URL in example/apps/client/.env and example/db/prisma/.env.

Install dependencies

# example/
npm install

Generate the schema.prisma file and Prisma Query engine

# example/
npx -w db/prisma prisma generate

Deploy the app

# example/
npx -w apps/client serverless

Screenshots/Code/Configuration/Logs

The error that I get is...

  17s › exampleApp › Error: Command failed with exit code 1: ../../node_modules/.bin/next build
(node:32204) [DEP_WEBPACK_CHUNK_HAS_ENTRY_MODULE] DeprecationWarning: Chunk.hasEntryModule: Use new ChunkGraph API
(Use `node --trace-deprecation ...` to show where the warning was created)
warn  - Compiled with warnings

../../node_modules/next/dist/server/load-components.js
Critical dependency: the request of a dependency is an expression

../../node_modules/next/dist/server/load-components.js
Critical dependency: the request of a dependency is an expression

../../node_modules/next/dist/server/load-components.js
Critical dependency: the request of a dependency is an expression

../../node_modules/next/dist/server/require.js
Critical dependency: the request of a dependency is an expression

../../node_modules/next/dist/server/require.js
Critical dependency: the request of a dependency is an expression

../../node_modules/next/dist/server/require.js
Critical dependency: the request of a dependency is an expression


> Build error occurred
Error: ENOENT: no such file or directory, open '/Users/nbuckner/git/nextjs-prisma-monorepo-serverless-deployment-issue/apps/client/.next/serverless/chunks/schema.prisma'
    at Object.openSync (fs.js:498:3)
    at Object.readFileSync (fs.js:394:35)
    at new LibraryEngine (/Users/nbuckner/git/nextjs-prisma-monorepo-serverless-deployment-issue/apps/client/.next/serverless/chunks/569.js:25128:42)
    at PrismaClient.getEngine (/Users/nbuckner/git/nextjs-prisma-monorepo-serverless-deployment-issue/apps/client/.next/serverless/chunks/569.js:37747:16)
    at new PrismaClient (/Users/nbuckner/git/nextjs-prisma-monorepo-serverless-deployment-issue/apps/client/.next/serverless/chunks/569.js:37716:29)
    at Module.1074 (/Users/nbuckner/git/nextjs-prisma-monorepo-serverless-deployment-issue/apps/client/.next/serverless/pages/index.js:26:16)
    at __webpack_require__ (/Users/nbuckner/git/nextjs-prisma-monorepo-serverless-deployment-issue/apps/client/.next/serverless/pages/index.js:465:43)
    at Module.6884 (/Users/nbuckner/git/nextjs-prisma-monorepo-serverless-deployment-issue/apps/client/.next/serverless/pages/index.js:154:23)
    at __webpack_require__ (/Users/nbuckner/git/nextjs-prisma-monorepo-serverless-deployment-issue/apps/client/.next/serverless/pages/index.js:465:43)
    at /Users/nbuckner/git/nextjs-prisma-monorepo-serverless-deployment-issue/apps/client/.next/serverless/pages/index.js:485:96 {
  type: 'Error',
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: '/Users/nbuckner/git/nextjs-prisma-monorepo-serverless-deployment-issue/apps/client/.next/serverless/chunks/schema.prisma',
  clientVersion: '3.1.1'
}
info  - Loaded env from /Users/nbuckner/git/nextjs-prisma-monorepo-serverless-deployment-issue/apps/client/.env
info  - Using webpack 5. Reason: Enabled by default https://nextjs.org/docs/messages/webpack5
info  - Checking validity of types...
info  - Creating an optimized production build...
info  - Collecting page data...

Versions

• OS/Environment: macOS Catalina 10.15.7
• Node v14.17.4
• NPM v7.20.5
• @sls-next/serverless-component version: 3.4.0-alpha.11
• Next.js version: 11.1.2
• @prisma/client version: 3.1.1
• prisma version: 3.1.1
• serverless version: 2.60.0

Additional context

This was working prior to upgrading to @sls-next/serverless-component 3.4.x. I went down this road, because I was running into some issues with next-auth and the @next-auth/prisma-adapter. I got everything to work together locally after all of the upgrades, but when I went to deploy the app to AWS it failed.

I started the example repo from the ground up to try and determine why the deployment no longer worked. It seems that something is different in the file resolution of schema.prisma, but I've been unable to figure out what.

To be fair there were issues in the previous version, but I could use the postBuildCommands to copy the schema.prisma file to where I needed it. The deployment appears to fail earlier in the process now.

Checklist

• You have reviewed the README and FAQs, which answers several common questions.
• You have reviewed our DEBUGGING wiki and have tried your best to include complete information and reproduction steps (including your configuration) as is possible.
• You have first tried using the most recent latest or alpha @sls-next/serverless-component release version, which may have already fixed your issue or implemented the feature you are trying to use. Note that the old serverless-next.js component and the serverless-next.js plugin are deprecated and no longer maintained.

[dphang]

Ya, this is a good use case to be adding e2e tests using Prisma and adding a small function to copy the right files for Prisma somewhere here: https://github.com/serverless-nextjs/serverless-next.js/tree/master/packages/libs/lambda-at-edge/src/build/third-party (I created a simple interface and did it for nexti18-next for example).

If you have some minimal repro you can create with Prisma, it will be faster for me to take a look.

[baronnoraz]

I have a minimal monorepo here https://github.com/baronnoraz/nextjs-prisma-monorepo-serverless-deployment-issue.

I could create one without the npm workspaces and put everything under a single folder. I didn't because this structure was working previously and I was trying to reproduce what I was seeing in my larger project.

If you checkout that repo, you should be able to reproduce by using NPM v7 and adding AWS API Keys to apps/client/.env and db/prisma/.env (you might need to set the DATABASE_URL, but hopefully not since you don't need it to work post deployment). Then run the following from the root of the project.

npm install
npx -w db/prisma prisma generate
npm -w apps/client serverless

The prisma generate command will put the .prisma folder in the top level node_modules folder. I assume this is similar to how yarn workspaces would work and somewhat similar to pnpm.

What mainly confused me is that my postBuildCommands were never executed. It seems like things are failing before the build gets that far. I'm going to look at your example and see if I can adapt that. I appreciate any help or insights. Thanks.

[baronnoraz]

@dphang sorry to bother you, but how do you get your third-party code included in the build process? Is it what you run as part of your postBuildCommands?

[baronnoraz]

I can confirm that the build command fails before any of the postBuildCommands are executed, so there's no way to copy the schema.prisma file to the correct spot as I was doing before, because the build fails too soon.

npx -w apps/client serverless 

# example/apps/client/serverless.yml
    build:
      cmd: ../../node_modules/.bin/next

---

I can build nextjs outside of serverless, but when I look at the `.next` folder there is no reference to `schema.prisma` in any of the transpiled javascript files and of course, no actual `schema.prisma` file. BUT it didn't give me an error.

npm -w apps/client run build

I tried to replicate running this command in the serverless build, BUT it did give me an error. The same error as before.

# example/apps/client/serverless.yml
    build:
      cmd: build.sh

# example/apps/client/build.sh
(cd ../../ && npm -w apps/client run build)

---

I tried adding that same command to `serverless.yml` directly, but I get a different error

# example/apps/client/serverless.yml
    build:
      cmd: (cd ../../ && npm -w apps/client run build)

  error:
  Error: Command failed with ENOENT: (cd ../../ && npm -w apps/client run build) build
spawn (cd ../../ && npm -w apps/client run build) ENOENT
    at Process.ChildProcess._handle.onexit (internal/child_process.js:269:19)
    at onErrorNT (internal/child_process.js:467:16)
    at processTicksAndRejections (internal/process/task_queues.js:82:21) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn (cd ../../ && npm -w apps/client run build)',
  path: '(cd ../../ && npm -w apps/client run build)',
  spawnargs: [ 'build' ],
  originalMessage: 'spawn (cd ../../ && npm -w apps/client run build) ENOENT',
  shortMessage: 'Command failed with ENOENT: (cd ../../ && npm -w apps/client run build) build\n' +
    'spawn (cd ../../ && npm -w apps/client run build) ENOENT',
  command: '(cd ../../ && npm -w apps/client run build) build',
  escapedCommand: '"(cd ../../ && npm -w apps/client run build)" build',

---

This makes me think that the next build may be working but something else is being executed before `postBuildCommands` is called. I'm not sure if that's right.

• CMD: ../../node_modules/.bin/next may work fine.
• Something runs here that breaks
• postBuildCommands never run.

[baronnoraz]

Ok, so this isn't the "fix", but I got it to work. Hopefully, it will help others.

I installed the copy-webpack-plugin and modified my next.config.js to

const CopyWebpackPlugin = require('copy-webpack-plugin');

module.exports = {
  reactStrictMode: true,
  webpack: (config, {isServer}) => {
    if (isServer) {
      config.externals.push('_http_common');
    }
    config.plugins.push(
      new CopyWebpackPlugin({
        patterns: [{
          from: '../../node_modules/.prisma/client/schema.prisma',
          to: './'
        }, {
          from: '../../node_modules/.prisma/client/libquery_engine-rhel-openssl-1.0.x.so.node',
          to: './'
        }]
      })
    )
    return config;
  }
};

You'll probably have to adjust those paths for your needs, because that's specific to my monorepo structure, but it makes it work for me.

[MGough]

Looks like we have a similar project structure! Yarn workspaces by any chance?

We had the same error, what's odd though is that we were using @sls-next/serverless-components 3.1.0 and started seeing this once we added prisma calls to our getServerSideProps. It didn't seem to be linked to upgrading any versions (serverless-components, next, prisma).

We saw it on Prisma 3.2.1 and 3.5.0. Serverless components version 3.1.0 and 3.6.0. It happened locally and on github actions.

It seems like it's primarily an issue between next and prisma, as we were seeing it locally if we ran next build with target: serverless, so potentially not actually related to serverless-next.js