Upgrades yarn v1 to v3. Improves CI times by 2x (#7738)

* Create env file workflow * Add env-cache * Fix env setter * Fix * Another fix * Fix * Fix * Fixes * FFS * Fix * Fix * Fix * Fix * Fix * Cache fixes * Fixes * Adds skipping steps * db-cache fixes * Test * Cache fixes * e2e * Possible caching conflicts * Running out of ideas * Caching is hard * One more time * cache-build not skipping * Fingers crossed * a * Test * Pls * Please * LFG * Build fix * fix * Whitespace!! * Zomars/cal 884 paid events not sending the link (#7318) * WIP * Sends correct emails for paid bookings * Update PaymentService.ts * Update webhook.ts * Update webhook.ts * Update settings back button redirect link (#7403) * fix(schedule): close on click #7143 * fix(EventSetupTab): validLocations length will always match validLocations length #7138 * fix(SettingsLayout): go back to right route #7379 * feat: get country code from ip geolocation (#6880) * feat: get coutnry code from ip geolocation Signed-off-by: Udit Takkar <udit.07814802719@cse.mait.ac.in> * fix: create new api route for fetching code Signed-off-by: Udit Takkar <udit.07814802719@cse.mait.ac.in> * chore: replace city with country Signed-off-by: Udit Takkar <udit.07814802719@cse.mait.ac.in> * refactor: create hook for country Signed-off-by: Udit Takkar <udit.07814802719@cse.mait.ac.in> --------- Signed-off-by: Udit Takkar <udit.07814802719@cse.mait.ac.in> * Team Workflows (#7038) Co-authored-by: Hariom Balhara <hariombalhara@gmail.com> Co-authored-by: CarinaWolli <wollencarina@gmail.com> Co-authored-by: zomars <zomars@me.com> Co-authored-by: Peer Richelsen <peeroke@gmail.com> * Add destination calendar name to DestinationCalendarSelector (#6701) * Add destination calendar name * Type fix * Search through calendars only for destination calendar credential * Refactor get connected calendars * Clean up --------- Co-authored-by: zomars <zomars@me.com> * Update viewer.tsx (#7428) * Fix - add team members to emails (#7207) * On booking add team members & translation * Add team members to round robin create * Only update calendars on reschedule if there is a calendar reference * Send email on reschedules * Send team email on cancelled event * Add team members to calendar event description * Clean up * Convert other emails to organizer & teams * Type check fixes * More type fixes * Change organizer scheduled input to an object * early return updateCalendarEvent * Introduce team member type * Fix type errors * Put team members before attendees * Remove lodash cloneDeep * Update packages/core/EventManager.ts Co-authored-by: Omar López <zomars@me.com> * Remove booking select object * Revert "Remove booking select object" This reverts commit 9f121ff. * Refactor email manager (#7270) Co-authored-by: zomars <zomars@me.com> * Type change * Remove conditional check for updateAllCalendarEvents --------- Co-authored-by: zomars <zomars@me.com> * Typefix * Updates webhook response * Update pr.yml * Update action.yml * Update action.yml * Update action.yml * Update action.yml * Update action.yml * Is this redundant? * Removed setup * Update action.yml * Update action.yml * Consolitades setup step * Revert "Consolitades setup step" This reverts commit 5e8d198. * Fix? * One more time * Revert "One more time" This reverts commit fd8b559. * Benchmarking buildjet * Update action.yml * Re-introduce setup * Adds embeds to missing pro cache * Lint fixes * Adds prettier ignore * Upgrades to yarn 3 * Updates lockfile * Reverts CI to ubuntu * Testing new yarn install * We cannot use immutable due to our private submodules * Adds CI skip * Fixes * Adds plugin * Forces local embed package * Moves eslint to root * Update yarn.lock * Playwright fixes * Embed test fixes * Splits embed react tests * Splits embed react tests * Removes install step to benchmark * Update playwright.config.ts * One playwright config for all * More test fixes * Update basic.e2e.ts * Added typescript as a global monorepo dev * Update to v18 * Update yarn.lock * Update env-create-file.yml * Update .github/workflows/pr.yml --------- Signed-off-by: Udit Takkar <udit.07814802719@cse.mait.ac.in> Co-authored-by: Esaú Morais <55207584+esau-morais@users.noreply.github.com> Co-authored-by: Udit Takkar <53316345+Udit-takkar@users.noreply.github.com> Co-authored-by: Carina Wollendorfer <30310907+CarinaWolli@users.noreply.github.com> Co-authored-by: Hariom Balhara <hariombalhara@gmail.com> Co-authored-by: CarinaWolli <wollencarina@gmail.com> Co-authored-by: Peer Richelsen <peeroke@gmail.com> Co-authored-by: Joe Au-Yeung <65426560+joeauyeung@users.noreply.github.com>
calcom · Mar 15, 2023 · 54cefcb · 54cefcb
1 parent f20d78b
commit 54cefcb
Show file tree

Hide file tree

Showing 41 changed files with 41,893 additions and 29,127 deletions.
diff --git a/.eslintignore b/.eslintignore
@@ -0,0 +1,6 @@
+node_modules
+**/**/node_modules
+**/**/.next
+**/**/public
+packages/prisma/zod
+apps/web/public/embed
diff --git a/.github/actions/cache-build/action.yml b/.github/actions/cache-build/action.yml
@@ -20,6 +20,7 @@ runs:
       with:
         path: |
           ${{ github.workspace }}/apps/web/.next
+          ${{ github.workspace }}/apps/web/public/embed
           **/.turbo/**
           **/dist/**
         key: ${{ runner.os }}-${{ env.cache-name }}-${{ env.key-1 }}-${{ env.key-2 }}-${{ env.key-3 }}-${{ env.key-4 }}

diff --git a/.github/actions/yarn-install/action.yml b/.github/actions/yarn-install/action.yml
@@ -1,19 +1,62 @@
-name: Yarn install
-description: "Install all NPM dependencies, caches them and restores if necessary"
+########################################################################################
+# "yarn install" composite action for yarn 2/3/4+ and "nodeLinker: node-modules"       #
+#--------------------------------------------------------------------------------------#
+# Cache:                                                                               #
+#   - Downloaded zip archive (multi-arch, preserved across yarn.lock changes)          #
+#   - Yarn install state (discarded on yarn.lock changes)                              #
+# References:                                                                          #
+#   - bench: https://gist.github.com/belgattitude/0ecd26155b47e7be1be6163ecfbb0f0b     #
+#   - vs @setup/node: https://github.com/actions/setup-node/issues/325                 #
+########################################################################################
+
+name: "Yarn install"
+description: "Run yarn install with node_modules linker and cache enabled"
 inputs:
   node_version:
     required: false
     default: v18.x
+
 runs:
   using: "composite"
   steps:
     - name: Use Node ${{ inputs.node_version }}
       uses: actions/setup-node@v3
       with:
         node-version: ${{ inputs.node_version }}
-        cache: "yarn"
-    - name: Yarn install
+    - name: Expose yarn config as "$GITHUB_OUTPUT"
+      id: yarn-config
       shell: bash
       run: |
-        yarn install --prefer-offline --frozen-lockfile
+        echo "CACHE_FOLDER=$(yarn config get cacheFolder)" >> $GITHUB_OUTPUT
+
+    # Yarn rotates the downloaded cache archives, @see https://github.com/actions/setup-node/issues/325
+    # Yarn cache is also reusable between arch and os.
+    - name: Restore yarn cache
+      uses: actions/cache@v3
+      id: yarn-download-cache
+      with:
+        path: ${{ steps.yarn-config.outputs.CACHE_FOLDER }}
+        key: yarn-download-cache-${{ hashFiles('yarn.lock') }}
+        restore-keys: |
+          yarn-download-cache-
+
+    # Invalidated on yarn.lock changes
+    - name: Restore yarn install state
+      id: yarn-install-state-cache
+      uses: actions/cache@v3
+      with:
+        path: .yarn/ci-cache/
+        key: ${{ runner.os }}-yarn-install-state-cache-${{ hashFiles('yarn.lock', '.yarnrc.yml') }}
+
+    - name: Install dependencies
+      shell: bash
+      run: |
+        yarn install --inline-builds
         yarn prisma generate
+      env:
+        # CI optimizations. Overrides yarnrc.yml options (or their defaults) in the CI action.
+        YARN_ENABLE_IMMUTABLE_INSTALLS: "false" # So it doesn't try to remove our private submodule deps
+        YARN_ENABLE_GLOBAL_CACHE: "false" # Use local cache folder to keep downloaded archives
+        YARN_INSTALL_STATE_PATH: .yarn/ci-cache/install-state.gz # Very small speedup when lock does not change
+        # Other environment variables
+        HUSKY: "0" # By default do not run HUSKY install
diff --git a/.github/workflows/cron-stale-issue.yml b/.github/workflows/cron-stale-issue.yml
@@ -18,5 +18,3 @@ jobs:
           days-before-stale: 60
           include-only-assigned: true
           days-before-close: -1
-
-
diff --git a/.github/workflows/e2e-embed-react.yml b/.github/workflows/e2e-embed-react.yml
@@ -0,0 +1,36 @@
+name: E2E Embed tests and booking flow(for non-embed as well)
+on:
+  workflow_call:
+
+jobs:
+  e2e-embed:
+    timeout-minutes: 20
+    runs-on: ubuntu-latest
+    services:
+      postgres:
+        image: postgres:12.1
+        env:
+          POSTGRES_USER: postgres
+          POSTGRES_DB: calendso
+        ports:
+          - 5432:5432
+
+    steps:
+      - uses: actions/checkout@v3
+      - uses: ./.github/actions/dangerous-git-checkout
+      - run: echo 'NODE_OPTIONS="--max_old_space_size=4096"' >> $GITHUB_ENV
+      - uses: ./.github/actions/yarn-install
+      - uses: ./.github/actions/yarn-playwright-install
+      - uses: ./.github/actions/env-read-file
+      - uses: ./.github/actions/cache-db
+      - uses: ./.github/actions/cache-build
+      - name: Run Tests
+        run: yarn test-e2e:embed-react
+        env:
+          DEPLOYSENTINEL_API_KEY: ${{ secrets.DEPLOYSENTINEL_API_KEY }}
+      - name: Upload Test Results
+        if: ${{ always() }}
+        uses: actions/upload-artifact@v2
+        with:
+          name: test-results
+          path: test-results
diff --git a/.github/workflows/e2e-embed.yml b/.github/workflows/e2e-embed.yml
@@ -25,19 +25,12 @@ jobs:
       - uses: ./.github/actions/cache-db
       - uses: ./.github/actions/cache-build
       - name: Run Tests
-        run: yarn turbo run embed-tests-update-snapshots:ci --scope=@calcom/embed-react --concurrency=1
+        run: yarn test-e2e:embed
         env:
           DEPLOYSENTINEL_API_KEY: ${{ secrets.DEPLOYSENTINEL_API_KEY }}
-      - name: Upload embed-core results
+      - name: Upload Test Results
         if: ${{ always() }}
         uses: actions/upload-artifact@v2
         with:
-          name: test-results-embed-core
-          path: packages/embeds/embed-core/playwright/results
-
-      - name: Upload embed-react results
-        if: ${{ always() }}
-        uses: actions/upload-artifact@v2
-        with:
-          name: test-results-embed-react
-          path: packages/embeds/embed-react/playwright/results
+          name: test-results
+          path: test-results
diff --git a/.github/workflows/env-create-file.yml b/.github/workflows/env-create-file.yml
@@ -20,7 +20,7 @@ env:
   INPUT_ENV_PAYMENT_FEE_FIXED: 10
   INPUT_ENV_SAML_DATABASE_URL: postgresql://postgres:@localhost:5432/calendso
   INPUT_ENV_SAML_ADMINS: pro@example.com
-  INPUT_ENV_NEXTAUTH_URL: http://localhost:3000/api/auth
+  INPUT_ENV_NEXTAUTH_URL: http://127.0.0.1:3000/api/auth
   INPUT_ENV_NEXT_PUBLIC_IS_E2E: 1
   # INPUT_ENV_EMAIL_FROM: e2e@cal.com
   # INPUT_ENV_EMAIL_SERVER_HOST: ${{ secrets.CI_EMAIL_SERVER_HOST }}

diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.yml
@@ -21,6 +21,7 @@ jobs:
     outputs:
       app-store: ${{ steps.filter.outputs.app-store }}
       embed: ${{ steps.filter.outputs.embed }}
+      embed-react: ${{ steps.filter.outputs.embed-react }}
     steps:
       - uses: actions/checkout@v3
       - uses: ./.github/actions/dangerous-git-checkout
@@ -34,6 +35,9 @@ jobs:
             embed:
               - 'apps/web/**'
               - 'packages/embeds/**'
+            embed-react:
+              - 'apps/web/**'
+              - 'packages/embeds/**'
 
   env:
     name: Create env file
@@ -81,6 +85,13 @@ jobs:
     uses: ./.github/workflows/e2e-embed.yml
     secrets: inherit
 
+  e2e-embed-react:
+    name: E2E React embeds tests
+    if: ${{ needs.changes.outputs.embed-react == 'true' }}
+    needs: [changes, lint, build]
+    uses: ./.github/workflows/e2e-embed-react.yml
+    secrets: inherit
+
   analyze:
     needs: build
     uses: ./.github/workflows/nextjs-bundle-analysis.yml

diff --git a/.gitignore b/.gitignore
@@ -87,3 +87,12 @@ apps/api
 apps/website
 apps/console
 apps/auth
+
+# Yarn Modern
+.pnp.*
+.yarn/*
+!.yarn/patches
+!.yarn/plugins
+!.yarn/releases
+!.yarn/sdks
+!.yarn/versions
diff --git a/.prettierignore b/.prettierignore
@@ -14,3 +14,4 @@ public
 .DS_Store
 .eslintignore
 packages/prisma/zod
+apps/web/public/embed
diff --git a/.yarn/plugins/@yarnpkg/plugin-interactive-tools.cjs b/.yarn/plugins/@yarnpkg/plugin-interactive-tools.cjs
diff --git a/.yarn/releases/yarn-3.4.1.cjs b/.yarn/releases/yarn-3.4.1.cjs
diff --git a/.yarnrc.yml b/.yarnrc.yml
@@ -0,0 +1,7 @@
+nodeLinker: node-modules
+
+plugins:
+  - path: .yarn/plugins/@yarnpkg/plugin-interactive-tools.cjs
+    spec: "@yarnpkg/plugin-interactive-tools"
+
+yarnPath: .yarn/releases/yarn-3.4.1.cjs
diff --git a/apps/docs/package.json b/apps/docs/package.json
@@ -24,7 +24,6 @@
     "react-dom": "^18.2.0"
   },
   "devDependencies": {
-    "@calcom/config": "*",
-    "eslint": "^8.34.0"
+    "@calcom/config": "*"
   }
 }
diff --git a/apps/web/.eslintignore b/apps/web/.eslintignore
@@ -1,2 +1,2 @@
 node_modules
-prisma/zod
+public/embed
diff --git a/apps/web/.prettierignore b/apps/web/.prettierignore
@@ -0,0 +1 @@
+public/embed
diff --git a/apps/web/package.json b/apps/web/package.json
@@ -28,9 +28,9 @@
     "@calcom/app-store-cli": "*",
     "@calcom/core": "*",
     "@calcom/dayjs": "*",
-    "@calcom/embed-core": "*",
-    "@calcom/embed-react": "*",
-    "@calcom/embed-snippet": "*",
+    "@calcom/embed-core": "workspace:*",
+    "@calcom/embed-react": "workspace:*",
+    "@calcom/embed-snippet": "workspace:*",
     "@calcom/features": "*",
     "@calcom/lib": "*",
     "@calcom/prisma": "*",
@@ -134,7 +134,7 @@
     "@calcom/config": "*",
     "@calcom/types": "*",
     "@microsoft/microsoft-graph-types-beta": "0.15.0-preview",
-    "@playwright/test": "^1.25.0",
+    "@playwright/test": "^1.31.2",
     "@testing-library/react": "^13.3.0",
     "@types/accept-language-parser": "1.5.2",
     "@types/async": "^3.2.15",
@@ -162,7 +162,6 @@
     "copy-webpack-plugin": "^11.0.0",
     "detect-port": "^1.3.0",
     "env-cmd": "^10.1.0",
-    "eslint": "^8.34.0",
     "mockdate": "^3.0.5",
     "module-alias": "^2.2.2",
     "msw": "^0.42.3",

diff --git a/apps/web/playwright/lib/fixtures.ts b/apps/web/playwright/lib/fixtures.ts
@@ -2,6 +2,7 @@ import { test as base } from "@playwright/test";
 
 import prisma from "@calcom/prisma";
 
+import type { ExpectedUrlDetails } from "../../../../playwright.config";
 import { createBookingsFixture } from "../fixtures/bookings";
 import { createEmbedsFixture, createGetActionFiredDetails } from "../fixtures/embeds";
 import { createPaymentsFixture } from "../fixtures/payments";
@@ -18,6 +19,21 @@ export interface Fixtures {
   prisma: typeof prisma;
 }
 
+declare global {
+  // eslint-disable-next-line @typescript-eslint/no-namespace
+  namespace PlaywrightTest {
+    //FIXME: how to restrict it to Frame only
+    interface Matchers<R> {
+      toBeEmbedCalLink(
+        calNamespace: string,
+        // eslint-disable-next-line
+        getActionFiredDetails: (a: { calNamespace: string; actionType: string }) => Promise<any>,
+        expectedUrlDetails?: ExpectedUrlDetails
+      ): Promise<R>;
+    }
+  }
+}
+
 /**
  *  @see https://playwright.dev/docs/test-fixtures
  */

diff --git a/package.json b/package.json
@@ -61,15 +61,20 @@
     "tdd": "jest --watch",
     "e2e": "NEXT_PUBLIC_IS_E2E=1 yarn playwright test --project=@calcom/web",
     "e2e:app-store": "QUICK=true yarn playwright test --project=@calcom/app-store",
+    "e2e:embed": "QUICK=true yarn playwright test --project=@calcom/embed-core",
+    "e2e:embed-react": "QUICK=true yarn playwright test --project=@calcom/embed-react",
     "test-e2e": "yarn db-seed && yarn e2e",
     "test-e2e:app-store": "yarn db-seed && yarn e2e:app-store",
+    "test-e2e:embed": "yarn db-seed && yarn e2e:embed",
+    "test-e2e:embed-react": "yarn db-seed && yarn e2e:embed-react",
     "test-playwright": "yarn playwright test --config=playwright.config.ts",
     "test": "jest",
     "type-check": "turbo run type-check",
     "web": "yarn workspace @calcom/web"
   },
   "devDependencies": {
     "@deploysentinel/playwright": "^0.3.3",
+    "@playwright/test": "^1.31.2",
     "@snaplet/copycat": "^0.3.0",
     "@types/dompurify": "^2.4.0",
     "@types/jest": "^28.1.7",
@@ -82,10 +87,12 @@
     "jest-watch-typeahead": "^2.0.0",
     "lint-staged": "^12.5.0",
     "prettier": "^2.8.4",
-    "ts-jest": "^28.0.8"
+    "ts-jest": "^28.0.8",
+    "typescript": "^4.9.4"
   },
   "dependencies": {
     "city-timezones": "^1.2.1",
+    "eslint": "^8.34.0",
     "turbo": "^1.4.3"
   },
   "resolutions": {
@@ -115,7 +122,7 @@
     "schema": "packages/prisma/schema.prisma",
     "seed": "ts-node --transpile-only ./packages/prisma/seed.ts"
   },
-  "packageManager": "yarn@1.22.17",
+  "packageManager": "yarn@3.4.1",
   "syncpack": {
     "filter": "^(?!@calcom).*",
     "semverRange": ""

diff --git a/packages/app-store/dailyvideo/package.json b/packages/app-store/dailyvideo/package.json
@@ -5,8 +5,8 @@
   "main": "./index.ts",
   "description": "Cal Video is the in-house web-based video conferencing platform powered by Daily.co, which is minimalistic and lightweight, but has most of the features you need.",
   "dependencies": {
-    "@calcom/prisma": "*",
-    "@calcom/lib": "*"
+    "@calcom/lib": "*",
+    "@calcom/prisma": "*"
   },
   "devDependencies": {
     "@calcom/types": "*"

diff --git a/packages/app-store/exchange2013calendar/package.json b/packages/app-store/exchange2013calendar/package.json
@@ -9,8 +9,8 @@
     "@calcom/lib": "*",
     "@calcom/prisma": "*",
     "@calcom/ui": "*",
-    "react-hook-form": "^7.43.3",
-    "ews-javascript-api": "^0.11.0"
+    "ews-javascript-api": "^0.11.0",
+    "react-hook-form": "^7.43.3"
   },
   "devDependencies": {
     "@calcom/types": "*"

diff --git a/packages/app-store/exchange2016calendar/package.json b/packages/app-store/exchange2016calendar/package.json
@@ -9,8 +9,8 @@
     "@calcom/lib": "*",
     "@calcom/prisma": "*",
     "@calcom/ui": "*",
-    "react-hook-form": "^7.43.3",
-    "ews-javascript-api": "^0.11.0"
+    "ews-javascript-api": "^0.11.0",
+    "react-hook-form": "^7.43.3"
   },
   "devDependencies": {
     "@calcom/types": "*"

diff --git a/packages/app-store/office365video/package.json b/packages/app-store/office365video/package.json
@@ -5,8 +5,8 @@
   "main": "./index.ts",
   "description": "Microsoft Teams is a business communication platform and collaborative workspace included in Microsoft 365. It offers workspace chat and video conferencing, file storage, and application integration. Both web versions and desktop/mobile applications are available. NOTE: MUST HAVE A WORK / SCHOOL ACCOUNT",
   "dependencies": {
-    "@calcom/prisma": "*",
-    "@calcom/lib": "*"
+    "@calcom/lib": "*",
+    "@calcom/prisma": "*"
   },
   "devDependencies": {
     "@calcom/types": "*"

diff --git a/packages/app-store/routing-forms/playwright/tests/basic.e2e.ts b/packages/app-store/routing-forms/playwright/tests/basic.e2e.ts
@@ -49,7 +49,7 @@ test.describe("Routing Forms", () => {
 
       await page.reload();
 
-      expect(await page.inputValue(`[data-testid="description"]`)).toMatch(description);
+      expect(await page.inputValue(`[data-testid="description"]`)).toBe(description);
       expect(await page.locator('[data-testid="field"]').count()).toBe(1);
 
       await expectCurrentFormToHaveFields(page, { 0: field }, types);

diff --git a/packages/app-store/tandemvideo/package.json b/packages/app-store/tandemvideo/package.json
@@ -5,8 +5,8 @@
   "main": "./index.ts",
   "description": "Tandem is a new virtual office space that allows teams to effortlessly connect as though they are in a physical office, online. Through co-working rooms, available statuses, live real-time video call, and chat options, you can see who’s around, talk and collaborate in one click. It works cross-platform with both desktop and mobile versions.",
   "dependencies": {
-    "@calcom/prisma": "*",
-    "@calcom/lib": "*"
+    "@calcom/lib": "*",
+    "@calcom/prisma": "*"
   },
   "devDependencies": {
     "@calcom/types": "*"