Skip to content

Commit f866733

Browse files
cursoragentBYK
cursoragent
and
BYK
committed
Add documentation for E2E testing setup and troubleshooting
Co-authored-by: burak.kaya <burak.kaya@sentry.io>
1 parent 4a5ba93 commit f866733

File tree

4 files changed

+281
-0
lines changed

4 files changed

+281
-0
lines changed

‎.cursor/rules/sdk_development.mdc‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,63 @@ Each package typically contains:
121121
- Integration tests use Playwright extensively
122122
- Never change the volta, yarn, or package manager setup in general unless explicitly asked for
123123

124+
### E2E Testing
125+
126+
E2E tests are located in `dev-packages/e2e-tests/` and verify SDK behavior in real-world framework scenarios.
127+
128+
#### How Verdaccio Registry Works
129+
130+
E2E tests use [Verdaccio](https://verdaccio.org/), a lightweight npm registry running in Docker. Before tests run:
131+
1. SDK packages are built and packed into tarballs (`yarn build && yarn build:tarball`)
132+
2. Tarballs are published to Verdaccio at `http://127.0.0.1:4873`
133+
3. Test applications install packages from Verdaccio instead of public npm
134+
135+
#### Critical `.npmrc` Requirement
136+
137+
**Every E2E test application MUST have an `.npmrc` file** with:
138+
```
139+
@sentry:registry=http://127.0.0.1:4873
140+
@sentry-internal:registry=http://127.0.0.1:4873
141+
```
142+
143+
Without this file, pnpm will install packages from the public npm registry, causing tests to use published versions instead of your local changes. This is a common cause of "tests pass in CI but fail locally" or vice versa.
144+
145+
#### Running a Single E2E Test
146+
147+
```bash
148+
# Build packages first
149+
yarn build && yarn build:tarball
150+
151+
# Run a specific test app
152+
cd dev-packages/e2e-tests
153+
yarn test:run <app-name>
154+
155+
# Run with a specific variant (e.g., Next.js 15)
156+
yarn test:run <app-name> --variant <variant-name>
157+
```
158+
159+
#### Common Pitfalls and Debugging
160+
161+
1. **Missing `.npmrc`**: Most common issue. Always verify the test app has the correct `.npmrc` file.
162+
163+
2. **Stale tarballs**: After SDK changes, must re-run `yarn build:tarball`.
164+
165+
3. **Bundler environment variable handling**:
166+
- Webpack's `DefinePlugin` doesn't replace values in `node_modules`
167+
- Vite's `define` doesn't replace values in `node_modules` either
168+
- For Vite apps needing env vars in dependencies, use `@rollup/plugin-replace`
169+
- Next.js injects `process.env` via webpack/turbopack automatically
170+
171+
4. **`import.meta.env` behavior**:
172+
- Vite replaces `import.meta.env.VITE_*` at build time
173+
- Other bundlers (webpack, turbopack) don't have `import.meta.env`
174+
- SDK code must use try-catch when accessing `import.meta.env`
175+
176+
5. **Debugging tips**:
177+
- Check browser console logs for SDK initialization errors
178+
- Use `debug: true` in Sentry config
179+
- Verify installed package version: check `node_modules/@sentry/*/package.json`
180+
124181
### Notes for Background Tasks
125182

126183
- Make sure to use [volta](https://volta.sh/) for development. Volta is used to manage the node, yarn and pnpm version used.

‎CLAUDE.md‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,63 @@ Each package typically contains:
120120
- Integration tests use Playwright extensively
121121
- Never change the volta, yarn, or package manager setup in general unless explicitly asked for
122122

123+
### E2E Testing
124+
125+
E2E tests are located in `dev-packages/e2e-tests/` and verify SDK behavior in real-world framework scenarios.
126+
127+
#### How Verdaccio Registry Works
128+
129+
E2E tests use [Verdaccio](https://verdaccio.org/), a lightweight npm registry running in Docker. Before tests run:
130+
1. SDK packages are built and packed into tarballs (`yarn build && yarn build:tarball`)
131+
2. Tarballs are published to Verdaccio at `http://127.0.0.1:4873`
132+
3. Test applications install packages from Verdaccio instead of public npm
133+
134+
#### Critical `.npmrc` Requirement
135+
136+
**Every E2E test application MUST have an `.npmrc` file** with:
137+
```
138+
@sentry:registry=http://127.0.0.1:4873
139+
@sentry-internal:registry=http://127.0.0.1:4873
140+
```
141+
142+
Without this file, pnpm will install packages from the public npm registry, causing tests to use published versions instead of your local changes. This is a common cause of "tests pass in CI but fail locally" or vice versa.
143+
144+
#### Running a Single E2E Test
145+
146+
```bash
147+
# Build packages first
148+
yarn build && yarn build:tarball
149+
150+
# Run a specific test app
151+
cd dev-packages/e2e-tests
152+
yarn test:run <app-name>
153+
154+
# Run with a specific variant (e.g., Next.js 15)
155+
yarn test:run <app-name> --variant <variant-name>
156+
```
157+
158+
#### Common Pitfalls and Debugging
159+
160+
1. **Missing `.npmrc`**: Most common issue. Always verify the test app has the correct `.npmrc` file.
161+
162+
2. **Stale tarballs**: After SDK changes, must re-run `yarn build:tarball`.
163+
164+
3. **Bundler environment variable handling**:
165+
- Webpack's `DefinePlugin` doesn't replace values in `node_modules`
166+
- Vite's `define` doesn't replace values in `node_modules` either
167+
- For Vite apps needing env vars in dependencies, use `@rollup/plugin-replace`
168+
- Next.js injects `process.env` via webpack/turbopack automatically
169+
170+
4. **`import.meta.env` behavior**:
171+
- Vite replaces `import.meta.env.VITE_*` at build time
172+
- Other bundlers (webpack, turbopack) don't have `import.meta.env`
173+
- SDK code must use try-catch when accessing `import.meta.env`
174+
175+
5. **Debugging tips**:
176+
- Check browser console logs for SDK initialization errors
177+
- Use `debug: true` in Sentry config
178+
- Verify installed package version: check `node_modules/@sentry/*/package.json`
179+
123180
### Notes for Background Tasks
124181

125182
- Make sure to use [volta](https://volta.sh/) for development. Volta is used to manage the node, yarn and pnpm version used.

‎CONTRIBUTING.md‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,81 @@ the tests in each location. Check out the `scripts` entry of the corresponding `
7373

7474
Note: you must run `yarn build` before `yarn test` will work.
7575

76+
## Running E2E Tests Locally
77+
78+
E2E tests verify SDK behavior in real-world framework scenarios using a local npm registry (Verdaccio).
79+
80+
### Prerequisites
81+
82+
1. **Docker**: Required to run the Verdaccio registry container
83+
2. **Volta with pnpm support**: Enable pnpm in Volta by setting `VOLTA_FEATURE_PNPM=1` in your environment. See [Volta pnpm docs](https://docs.volta.sh/advanced/pnpm).
84+
85+
### Step-by-Step Instructions
86+
87+
1. **Build the SDK packages and create tarballs:**
88+
```bash
89+
yarn build
90+
yarn build:tarball
91+
```
92+
Note: You must re-run `yarn build:tarball` after any changes to packages.
93+
94+
2. **Set up environment (optional):**
95+
```bash
96+
cd dev-packages/e2e-tests
97+
cp .env.example .env
98+
# Fill in Sentry project auth info if running tests that send data to Sentry
99+
```
100+
101+
3. **Run all E2E tests:**
102+
```bash
103+
yarn test:e2e
104+
```
105+
106+
4. **Or run a specific test application:**
107+
```bash
108+
yarn test:run <app-name>
109+
# Example: yarn test:run nextjs-app-dir
110+
```
111+
112+
5. **Run with a specific variant:**
113+
```bash
114+
yarn test:run <app-name> --variant <variant-name>
115+
# Example: yarn test:run nextjs-pages-dir --variant 15
116+
```
117+
118+
### Common Issues and Troubleshooting
119+
120+
#### Packages install from public npm instead of Verdaccio
121+
122+
Every E2E test application **must** have an `.npmrc` file with:
123+
```
124+
@sentry:registry=http://127.0.0.1:4873
125+
@sentry-internal:registry=http://127.0.0.1:4873
126+
```
127+
Without this, pnpm will fetch packages from the public npm registry instead of the local Verdaccio instance, causing tests to use outdated/published versions instead of your local changes.
128+
129+
#### Tests fail after making SDK changes
130+
131+
Make sure to rebuild tarballs:
132+
```bash
133+
yarn build
134+
yarn build:tarball
135+
```
136+
137+
#### Docker-related issues
138+
139+
- Ensure Docker daemon is running
140+
- Check that port 4873 is not in use by another process
141+
- Try stopping and removing existing Verdaccio containers
142+
143+
#### Debugging test failures
144+
145+
1. Check browser console logs for SDK initialization errors
146+
2. Enable debug mode in the test app's Sentry config: `debug: true`
147+
3. Verify packages are installed from Verdaccio by checking the version in `node_modules/@sentry/*/package.json`
148+
149+
For more details, see [dev-packages/e2e-tests/README.md](dev-packages/e2e-tests/README.md).
150+
76151
## Debug Build Flags
77152

78153
Throughout the codebase, you will find a `__DEBUG_BUILD__` constant. This flag serves two purposes:

‎dev-packages/e2e-tests/README.md‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,98 @@ EOF
8888

8989
Make sure to add a `test:build` and `test:assert` command to the new app's `package.json` file.
9090

91+
### Critical: The `.npmrc` File
92+
93+
**Every E2E test application MUST have an `.npmrc` file.** This file tells pnpm to fetch `@sentry/*` and `@sentry-internal/*` packages from the local Verdaccio registry instead of the public npm registry.
94+
95+
The `.npmrc` file must contain:
96+
97+
```
98+
@sentry:registry=http://127.0.0.1:4873
99+
@sentry-internal:registry=http://127.0.0.1:4873
100+
```
101+
102+
**Why is this critical?**
103+
104+
Without this file:
105+
- pnpm will install packages from the public npm registry
106+
- Your local SDK changes will NOT be tested
107+
- Tests may pass or fail based on the published version, not your changes
108+
- This is one of the most common causes of confusing test failures
109+
110+
**How to verify packages are installed from Verdaccio:**
111+
112+
Check the version in `node_modules/@sentry/*/package.json`. If it shows a version like `0.0.0-pr.12345` or matches your local build, Verdaccio is working correctly. If it shows a released version (e.g., `8.0.0`), the `.npmrc` is missing or incorrect.
113+
114+
## Troubleshooting
115+
116+
### Common Issues
117+
118+
#### Tests fail with "Cannot find module '@sentry/...'" or use wrong package version
119+
120+
1. Verify the test application has an `.npmrc` file (see above)
121+
2. Rebuild tarballs: `yarn build && yarn build:tarball`
122+
3. Delete `node_modules` in the test application and re-run the test
123+
124+
#### Docker/Verdaccio issues
125+
126+
- Ensure Docker daemon is running
127+
- Check that port 4873 is not already in use: `lsof -i :4873`
128+
- Stop any existing Verdaccio containers: `docker ps` and `docker stop <container-id>`
129+
- Check Verdaccio logs for errors
130+
131+
#### Tests pass locally but fail in CI (or vice versa)
132+
133+
- Most likely cause: missing `.npmrc` file
134+
- Verify all `@sentry/*` dependencies use `latest || *` version specifier
135+
- Check if the test relies on environment-specific behavior
136+
137+
### Debugging Tips
138+
139+
1. **Enable Sentry debug mode**: Add `debug: true` to the Sentry init config to see detailed SDK logs
140+
2. **Check browser console**: Look for SDK initialization errors or warnings
141+
3. **Inspect network requests**: Verify events are being sent to the expected endpoint
142+
4. **Check installed versions**: `cat node_modules/@sentry/browser/package.json | grep version`
143+
144+
## Bundler-Specific Behavior
145+
146+
Different bundlers handle environment variables and code replacement differently. This is important when writing tests or SDK code that relies on build-time constants.
147+
148+
### Webpack
149+
150+
- `DefinePlugin` replaces variables in your application code
151+
- **Does NOT replace values inside `node_modules`**
152+
- Environment variables must be explicitly defined
153+
154+
### Vite
155+
156+
- `define` option replaces variables in your application code
157+
- **Does NOT replace values inside `node_modules`**
158+
- `import.meta.env.VITE_*` variables are replaced at build time
159+
- For replacing values in dependencies, use `@rollup/plugin-replace`
160+
161+
### Next.js
162+
163+
- Automatically injects `process.env` via webpack/turbopack
164+
- Handles environment variables more seamlessly than raw webpack/Vite
165+
- Server and client bundles may have different environment variable access
166+
167+
### `import.meta.env` Considerations
168+
169+
- Only available in Vite and ES modules
170+
- Webpack and Turbopack do not have `import.meta.env`
171+
- SDK code accessing `import.meta.env` must use try-catch to handle environments where it doesn't exist
172+
173+
```typescript
174+
// Safe pattern for SDK code
175+
let envValue: string | undefined;
176+
try {
177+
envValue = import.meta.env.VITE_SOME_VAR;
178+
} catch {
179+
// import.meta.env not available in this bundler
180+
}
181+
```
182+
91183
Test apps in the folder `test-applications` will be automatically picked up by CI in the job `job_e2e_tests` (in `.github/workflows/build.yml`).
92184
The test matrix for CI is generated in `dev-packages/e2e-tests/lib/getTestMatrix.ts`.
93185

0 commit comments

Comments
 (0)