-
Notifications
You must be signed in to change notification settings - Fork 307
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Development
: Playwright E2E client coverage
#10466
Draft
muradium
wants to merge
35
commits into
develop
Choose a base branch
from
feature/playwright/test-coverage
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
35 commits
Select commit
Hold shift + click to select a range
3d95c6f
Use monocart-reporter and auto-fixture for collecting Playwright cove…
muradium 03c2621
Add script for archiving coverage reports for artifact extraction
muradium b42ed3e
Add script for merging coverage reports
muradium 00e0592
Zip coverage results after merging, disable css coverage, disable raw…
muradium 1d14b3d
Extra logs
muradium 9784e14
Change merge coverage script into mjs script
muradium ee0dc3e
Fix the names of cleaned-up reports on compose command
muradium 6c7900a
Use istanbul to efficiently merge json coverage reports
muradium 67c12a9
Upload only the coverage of parallel tests as artifact to check if th…
muradium 4421b5a
Merge branch 'develop' into feature/playwright/test-coverage
muradium db814e2
Try using artemis-app instead of nginx as the base url of tests
muradium 759b0f2
Revert "Try using artemis-app instead of nginx as the base url of tests"
muradium 12bcab1
Enable inlineSources to make sure webapp sources are still present in…
muradium c15657c
Revert "Upload only the coverage of parallel tests as artifact to che…
muradium 0f5046e
Revert "Enable inlineSources to make sure webapp sources are still pr…
muradium 5967396
Execute only a small portion of tests to avoid heap memory limits
muradium 4e68100
Enable inlineSources to make sure webapp sources are still present in…
muradium 7b63c9b
Try increased heap size for the whole test set
muradium 5e29c1d
Remove raw report remains
muradium 6a31344
Archive the original unmerged report
muradium 55b5b85
Use artemis-app files for coverage generation
muradium f113623
Fix base url by using the value from env
muradium 923c394
Enable all tests back to test coverage performance
muradium a39c735
Revert to archiving the whole coverage report
muradium 991692f
Zip all client coverage reports separately
muradium 0b2baf6
Archive the whole report
muradium 3e7bffe
Improve the merge coverage script
muradium 0189964
Try coverage without inlining sources
f298e61
Minor improvements
dcfed29
Merge branch 'develop' into feature/playwright/test-coverage
muradium 8d36858
Merge branch 'develop' into feature/playwright/test-coverage
muradium 1c9d3a3
Filtering coverage after report generation and mapping sources
muradium f87438b
Add config file extension on import
muradium 58d7bdc
Move coverage filters to merge script
muradium 5063b26
Refactor coverage filtering logic in merge-coverage-reports
muradium File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,105 @@ | ||
// merge-coverage-reports.mjs | ||
import path from 'path'; | ||
import { fileURLToPath } from 'url'; | ||
import archiver from 'archiver'; | ||
import coverage from 'istanbul-lib-coverage'; | ||
import reports from 'istanbul-reports'; | ||
import libReport from 'istanbul-lib-report'; | ||
import fsAsync from 'fs/promises'; | ||
import fs from 'fs'; | ||
|
||
/** | ||
* Represents a configuration object used for filtering file paths during coverage report generation. | ||
* | ||
* @property {string[]} includePaths - An array of file path patterns to include in the coverage analysis. | ||
* @property {string[]} excludePaths - An array of file path patterns to exclude from the coverage analysis. | ||
*/ | ||
const coverageFilters = { | ||
includePaths: ['src/main/webapp'], | ||
excludePaths: [], | ||
}; | ||
|
||
const __filename = fileURLToPath(import.meta.url); | ||
const __dirname = path.dirname(__filename); | ||
|
||
const coverageParallelDir = path.join(__dirname, 'test-reports/monocart-report-parallel'); | ||
const coverageSequentialDir = path.join(__dirname, 'test-reports/monocart-report-sequential'); | ||
const coverageDir = path.join(__dirname, 'test-reports/client-coverage'); | ||
const lcovDir = path.join(coverageDir, 'lcov-report'); | ||
|
||
console.log(`Merging coverage reports`); | ||
|
||
const coverageParallel = JSON.parse(fs.readFileSync(path.join(coverageParallelDir, '/coverage/coverage-final.json'), 'utf8')); | ||
const coverageSequential = JSON.parse(fs.readFileSync(path.join(coverageSequentialDir, '/coverage/coverage-final.json'), 'utf8')); | ||
|
||
// Apply filters to coverage data before merging | ||
const filteredCoverageParallel = filterCoverageData(coverageParallel, coverageFilters); | ||
const filteredCoverageSequential = filterCoverageData(coverageSequential, coverageFilters); | ||
|
||
const mapA = coverage.createCoverageMap(filteredCoverageParallel); | ||
const mapB = coverage.createCoverageMap(filteredCoverageSequential); | ||
mapA.merge(mapB); | ||
|
||
const context = libReport.createContext({ | ||
dir: lcovDir, | ||
coverageMap: mapA, | ||
}); | ||
|
||
const htmlReport = reports.create('html'); | ||
const lcovReport = reports.create('lcovonly', { file: 'lcov.info' }); | ||
|
||
htmlReport.execute(context); | ||
lcovReport.execute(context); | ||
|
||
console.log(`Merged coverage reports successfully`); | ||
|
||
await fsAsync.rm(coverageParallelDir, { recursive: true, force: true }); | ||
await fsAsync.rm(coverageSequentialDir, { recursive: true, force: true }); | ||
|
||
// Bamboo can upload only files as an artifact, not directories | ||
// That's why we archive the lcov coverage directory on CI to prepare it as an artifact | ||
if (process.env.CI === 'true') { | ||
try { | ||
await createArchive(path.join(coverageDir, 'e2e-client-coverage.zip'), lcovDir); | ||
} catch (err) { | ||
console.error('Error while creating archives:', err); | ||
} | ||
} | ||
|
||
// Archives the directory | ||
async function createArchive(outputPath, inputDirectory) { | ||
const output = await fs.createWriteStream(outputPath); | ||
const archive = archiver('zip', { zlib: { level: 9 } }); | ||
|
||
output.on('close', () => { | ||
console.log(`Coverage report archived on: ${outputPath}`); | ||
}); | ||
|
||
archive.on('error', (err) => { | ||
throw err; | ||
}); | ||
|
||
archive.pipe(output); | ||
archive.directory(inputDirectory, '', (entry) => entry); | ||
await archive.finalize(); | ||
} | ||
|
||
/** | ||
* Filter coverage data based on include and exclude path filters | ||
* @param {Object} coverageData - The coverage data to filter | ||
* @param {Object} filters - The filters to apply | ||
* @returns {Object} - The filtered coverage data | ||
*/ | ||
function filterCoverageData(coverageData, filters) { | ||
Object.keys(coverageData).forEach(filePath => { | ||
const shouldInclude = filters.includePaths.length === 0 || | ||
filters.includePaths.some(includePath => filePath.includes(includePath)); | ||
const shouldExclude = filters.excludePaths.some(excludePath => filePath.includes(excludePath)); | ||
|
||
if (!(shouldInclude && !shouldExclude)) { | ||
delete coverageData[filePath]; | ||
} | ||
}); | ||
|
||
return coverageData; | ||
} |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a quick note: since Bamboo will soon be removed, we’re actively migrating our E2E test runs to GitHub Actions. In a couple of weeks, the tests should be fully running here. After that, Helios will parse the final test results (
results.xml
). So, IMO there’s not much value in adding any new Bamboo-specific functionality.From what I can see, you’re still generating
results.xml
as the final artifact in JUnit format, and that logic hasn’t changed—please correct me if I’m wrong.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, test results are still in JUnit format, no changes there.
This PR is for coverage report. The part you commented on is only for preparing artifact for Bamboo. On GitHub actions, we can just upload the lcov report directory as an artifact. Do you think it makes sense to skip the CI part for now until we transition to new CI pipeline?