-
Notifications
You must be signed in to change notification settings - Fork 27
FEAT: Full Code Coverage & Workflow Integration #238
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 2 commits
Commits
Show all changes
44 commits
Select commit
Hold shift + click to select a range
40deeba
FEAT: Full Stack Code Coverage
bewithgaurav f11fa41
Link to HTML and summary only
bewithgaurav a7aeb06
Potential fix for code scanning alert no. 310: Workflow does not cont…
bewithgaurav acaa2f3
port to linux since macOS doesnt support docker in GH runner
bewithgaurav 9d02e9b
correct the sh filename
bewithgaurav c04b019
Merge branch 'main' of https://github.com/microsoft/mssql-python into…
bewithgaurav b369afe
codecov sh file to linux
bewithgaurav 89266e9
fixing bugs
bewithgaurav b55d146
do things only once
bewithgaurav c4c47a4
add this in ADO
bewithgaurav 6c977ad
fix Github comment
bewithgaurav 541945b
fix Github comment, core logic is now in PR validation
bewithgaurav fa3d6cc
fix spamming in GH workflow
bewithgaurav 3d67133
fix spamming in GH workflow
bewithgaurav aed05f0
line coverage only for cpp
bewithgaurav a65c52d
line coverage only for cpp
bewithgaurav 2173309
refactor
bewithgaurav 054c098
added a nice GH comment
bewithgaurav 014099d
added a nice GH comment
bewithgaurav 39cbd93
comment is prettier now
bewithgaurav 67de428
added graceful handling of failures
bewithgaurav 8e5c604
comment final
bewithgaurav 79373c9
Merge branch 'main' of https://github.com/microsoft/mssql-python into…
bewithgaurav d844473
trying patch coverage
bewithgaurav b7b5c9c
Merge branch 'main' of https://github.com/microsoft/mssql-python into…
bewithgaurav 6ad5087
fix xml search and dont trigger PR validation anymore
bewithgaurav 45747b0
undo trigger PR validation and use a specified build for code coevrag…
bewithgaurav 6e03369
specific build id
bewithgaurav 66139e0
specific build id
bewithgaurav b3329a0
specific build id
bewithgaurav 2b4927d
unzip error
bewithgaurav 2103aaa
git main
bewithgaurav 7f26846
pull all branches
bewithgaurav 67fb0d2
polling for artifact instead of build status
bewithgaurav f05ef05
test method increased, patch coverage % debugging
bewithgaurav 049b611
fix % extraction, make format better
bewithgaurav da0c0c5
add something in cpp
bewithgaurav c250305
change name from test to dummy in cursor
bewithgaurav 8ad00e9
remove redundant code and add a re-label check on pr-format workflow
bewithgaurav 328b35c
cleanup
bewithgaurav 2176fba
rename PR validation task
bewithgaurav da9651c
Merge branch 'main' of https://github.com/microsoft/mssql-python into…
bewithgaurav dfb2537
Reformatted Comment
bewithgaurav 867ae79
Merge branch 'main' into bewithgaurav/universal_codecov
bewithgaurav File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| name: PR Validation (macOS with Unified Coverage) | ||
|
|
||
| on: | ||
| pull_request: | ||
| branches: | ||
| - main | ||
|
|
||
| jobs: | ||
| pytest-macos: | ||
| name: macOS Unified Coverage | ||
| runs-on: macos-latest | ||
|
|
||
| env: | ||
| DB_PASSWORD: ${{ secrets.DB_PASSWORD }} | ||
| DB_CONNECTION_STRING: ${{ secrets.DB_CONNECTION_STRING }} | ||
|
|
||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Python 3.13 | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3.13' | ||
|
|
||
| - name: Install CMake | ||
| run: | | ||
| brew update | ||
| brew uninstall cmake --ignore-dependencies || echo "CMake not installed" | ||
| brew install cmake | ||
|
|
||
| - name: Install and start Colima-based Docker | ||
| run: | | ||
| brew install docker colima | ||
| colima start --cpu 4 --memory 8 --disk 50 | ||
| docker context use colima >/dev/null || true | ||
| docker version | ||
| docker ps | ||
|
|
||
| - name: Start SQL Server container | ||
| run: | | ||
| docker pull mcr.microsoft.com/mssql/server:2022-latest | ||
| docker run \ | ||
| --name sqlserver \ | ||
| -e ACCEPT_EULA=Y \ | ||
| -e MSSQL_SA_PASSWORD="${DB_PASSWORD}" \ | ||
| -p 1433:1433 \ | ||
| -d mcr.microsoft.com/mssql/server:2022-latest | ||
|
|
||
| # Wait until SQL Server is ready | ||
| for i in {1..30}; do | ||
| docker exec sqlserver \ | ||
| /opt/mssql-tools18/bin/sqlcmd \ | ||
| -S localhost \ | ||
| -U SA \ | ||
| -P "$DB_PASSWORD" \ | ||
| -C -Q "SELECT 1" && break | ||
| sleep 2 | ||
| done | ||
|
|
||
| - name: Install Python dependencies | ||
| run: | | ||
| python -m pip install --upgrade pip | ||
| pip install -r requirements.txt | ||
|
|
||
| - name: Build pybind bindings | ||
| run: | | ||
| cd mssql_python/pybind | ||
| ./build.sh | ||
|
|
||
| - name: Run pytest (with raw coverage) | ||
| run: | | ||
| echo "Running pytest..." | ||
| python -m pytest -v --junitxml=test-results.xml --cov=. --cov-report=xml --capture=tee-sys --cache-clear | ||
|
|
||
| - name: Generate unified coverage (Python + C++) | ||
| run: | | ||
| chmod +x ./generate_codecov.sh | ||
| ./generate_codecov.sh | ||
| # Create summary text file from LCOV report | ||
| genhtml total.info \ | ||
| --output-directory unified-coverage \ | ||
| --quiet \ | ||
| --title "Unified Coverage Report" \ | ||
| --summary-only > unified-coverage/summary.txt | ||
|
|
||
| - name: Upload full HTML coverage artifact | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: UnifiedCoverageHTML | ||
| path: unified-coverage | ||
|
|
||
| - name: Comment unified coverage summary on PR | ||
| uses: marocchino/sticky-pull-request-comment@v2 | ||
| with: | ||
| header: Unified Coverage Report | ||
| message: | | ||
| $(cat unified-coverage/summary.txt) | ||
|
|
||
| 👉 [Download full HTML report](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}#artifacts) | ||
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| #!/bin/bash | ||
| set -euo pipefail | ||
|
|
||
| echo "===================================" | ||
| echo "[STEP 1] Installing dependencies" | ||
| echo "===================================" | ||
|
|
||
| # Ensure Homebrew exists | ||
| if ! command -v brew &>/dev/null; then | ||
| echo "[ERROR] Homebrew is required. Install from https://brew.sh/" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Install LLVM (for llvm-profdata, llvm-cov) | ||
| if ! command -v llvm-profdata &>/dev/null; then | ||
| echo "[ACTION] Installing LLVM via Homebrew" | ||
| brew install llvm | ||
| fi | ||
| export PATH="/opt/homebrew/opt/llvm/bin:$PATH" | ||
bewithgaurav marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| # Install lcov (provides lcov + genhtml) | ||
| if ! command -v genhtml &>/dev/null; then | ||
| echo "[ACTION] Installing lcov via Homebrew" | ||
| brew install lcov | ||
| fi | ||
|
|
||
| # Install Python plugin for LCOV export | ||
| if ! python -m pip show coverage-lcov &>/dev/null; then | ||
| echo "[ACTION] Installing coverage-lcov via pip" | ||
| python -m pip install coverage-lcov | ||
| fi | ||
|
|
||
| echo "===================================" | ||
| echo "[STEP 2] Running pytest with Python coverage" | ||
| echo "===================================" | ||
|
|
||
| # Cleanup old coverage | ||
| rm -f .coverage coverage.xml python-coverage.info cpp-coverage.info total.info | ||
| rm -rf htmlcov unified-coverage | ||
|
|
||
| # Run pytest with Python coverage (XML + HTML output) | ||
| python -m pytest -v \ | ||
| --junitxml=test-results.xml \ | ||
| --cov=mssql_python \ | ||
| --cov-report=xml \ | ||
| --cov-report=html \ | ||
| --capture=tee-sys \ | ||
| --cache-clear | ||
|
|
||
| # Convert Python coverage to LCOV format (restrict to your repo only) | ||
| echo "[ACTION] Converting Python coverage to LCOV" | ||
| coverage lcov -o python-coverage.info --include="mssql_python/*" | ||
|
|
||
| echo "===================================" | ||
| echo "[STEP 3] Processing C++ coverage (Clang/LLVM)" | ||
| echo "===================================" | ||
|
|
||
| # Merge raw profile data from pybind runs | ||
| if [ ! -f default.profraw ]; then | ||
| echo "[ERROR] default.profraw not found. Did you build with -fprofile-instr-generate?" | ||
| exit 1 | ||
| fi | ||
|
|
||
| llvm-profdata merge -sparse default.profraw -o default.profdata | ||
|
|
||
| # Find the pybind .so (assuming universal2 build) | ||
| PYBIND_SO=$(find mssql_python -name "*.so" | grep "universal2" | head -n 1) | ||
| if [ -z "$PYBIND_SO" ]; then | ||
| echo "[ERROR] Could not find pybind .so (universal2 build)." | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "[INFO] Using pybind module: $PYBIND_SO" | ||
|
|
||
| # Export C++ coverage, excluding Python headers, pybind11, and Homebrew includes | ||
| llvm-cov export "$PYBIND_SO" \ | ||
| -instr-profile=default.profdata \ | ||
| -arch arm64 \ | ||
bewithgaurav marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| -ignore-filename-regex='(python3\.13|cpython|pybind11|homebrew)' \ | ||
bewithgaurav marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| -format=lcov > cpp-coverage.info | ||
|
|
||
| echo "===================================" | ||
| echo "[STEP 4] Merging Python + C++ coverage" | ||
| echo "===================================" | ||
|
|
||
| # Merge LCOV reports (ignore minor inconsistencies in Python LCOV export) | ||
| lcov -a python-coverage.info -a cpp-coverage.info -o total.info \ | ||
| --ignore-errors inconsistent,corrupt | ||
|
|
||
| # Generate unified HTML report | ||
| genhtml total.info --output-directory unified-coverage | ||
|
|
||
| echo "[SUCCESS] Unified coverage report generated at unified-coverage/index.html" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.