-
Notifications
You must be signed in to change notification settings - Fork 22
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adds a new github CI Action to generate coverage and upload to coveralls #1373
Merged
oleksandr-pavlyk
merged 19 commits into
IntelPython:use-skbuild-and-cmake
from
vlad-perevezentsev:add_gen_coverage
Apr 7, 2023
Merged
Changes from 18 commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
88f9f82
Add gen_coverage script
vlad-perevezentsev b6ce69a
Add generate_coverage.yaml
vlad-perevezentsev 9bf1cc4
set env
vlad-perevezentsev 15f2855
Set OSL_ICD_FILENAMES env
vlad-perevezentsev bc60d08
add coveralls support
vlad-perevezentsev 58135be
Update generage_coverage.yaml
vlad-perevezentsev b9dd88e
Enable running all tests for coverage
vlad-perevezentsev 63b2571
ignore test_random.py
vlad-perevezentsev ee17454
Skip falling tests on cpu
vlad-perevezentsev b419bce
Skip falling tests in test_random and remove --ignore flag in pytest
vlad-perevezentsev e9cfc00
Add parser for command-line options
vlad-perevezentsev ee6d2e4
Fix a mistake
vlad-perevezentsev a53fd6b
Merge use-skbuild-and-cmake into add_gen_coverage
vlad-perevezentsev 67b488c
Change branch name in generate_coverage.yaml
vlad-perevezentsev 554e978
Merge use-skbuild-and-cmake into add_gen_coverage
vlad-perevezentsev a8950c5
Fix remarks in CMakeLists
vlad-perevezentsev a9d7906
Add --ignore test_strides.py in yaml
vlad-perevezentsev 9543c23
Unset OCL_ICD_FILENAMES
vlad-perevezentsev d2c3fbc
Update scripts/gen_coverage.py
oleksandr-pavlyk File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,81 @@ | ||
name: Generate coverage data for dpnp | ||
on: | ||
pull_request: | ||
push: | ||
branches: [use-skbuild-and-cmake] | ||
|
||
jobs: | ||
generate-coverage: | ||
name: Generate coverage and push to Coveralls.io | ||
runs-on: ubuntu-20.04 | ||
|
||
defaults: | ||
run: | ||
shell: bash -l {0} | ||
|
||
env: | ||
python-ver: '3.10' | ||
|
||
steps: | ||
- name: Cancel Previous Runs | ||
uses: styfle/cancel-workflow-action@0.11.0 | ||
with: | ||
access_token: ${{ github.token }} | ||
|
||
- name: Checkout repo | ||
uses: actions/checkout@v3 | ||
with: | ||
fetch-depth: 0 | ||
|
||
- name: Setup miniconda | ||
uses: conda-incubator/setup-miniconda@v2.2.0 | ||
with: | ||
auto-update-conda: true | ||
python-version: ${{ env.python-ver }} | ||
miniconda-version: 'latest' | ||
activate-environment: 'coverage' | ||
channels: intel, conda-forge | ||
|
||
- name: Install Lcov | ||
run: | | ||
sudo apt-get install lcov | ||
- name: Install dpnp dependencies | ||
run: | | ||
conda install cython llvm cmake scikit-build ninja pytest pytest-cov coverage[toml] \ | ||
dppy/label/dev::dpctl dpcpp_linux-64 mkl-devel-dpcpp tbb-devel onedpl-devel | ||
- name: Conda info | ||
run: | | ||
conda info | ||
conda list | ||
- name: Build dpnp with coverage | ||
run: | | ||
python scripts/gen_coverage.py --pytest-opts="--ignore tests/test_random.py \ | ||
--ignore tests/test_strides.py" | ||
- name: Install coverall dependencies | ||
run: | | ||
sudo gem install coveralls-lcov | ||
conda install coveralls | ||
- name: Upload coverage data to coveralls.io | ||
run: | | ||
echo "Processing pytest-coverage" | ||
export DPNP_PYTEST_LCOV=$(find . -name dpnp_pytest.lcov) | ||
coveralls-lcov -v -n $DPNP_PYTEST_LCOV > pytest-dpnp-c-api-coverage.json | ||
# merge file with coverage data and upload | ||
echo "Merging files with coverage data" | ||
coveralls --service=github --merge=pytest-dpnp-c-api-coverage.json | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
COVERALLS_PARALLEL: true | ||
|
||
coveralls: | ||
name: Indicate completion to coveralls.io | ||
needs: generate-coverage | ||
runs-on: ubuntu-latest | ||
container: python:3-slim | ||
steps: | ||
- name: Finished | ||
run: | | ||
pip3 install --upgrade coveralls | ||
coveralls --finish | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
if (EXISTS ${SOURCE_FILE}) | ||
configure_file(${SOURCE_FILE} ${DEST} COPYONLY) | ||
endif() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
[tool.coverage.run] | ||
plugins = [ | ||
"Cython.Coverage" | ||
] | ||
branch = true | ||
source = [ | ||
"dpnp", | ||
] | ||
omit = [ | ||
"tests/*", | ||
"dpnp/_version.py", | ||
] | ||
|
||
[tool.coverage.report] | ||
omit = [ | ||
"tests/*", | ||
"dpnp/_version.py", | ||
] | ||
|
||
[tool.pytest.ini.options] | ||
minversion = "6.0" | ||
norecursedirs= [ | ||
".*", "*.egg*", "build", "dist", "conda-recipe", | ||
] | ||
addopts = [ | ||
"--junitxml=junit.xml", | ||
"--ignore setup.py", | ||
"--ignore run_test.py", | ||
"--cov-report term-missing", | ||
"--tb native", | ||
"--strict", | ||
"--durations=20", | ||
"-q -ra", | ||
] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,145 @@ | ||
import os | ||
import subprocess | ||
import sys | ||
|
||
def run( | ||
c_compiler=None, | ||
cxx_compiler=None, | ||
bin_llvm=None, | ||
pytest_opts = "", | ||
): | ||
|
||
IS_LIN = False | ||
|
||
if "linux" in sys.platform: | ||
IS_LIN = True | ||
elif sys.platform in ["win32", "cygwin"]: | ||
pass | ||
else: | ||
assert False, sys.platform + " not supported" | ||
|
||
if not IS_LIN: | ||
raise RuntimeError( | ||
"This scripts only supports coverage collection on Linux" | ||
) | ||
|
||
setup_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) | ||
dpctl_cmake_dir = subprocess.check_output(["python", "-m", "dpctl", "--cmakedir"]) | ||
oleksandr-pavlyk marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
cmake_args = [ | ||
sys.executable, | ||
"setup.py", | ||
"develop", | ||
"-G=Ninja", | ||
"--", | ||
"-DCMAKE_C_COMPILER:PATH=" + c_compiler, | ||
"-DCMAKE_CXX_COMPILER:PATH=" + cxx_compiler, | ||
"-DDPCTL_MODULE_PATH=" + dpctl_cmake_dir.decode().rstrip(), | ||
"-DCMAKE_VERBOSE_MAKEFILE=ON", | ||
"-DDPNP_GENERATE_COVERAGE=ON", | ||
] | ||
|
||
env = None | ||
if bin_llvm: | ||
env = { | ||
"PATH": ":".join((os.environ.get("PATH", ""), bin_llvm)), | ||
"LLVM_TOOLS_HOME": bin_llvm, | ||
} | ||
env.update({k: v for k, v in os.environ.items() if k != "PATH"}) | ||
|
||
|
||
subprocess.check_call(cmake_args, shell=False, cwd=setup_dir, env=env) | ||
|
||
env["LLVM_PROFILE_FILE"] = "dpnp_pytest.profraw" | ||
subprocess.check_call( | ||
[ | ||
"pytest", | ||
"-q", | ||
"-ra", | ||
"--disable-warnings", | ||
"--cov-config", | ||
"pyproject.toml", | ||
"--cov", | ||
"dpnp", | ||
"--cov-report", | ||
"term-missing", | ||
"--pyargs", | ||
"tests", | ||
"-vv", | ||
*pytest_opts.split(), | ||
], | ||
cwd=setup_dir, | ||
shell=False, | ||
env=env, | ||
) | ||
|
||
def find_objects(): | ||
import os | ||
|
||
objects = [] | ||
dpnp_path = os.getcwd() | ||
search_path = os.path.join(dpnp_path, "dpnp") | ||
files = os.listdir(search_path) | ||
for file in files: | ||
if file.endswith("_c.so"): | ||
objects.extend(["-object", os.path.join(search_path, file)]) | ||
return objects | ||
|
||
objects = find_objects() | ||
instr_profile_fn = "dpnp_pytest.profdata" | ||
# generate instrumentation profile data | ||
subprocess.check_call( | ||
[ | ||
os.path.join(bin_llvm, "llvm-profdata"), | ||
"merge", | ||
"-sparse", | ||
env["LLVM_PROFILE_FILE"], | ||
"-o", | ||
instr_profile_fn, | ||
] | ||
) | ||
|
||
# export lcov | ||
with open("dpnp_pytest.lcov", "w") as fh: | ||
subprocess.check_call( | ||
[ | ||
os.path.join(bin_llvm, "llvm-cov"), | ||
"export", | ||
"-format=lcov", | ||
"-ignore-filename-regex=/tmp/icpx*", | ||
"-instr-profile=" + instr_profile_fn, | ||
] | ||
+ objects, | ||
stdout=fh, | ||
) | ||
|
||
if __name__ == "__main__": | ||
import argparse | ||
|
||
parser = argparse.ArgumentParser( | ||
description="Driver to build dpnp and generate coverage" | ||
) | ||
driver = parser.add_argument_group(title="Coverage driver arguments") | ||
driver.add_argument( | ||
"--pytest-opts", | ||
help="Channels through additional pytest options", | ||
dest="pytest_opts", | ||
default="", | ||
type=str, | ||
) | ||
|
||
args = parser.parse_args() | ||
|
||
c_compiler = "icx" | ||
cxx_compiler = "icpx" | ||
oleksandr-pavlyk marked this conversation as resolved.
Show resolved
Hide resolved
|
||
icx_path = subprocess.check_output(["which", "icx"]) | ||
bin_dir = os.path.dirname(os.path.dirname(icx_path)) | ||
bin_llvm = os.path.join(bin_dir.decode("utf-8"), "bin-llvm") | ||
|
||
|
||
run( | ||
c_compiler=c_compiler, | ||
cxx_compiler=cxx_compiler, | ||
bin_llvm=bin_llvm, | ||
pytest_opts = args.pytest_opts, | ||
) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably it would be good to have a separate variable with a list of ignoring tests.