Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
5ada2dd
Remove deprecated images from CI and release-script
AyanSinhaMahapatra Sep 9, 2022
b3aaa47
Bump versions to 32.0.0b1
AyanSinhaMahapatra Nov 17, 2022
27b15fc
Add license detection rules
AyanSinhaMahapatra Nov 17, 2022
969f5cf
Fix test failures
AyanSinhaMahapatra Nov 18, 2022
e5d3a19
Fix license/rules dump format
AyanSinhaMahapatra Dec 12, 2022
fd60bb8
Merge branch 'develop' into release-32-rc1-prep
AyanSinhaMahapatra Dec 12, 2022
7cbdb17
Reformat licenses with a load/dump
AyanSinhaMahapatra Dec 13, 2022
279732f
Reformat rules with a load/dump
AyanSinhaMahapatra Dec 14, 2022
84186d0
Add license updates from spdx license list 3.19
AyanSinhaMahapatra Dec 14, 2022
162cffb
Vendor python-forntmatter completely
AyanSinhaMahapatra Dec 15, 2022
462d0e6
Update synclic script and misc test expectations
AyanSinhaMahapatra Dec 15, 2022
1ea6303
Add new license category "CLA" #3147 #3038
AyanSinhaMahapatra Dec 15, 2022
826b261
Deprecate licenses and rules
AyanSinhaMahapatra Dec 15, 2022
6921c9d
Add new and updated licenses and rules
AyanSinhaMahapatra Dec 15, 2022
d0ebced
Fix test failures
AyanSinhaMahapatra Dec 16, 2022
755f28b
Merge branch 'license-detection-follow-up' into release-32-rc1-prep
AyanSinhaMahapatra Dec 20, 2022
15318be
Update debian and spdx output plugins
AyanSinhaMahapatra Dec 22, 2022
e9886d4
Add license dump CLI option back
AyanSinhaMahapatra Dec 22, 2022
2b67d0b
Add new docstrings and update old docstrings
AyanSinhaMahapatra Dec 22, 2022
f93f880
Update rule_identifiers for synthetic rules
AyanSinhaMahapatra Dec 22, 2022
7597fbc
Update CHANGELOG
AyanSinhaMahapatra Dec 22, 2022
1477742
Update test expecations for recent changes
AyanSinhaMahapatra Dec 22, 2022
b300cf7
Update documentation
AyanSinhaMahapatra Dec 22, 2022
466084d
Ensure license dump test does not regen footer
AyanSinhaMahapatra Dec 22, 2022
51e40bf
Remove dangling code using mappings
AyanSinhaMahapatra Dec 23, 2022
23ee46c
Fix test expectations
AyanSinhaMahapatra Dec 23, 2022
b4f497c
Do not use packaging.LegacyVersion #3171 #3177
pombredanne Dec 21, 2022
7e64e73
Merge branch 'release-32-rc1-prep' into v31.2.2-branch-hotfix
AyanSinhaMahapatra Dec 23, 2022
96b16c3
Merge pull request #3180 from nexB/v31.2.2-branch-hotfix
AyanSinhaMahapatra Dec 23, 2022
33503bc
Add new BSD simplified variant
pombredanne Dec 26, 2022
950c6c8
Format license text and remove copyright notice
pombredanne Dec 28, 2022
843c285
Format license text and remove copyright notice
pombredanne Dec 28, 2022
b7a0309
Restore correct test expectation
pombredanne Dec 28, 2022
3937aa0
Align ignorables with copyright removal from text
pombredanne Dec 28, 2022
92b6335
Generate correct "rule file" paths
pombredanne Dec 28, 2022
e2c41bc
Format code
pombredanne Dec 28, 2022
9f0b475
Merge remote-tracking branch 'upstream/develop' into release-32-rc1-prep
pombredanne Dec 28, 2022
4503367
Streamline license_db dump command
pombredanne Dec 28, 2022
d763f7a
Bump to 32.0.0rc1
pombredanne Dec 28, 2022
e47b3b4
Remove duplicate class
pombredanne Dec 28, 2022
3def6a8
Organize imports
pombredanne Dec 28, 2022
ff65a36
Guard tracing with TRACE flag
pombredanne Dec 28, 2022
950d429
Apply cosmetic refactorings
pombredanne Dec 28, 2022
28942bc
Remove duplicated function
pombredanne Dec 28, 2022
ec7a7a6
Remove duplicated import
pombredanne Dec 28, 2022
2cd7d98
Guard tracing with TRACE flag
pombredanne Dec 28, 2022
a783162
Streamline imports
pombredanne Dec 28, 2022
e0a08c4
Streamline imports
pombredanne Dec 28, 2022
0547a87
Streamline imports
pombredanne Dec 28, 2022
f2568b0
Streamline imports
pombredanne Dec 28, 2022
90136e0
Streamline imports
pombredanne Dec 28, 2022
6842e2d
Streamline imports and unused variables
pombredanne Dec 28, 2022
8b3d0e4
Use correct Package factory
pombredanne Dec 28, 2022
c1d55c2
Remove unused code
pombredanne Dec 28, 2022
68d77a7
Use correct function name
pombredanne Dec 28, 2022
2bbeea8
Rename function name to avoid duplication
pombredanne Dec 28, 2022
6d78b2e
Streamline imports
pombredanne Dec 28, 2022
dd6d4b9
Streamline imports
pombredanne Dec 28, 2022
b6d0162
Format code
pombredanne Dec 28, 2022
7cb42d4
Streamline imports
pombredanne Dec 28, 2022
11af6f3
Streamline imports
pombredanne Dec 28, 2022
5112b25
Guard imports from cache inside functions
pombredanne Dec 28, 2022
e8c0df6
Add support for FIPS
pombredanne Dec 28, 2022
a959b49
Ensue rule id is unique also for rule without texts
pombredanne Dec 28, 2022
fdde6d6
Restore correct license expression mapping
pombredanne Dec 28, 2022
ce5be3a
Import sys at the top
pombredanne Dec 28, 2022
ae4c63f
Add support for FIPS
pombredanne Dec 29, 2022
021ad24
Regen test with new rule identifier
pombredanne Dec 29, 2022
d2b4fc7
Update tests expectations
pombredanne Jan 2, 2023
c0590ca
Apply cosmetic refactoring
pombredanne Jan 2, 2023
ac4f7dc
Improve Rule check with is_synthetic check
pombredanne Jan 2, 2023
080e163
Use correct imports in tests
pombredanne Jan 2, 2023
83a0db7
Avoit using copyright as a varibale name
pombredanne Jan 2, 2023
8ee4406
Rename and streamline get_rule_object_from_match
pombredanne Jan 2, 2023
ccc79a4
Make license reference its own plugin
pombredanne Jan 3, 2023
d7c682d
Remove deprecated is_license_text plugin
pombredanne Jan 3, 2023
e6fca98
Use correct planned release date
pombredanne Jan 3, 2023
d292aac
Improve license and rule references handling
pombredanne Jan 3, 2023
ccef6f8
Do not include SPDX expression in low level matches
pombredanne Jan 3, 2023
f27e61c
Update test expectations for RPMs
pombredanne Jan 3, 2023
0298a5c
Update test expectations
pombredanne Jan 3, 2023
edaef02
Do not keep is_deprecated License flag
pombredanne Jan 3, 2023
b3acde0
Enable LicenseReference plugin correctly
pombredanne Jan 3, 2023
857962c
Update test expectations
pombredanne Jan 3, 2023
299790f
Merge upstream branch update-ci-images
pombredanne Jan 3, 2023
88b5144
Generate correct Rule URL
pombredanne Jan 3, 2023
2b64449
Use plain JSON test file
pombredanne Jan 3, 2023
2548f43
Correct license mappings for Alpine
pombredanne Jan 4, 2023
676a471
Generate correct URLs for license and rules
pombredanne Jan 4, 2023
993d4af
Use correct URLs for license and rules
pombredanne Jan 4, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
The diff you're trying to view is too large. We only load the first 3000 changed files.
171 changes: 71 additions & 100 deletions .github/workflows/scancode-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ on:
permissions: {}
jobs:

build_scancode_for_pypi:
build_scancode_wheel_for_pypi:
permissions:
contents: read # to fetch code (actions/checkout)

name: Build PyPI archives
name: Build PyPI wheels
runs-on: ubuntu-20.04

defaults:
Expand All @@ -33,51 +33,65 @@ jobs:

strategy:
fail-fast: true
matrix:
pyver: ["3.7", "3.8", "3.9", "3.10"]

steps:
- uses: actions/checkout@v2

- name: Set up Python
- name: Set up Python on ${{ matrix.pyver }}
uses: actions/setup-python@v1
with:
python-version: 3.8
python-version: ${{ matrix.pyver }}

- name: Install requirements then build main and mini wheel and sdist
run: etc/release/scancode-create-pypi-dist.sh
- name: Install requirements then build main and mini wheel
run: etc/release/scancode-create-pypi-wheel.sh

- name: Collect built main wheel
- name: Collect built wheels for ${{ matrix.pyver }}
uses: actions/upload-artifact@v3
with:
name: main_wheel
path: dist/scancode_toolkit-3*.whl
name: wheels-${{ matrix.pyver }}
path: dist/*.whl

- name: Collect built main sdist
uses: actions/upload-artifact@v3
with:
name: main_sdist
path: dist/scancode-toolkit-3*.tar.gz

- name: Collect built mini wheel
uses: actions/upload-artifact@v3
build_scancode_sdist_for_pypi:
permissions:
contents: read # to fetch code (actions/checkout)

name: Build PyPI sdist archives
runs-on: ubuntu-20.04

defaults:
run:
shell: bash

strategy:
fail-fast: true

steps:
- uses: actions/checkout@v2

- name: Set up Python
uses: actions/setup-python@v1
with:
name: mini_wheel
path: dist/scancode_toolkit_mini-3*.whl
python-version: "3.10"

- name: Collect built mini sdist
- name: Install requirements then build main and mini sdist
run: etc/release/scancode-create-pypi-sdist.sh

- name: Collect built sdist
uses: actions/upload-artifact@v3
with:
name: mini_sdist
path: dist/scancode-toolkit-mini*.tar.gz
name: sdists
path: dist/*.tar.gz


build_scancode_for_release_linux:
permissions:
contents: read # to fetch code (actions/checkout)

name: Build Release for linux
name: Build app Release for linux
runs-on: ubuntu-20.04
needs:
- build_scancode_for_pypi

defaults:
run:
Expand All @@ -92,18 +106,10 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v1
with:
python-version: 3.8

- name: Download a the main wheel
uses: actions/download-artifact@v3
with:
name: main_wheel
path: dist
python-version: "3.8"

- name: Build linux app archive
run: |
./configure --rel
etc/release/scancode-create-release-app-linux.sh
run: etc/release/scancode-create-release-app-linux.sh

- name: Collect built linux app
uses: actions/upload-artifact@v3
Expand All @@ -116,10 +122,8 @@ jobs:
permissions:
contents: read # to fetch code (actions/checkout)

name: Build Release for mac
name: Build app Release for mac
runs-on: ubuntu-20.04
needs:
- build_scancode_for_pypi

defaults:
run:
Expand All @@ -134,18 +138,10 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v1
with:
python-version: 3.8

- name: Download a the main wheel
uses: actions/download-artifact@v3
with:
name: main_wheel
path: dist
python-version: "3.8"

- name: Build mac app archive
run: |
./configure --rel
etc/release/scancode-create-release-app-macos.sh
run: etc/release/scancode-create-release-app-macos.sh

- name: Collect built mac app
uses: actions/upload-artifact@v3
Expand All @@ -158,10 +154,8 @@ jobs:
permissions:
contents: read # to fetch code (actions/checkout)

name: Build Release for windows
name: Build app Release for windows
runs-on: ubuntu-20.04
needs:
- build_scancode_for_pypi

defaults:
run:
Expand All @@ -175,18 +169,10 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v1
with:
python-version: 3.8

- name: Download a the main wheel
uses: actions/download-artifact@v3
with:
name: main_wheel
path: dist
python-version: "3.8"

- name: Build windows app archive
run: |
./configure --rel
etc/release/scancode-create-release-app-windows.sh
run: etc/release/scancode-create-release-app-windows.sh

- name: Collect built windows app
uses: actions/upload-artifact@v3
Expand All @@ -199,10 +185,8 @@ jobs:
permissions:
contents: read # to fetch code (actions/checkout)

name: Build source
name: Build app source
runs-on: ubuntu-20.04
needs:
- build_scancode_for_pypi

defaults:
run:
Expand All @@ -216,18 +200,10 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v1
with:
python-version: 3.8

- name: Download a the main sdist
uses: actions/download-artifact@v3
with:
name: main_sdist
path: dist
python-version: "3.10"

- name: Build source archive with deps
run: |
./configure --rel
etc/release/scancode-create-release-app-sources.sh
run: etc/release/scancode-create-release-app-sources.sh

- name: Collect built source app tarball
uses: actions/upload-artifact@v3
Expand All @@ -236,13 +212,13 @@ jobs:
path: release/*


smoke_test_install_and_run_pypi_dists_posix:
smoke_test_install_and_run_pypi_wheels_on_posix:
permissions:
contents: read # to fetch code (actions/checkout)

name: Test POSIX PyPI wheels
name: Test PyPI wheels on linux and mac
needs:
- build_scancode_for_pypi
- build_scancode_wheel_for_pypi
runs-on: ${{ matrix.os }}

defaults:
Expand All @@ -252,10 +228,8 @@ jobs:
strategy:
fail-fast: true
matrix:
os: [ubuntu-22.04, macos-12]
pyver: ["3.9", "3.10"]
# os: [ubuntu-18.04, ubuntu-20.04, ubuntu-22.04, macos-10.15, macos-11, macos-12]
# pyver: ["3.7", "3.8", "3.9", "3.10"]
os: [ubuntu-20.04, ubuntu-22.04, macos-11, macos-12]
pyver: ["3.7", "3.8", "3.9", "3.10"]

steps:
- uses: actions/checkout@v2
Expand All @@ -265,10 +239,10 @@ jobs:
with:
python-version: ${{ matrix.pyver }}

- name: Download a single artifact mainw
- name: Download wheels artifact
uses: actions/download-artifact@v3
with:
name: main_wheel
name: wheels-${{ matrix.pyver }}
path: dist

- name: test install wheels
Expand All @@ -277,21 +251,20 @@ jobs:
rm -rf venv; \
python -m venv venv
venv/bin/python -m pip install --upgrade pip wheel setuptools
for f in `find dist -type f`; \
for f in `find dist -type f -name "*${{ matrix.pyver }}*"`; \
do \
venv/bin/python -m pip install --force-reinstall "$f[full]" ; \
venv/bin/scancode -clipeu --json-pp - some.file ; \
venv/bin/python -m pip uninstall --yes scancode-toolkit ; \
done


smoke_test_install_and_run_pypi_dists_windows:
smoke_test_install_and_run_pypi_wheels_on_windows:
permissions:
contents: read # to fetch code (actions/checkout)

name: Test Windows PyPI wheels
name: Test PyPI wheels on Windows
needs:
- build_scancode_for_pypi
- build_scancode_wheel_for_pypi
runs-on: ${{ matrix.os }}

defaults:
Expand All @@ -301,10 +274,8 @@ jobs:
strategy:
fail-fast: true
matrix:
os: [windows-2022]
pyver: ["3.9", "3.10"]
#os: [windows-2019, windows-2022]
#pyver: ["3.7", "3.8", "3.9", "3.10"]
os: [windows-2019, windows-2022]
pyver: ["3.7", "3.8", "3.9", "3.10"]

steps:
- uses: actions/checkout@v2
Expand All @@ -314,22 +285,21 @@ jobs:
with:
python-version: ${{ matrix.pyver }}

- name: Download a single artifact mainw
- name: Download wheels artifact
uses: actions/download-artifact@v3
with:
name: main_wheel
name: wheels-${{ matrix.pyver }}
path: dist

- name: test install wheel
run: |
echo "license: gpl-2.0" > some.file
python -m venv venv
venv/Scripts/python.exe -m pip install --upgrade pip
for f in `find dist -type f`; \
for f in `find dist -type f -name "*${{ matrix.pyver }}*"`; \
do \
venv/Scripts/python.exe -m pip install --force-reinstall "$f[full]" ; \
venv/Scripts/scancode -clipeu --json-pp - some.file ; \
venv/Scripts/python.exe -m pip uninstall --yes scancode-toolkit ; \
done


Expand Down Expand Up @@ -390,7 +360,7 @@ jobs:
strategy:
fail-fast: true
matrix:
os: [macos-10.15, macos-11, macos-12]
os: [macos-11, macos-12]
pyver: [3.8]

steps:
Expand Down Expand Up @@ -509,9 +479,10 @@ jobs:
publish_to_pypi:
name: Publish to PyPI
needs:
- build_scancode_for_pypi
- smoke_test_install_and_run_pypi_dists_windows
- smoke_test_install_and_run_pypi_dists_posix
- build_scancode_wheel_for_pypi
- build_scancode_sdist_for_pypi
- smoke_test_install_and_run_pypi_wheels_on_windows
- smoke_test_install_and_run_pypi_wheels_on_posix
- publish_to_gh_release
runs-on: ubuntu-20.04
defaults:
Expand All @@ -520,7 +491,7 @@ jobs:
strategy:
fail-fast: true
matrix:
dist_names: [main_wheel, main_sdist, mini_wheel, mini_sdist]
dist_names: ["wheels-3.7", "wheels-3.8", "wheels-3.9", "wheels-3.10", sdists]

steps:
- name: Set up Python
Expand Down
Loading