Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: foss scan #168

Merged
merged 2 commits into from
Sep 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@ jobs:
name: lint and test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
fetch-depth: 0

- uses: actions/setup-go@v5
- uses: actions/setup-go@0a12ed9d6a96ab950c8f026ed9f722fe0da7ef32 # v5.0.2
with:
go-version: '1.22.5'

Expand All @@ -35,7 +35,7 @@ jobs:
- name: make sbom
run: make sbom

- uses: actions/upload-artifact@v3
- uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
with:
name: sbom
path: tmp/garm-operator.bom.spdx
9 changes: 5 additions & 4 deletions .github/workflows/foss.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
# Jobs to run Blackduck based FOSS scan
name: FOSS Scan
on:
workflow_dispatch: {}
push:
branches:
- main
Expand All @@ -20,7 +21,7 @@ jobs:
id: go

- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7

- name: Synopsys Detect
run: |
Expand All @@ -38,13 +39,13 @@ jobs:
BLACKDUCK_TOKEN: ${{ secrets.BLACKDUCK_TOKEN }}

- name: Archive foss scan notices report
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
with:
name: 3RD_PARTY_LICENSES.txt
path: tmp/Black_Duck_Notices_Report.txt

- name: Archive foss scan risk report
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
with:
name: foss-scan-risk-report
path: tmp/BlackDuck_RiskReport.pdf
path: tmp/BlackDuck_RiskReport.pdf
2 changes: 1 addition & 1 deletion .github/workflows/pull-request-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,6 @@ jobs:
name: Validate PR title
runs-on: ubuntu-latest
steps:
- uses: amannn/action-semantic-pull-request@v5
- uses: amannn/action-semantic-pull-request@0723387faaf9b38adef4775cd42cfd5155ed6017 # v5.5.3
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
4 changes: 2 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,11 @@ jobs:
username: ${{ secrets.REGISTRY_USERNAME }}
password: ${{ secrets.REGISTRY_PASSWORD }}

- uses: actions/checkout@v4
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
fetch-depth: 0
- run: git fetch --force --tags
- uses: actions/setup-go@v5
- uses: actions/setup-go@0a12ed9d6a96ab950c8f026ed9f722fe0da7ef32 # v5.0.2
with:
go-version: '1.22.5'

Expand Down
12 changes: 9 additions & 3 deletions hack/foss-scan.sh
Original file line number Diff line number Diff line change
Expand Up @@ -89,9 +89,16 @@ bash <(curl -s -L https://detect.synopsys.com/detect8.sh) \
--detect.cleanup=false \
--detect.risk.report.pdf=true \
--detect.risk.report.pdf.path=tmp/ \
--detect.notices.report=true \
--detect.notices.report.path=tmp/ \
--detect.wait.for.results=true \
--insecure

chmod +x ./hack/generate-notices-file.sh

if ! ./hack/generate-notices-file.sh; then
echo "Error: generate-notices-file.sh execution failed!"
exit 1
fi

RC=$?

# Delete the scan if it completed successfully.
Expand All @@ -101,7 +108,6 @@ fi
set -e

mv tmp/*BlackDuck_RiskReport.pdf tmp/BlackDuck_RiskReport.pdf
mv tmp/*Black_Duck_Notices_Report.txt tmp/Black_Duck_Notices_Report.txt

exit $RC

Expand Down
147 changes: 147 additions & 0 deletions hack/generate-notices-file.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
#! /bin/bash
# SPDX-License-Identifier: MIT

if [ -z "${BLACKDUCK_TOKEN}" ]; then
echo "BLACKDUCK_TOKEN must be set" && exit 1
fi

if [ -z "${BLACKDUCK_URL}" ]; then
echo "BLACKDUCK_URL must be set" && exit 1
fi

if [ -z "${BLACKDUCK_PROJECT_NAME}" ]; then
echo "BLACKDUCK_PROJECT_NAME must be set" && exit 1
fi

if [ -z "${BLACKDUCK_SCAN_VERSION_NAME}" ]; then
echo "BLACKDUCK_SCAN_VERSION_NAME must be set" && exit 1
fi

if [ -z "${MAX_RETRY_COUNT}" ]; then
MAX_RETRY_COUNT=12
fi


echo "Get Bearer Token ..."
bearer_token=$(curl -s -S -X POST "${BLACKDUCK_URL}/api/tokens/authenticate" \
-H "Authorization: token ${BLACKDUCK_TOKEN}" \
-H "Accept: application/vnd.blackducksoftware.user-4+json" \
| jq -rc '.bearerToken')
echo "Lookup Project ..."
encoded_project_name=$(jq -rn --arg name "${BLACKDUCK_PROJECT_NAME}" '$name|@uri')
project_response=$(curl -s -S -X GET "${BLACKDUCK_URL}/api/projects?q=name:${encoded_project_name}" \
-H "Authorization: Bearer ${bearer_token}" \
-H "Accept: application/json" \
-H "Content-Type: application/vnd.blackducksoftware.report-4+json")

project_count=$(echo "${project_response}" | jq '.totalCount')
project_url=""
if [ "${project_count}" -gt 0 ]; then
project_url=$(echo "${project_response}" | jq -r --arg PROJECT_NAME "${BLACKDUCK_PROJECT_NAME}" '.items[] | select(.name==$PROJECT_NAME)._meta.href' | head -n 1)
if [ -z "${project_url}" ]; then
echo "No matching project with name ${BLACKDUCK_PROJECT_NAME} found."
exit 1
fi
else
echo "Project lookup returns 0 items."
exit 1
fi

echo "Lookup Version"
encoded_version_name=$(jq -rn --arg name "${BLACKDUCK_SCAN_VERSION_NAME}" '$name|@uri')
version_response=$(curl -s -S -X GET "${project_url}/versions?q=name:${encoded_version_name}" \
-H "Authorization: Bearer ${bearer_token}" \
-H "Accept: application/json" \
-H "Content-Type: application/vnd.blackducksoftware.report-4+json")

version_count=$(echo "${version_response}" | jq '.totalCount')
version_links=""
if [ "${version_count}" -gt 0 ]; then
version_links=$(echo "${version_response}" | jq -r --arg VERSION_NAME "${BLACKDUCK_SCAN_VERSION_NAME}" '.items[] | select(.versionName==$VERSION_NAME) | ._meta')
if [ -z "${version_links}" ]; then
echo "No matching project version with name ${BLACKDUCK_SCAN_VERSION_NAME} found."
exit 1
fi
else
echo "Version lookup returns 0 items."
exit 1
fi

echo "Get License Report URL ..."
license_report_url=$(echo ${version_links} | jq -r '.links[] | select(.rel=="licenseReports") | .href')

if [ -z "${license_report_url}" ]; then
echo "License report URL could not be determined!"
exit 1
fi
echo "License Report URL: ${license_report_url}"

echo "Trigger Report Creation ..."
report_create_response=$(curl -s -S -i -X POST ${license_report_url} \
-H "Accept: */*" \
-H "Authorization: Bearer ${bearer_token}" \
-H "Content-Type: application/json" \
-d '{"reportFormat":"TEXT","categories":["LICENSE_DATA","LICENSE_TEXT","COPYRIGHT_TEXT"]}')

# Check if the response is okay (200 or 201)
http_status=$(echo "${report_create_response}" | grep HTTP/ | tail -1 | awk '{print $2}')
echo "HTTP Status: ${http_status}"

if [ "${http_status}" -ne 200 ] && [ "${http_status}" -ne 201 ]; then
echo "Failed to create Report, HTTP status: ${http_status}"
exit 1
fi

# get report location
report_location=""
report_location=$(echo "${report_create_response}" | grep location | tail -1 | awk '{print $2}' | tr -d '\r')
if [ -z "${report_location}" ]; then
echo "Unable to resolve Report location url from create report response"
fi
echo "Got Report Location URL: ${report_location}"

# check report status to be completed
retry_count=0
report_status=""

while [ "${report_status}" != "COMPLETED" ] && [ "${report_status}" != "FAILED" ] && [ ${retry_count} -lt ${MAX_RETRY_COUNT} ]; do
sleep 10
report_status=$(curl -s -S -X GET ${report_location} \
-H "Accept: */*" -H "Authorization: Bearer ${bearer_token}" \
| jq -r '.status')
echo Retry ${retry_count}: Current report status is: ${report_status}
let retry_count++ || true
done

if [ "${report_status}" == "FAILED" ]; then
echo "Report creation failed after ${retry_count} retries!"
exit 1
fi

if [ "${report_status}" != "COMPLETED" ]; then
echo "Report creation is not finished after ${MAX_RETRY_COUNT} retries!"
echo "Deleting stuck report..."
curl -s -S -X DELETE ${report_location} \
-H "Accept: */*" \
-H "Authorization: Bearer ${bearer_token}"
echo "Stuck report deleted."
exit 1
fi

echo "Get URL for Report Download ..."
report_download_url=$(curl -s -S -X GET ${report_location} \
-H "Accept: */*" \
-H "Authorization: Bearer ${bearer_token}" \
| jq -r '._meta.links[] | select(.rel=="download") | .href')

echo "Got Report Download URL: ${report_download_url}"

# download licenses.zip
curl -s -S -X GET ${report_download_url} \
-H "Accept: */*" \
-H "Authorization: Bearer ${bearer_token}" \
-o ${BLACKDUCK_PROJECT_NAME}-licenses.zip

unzip -j ${BLACKDUCK_PROJECT_NAME}-licenses.zip

mv version-license_*.txt tmp/Black_Duck_Notices_Report.txt