Skip to content

Commit 8fcbaea

Browse files
erik-krogherik-krogh
committed
Merge branch 'main' into labelNaming
2 parents ae3b324 + c015ef6 commit 8fcbaea

File tree

732 files changed

+24834
-22975
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

732 files changed

+24834
-22975
lines changed

‎.bazelrc

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
build --copt="-std=c++17"
2+
3+
try-import %workspace%/local.bazelrc

‎.bazelversion

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
5.0.0

‎.github/actions/fetch-codeql/action.yml

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,22 @@ description: Fetches the latest version of CodeQL
33
runs:
44
using: composite
55
steps:
6+
- name: Select platform - Linux
7+
if: runner.os == 'Linux'
8+
shell: bash
9+
run: echo "GA_CODEQL_CLI_PLATFORM=linux64" >> $GITHUB_ENV
10+
11+
- name: Select platform - MacOS
12+
if: runner.os == 'MacOS'
13+
shell: bash
14+
run: echo "GA_CODEQL_CLI_PLATFORM=osx64" >> $GITHUB_ENV
15+
616
- name: Fetch CodeQL
717
shell: bash
818
run: |
919
LATEST=$(gh release list --repo https://github.com/github/codeql-cli-binaries | cut -f 1 | grep -v beta | sort --version-sort | tail -1)
10-
gh release download --repo https://github.com/github/codeql-cli-binaries --pattern codeql-linux64.zip "$LATEST"
11-
unzip -q -d "${RUNNER_TEMP}" codeql-linux64.zip
20+
gh release download --repo https://github.com/github/codeql-cli-binaries --pattern codeql-$GA_CODEQL_CLI_PLATFORM.zip "$LATEST"
21+
unzip -q -d "${RUNNER_TEMP}" codeql-$GA_CODEQL_CLI_PLATFORM.zip
1222
echo "${RUNNER_TEMP}/codeql" >> "${GITHUB_PATH}"
1323
env:
1424
GITHUB_TOKEN: ${{ github.token }}

‎.github/workflows/check-qldoc.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,9 @@ jobs:
3737
done
3838
git checkout HEAD^
3939
for pack_dir in ${changed_lib_packs}; do
40+
# When we add a new language, pack_dir would not exist in HEAD^.
41+
# In this case the right thing to do is to skip the check.
42+
[[ ! -d "${pack_dir}" ]] && continue
4043
lang="${pack_dir%/ql/lib}"
4144
gh codeql generate library-doc-coverage --output="${RUNNER_TEMP}/${lang}-baseline.txt" --dir="${pack_dir}"
4245
awk -F, '{gsub(/"/,""); if ($4==0 && $6=="public") print "\""$3"\"" }' "${RUNNER_TEMP}/${lang}-current.txt" | sort -u > "${RUNNER_TEMP}/current-undocumented.txt"

‎.github/workflows/post-pr-comment.yml

Lines changed: 48 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,17 @@
1-
name: Post pull-request comment
1+
# This workflow is the second part of the process described in
2+
# .github/workflows/qhelp-pr-preview.yml
3+
# See that file for more info.
4+
5+
name: Post PR comment
26
on:
37
workflow_run:
4-
workflows: ["Query help preview"]
8+
workflows: [Render QHelp changes]
59
types:
610
- completed
711

812
permissions:
913
pull-requests: write
14+
actions: read
1015

1116
jobs:
1217
post_comment:
@@ -17,15 +22,53 @@ jobs:
1722
env:
1823
GITHUB_TOKEN: ${{ github.token }}
1924
WORKFLOW_RUN_ID: ${{ github.event.workflow_run.id }}
20-
- run: |
21-
PR="$(grep -o '^[0-9]\+$' pr.txt)"
25+
26+
- name: Check that PR SHA matches workflow SHA
27+
run: |
28+
PR="$(grep -o '^[0-9]\+$' pr_number.txt)"
2229
PR_HEAD_SHA="$(gh api "/repos/${GITHUB_REPOSITORY}/pulls/${PR}" --jq .head.sha)"
2330
# Check that the pull-request head SHA matches the head SHA of the workflow run
2431
if [ "${WORKFLOW_RUN_HEAD_SHA}" != "${PR_HEAD_SHA}" ]; then
2532
echo "PR head SHA ${PR_HEAD_SHA} does not match workflow_run event SHA ${WORKFLOW_RUN_HEAD_SHA}. Stopping." 1>&2
2633
exit 1
2734
fi
28-
gh pr comment "${PR}" --repo "${GITHUB_REPOSITORY}" -F comment.txt
2935
env:
3036
GITHUB_TOKEN: ${{ github.token }}
3137
WORKFLOW_RUN_HEAD_SHA: ${{ github.event.workflow_run.head_commit.id }}
38+
39+
- name: Create or update comment
40+
run: |
41+
COMMENT_PREFIX="QHelp previews"
42+
COMMENT_AUTHOR="github-actions[bot]"
43+
PR_NUMBER="$(grep -o '^[0-9]\+$' pr_number.txt)"
44+
45+
# If there is no existing comment, comment_id.txt will contain just a
46+
# newline (due to jq & gh behaviour). This will cause grep to fail, so
47+
# we catch that.
48+
RAW_COMMENT_ID=$(grep -o '^[0-9]\+$' comment_id.txt || true)
49+
50+
if [ $RAW_COMMENT_ID ]
51+
then
52+
# Fetch existing comment, and validate:
53+
# - comment belongs to the PR with number $PR_NUMBER
54+
# - comment starts with the expected prefix ("QHelp previews")
55+
# - comment author is github-actions[bot]
56+
FILTER='select(.issue_url | endswith($repo+"/issues/"+$pr))
57+
| select(.body | startswith($prefix))
58+
| select(.user.login == $author)
59+
| .id'
60+
COMMENT_ID=$(gh api "repos/${GITHUB_REPOSITORY}/issues/comments/${RAW_COMMENT_ID}" | jq --arg repo "${GITHUB_REPOSITORY}" --arg pr "${PR_NUMBER}" --arg prefix "${COMMENT_PREFIX}" --arg author "${COMMENT_AUTHOR}" "${FILTER}")
61+
if [ $COMMENT_ID ]
62+
then
63+
# Update existing comment
64+
jq --rawfile body comment_body.txt '{"body":$body}' -n | gh api "repos/${GITHUB_REPOSITORY}/issues/comments/${COMMENT_ID}" -X PATCH --input -
65+
else
66+
echo "Comment ${RAW_COMMENT_ID} did not pass validations: not editing." >&2
67+
exit 1
68+
fi
69+
else
70+
# Create new comment
71+
jq --rawfile body comment_body.txt '{"body":$body}' -n | gh api "repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments" -X POST --input -
72+
fi
73+
env:
74+
GITHUB_TOKEN: ${{ github.token }}

‎.github/workflows/qhelp-pr-preview.yml

Lines changed: 45 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,25 @@
1-
name: Query help preview
1+
# This workflow checks for any changes in .qhelp files in pull requests.
2+
# For any changed files, it renders them to markdown in a file called `comment_body.txt`.
3+
# It then checks if there's an existing comment on the pull request generated by
4+
# this workflow, and writes the comment ID to `comment_id.txt`.
5+
# It also writes the PR number to `pr_number.txt`.
6+
# These three files are uploaded as an artifact.
7+
8+
# When this workflow completes, the workflow "Post PR comment" runs.
9+
# It downloads the artifact and adds a comment to the PR with the rendered
10+
# QHelp.
11+
12+
# The task is split like this because creating PR comments requires extra
13+
# permissions that we don't want to expose to PRs from external forks.
14+
15+
# For more info see:
16+
# https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_run
17+
# https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
18+
name: Render QHelp changes
219

320
permissions:
421
contents: read
22+
pull-requests: read
523

624
on:
725
pull_request:
@@ -15,12 +33,16 @@ jobs:
1533
qhelp:
1634
runs-on: ubuntu-latest
1735
steps:
18-
- run: echo "${{ github.event.number }}" > pr.txt
36+
- run: echo "${PR_NUMBER}" > pr_number.txt
37+
env:
38+
PR_NUMBER: ${{ github.event.number }}
1939
- uses: actions/upload-artifact@v2
2040
with:
2141
name: comment
22-
path: pr.txt
42+
path: pr_number.txt
43+
if-no-files-found: error
2344
retention-days: 1
45+
2446
- uses: actions/checkout@v2
2547
with:
2648
fetch-depth: 2
@@ -36,7 +58,7 @@ jobs:
3658
- name: QHelp preview
3759
run: |
3860
EXIT_CODE=0
39-
echo "QHelp previews:" > comment.txt
61+
echo "QHelp previews:" > comment_body.txt
4062
while read -r -d $'\0' path; do
4163
if [ ! -f "${path}" ]; then
4264
exit 1
@@ -52,12 +74,29 @@ jobs:
5274
echo '```'
5375
fi
5476
echo "</details>"
55-
done < "${RUNNER_TEMP}/paths.txt" >> comment.txt
77+
done < "${RUNNER_TEMP}/paths.txt" >> comment_body.txt
5678
exit "${EXIT_CODE}"
5779
5880
- if: always()
5981
uses: actions/upload-artifact@v2
6082
with:
6183
name: comment
62-
path: comment.txt
84+
path: comment_body.txt
85+
if-no-files-found: error
86+
retention-days: 1
87+
88+
- name: Save ID of existing QHelp comment (if it exists)
89+
run: |
90+
# Find the latest comment starting with "QHelp previews"
91+
COMMENT_PREFIX="QHelp previews"
92+
gh api "repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments" --paginate | jq --arg prefix "${COMMENT_PREFIX}" '[.[] | select(.body|startswith($prefix)) | .id] | max' > comment_id.txt
93+
env:
94+
GITHUB_TOKEN: ${{ github.token }}
95+
PR_NUMBER: ${{ github.event.number }}
96+
97+
- uses: actions/upload-artifact@v2
98+
with:
99+
name: comment
100+
path: comment_id.txt
101+
if-no-files-found: error
63102
retention-days: 1

‎.github/workflows/swift-qltest.yml

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
name: "Swift: Run QL Tests"
2+
3+
on:
4+
pull_request:
5+
paths:
6+
- "swift/**"
7+
- .github/workflows/swift-qltest.yml
8+
branches:
9+
- main
10+
defaults:
11+
run:
12+
working-directory: swift
13+
14+
jobs:
15+
qlformat:
16+
runs-on: ubuntu-latest
17+
steps:
18+
- uses: actions/checkout@v2
19+
- uses: ./.github/actions/fetch-codeql
20+
- name: Check QL formatting
21+
run: find ql "(" -name "*.ql" -or -name "*.qll" ")" -print0 | xargs -0 codeql query format --check-only
22+
qltest:
23+
runs-on: ${{ matrix.os }}
24+
strategy:
25+
fail-fast: false
26+
matrix:
27+
os : [ubuntu-20.04, macos-latest]
28+
steps:
29+
- uses: actions/checkout@v2
30+
- uses: ./.github/actions/fetch-codeql
31+
- name: Install bazelisk - Linux
32+
if: runner.os == 'Linux'
33+
run: |
34+
sudo apt-get update
35+
sudo apt-get install -y wget
36+
wget https://github.com/bazelbuild/bazelisk/releases/download/v1.11.0/bazelisk-linux-amd64
37+
mv bazelisk-linux-amd64 /usr/local/bin/bazel
38+
chmod +x /usr/local/bin/bazel
39+
- name: Install bazelisk - macOS
40+
if: runner.os == 'MacOS'
41+
run: |
42+
brew install bazelisk
43+
- name: Build Swift extractor
44+
run: |
45+
bazel run //swift:create-extractor-pack
46+
- name: Run QL tests
47+
run: |
48+
codeql test run --threads=0 --ram 5000 --search-path "${{ github.workspace }}/swift/extractor-pack" --check-databases --check-unused-labels --check-repeated-labels --check-redefined-labels --check-use-before-definition ql/test
49+
env:
50+
GITHUB_TOKEN: ${{ github.token }}
51+

‎.gitignore

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
# Byte-compiled python files
1818
*.pyc
1919

20-
# python virtual environment folder
20+
# python virtual environment folder
2121
.venv/
2222

2323
# It's useful (though not required) to be able to unpack codeql in the ql checkout itself
@@ -29,4 +29,10 @@ csharp/extractor/Semmle.Extraction.CSharp.Driver/Properties/launchSettings.json
2929
.codeql
3030

3131
# Compiled class file
32-
*.class
32+
*.class
33+
34+
# links created by bazel
35+
/bazel-*
36+
37+
# CLion project files
38+
/.clwb

‎.pre-commit-config.yaml

Lines changed: 20 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2,28 +2,41 @@
22
# See https://pre-commit.com/hooks.html for more hooks
33
exclude: /test/.*$(?<!\.ql)(?<!\.qll)(?<!\.qlref)
44
repos:
5-
- repo: https://github.com/pre-commit/pre-commit-hooks
5+
- repo: https://github.com/pre-commit/pre-commit-hooks
66
rev: v3.2.0
77
hooks:
8-
- id: trailing-whitespace
9-
- id: end-of-file-fixer
8+
- id: trailing-whitespace
9+
- id: end-of-file-fixer
1010

11-
- repo: local
11+
- repo: https://github.com/pre-commit/mirrors-clang-format
12+
rev: v13.0.1
1213
hooks:
13-
- id: codeql-format
14+
- id: clang-format
15+
files: ^swift/.*\.(h|c|cpp)$
16+
17+
- repo: local
18+
hooks:
19+
- id: codeql-format
1420
name: Fix QL file formatting
1521
files: \.qll?$
1622
language: system
1723
entry: codeql query format --in-place
1824

19-
- id: sync-files
25+
- id: sync-files
2026
name: Fix files required to be identical
2127
language: system
2228
entry: python3 config/sync-files.py --latest
2329
pass_filenames: false
2430

25-
- id: qhelp
31+
- id: qhelp
2632
name: Check query help generation
2733
files: \.qhelp$
2834
language: system
2935
entry: python3 misc/scripts/check-qhelp.py
36+
37+
- id: swift-codegen
38+
name: Run Swift checked in code generation
39+
files: ^swift/(codegen/|.*/generated/|ql/lib/swift\.dbscheme$)
40+
language: system
41+
entry: bazel run //swift/codegen
42+
pass_filenames: false

‎BUILD.bazel

Whitespace-only changes.

‎CODEOWNERS

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -5,14 +5,6 @@
55
/python/ @github/codeql-python
66
/ruby/ @github/codeql-ruby
77

8-
# Make @xcorail (GitHub Security Lab) a code owner for experimental queries so he gets pinged when we promote a query out of experimental
9-
/cpp/**/experimental/**/* @github/codeql-c-analysis @xcorail
10-
/csharp/**/experimental/**/* @github/codeql-csharp @xcorail
11-
/java/**/experimental/**/* @github/codeql-java @xcorail
12-
/javascript/**/experimental/**/* @github/codeql-javascript @xcorail
13-
/python/**/experimental/**/* @github/codeql-python @xcorail
14-
/ruby/**/experimental/**/* @github/codeql-ruby @xcorail
15-
168
# ML-powered queries
179
/javascript/ql/experimental/adaptivethreatmodeling/ @github/codeql-ml-powered-queries-reviewers
1810

@@ -31,3 +23,11 @@
3123

3224
# QL for QL reviewers
3325
/ql/ @github/codeql-ql-for-ql-reviewers
26+
27+
# Bazel
28+
**/*.bazel @github/codeql-ci-reviewers
29+
**/*.bzl @github/codeql-ci-reviewers
30+
31+
# Documentation etc
32+
/*.md @github/code-scanning-product
33+
/LICENSE @github/code-scanning-product

‎CONTRIBUTING.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,3 +70,7 @@ After the experimental query is merged, we welcome pull requests to improve it.
7070
If you contribute to this project, we will record your name and email address (as provided by you with your contributions) as part of the code repositories, which are public. We might also use this information to contact you in relation to your contributions, as well as in the normal course of software development. We also store records of CLA agreements signed in the past, but no longer require contributors to sign a CLA. Under GDPR legislation, we do this on the basis of our legitimate interest in creating the CodeQL product.
7171

7272
Please do get in touch (privacy@github.com) if you have any questions about this or our data protection policies.
73+
74+
## Bazel
75+
Please notice that any bazel targets and definitions in this repository are currently experimental
76+
and for internal use only.

‎README.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,9 @@ We welcome contributions to our standard library and standard checks. Do you hav
1313

1414
## License
1515

16-
The code in this repository is licensed under the [MIT License](LICENSE) by [GitHub](https://github.com). The use of CodeQL on open source code is licensed under specific [Terms & Conditions](https://securitylab.github.com/tools/codeql/license/) UNLESS you have a commercial license in place. If you'd like to use CodeQL with a commercial codebase, please [contact us](https://github.com/enterprise/contact) for further help.
16+
The code in this repository is licensed under the [MIT License](LICENSE) by [GitHub](https://github.com).
17+
18+
The CodeQL CLI (including the CodeQL engine) is hosted in a [different repository](https://github.com/github/codeql-cli-binaries) and is [licensed separately](https://github.com/github/codeql-cli-binaries/blob/main/LICENSE.md). If you'd like to use the CodeQL CLI to analyze closed-source code, you will need a separate commercial license; please [contact us](https://github.com/enterprise/contact) for further help.
1719

1820
## Visual Studio Code integration
1921

‎WORKSPACE.bazel

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# Please notice that any bazel targets and definitions in this repository are currently experimental
2+
# and for internal use only.
3+
4+
workspace(name = "codeql")
5+
6+
load("//misc/bazel:workspace.bzl", "codeql_workspace")
7+
8+
codeql_workspace()
9+
10+
load("//misc/bazel:workspace_deps.bzl", "codeql_workspace_deps")
11+
12+
codeql_workspace_deps()

0 commit comments

Comments
 (0)