Permission issues with globally installed codeql

[dbeer1]

We are using codeql-bundle to bundle up our custom query packs, but the way we build and install the bundle is posing a problem with the way this tool and codeql work.

We install the base codeql bundle in a global read-only location (e.g. /opt/codeql) then build the custom bundle on top of that. Once built, we install the custom bundle in a similar location (e.g. /opt/custom-codeql) and run it from there.

This poses a number of challenges which we have had to work around:

1. codeql-bundle tries to write to the directory that you pass in --bundle, though it seems like it should only write to --output. This causes permission issues when we build the bundle, which can be worked around by copying the base codeql bundle to a temp dir.

2. codeql-bundle packages but doesn't compile the query packs from the workspace. As they are installed in a read-only location, the cache can't be updated after install so the queries are recompiled each run. We work around this by running codeql query compile on each .qls file we find after bundling but before installation

3. codeql database analyze won't read cache directories if they are read-only. So if installed in /opt/custom-codeql with queries compiled into the cache, the queries will still be recompiled each run. It seems like codeql should be able to load the cache even if it can't write it. We work around this by copying the installed custom codeql bundle to a temp dir and making it writable before running each time.

From the execute-queries log in the database:

[2023-01-06 08:33:07] [DETAILS] resolve library-path> Not adding per-pack compilation cache, since /opt/custom-codeql/qlpacks/mypack/0.1.0 is not writable.

Is it possible to make this workflow require fewer custom steps?

[rvermeulen]

Hi @dbeer1,

In its current form the tool indeed writes to the --bundle location to build the custom bundle. I can adjust it to work on a temporary copy. The tool is already doing this if the base bundle passed to --bundle is an archive so the change is minor.

I have to further investigate 2 and 3.

[rvermeulen]

For two we can copy the packs to a location where we can write the cache so it will be included in the bundle, however it is easier to that with the whole workspace before bundling.

Point three is a limitation of the CodeQL CLI. I will report this to see if they want to address this.