Skip to content

Commit

Permalink
Merge pull request github#12077 from github/codeql-cli-articles-migra…
Browse files Browse the repository at this point in the history
…tion-update

Update CodeQL CLI docs articles and links to point to new location on GitHub Docs site
  • Loading branch information
saritai authored Feb 3, 2023
2 parents cd660e1 + 75b0676 commit 8edd378
Show file tree
Hide file tree
Showing 44 changed files with 276 additions and 3,311 deletions.
8 changes: 3 additions & 5 deletions docs/codeql/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,13 @@ We welcome contributions to our CodeQL docs. Want to improve existing docs or ad

## Contributing to CodeQL docs on `codeql.github.com`

To make changes to the documentation on [codeql.github.com](https://codeql.github.com/docs/codeql-overview/), you can make changes to the documentation files using the GitHub UI, a codespace, or a local text editor, and then open a pull request for review. For more information about the format and structure of the CodeQL documentation on [codeql.github.com](https://codeql.github.com/docs/codeql-overview/), please see the [README](docs/codeql/README.rst).
To make changes to the documentation on [codeql.github.com](https://codeql.github.com/docs/codeql-overview/), you can make changes to the documentation files using the GitHub UI, a codespace, or a local text editor, and then open a pull request for review. For more information about the format and structure of the CodeQL documentation on [codeql.github.com](https://codeql.github.com/docs/codeql-overview/), please see the [README](README.rst).

## Contributing to CodeQL CLI docs on `docs.github.com`

We are in the process of moving all documentation about the CodeQL CLI from [github/codeql](docs/codeql) to the public [github/docs](https://github.com/github/docs) repository so that this documentation is published on the [GitHub Docs](https://docs.github.com/en/code-security/code-scanning) site. This includes all articles that are currently published under "[Using the CodeQL CLI](https://codeql.github.com/docs/codeql-cli/using-the-codeql-cli/)" and "[CodeQL CLI reference](https://codeql.github.com/docs/codeql-cli/codeql-cli-reference/)" categories on the CodeQL microsite. This will make it easier for code scanning users to find information about using CodeQL to query their codebases.
We have moved documentation about the CodeQL CLI from [github/codeql](docs/codeql) to the public [github/docs](https://github.com/github/docs) repository so that this documentation is published on the [GitHub Docs](https://docs.github.com/en/code-security/codeql-cli) site. This includes all articles that were under the "[Using the CodeQL CLI](https://codeql.github.com/docs/codeql-cli/using-the-codeql-cli/)" and "[CodeQL CLI reference](https://codeql.github.com/docs/codeql-cli/codeql-cli-reference/)" categories on the CodeQL microsite. This will make it easier for code scanning users to find information about using CodeQL to query their codebases.

**Note**: For a brief time, we will have source files for CodeQL CLI documentation in two locations. During this period we will not accept changes to the old files in the `codeql` repository, only to the new files in the `docs` repository.

To contribute to these docs, which are located in the [`code-scanning`](https://github.com/github/docs/tree/main/content/code-security/code-scanning) directory, please refer to the [CONTRIBUTING.md](CONTRIBUTING.md) file in the `docs` repository.
To contribute to these docs, which are located in the [`codeql-cli`](https://github.com/github/docs/tree/main/content/code-security/codeql-cli) directory, please refer to the [CONTRIBUTING.md](https://github.com/github/docs/blob/main/CONTRIBUTING.md) file in the `docs` repository.



408 changes: 20 additions & 388 deletions docs/codeql/codeql-cli/about-codeql-packs.rst

Large diffs are not rendered by default.

104 changes: 9 additions & 95 deletions docs/codeql/codeql-cli/about-codeql-workspaces.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,100 +3,14 @@
About CodeQL workspaces
=======================

.. include:: ../reusables/beta-note-package-management.rst
.. pull-quote::
This article was moved to "`About CodeQL workspaces <https://docs.github.com/en/code-security/codeql-cli/codeql-cli-reference/about-codeql-workspaces>`__" on the `GitHub Docs <https://docs.github.com/en/code-security/codeql-cli>`__ site as of January 2023.

.. include:: ../reusables/codeql-cli-articles-migration-note.rst

CodeQL workspaces are used to group multiple CodeQL packs together. A typical use case for a CodeQL workspace is to develop a set of CodeQL library and query packs that are mutually dependent. For more information on CodeQL packs, see ":doc:`About CodeQL packs <about-codeql-packs>`."
.. include:: ../reusables/codeql-cli-migration-toc-note.rst

The main benefit of a CodeQL workspace is that it makes it easier for you to develop and maintain multiple CodeQL packs. When you use a CodeQL workspace, all the CodeQL packs in the workspace are available as *source dependencies* for each other when you run a CodeQL command that resolves queries. This makes it easier to develop, maintain, and publish multiple, related CodeQL packs.

In most cases, you should store the CodeQL workspace and the CodeQL packs contained in it in one git repository. This makes it easier to share your CodeQL development environment.

The ``codeql-workspace.yml`` file
---------------------------------

A CodeQL workspace is defined by a ``codeql-workspace.yml`` yaml file. This file contains a ``provide`` block, and optionally ``ignore`` and ``registries`` blocks.

* The ``provide`` block contains a list of glob patterns that define the CodeQL packs that are available in the workspace.
* The ``ignore`` block contains a list of glob patterns that define CodeQL packs that are not available in the workspace.
* The ``registries`` block contains a list of GHES URLs and package patterns that control which container registry is used for publishing CodeQL packs. For more information, see :ref:`Working with CodeQL packs on GitHub Enterprise Server <working-with-codeql-packs-on-ghes>`.

Each entry in the ``provide`` or ``ignore`` section must map to the location of a ``qlpack.yml`` file. All glob patterns are defined relative to the directory that contains the workspace file. For a list of patterns accepted in this file, see "`@actions/glob <https://github.com/actions/toolkit/tree/main/packages/glob#patterns>`__ ."

For example, the following ``codeql-workspace.yml`` file defines a workspace that contains all the CodeQL packs recursively found in the ``codeql-packs`` directory, except for the packs in the ``experimental`` directory. The ``registries`` block specifies that ``codeql/*`` packs should be downloaded from https://ghcr.io/v2/, which is GitHub's default container registry. All other packs should be downloaded from and published to the regsitry at ``GHE_HOSTNAME``.

.. code-block:: yaml
provide:
- "*/codeql-packs/**/qlpack.yml"
ignore:
- "*/codeql-packs/**/experimental/**/qlpack.yml"
registries:
- packages: 'codeql/*'
url: https://ghcr.io/v2/
- packages: '*'
url: https://containers.GHE_HOSTNAME/v2/
To verify that your ``codeql-workspace.yml`` file includes the CodeQL packs that you expect, run the ``codeql pack ls`` command in the same directory as your workspace. The result of the command is a list of all CodeQL packs in the workspace.

.. _source-dependencies:

Source Dependencies
-------------------

Source dependencies are CodeQL packs that are resolved from the local file system outside of the CodeQL package cache. These dependencies can be in the same CodeQL workspace, or specified as a path option using the ``--additional-packs`` argument. When you compile and run queries locally, source dependencies override any dependencies found in the CodeQL package cache as well as version constraints defined in the ``qlpack.yml``. All references to CodeQL packs in the same workspace are resolved as source dependencies.

This is particularly useful in the following situations:

- One of the dependencies of the query pack you are running is not yet published. Resolving from source is the only way to reference that pack.
- You are making changes to multiple packs at the same time and want to test them together. Resolving from source ensures that you are using the version of the pack with your changes in it.

CodeQL workspaces and query resolution
--------------------------------------

All CodeQL packs in a workspace are available as source dependencies for each other when you run any CodeQL command that resolves queries or packs. For example, when you run ``codeql pack install`` in a pack directory in a workspace, any dependency that can be found in the workspace will be used instead of downloading that dependency to the package cache and adding it to the ``codeql-pack.lock.yml`` file. For more information, see ":ref:`Adding and Installing Dependencies <adding-and-installing-dependencies>`."

Similarly, when you publish a CodeQL query pack to the GitHub container registry using ``codeql pack publish`` the command will always use the dependencies from the workspace instead of using dependencies found in the local package cache.

This ensures that any local changes you make to a query library in a dependency are automatically reflected in any query packs you publish from that workspace.

Example
~~~~~~~

Consider the following ``codeql-workspace.yml`` file:

.. code-block:: yaml
provide:
- "**/qlpack.yml"
And the following CodeQL library pack ``qlpack.yml`` file in the workspace:

.. code-block:: yaml
name: my-company/my-library
library: true
version: 1.0.0
And the following CodeQL query pack ``qlpack.yml`` file in the workspace:

.. code-block:: yaml
name: my-company/my-queries
version: 1.0.0
dependencies:
my-company/my-library: "*"
codeql/cpp-all: ~0.2.0
Notice that the ``dependencies`` block for the CodeQL query pack, ``my-company/my-queries``, specifies ``"*"`` as the version of the library pack. Since the library pack is already defined as a source dependency in ``codeql-workspace.yml``, the library pack's content is always resolved from inside the workspace. Any version constraint you define will be ignored in this case. We recommend that you use ``"*"`` for source dependencies to make it clear that the version is inherited from the workspace.

When you execute ``codeql pack install`` from the query pack directory, an appropriate version of ``codeql/cpp-all`` is downloaded to the local package cache. Also, a ``codeql-pack.lock.yml`` file is created that contains the resolved version of ``codeql/cpp-all``. The lock file won't contain an entry for ``my-company/my-library`` since it is resolved from source dependencies. The ``codeql-pack.lock.yml`` file will look something like this:

.. code-block:: yaml
dependencies:
codeql/cpp-all:
version: 0.2.2
When you execute ``codeql pack publish`` from the query pack directory, the ``codeql/cpp-all`` dependency from the package cache and the ``my-company/my-library`` from the workspace are bundled with ``my-company/my-queries`` and published to the GitHub container registry.
* `The codeql-workspace.yml file <https://docs.github.com/en/code-security/codeql-cli/codeql-cli-reference/about-codeql-workspaces#the-codeql-workspaceyml-file>`__
* `Source dependencies <https://docs.github.com/en/code-security/codeql-cli/codeql-cli-reference/about-codeql-workspaces#source-dependencies>`__
* `CodeQL workspaces and query resolution <https://docs.github.com/en/code-security/codeql-cli/codeql-cli-reference/about-codeql-workspaces#codeql-workspaces-and-query-resolution>`__
* `Example <https://docs.github.com/en/code-security/codeql-cli/codeql-cli-reference/about-codeql-workspaces#example>`__
6 changes: 4 additions & 2 deletions docs/codeql/codeql-cli/about-ql-packs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,10 @@
About QL packs
==============

This page has been moved to ":doc:`About CodeQL packs <about-codeql-packs>`."

.. pull-quote::
This page was moved to "`About CodeQL packs <https://docs.github.com/en/code-security/codeql-cli/codeql-cli-reference/about-codeql-packs>`__" on the `GitHub Docs <https://docs.github.com/en/code-security/codeql-cli>`__ site as of January 2023.

.. include:: ../reusables/codeql-cli-articles-migration-note.rst

.. toctree::
:hidden:
Expand Down
28 changes: 6 additions & 22 deletions docs/codeql/codeql-cli/about-the-codeql-cli.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,27 +3,11 @@
About the CodeQL CLI
====================

Software developers and security researchers can secure their code
using the CodeQL CLI.
.. pull-quote::
This article was moved to "`About the CodeQL CLI <https://docs.github.com/en/code-security/codeql-cli/using-the-codeql-cli/about-the-codeql-cli>`__" on the `GitHub Docs <https://docs.github.com/en/code-security/codeql-cli>`__ site as of January 2023.

.. include:: ../reusables/codeql-cli-articles-migration-note.rst

The CodeQL CLI is a command-line tool used to run CodeQL processes locally on
open source software projects. You can use the CodeQL CLI to:
.. include:: ../reusables/codeql-cli-migration-toc-note.rst

- Run CodeQL analyses using queries provided by GitHub engineers and the open
source community
- Create CodeQL databases to use in the CodeQL for Visual Studio Code
- Develop and test custom CodeQL queries to use in your own analyses

For information about using the CodeQL CLI, see
":ref:`Getting started with the CodeQL CLI <getting-started-with-the-codeql-cli>`."

CodeQL CLI commands
-------------------

The CodeQL CLI includes commands to create and analyze CodeQL databases from the
command line. To run a command, use::

codeql [command] [subcommand]

To view the reference documentation for a command, add the ``--help`` flag, or visit the
"`CodeQL CLI manual <../manual>`__."
* `CodeQL CLI commands <https://docs.github.com/en/code-security/codeql-cli/using-the-codeql-cli/about-the-codeql-cli#codeql-cli-commands>`__
Loading

0 comments on commit 8edd378

Please sign in to comment.