Skip to content

Commit

Permalink
Update docs to add build-mode: none analysis for C# (#51258)
Browse files Browse the repository at this point in the history
Co-authored-by: Ben Ahmady <32935794+subatoi@users.noreply.github.com>
  • Loading branch information
felicitymay and subatoi authored Jun 20, 2024
1 parent 9415cf1 commit e252be4
Show file tree
Hide file tree
Showing 4 changed files with 78 additions and 28 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -128,25 +128,18 @@ For information about the languages, libraries, and frameworks that are supporte
{% ifversion codeql-no-build %}
## About creating a {% data variables.product.prodname_codeql %} Java database without a build
## About build mode None for {% data variables.product.prodname_codeql %}
{% data variables.product.prodname_codeql %} creates a Java database without requiring a build when one or both of the following conditions is true:
For {% data variables.code-scanning.no_build_support %}, {% data variables.product.prodname_codeql %} creates a database without requiring a build when you enable default setup for {% data variables.product.prodname_code_scanning %} unless the repository also includes Kotlin code. If a repository contains Kotlin code in addition to Java code, default setup is enabled with the autobuild process because Kotlin analysis requires a build.
-`build-mode` is set to `none`.
* Default setup is configured and Java code (but not Kotlin code) was present when {% data variables.product.prodname_codeql %} was enabled.
Creating a {% data variables.product.prodname_codeql %} database without a build may produce less accurate results than using `autobuild` or manual build steps if:

{% data variables.product.prodname_codeql %} will attempt to run Gradle or Maven first in order to extract accurate dependency information (but not to invoke a build), before creating a database from all Java files present.

You can read about the system requirements to run Maven or Gradle, and the rules for selecting between them, in {% ifversion codeql-kotlin-beta %}"[Building Java and Kotlin](#building-java--and-kotlin)"{% else %}"[Building Java](#building-java)"{% endif %}. If you're creating a Java database without a build, these are the same, but note that multiple root Maven or Gradle scripts are allowed. Every root Maven or Gradle project file (a build script without any build script present in an ancestor directory) is queried for dependency information, and more recent dependency versions are preferred if there is a clash.

Creating a {% data variables.product.prodname_codeql %} Java database without a build may produce less accurate results than using `autobuild` or manual build steps if:

* Gradle or Maven build scripts cannot be queried for dependency information, and dependency guesses (based on Java package names) are inaccurate.
* The analyzed code includes code that was generated during the build process.
* The build scripts cannot be queried for dependency information, and dependency guesses are inaccurate.
* The repository normally generates code during the build process.

To use `autobuild` or manual build steps, you can use advanced setup.

If `build-mode` is set to `none` and Kotlin code is found in the repository, it will not be analyzed and a warning will be produced. See {% ifversion codeql-kotlin-beta %}"[Building Java and Kotlin](#building-java--and-kotlin)"{% else %}"[Building Java](#building-java)"{% endif %}.
>[!NOTE] For Java analysis, if `build-mode` is set to `none` and Kotlin code is found in the repository, the Kotlin code will not be analyzed and a warning will be produced. See {% ifversion codeql-kotlin-beta %}"[Building Java and Kotlin](#building-java--and-kotlin)"{% else %}"[Building Java](#building-java)"{% endif %}.

{% endif %}

Expand Down Expand Up @@ -297,7 +290,7 @@ If you added manual build steps for compiled languages and {% data variables.pro

{% ifversion codeql-no-build %}{% data variables.product.prodname_codeql %} supports build modes `autobuild` or `manual` for C/C++ code.

### Autobuild summary{% endif %}
### Autobuild summary for C/C++{% endif %}

| Supported system type | System name |
|----|----|
Expand All @@ -322,7 +315,7 @@ On Linux and macOS, the `autobuild` step reviews the files present in the reposi
1. If none are found, search subdirectories for a unique directory with a build system for C/C++.
1. Run an appropriate command to configure the system.

#### Runner requirements
#### Runner requirements for C/C++

{% ifversion codeql-cpp-autoinstall-dependencies %}
On Ubuntu Linux runners, `autobuild` may try to automatically install dependencies required by the detected configuration and build steps. By default, this behavior is enabled on {% data variables.product.prodname_dotcom %}-hosted runners and disabled on self-hosted runners. You can enable or disable this feature explicitly by setting `CODEQL_EXTRACTOR_CPP_AUTOINSTALL_DEPENDENCIES` to `true` or `false` in the environment. For more information about defining environment variables, see "[AUTOTITLE](/actions/learn-github-actions/variables#defining-environment-variables-for-a-single-workflow)."
Expand All @@ -337,9 +330,37 @@ Windows runners require `powershell.exe` to be on the `PATH`.

## Building C#

{% ifversion codeql-no-build %}{% data variables.product.prodname_codeql %} supports build modes `autobuild` or `manual` for C# code.
{% ifversion codeql-no-build %}{% data variables.product.prodname_codeql %} supports build modes {% ifversion codeql-no-build-csharp %}`none`, {% endif %}`autobuild` or `manual` for C# code.{% endif %}

{% ifversion codeql-no-build-csharp %}

When you enable default setup for a repository that contains C# code, the build mode is set to `none` automatically.

### No build for C#

{% data variables.product.prodname_codeql %} restores dependencies and generates a few additional source files, to give more accurate results, before creating a database from all the source files and dependencies.

Dependencies are restored using multiple heuristics and strategies. The following files are the primary source of information: `*.csproj`, `*.sln`, `nuget.config`, `packages.config`, `global.json`, and `project.assets.json`.

The following generated source files are optional, but significantly increase the correctness of the {% data variables.product.prodname_codeql %} database:

* `global` generated `using` directives to handle the implicit `using` feature of MSbuild.
* ASP.NET core view files, `.cshtml` files are converted to `.cs` files.

The information from the dependency assembly names, generated source files, and the source files in the repository is compiled and used to create a {% data variables.product.prodname_codeql %} database.

#### Accuracy of no build analysis for C#

Creating a {% data variables.product.prodname_codeql %} database without building the full code relies on being able to restore dependencies and being able to compile together the source files in the repository. When there are problems restoring dependencies or compiling the source code, this can affect the accuracy of the {% data variables.product.prodname_codeql %} database and {% data variables.product.prodname_code_scanning %} analysis results.

You can ensure a more accurate analysis by taking the following steps:

### Autobuild summary{% endif %}
* Provide access to the public internet or ensure that access to a private Nuget feed is available.
* Check whether the repository requires multiple versions of the same Nuget dependency. {% data variables.product.prodname_codeql %} can use only one version and usually chooses the newer version where there are multiple versions. This approach may not work for all repositories.
* Check whether multiple versions of .NET are referenced, for example, `net48`, `net5.0`, and `netstandard1.6`. {% data variables.product.prodname_codeql %} can use only one version and this may affect accuracy.
* Avoid colliding class names, otherwise this may cause missing method call targets, which has an impact on dataflow analysis.

### Autobuild summary for C#{% endif %}

| Supported system type | System name |
|----|----|
Expand All @@ -355,41 +376,53 @@ The `autobuild` process attempts to autodetect a suitable build method for C# us
If `autobuild` detects multiple solution or project files at the same (shortest) depth from the top level directory, it will attempt to build all of them.
1. Invoke a script that looks like a build script—`build.bat`, `build.cmd`, and `build.exe` (in that order).

#### Runner requirements
#### Runner requirements for C# on Windows

For .NET Core application development on self-hosted runners, the .NET SDK is required (for `dotnet`).

For .NET Framework application development, you will need Microsoft Build Tools (for `msbuild`) and Nuget CLI (for `nuget`).

Windows runners require `powershell.exe` to be on the `PATH`.

{% ifversion codeql-no-build-csharp %}

If you plan to create {% data variables.product.prodname_codeql %} databases using `build-mode: none`, you also need to provide access to the public internet, or you must ensure that access to a private Nuget feed is available.

{% endif %}

### Linux and macOS autodetection

1. Invoke `dotnet build` on the solution (`.sln`) or project (`.csproj`) file closest to the root.
1. Invoke `MSbuild` on the solution or project file closest to the root.
If `autobuild` detects multiple solution or project files at the same (shortest) depth from the top level directory, it will attempt to build all of them.
1. Invoke a script that looks like a build script—`build` and `build.sh` (in that order).

#### Runner requirements
#### Runner requirements for C# on Linux and macOS

For .NET Core application development on self-hosted runners, the .NET SDK is required (for `dotnet`).

For .NET Framework application development, you will require Mono Runtime (to run `mono`, `msbuild`, or `nuget`).

{% ifversion codeql-no-build-csharp %}

If you plan to create {% data variables.product.prodname_codeql %} databases using `build-mode: none`, you also need to provide access to the public internet, or you must ensure that access to a private Nuget feed is available.

{% endif %}

{% ifversion codeql-go-autobuild %}

## Building Go

{% ifversion codeql-no-build %}{% data variables.product.prodname_codeql %} supports build modes `autobuild` or `manual` for Go code.

### Autobuild summary{% endif %}
### Autobuild summary for Go{% endif %}

| Supported system type | System name |
|----|----|
| Operating system | Windows, macOS, and Linux |
| Build system | Go modules, `dep` and Glide, as well as build scripts including Makefiles and Ninja scripts |

### Autodetection
### Autodetection for Go

The `autobuild` process attempts to autodetect a suitable way to install the dependencies needed by a Go repository before extracting all `.go` files:

Expand Down Expand Up @@ -417,22 +450,33 @@ When you first enable default setup for a repository, if only Java code is detec

If you later add Kotlin code to a repository that uses the `none` build mode, {% data variables.product.prodname_codeql %} analysis reports a warning message explaining that Kotlin is not supported. You will need to disable default setup and re-enable it. When you re-enable default setup, the build mode will change to `autobuild` so that both languages can be analyzed. Alternatively, you can change to an advanced setup. For more information, see "[AUTOTITLE](/code-security/code-scanning/troubleshooting-code-scanning/kotlin-detected-in-no-build)."

### Autobuild summary{% endif %}
### No build for Java

{% data variables.product.prodname_codeql %} will attempt to run Gradle or Maven to extract accurate dependency information (but not to invoke a build), before creating a database from all Java files present. Every root Maven or Gradle project file (a build script without any build script present in an ancestor directory) is queried for dependency information, and more recent dependency versions are preferred if there is a clash. For information about the runner requirements to run Maven or Gradle, see "[Runner requirements for Java](#runner-requirements-for-java)."

#### Accuracy of no build analysis for Java

Creating a {% data variables.product.prodname_codeql %} Java database without a build may produce less accurate results than using `autobuild` or manual build steps if:

* Gradle or Maven build scripts cannot be queried for dependency information, and dependency guesses (based on Java package names) are inaccurate.
* The repository normally generates code during the build process. This would be analyzed if you created the {% data variables.product.prodname_codeql %} database using a different mode.

### Autobuild summary for Java{% endif %}

| Supported system type | System name |
|----|----|
| Operating system | Windows, macOS, and Linux (no restriction) |
| Build system | Gradle, Maven and Ant |

### Autodetection
### Autodetection for Java

The `autobuild` process tries to determine the build system for Java codebases by applying this strategy:

1. Search for a build file in the root directory. Check for Gradle then Maven then Ant build files.
1. Run the first build file found. If both Gradle and Maven files are present, the Gradle file is used.
1. Otherwise, search for build files in direct subdirectories of the root directory. If only one subdirectory contains build files, run the first file identified in that subdirectory (using the same preference as for 1). If more than one subdirectory contains build files, report an error.

### Runner requirements
### Runner requirements for Java

If you're using self-hosted runners, the required version(s) of Java should be present:

Expand All @@ -456,7 +500,7 @@ Windows runners require `powershell.exe` to be on the `PATH`.

{% ifversion codeql-no-build %}{% data variables.product.prodname_codeql %} supports build modes `autobuild` or `manual` for Swift code.

### Autobuild summary{% endif %}
### Autobuild summary for Swift{% endif %}

| Supported system type | System name |
|----|----|
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -208,10 +208,10 @@ In addition, for {% data variables.code-scanning.no_build_support %}, there is a
The {% data variables.product.prodname_codeql_cli %} includes autobuilders for {% data variables.code-scanning.compiled_languages %} code. {% data variables.product.prodname_codeql %} autobuilders allow you to build projects for compiled languages without specifying any build commands. When an autobuilder is invoked, {% data variables.product.prodname_codeql %} examines the source for evidence of a build system and attempts to run the optimal set of commands required to extract a database. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#about-autobuild)."

An autobuilder is invoked automatically when you execute `codeql database create` for a compiled language if you don’t include a
`--command` option{% ifversion codeql-no-build %} or set `--build-mode none`{% endif %}. For example, for a C# codebase, you could simply run:
`--command` option{% ifversion codeql-no-build %} or set `--build-mode none`{% endif %}. For example, for a C/C++ codebase, you could simply run:

```shell
codeql database create --language=csharp <output-folder>/csharp-database
codeql database create --language=cpp <output-folder>/cpp-database
```

If a codebase uses a standard build system, relying on an autobuilder is often the simplest way to create a database. For sources that require non-standard build steps, you may need to explicitly define each step in the command line.
Expand Down
6 changes: 6 additions & 0 deletions data/features/codeql-no-build-csharp.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Reference: #14183 (Add C# to existing Java support)

versions:
fpt: '*'
ghec: '*'
ghes: '>=3.15'
2 changes: 1 addition & 1 deletion data/variables/code-scanning.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ codeql_workflow: 'CodeQL analysis workflow'
tool_status_page: 'tool status page'

# List of compiled languages supported for `no-build` extraction
no_build_support: 'Java'
no_build_support: 'Java{% ifversion codeql-no-build-csharp %} and C#{% endif %}'

# List of compiled languages
compiled_languages: 'C/C++, C#, {% ifversion codeql-go-autobuild %} Go,{% endif %} {% ifversion codeql-swift-beta %} Java, and Swift{% else %} and Java{% endif %}'
Expand Down

0 comments on commit e252be4

Please sign in to comment.