Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 7 additions & 22 deletions Building-Castles/DevSecOps-Pipeline-Full-Implementation.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ was correctly completed, the pipeline will be automatically integrated with
the `semgrep-sast` test stage, which means that GitLab discovered the `.java`
file and processed it with the ideal tester:

![DevSecOps-Pipeline-Simulate-Code-Problem-GitLab-CI.png](images/DevSecOps-Pipeline-Simulate-Code-Problem-GitLab-CI.png)
![DevSecOps-Pipeline-Nexus-GitLab-CI.png](images/DevSecOps-Pipeline-Nexus-GitLab-CI.png)

Second thing to note is that, unexpectedly, the pipeline will complete with
no errors. The SonarQube stage completes as follows:
Expand All @@ -115,7 +115,7 @@ Job succeeded

But by looking into the project details we will find a Failed status:

![DevSecOps-Pipeline-Simulate-Code-Problem-Sonarqube-Code-Failure.png](images/DevSecOps-Pipeline-Simulate-Code-Problem-Sonarqube-Code-Failure.png)
![DevSecOps-Pipeline-Sonarqube-Code-Failure.png](images/DevSecOps-Pipeline-Sonarqube-Code-Failure.png)

This happens because we don't have a blocker for the pipeline.

Expand Down Expand Up @@ -163,22 +163,7 @@ To ssh://172.16.99.1:2222/devsecops/myproject.git
A new pipeline will be started, and this should fail with a message like
this:

```console
INFO: Analysis report uploaded in 13ms
INFO: ------------- Check Quality Gate status
INFO: Waiting for the analysis report to be processed (max 300s)
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 10.559s
INFO: Final Memory: 21M/114M
INFO: ------------------------------------------------------------------------
ERROR: Error during SonarScanner execution
ERROR: QUALITY GATE STATUS: FAILED - View details on http://172.16.99.1:9000/dashboard?id=myproject
ERROR:
ERROR: Re-run SonarScanner using the -X switch to enable full debug logging.
ERROR: Job failed: exit code 1
```
![DevSecOps-Pipeline-GitLab-Sonarqube-Quality-Gate-Failure.png](images/DevSecOps-Pipeline-GitLab-Sonarqube-Quality-Gate-Failure.png)

## Fix the code

Expand Down Expand Up @@ -217,13 +202,13 @@ To ssh://172.16.99.1:2222/devsecops/myproject.git
d582dcf..00f0947 main -> main
```

The original error is not there anymore, and the pipeline should now be
green.
The original error is not there anymore, but there's a possibility for the
pipeline to not (yet) be green.

### Code coverage

If Sonarqube fails again, check the quality gates, and create a custom quality
gate associated to the project that will have `Conditions on New Code` ->
If Sonarqube fails again, check the **Quality Gates**, and create a custom
quality gate associated to the project that will have `Conditions on New Code` ->
`Coverage` set to zero.

Another viable solution would be to implement a unit test related to the example
Expand Down
126 changes: 71 additions & 55 deletions Building-Castles/DevSecOps-Pipeline-GitLab-SAST.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,7 @@ capabilities that will affect the CI process.
## Add some code into the repo

We will need some code inside `myproject` to check GitLab's SAST
functionalities, so we will create two kind of sources.

A faulty Kubernetes Pod manifest:
functionalities, so we will create a faulty Kubernetes Pod manifest:

```console
$ pwd
Expand All @@ -18,6 +16,8 @@ $ mkdir -v manifests
mkdir: created directory 'manifests'

$ cat <<EOF > manifests/Pod.yml
# Intentionally insecure minimal example for Kubesec/GitLab SAST
# Kubesec commonly flags allowPrivilegeEscalation: true as critical
apiVersion: v1
kind: Pod
metadata:
Expand All @@ -31,31 +31,21 @@ spec:
EOF
```

A faulty php source code:
And a faulty Python code:

```console
$ mkdir -v php
mkdir: created directory 'php'

$ cat <<EOF > php/faulty.php
<?php
// This code contains a security vulnerability

// Extract user input from GET parameters
\$id = \$_GET['id'];

// SQL query without proper input validation
\$query = "SELECT * FROM users WHERE id = \$id";

// Execute the SQL query
\$result = mysqli_query(\$connection, \$query);
$ pwd
/home/kirater/myproject

// Fetch the data
\$data = mysqli_fetch_assoc($result);
$ mkdir -v python
mkdir: created directory 'python'

// Display the user's details
echo "Username: " . \$data['username'] . "<br>";
echo "Email: " . \$data['email'] . "<br>";
$ cat <<EOF > python/script.py
# Intentionally insecure minimal example for Semgrep/GitLab SAST
# Semgrep commonly flags use of eval on untrusted input.
user_input = input("data: ")
result = eval(user_input) # <-- insecure: Semgrep should flag this
print(result)
EOF
```

Expand Down Expand Up @@ -102,7 +92,7 @@ $ git add . && git commit -m "Activate CI"
3 files changed, 43 insertions(+)
create mode 100644 .gitlab-ci.yml
create mode 100644 manifests/Pod.yml
create mode 100644 php/faulty.php
create mode 100644 python/script.py

$ git push
Enumerating objects: 8, done.
Expand All @@ -127,14 +117,15 @@ And after some time everything should be green:
![DevSecOps-Pipeline-GitLab-SAST-CI-2.png](images/DevSecOps-Pipeline-GitLab-SAST-CI-2.png)

Which it seems good, apart from the fact that we introduced faulty code for
both php sources and Kubernetes manifests, and by looking at the details of
both Python script and Kubernetes manifests, and by looking at the details of
the pipelines we should see something like this:

![DevSecOps-Pipeline-GitLab-SAST-CI-Kube-Critical.png](images/DevSecOps-Pipeline-GitLab-SAST-CI-Kube-Critical.png)

And the same applies for PHP, but with no `Critical` vulnerabilities.
And the same applies for Python, with an `High` vulnerability found.
So, if we want to make this worthy, we need to instruct our pipeline to stop
if it finds `Critical` vulnerabilites.
if it finds `Critical` vulnerabilites for Kubesec and `High` vulnerabilites for
Semgrep.

## Analyze reports

Expand All @@ -150,48 +141,43 @@ stages:
- sast-analysis
```

We will first define a variable named `BLOCKING_VULNERABILITY_SEVERITY` that
will be set to the defined severity, in this case `Critical`:
We will first define two variables named `KUBESEC_SEVERITY` and
`SEMGREP_SEVERITY` that will be set to the relative severity value:

```yaml
variables:
...
BLOCKING_VULNERABILITY_SEVERITY: "Critical"
SEMGREP_SEVERITY: "High"
KUBESEC_SEVERITY: "Critical"
```

Then for each of the analysis we want to check we will define a related and
connected job.

For php:
Then for the analysis we want to check we will define a related and connected
job:

```yaml
sast-analysis-php:
sast-analysis-semgrep:
stage: sast-analysis
image: registry.gitlab.com/gitlab-ci-utils/curl-jq
needs:
- job: phpcs-security-audit-sast
- job: semgrep-sast
script:
- JQRESULT=$(jq -r --arg SEV "${BLOCKING_VULNERABILITY_SEVERITY}" '.vulnerabilities[] | select(.severity==$SEV).description' gl-sast-report.json)
- JQRESULT=$(jq -r --arg SEV "${SEMGREP_SEVERITY}" '.vulnerabilities[] | select(.severity==$SEV).description' gl-sast-report.json)
- echo "JQRESULT -> $JQRESULT"
- if [ "x$JQRESULT" != "x" ]; then echo -e "One or more ${BLOCKING_VULNERABILITY_SEVERITY} vulnerabilities have been found:\n$JQRESULT"; exit 1; fi
```

Then for `kubesec`:
- if [ "x$JQRESULT" != "x" ]; then echo -e "One or more ${SEMGREP_SEVERITY} vulnerabilities have been found:\n$JQRESULT"; exit 1; fi

```yaml
sast-analysis-kubesec:
stage: sast-analysis
image: registry.gitlab.com/gitlab-ci-utils/curl-jq
needs:
- job: kubesec-sast
script:
- JQRESULT=$(jq -r --arg SEV "${BLOCKING_VULNERABILITY_SEVERITY}" '.vulnerabilities[] | select(.severity==$SEV).description' gl-sast-report.json)
- JQRESULT=$(jq -r --arg SEV "${KUBESEC_SEVERITY}" '.vulnerabilities[] | select(.severity==$SEV).description' gl-sast-report.json)
- echo "JQRESULT -> $JQRESULT"
- if [ "x$JQRESULT" != "x" ]; then echo -e "One or more ${BLOCKING_VULNERABILITY_SEVERITY} vulnerabilities have been found:\n$JQRESULT"; exit 1; fi
- if [ "x$JQRESULT" != "x" ]; then echo -e "One or more ${KUBESEC_SEVERITY} vulnerabilities have been found:\n$JQRESULT"; exit 1; fi
```

In both cases (note that are both called `sast-analysis`), if there will be
`Critical` vulnerabilities inside the json the pipeline will fail.
Note that both stages are called `sast-analysis`, if there will be
vulnerabilities inside the relative json, then the pipeline will fail.

The resulting and final yaml will be something like this:

Expand All @@ -202,7 +188,8 @@ include:
variables:
SECURE_LOG_LEVEL: "debug"
SCAN_KUBERNETES_MANIFESTS: "true"
BLOCKING_VULNERABILITY_SEVERITY: "Critical"
SEMGREP_SEVERITY: "High"
KUBESEC_SEVERITY: "Critical"

stages:
- test
Expand All @@ -214,25 +201,25 @@ sast:
paths:
- gl-sast-report.json

sast-analysis-php:
sast-analysis-semgrep:
stage: sast-analysis
image: registry.gitlab.com/gitlab-ci-utils/curl-jq
needs:
- job: phpcs-security-audit-sast
- job: semgrep-sast
script:
- JQRESULT=$(jq -r --arg SEV "${BLOCKING_VULNERABILITY_SEVERITY}" '.vulnerabilities[] | select(.severity==$SEV).description' gl-sast-report.json)
- JQRESULT=$(jq -r --arg SEV "${SEMGREP_SEVERITY}" '.vulnerabilities[] | select(.severity==$SEV).description' gl-sast-report.json)
- echo "JQRESULT -> $JQRESULT"
- if [ "x$JQRESULT" != "x" ]; then echo -e "One or more ${BLOCKING_VULNERABILITY_SEVERITY} vulnerabilities have been found:\n$JQRESULT"; exit 1; fi
- if [ "x$JQRESULT" != "x" ]; then echo -e "One or more ${SEMGREP_SEVERITY} vulnerabilities have been found:\n$JQRESULT"; exit 1; fi

sast-analysis-kubesec:
stage: sast-analysis
image: registry.gitlab.com/gitlab-ci-utils/curl-jq
needs:
- job: kubesec-sast
script:
- JQRESULT=$(jq -r --arg SEV "${BLOCKING_VULNERABILITY_SEVERITY}" '.vulnerabilities[] | select(.severity==$SEV).description' gl-sast-report.json)
- JQRESULT=$(jq -r --arg SEV "${KUBESEC_SEVERITY}" '.vulnerabilities[] | select(.severity==$SEV).description' gl-sast-report.json)
- echo "JQRESULT -> $JQRESULT"
- if [ "x$JQRESULT" != "x" ]; then echo -e "One or more ${BLOCKING_VULNERABILITY_SEVERITY} vulnerabilities have been found:\n$JQRESULT"; exit 1; fi
- if [ "x$JQRESULT" != "x" ]; then echo -e "One or more ${KUBESEC_SEVERITY} vulnerabilities have been found:\n$JQRESULT"; exit 1; fi
```

Committing the code and pushing the changes:
Expand Down Expand Up @@ -264,7 +251,10 @@ that was defined in the stage:

The `Critical` vulnerability was detected.

## Fix the problem
The same applies for the Semgrep analysis, where there's a clear indication of
the `eval` function usage inside the Python script.

## Fix the problems

To fix the detected problem we need to understand it from the analysis:

Expand Down Expand Up @@ -302,6 +292,32 @@ To ssh://172.16.99.1:2222/devsecops/myproject.git
12e7259..d9638d7 main -> main
```

Same will be done for the Python script:

```python
import ast
user_input = input("data: ")
print(ast.literal_eval(user_input)) # safe: only evaluates Python literals, not arbitrary code
```

This modification will be pushed as well:

```console
$ git add . && git commit -m "Fix Python script"
[main a17809b] Fix Python script
1 file changed, 2 insertions(+), 4 deletions(-)

$ git push
Enumerating objects: 7, done.
Counting objects: 100% (7/7), done.
Delta compression using up to 4 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (4/4), 413 bytes | 413.00 KiB/s, done.
Total 4 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)
To ssh://172.16.99.1:2222/devsecops/myproject.git
611b468..a17809b main -> main
```

Finally, after some time, the pipeline should be all green:

![DevSecOps-Pipeline-GitLab-SAST-CI-Pipeline-With-Check-Fixed.png](images/DevSecOps-Pipeline-GitLab-SAST-CI-Pipeline-With-Check-Fixed.png)
6 changes: 3 additions & 3 deletions Building-Castles/DevSecOps-Pipeline-GitLab.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ $ docker run --detach \
--publish 8080:80 \
--publish 8443:443 \
--publish 2222:22 \
gitlab/gitlab-ce:16.11.8-ce.0
gitlab/gitlab-ce:18.4.1-ce.0
...
```

Expand All @@ -26,7 +26,7 @@ Check the progresses, until the web interface comes up:
```console
$ docker logs -f gitlab
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=16.11.8-ce.0
Current version: gitlab-ce=18.4.1-ce.0
...
```

Expand Down Expand Up @@ -181,7 +181,7 @@ $ docker run --detach \
--privileged \
--volume /var/run/docker.sock:/var/run/docker.sock \
--volume $PWD/gitlab-runner:/etc/gitlab-runner \
gitlab/gitlab-runner:v16.11.3
gitlab/gitlab-runner:v18.4.0
...
```

Expand Down
12 changes: 6 additions & 6 deletions Building-Castles/DevSecOps-Pipeline-Nexus.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ changed ownership of 'nexus' from kirater:docker to 200:200
```

Then it will be possible to launch the Nexus instance using the
`sonatype/nexus3` container, exposing these ports (Host/Container):
`sonatype/nexus3:3.84.1` container, exposing these ports (Host/Container):

- 8081:8081
- 9443:9443
Expand All @@ -30,8 +30,8 @@ $ docker run --detach \
--publish 9443:9443 \
--publish 5000:5000 \
--volume $PWD/nexus:/opt/sonatype/sonatype-work \
sonatype/nexus3
Unable to find image 'sonatype/nexus3:latest' locally
sonatype/nexus3:3.84.1
Unable to find image 'sonatype/nexus3:3.84.1' locally
latest: Pulling from sonatype/nexus3
36c12cb044ac: Pull complete
8a1b09f0eced: Pull complete
Expand All @@ -41,7 +41,7 @@ f47ff8368d44: Pull complete
cd8e246a9663: Pull complete
68721705c831: Pull complete
Digest: sha256:88044c8cdbbf1fea42b65b6c785bb88e4e2b2e96b3becd2bfce22f481216a951
Status: Downloaded newer image for sonatype/nexus3:latest
Status: Downloaded newer image for sonatype/nexus3:3.84.1
1bfde8b8401841bb82e9c00ab6e0efc90860dbbe44771e74cc13e4218d4da162
```

Expand All @@ -52,7 +52,7 @@ $ docker logs -f nexus
...
-------------------------------------------------

Started Sonatype Nexus OSS 3.72.0-04
Started Sonatype Nexus COMMUNITY 3.84.1-01

-------------------------------------------------
```
Expand Down Expand Up @@ -263,7 +263,7 @@ nothing to commit, working tree clean

And then follow the progress from the GitLab interface:

![DevSecOps-Pipeline-Nexus-GitLab-CI.png](images/DevSecOps-Pipeline-Nexus-GitLab-CI.png)
![DevSecOps-Pipeline-GitLab-Nexus-CI.png](images/DevSecOps-Pipeline-GitLab-Nexus-CI.png)

The final result should be the presence of the image inside the Nexus
interface:
Expand Down
4 changes: 2 additions & 2 deletions Building-Castles/DevSecOps-Pipeline-SonarQube.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ exposing these ports (Host/Container):
$ cd && docker run --detach \
--name sonarqube \
--publish 9000:9000 \
sonarqube:lts
sonarqube:25.9.0.112764-community
```

Check the progresses, until the web interface comes up:
Expand Down Expand Up @@ -104,7 +104,7 @@ To ssh://172.16.99.1:2222/devsecops/myproject.git
And then follow the progress from the GitLab interface, under the CI
pipelines a new stage should be visible:

![DevSecOps-Pipeline-Sonarqube-GitLab-CI.png](images/DevSecOps-Pipeline-Sonarqube-GitLab-CI.png)
![DevSecOps-Pipeline-GitLab-Sonarqube-CI.png](images/DevSecOps-Pipeline-GitLab-Sonarqube-CI.png)

The project `myproject` will now be visible also in the Sonarqube web
interface.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified Building-Castles/images/DevSecOps-Pipeline-GitLab-SAST-CI-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified Building-Castles/images/DevSecOps-Pipeline-GitLab-SAST-CI-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Binary file modified Building-Castles/images/DevSecOps-Pipeline-Nexus-Image.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Loading