feat: support RPM archives #7628

knqyf263 · 2024-10-01T21:44:08Z

Description

Add support for RPM files. It's disabled unless TRIVY_EXPERIMENTAL_RPM_ARCHIVE is set as it's experimental.

$ TRIVY_EXPERIMENTAL_RPM_ARCHIVE=true ./trivy fs ./redhat-7.9-rpms -f cyclonedx

Details

2024-10-02T13:46:05+04:00       INFO    "--format cyclonedx" disables security scanning. Specify "--scanners vuln" explicitly if you want to include vulnerabilities in the "cyclonedx" report.
2024-10-02T13:46:05+04:00       INFO    Detected OS     family="none" version=""
2024-10-02T13:46:05+04:00       INFO    Number of language-specific files       num=0
{
  "$schema": "http://cyclonedx.org/schema/bom-1.6.schema.json",
  "bomFormat": "CycloneDX",
  "specVersion": "1.6",
  "serialNumber": "urn:uuid:ca46e752-2b04-49fc-a237-067b200be2e4",
  "version": 1,
  "metadata": {
    "timestamp": "2024-10-02T09:46:05+00:00",
    "tools": {
      "components": [
        {
          "type": "application",
          "group": "aquasecurity",
          "name": "trivy",
          "version": "0.55.0-49-g981a8b3e9"
        }
      ]
    },
    "component": {
      "bom-ref": "48aad1f0-9af0-445e-afd4-26a44dc2acda",
      "type": "application",
      "name": "/tmp/rpms/redhat-7.9-rpm-for-trivy",
      "properties": [
        {
          "name": "aquasecurity:trivy:SchemaVersion",
          "value": "2"
        }
      ]
    }
  },
  "components": [
    {
      "bom-ref": "4194564a-f44c-4a33-b848-6e570af8c7b5",
      "type": "operating-system",
      "name": "none",
      "properties": [
        {
          "name": "aquasecurity:trivy:Class",
          "value": "os-pkgs"
        },
        {
          "name": "aquasecurity:trivy:Type",
          "value": "none"
        }
      ]
    },
    {
      "bom-ref": "pkg:rpm/redhat/adcli-doc@0.8.2-12.el8",
      "type": "library",
      "supplier": {
        "name": "Red Hat, Inc."
      },
      "name": "adcli-doc",
      "version": "0.8.2-12.el8",
      "licenses": [
        {
          "license": {
            "name": "LGPLv2+"
          }
        }
      ],

TODOs

Parse RPM files
Enable/Disable the analyzer
Add tests

Checklist

I've read the guidelines for contributing to this repository.
I've followed the conventions in the PR title.
I've added tests that prove my fix is effective or that my feature works.
I've updated the documentation with the relevant information (if needed).
I've added usage information (if the PR introduces new options)
I've included a "before" and "after" example to the description (if the PR is a user interface change).

Signed-off-by: knqyf263 <knqyf263@gmail.com>

knqyf263 · 2024-10-02T10:07:11Z

@DmitriyLewen Some tests are failing due to 429 from GHCR, but it's ready for review.

Signed-off-by: knqyf263 <knqyf263@gmail.com>

DmitriyLewen · 2024-10-02T10:33:14Z

pkg/fanal/analyzer/pkg/rpm/archive.go

+	sourceRpm, err := h.GetString(rpmutils.SOURCERPM)
+	if a.unexpectedError(err) != nil {
+		return types.Package{}, xerrors.Errorf("failed to get source rpm: %w", err)
+	}
+	srcName, srcVer, srcRel, err := splitFileName(sourceRpm)
+	if err != nil {
+		a.logger.Debug("Invalid Source RPM Found", log.String("sourcerpm", sourceRpm))
+	}


can we skip splitFileName If SOURCERPM is not found?
(to avoid splitFileName error)
as for rpm:

trivy/pkg/fanal/analyzer/pkg/rpm/rpm.go

Lines 130 to 136 in 096fcd0

if pkg.SourceRpm != "(none)" && pkg.SourceRpm != "" {

// source epoch is not included in SOURCERPM

srcName, srcVer, srcRel, err = splitFileName(pkg.SourceRpm)

if err != nil {

log.DebugContext(ctx, "Invalid Source RPM Found", log.String("sourcerpm", pkg.SourceRpm))

}

}

Fixed in c1fa209

DmitriyLewen · 2024-10-02T10:42:35Z

pkg/fanal/analyzer/pkg/rpm/archive.go

+func (a *rpmArchiveAnalyzer) generatePURL(pkg *types.Package) *packageurl.PackageURL {
+	vendor := strings.ToLower(pkg.Maintainer)
+
+	// TODO: Handle more vendors
+	var ns string
+	switch {
+	case strings.Contains(vendor, "red hat"):
+		ns = "redhat"
+	case strings.Contains(vendor, "fedora"):
+		ns = "fedora"
+	case strings.Contains(vendor, "opensuse"):
+		ns = "opensuse"
+	case strings.Contains(vendor, "suse"):
+		ns = "suse"
+	}
+	return packageurl.NewPackageURL(packageurl.TypeRPM, ns, pkg.Name, utils.FormatVersion(*pkg), nil, "")
+}


IIUC we will overwrite this purl if OS is detected:

trivy/pkg/fanal/applier/docker.go

Lines 223 to 225 in 096fcd0

if mergedLayer.OS.Family != "" {

mergedLayer.Packages[i].Identifier.PURL = newPURL(mergedLayer.OS.Family, types.Metadata{OS: &mergedLayer.OS}, pkg)

}

We need to generate PURL here as OS is not detected when scanning RPM files.

I agree that we need to generate PURL here.
I am worried about case when we scan image that containing *.rpm file, and OS and vendor of this file are not equal.
In this case we will use namespace from OS.
(example): fedora image contains socat-1.7.3.2-2.el7.x86_64.rpm file (for RedHat).
in analyzer we use redhat namespace.
But in applier we will overwrite purl and use fedora namespace.

I meant that perhaps we need to add purl == nil check in applier.

UPD:

➜ cat Dockerfile FROM centos COPY socat-1.7.3.2-2.el7.x86_64.rpm socat-1.7.3.2-2.el7.x86_64.rpm ➜ TRIVY_EXPERIMENTAL_RPM_ARCHIVE=true ./trivy -q image -f json --list-all-pkgs rpm-archives | grep socat@1.7.3.2 "PURL": "pkg:rpm/centos/socat@1.7.3.2-2.el7?arch=x86_64\u0026distro=centos-8.4.2105",

It makes sense. Added 9c26be0

Signed-off-by: knqyf263 <knqyf263@gmail.com>

DmitriyLewen · 2024-10-02T11:20:51Z

I found another problem:
we can't detect vulnerabilities if OS is not found and we remove rpm packages:

➜ TRIVY_EXPERIMENTAL_RPM_ARCHIVE=true ./trivy -q fs -f json --list-all-pkgs ./pkg/fanal/analyzer/pkg/rpm/testdata/socat-1.7.3.2-2.el7.x86_64.rpm
{
  "SchemaVersion": 2,
  "CreatedAt": "2024-10-02T17:16:45.768306+06:00",
  "ArtifactName": "pkg/fanal/analyzer/pkg/rpm/testdata/socat-1.7.3.2-2.el7.x86_64.rpm",
  "ArtifactType": "filesystem",
  "Metadata": {
    "OS": {
      "Family": "none",
      "Name": ""
    },
    "ImageConfig": {
      "architecture": "",
      "created": "0001-01-01T00:00:00Z",
      "os": "",
      "rootfs": {
        "type": "",
        "diff_ids": null
      },
      "config": {}
    }
  }
}

It works only when vuln scanner is disable.

➜  TRIVY_EXPERIMENTAL_RPM_ARCHIVE=true ./trivy -q fs -f cyclonedx  ./pkg/fanal/analyzer/pkg/rpm/testdata/socat-1.7.3.2-2.el7.x86_64.rpm | grep socat@1.7.3.2-2
      "bom-ref": "pkg:rpm/redhat/socat@1.7.3.2-2.el7",
      "purl": "pkg:rpm/redhat/socat@1.7.3.2-2.el7",
        "pkg:rpm/redhat/socat@1.7.3.2-2.el7"
      "ref": "pkg:rpm/redhat/socat@1.7.3.2-2.el7",

See

trivy/pkg/scanner/local/scan.go

Lines 165 to 166 in 4926da7

    
           case errors.Is(err, ospkgDetector.ErrUnsupportedOS): 
        
           // do nothing

Signed-off-by: knqyf263 <knqyf263@gmail.com>

knqyf263 · 2024-10-02T11:25:45Z

Nice catch. I added a note.
ab37b6a

DmitriyLewen

LGTM

feat: support rpm archives

e83eee9

Signed-off-by: knqyf263 <knqyf263@gmail.com>

knqyf263 self-assigned this Oct 1, 2024

knqyf263 changed the title ~~feat: support rpm archives~~ feat: support RPM archives Oct 2, 2024

knqyf263 added 5 commits October 2, 2024 12:15

test: add tests

4f4dd1d

Signed-off-by: knqyf263 <knqyf263@gmail.com>

fix: lint issues

2f34afc

Signed-off-by: knqyf263 <knqyf263@gmail.com>

feat: disable RPM archive analyzer unless env is set

981a8b3

Signed-off-by: knqyf263 <knqyf263@gmail.com>

feat: add file path

782d8ac

Signed-off-by: knqyf263 <knqyf263@gmail.com>

test: use strings.Reader

6b4a3a3

Signed-off-by: knqyf263 <knqyf263@gmail.com>

knqyf263 marked this pull request as ready for review October 2, 2024 10:06

knqyf263 requested a review from DmitriyLewen as a code owner October 2, 2024 10:06

test: fix expected file path

096fcd0

Signed-off-by: knqyf263 <knqyf263@gmail.com>

DmitriyLewen reviewed Oct 2, 2024

View reviewed changes

knqyf263 added 3 commits October 2, 2024 14:51

fix: skip source rpm if empty

c1fa209

Signed-off-by: knqyf263 <knqyf263@gmail.com>

docs: add RPM archives

9ed5d88

Signed-off-by: knqyf263 <knqyf263@gmail.com>

fix: check if PURL is nil

9c26be0

Signed-off-by: knqyf263 <knqyf263@gmail.com>

docs: add a note

ab37b6a

Signed-off-by: knqyf263 <knqyf263@gmail.com>

DmitriyLewen approved these changes Oct 2, 2024

View reviewed changes

knqyf263 enabled auto-merge October 2, 2024 11:28

knqyf263 added this pull request to the merge queue Oct 2, 2024

Merged via the queue into aquasecurity:main with commit 69bf7e0 Oct 2, 2024
17 checks passed

knqyf263 deleted the feat/rpm_files branch October 2, 2024 12:04

aqua-bot mentioned this pull request Oct 2, 2024

release: v0.56.0 [main] #7447

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: support RPM archives #7628

feat: support RPM archives #7628

knqyf263 commented Oct 1, 2024 •

edited

Loading

knqyf263 commented Oct 2, 2024

DmitriyLewen Oct 2, 2024

knqyf263 Oct 2, 2024

DmitriyLewen Oct 2, 2024

knqyf263 Oct 2, 2024

DmitriyLewen Oct 2, 2024 •

edited

Loading

knqyf263 Oct 2, 2024

DmitriyLewen commented Oct 2, 2024

knqyf263 commented Oct 2, 2024

DmitriyLewen left a comment

	if pkg.SourceRpm != "(none)" && pkg.SourceRpm != "" {
	// source epoch is not included in SOURCERPM
	srcName, srcVer, srcRel, err = splitFileName(pkg.SourceRpm)
	if err != nil {
	log.DebugContext(ctx, "Invalid Source RPM Found", log.String("sourcerpm", pkg.SourceRpm))
	}
	}

	if mergedLayer.OS.Family != "" {
	mergedLayer.Packages[i].Identifier.PURL = newPURL(mergedLayer.OS.Family, types.Metadata{OS: &mergedLayer.OS}, pkg)
	}

feat: support RPM archives #7628

feat: support RPM archives #7628

Conversation

knqyf263 commented Oct 1, 2024 • edited Loading

Description

TODOs

Checklist

knqyf263 commented Oct 2, 2024

DmitriyLewen Oct 2, 2024

Choose a reason for hiding this comment

knqyf263 Oct 2, 2024

Choose a reason for hiding this comment

DmitriyLewen Oct 2, 2024

Choose a reason for hiding this comment

knqyf263 Oct 2, 2024

Choose a reason for hiding this comment

DmitriyLewen Oct 2, 2024 • edited Loading

Choose a reason for hiding this comment

knqyf263 Oct 2, 2024

Choose a reason for hiding this comment

DmitriyLewen commented Oct 2, 2024

knqyf263 commented Oct 2, 2024

DmitriyLewen left a comment

Choose a reason for hiding this comment

knqyf263 commented Oct 1, 2024 •

edited

Loading

DmitriyLewen Oct 2, 2024 •

edited

Loading