Performance issues from scanSBOMFile reading every file multiple times

Scorecard utilizes `osvscanner.DoScan` when performing its `Vulnerabilities` check. The time to complete the check is more than an order of magnitude higher than other checks. Running `pprof` shows a hot spot in `scanSBOMFile`.

It looks like when walking a directory, every file is potentially parsed as an SBOM:
https://github.com/google/osv-scanner/blob/6eab4504663395b4f5983ebe4570e8b816844ab5/pkg/osvscanner/osvscanner.go#L114-L117

There are currently two providers, SPDX and CycloneDX. While SPDX checks for a filename, there's no such check for CycloneDX:
https://github.com/google/osv-scanner/blob/6eab4504663395b4f5983ebe4570e8b816844ab5/pkg/osvscanner/osvscanner.go#L231-L238

I believe this means attempting to parse every file as a CycloneDX SBOM, twice. In large repositories, this adds up ( longest observed is 5 minutes) :
https://github.com/google/osv-scanner/blob/6eab4504663395b4f5983ebe4570e8b816844ab5/internal/sbom/cyclonedx.go#L50-L59

![8sg3cyNvvMYG5HH](https://user-images.githubusercontent.com/6064089/222297330-4772838c-a0ae-4b29-ae38-51fb938ff3a1.png)


	// No need to check for error
	// If scan fails, it means it isn't a valid SBOM file,
	// so just move onto the next file
	_ = scanSBOMFile(r, query, path)

	for _, provider := range sbom.Providers {
	if provider.Name() == "SPDX" &&
	!strings.Contains(strings.ToLower(filepath.Base(path)), ".spdx") {
	// All spdx files should have the .spdx in the filename, even if
	// it's not the extension: https://spdx.github.io/spdx-spec/v2.3/conformance/
	// Skip if this isn't the case to avoid panics
	continue
	}

	func (c *CycloneDX) GetPackages(r io.ReadSeeker, callback func(Identifier) error) error {
	var bom cyclonedx.BOM

	for _, formatType := range cycloneDXTypes {
	_, err := r.Seek(0, io.SeekStart)
	if err != nil {
	return fmt.Errorf("failed to seek to start of file: %w", err)
	}
	decoder := cyclonedx.NewBOMDecoder(r, formatType)
	err = decoder.Decode(&bom)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Performance issues from scanSBOMFile reading every file multiple times #257

spencerschrock
openedon Mar 2, 2023

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Performance issues from scanSBOMFile reading every file multiple times #257

Description

spencerschrockopenedon Mar 2, 2023

Metadata