v5.1.1

What's Changed

• Fixed a data race when analyzing repositories with --file-mode git (#4522, @spencerschrock)

Full Changelog: v5.1.0...v5.1.1

v5.1.0

What's Changed

• There is a new --file-mode flag to control how repository files are fetched. (#4474, @spencerschrock)
  • The default method if unspecified is --file-mode archive which is what older versions of Scorecard always used.
  • --file-mode git produces the most accurate results for repositories with .gitattributes files at the cost of analysis speed. This currently only supports GitHub but can be expanded to other forges in the future.
• The Scorecard binary name in releases is now consistent across all platforms (#4520, @timothysparg)
• Scorecard now prints a warning if multiple GitHub PAT environment variables are set with different values. (#4483, @aunovis-heidrich)

Azure DevOps Support (Experimental)

There is now experimental support for Azure DevOps repositories (thanks @JamieMagee). Most checks should work, but the following checks do not: Branch-Protection, SBOM, and Signed-Releases.

To analyze a repository:

1. Set the SCORECARD_EXPERIMENTAL environment variable to any value.
2. Put your access token in the AZURE_DEVOPS_AUTH_TOKEN environment variable.

Checks

• Enabled Fuzzing, License, Packaging, SAST, and Security-Policy checks when using --local option (#4423, @lharrison13)

CI-Tests

• ✨ Add woodpecker as known CI by @6543 in #4336

Contributors

• Contributing organizations are now lexicographically sorted. (#4436, @spencerschrock)

Dangerous-Workflow

• When detecting a potential script injection in a GitHub workflow, Scorecard now adds a machine-readable patch to fix the vulnerability. This patch can be applied to your project using git apply or patch -p1 from the repository's root. The patch is currently only visible when running the hasDangerousWorkflowScriptInjection probe directly. (#4218, @pnacht)

Fuzzing

• Support detection of fuzzing in Elixir and Gleam through the import of property-based testing modules (#4408, @kikofernandez)
• Support detection of fuzzing in Erlang through the import of property-based testing modules (#4406, @kikofernandez)

License

• Updates list of supported SDPX licenses to include latest data. (#4323, @lelia)

Pinned-Dependencies

• ✨ Support Nuget Pinned Dependency with RestoreLockedMode attribute by @balteravishay in #4351
• ✨ Support Nuget Central Package Management by @balteravishay in #4369

Security-Policy

• Fixed an issue where an org's .github repository was checked for a security policy without the proper authenticated transport by @jeffmendoza in #4259

Signed-Releases

• Included links now link to the artifacts instead of the API details about the artifacts by @klbynum in #4290

Docs

• 📖 Fix typo in branch protections details by @martincostello in #4270
• 📖 Updated Scorecard link in README.md by @Wavyeli32 in #4262
• 📖 Mention rulesets for GitHub Branch-Protection remediation by @pethers in #4316
• 📖 explicitly state both check documentation files are committed by @spencerschrock in #4317
• 📖 clarify project goals and non-goals by @spencerschrock in #4318
• 📖 governance: Add Incubation application submission by @justaugustus in #4200
• 📖 Fix SBOM-Everywhere link by @evankanderson in #4334
• 📖 governance: Add meeting note archives from 2021 through 2024 by @justaugustus in #4482

New Contributors

• @Wavyeli32 made their first contribution in #4262
• @klbynum made their first contribution in #4290
• @6543 made their first contribution in #4336
• @evankanderson made their first contribution in #4334
• @kikofernandez made their first contribution in #4406
• @lharrison13 made their first contribution in #4423
• @renewitt made their first contribution in #4476
• @aunovis-heidrich made their first contribution in #4483
• @timothysparg made their first contribution in #4520

Full Changelog: v5.0.0...v5.1.0

v5.0.0

What's Changed

We’ll highlight the major changes between v4.13.1 and v5.0.0 below, as well as some of the changes between v5.0.0-rc2 and v5.0.0. For a more complete picture, see the v5.0.0-rc1 and v5.0.0-rc2 changelogs as well.

Structured Results

Structured Results is the main feature from this release. At a high level, structured results involve breaking the existing 19 Scorecard Checks into individual heuristics so users can pick and choose which ones they care about. You can see a list of all supported probes by checking out our documentation (paying attention to lifecycle / stability guarantees). To run individual probes, use the --probes CLI flag with a comma separated list of names. You must also specify the --format probe option to see the results. Please run scorecard --help if you need more details.
Example:

scorecard --repo github.com/ossf/scorecard --probes archived,fuzzed,hasLicenseFile --format probe

For more details on the feature, please check out our blog post or the talk given at Open Source Summit NA 2024: Structured Scorecard Results: Tailor Your Own Supply-Chain Security Policies.

Maintainer Annotations

Maintainer Annotations let maintainers add context to display alongside Scorecard check results. Annotations can provide users additional information when Scorecard has an incomplete assessment of a project's security practices. To see the maintainers annotations for each check, if present, use the --show-annotations option. For example, the not-detected annotation can annotate when a maintainer fulfills a check or probe in a way that is supported by Scorecard but not identified.

For more details, check out our documentation for the feature.

Breaking Changes

API changes

The biggest change is that everything in github.com/ossf/scorecard/v4/pkg now lives in github.com/ossf/scorecard/v5/pkg/scorecard. This allows renaming of some function names and types to be less repetitive.

• RunScorecard is now Run
• ScorecardResult is now Result

Expected changes:

pkg.ScorecardRun() -> scorecard.Run()
pkg.ScorecardResult -> scorecard.Result

The signature of RunScorecard (now called Run) has changed to allow for fewer breaking changes in the future. For full motivation, see the associated issue. There should be fewer setup code needed than before. Callers no longer need to pass in all clients and arguments and can rely on sensible default behavior. Callers that want to customize the analysis can influence the results with our Option types.

A similar change was done with formatting the results, which now accept an option struct pointer. Using a nil pointer will use default values.

Unlikely to cause issues

These changes are technically breaking in a semver sense, but we don’t expect most users to depend on them or require changes.

• Dependency diff functionality has been removed in #4146.
  • For GitHub replacements, consider actions/dependency-review-action which now supports showing Scorecard data.
• clients.Repo must now implement Path in #4104
• clients.RepoClient now returns a read closer in #3912
• Some branch protection types were renamed in #3879
• Some sentinel errors were renamed in #4040

Checks

Binary-Artifacts

• Binary-Artifacts check now supports the new gradle wrapper validation action, and can be pinned to a hash. (#4097, @spencerschrock)

Dependency-Update-Tool

• ✨ Recognize scala-steward as dependency update tool by @raboof in #4130

Fuzzing

• ⚠️ OneFuzz detection removed due to deprecation #3666

License

• 🐛 fix Unlicense detection by @spencerschrock in #4145
• 📖 Improve the REUSE parts of the License check by @mxmehl in #4155

Packaging

• ✨ detect sbt ci-release packaging workflows by @raboof in #4135

Pinned-Dependencies

• Improve Pinned-Dependencies remediation creation performance (#4131, @raboof)
• ✨ Add support for Nuget restore by @balteravishay in #4157

SBOM

• ✨ Add experimental check for published SBOM by @ashearin in #3903

Vulnerabilities

• 🌱 add stack info to osv-scanner error by @Zxilly in #4172

Probes

• ✨ probe: releases with verified provenance by @raghavkaul in #4141

Other

• 🐛 Use direct endpoint instead of search to find repository URL from npm database by @aklevans in #4118
• ✨ move to cgr base image by @naveensrinivasan in #4113
• 🐛 fix: correct sarif json schema url by @Zxilly in #4170

New Contributors

• @petermetz made their first contribution in #4111
• @aklevans made their first contribution in #4118
• @mxmehl made their first contribution in #4155
• @Zxilly made their first contribution in #4170

Full Changelog: v4.13.1...v5.0.0

v5.0.0-rc2

Important

This is a v5 prerelease candidate. There may be more breaking changes before the official v5.0.0 release.

What's Changed

Structured Results

• ✨ allow probes to collect their own data from repo clients by @spencerschrock in #4052

Check Enhancements and Bug Fixes

• Signed-Releases
  • 🐛 Fixed a Signed-Releases bug where more releases were being analyzed than intended. (#4060, @spencerschrock)
• Code-Review
  • 🐛 Fixed an issue where Phabricator reviews weren't being parsed properly. (#4086, @spencerschrock)

Breaking Changes

• ⚠️ errors in ErrXXX format by @cmwylie19 in #4040
• ⚠️ Enables maintainers to write annotations for Scorecard checks and consumers to view these annotations in Scorecard UI. (experimental #3905, @gabibguti)

Docs

• 📖 Docs: update website by @raghavkaul in #4041
• 📖 governance: Adopt Scorecard project charter by @justaugustus in #4054

Other

• 🌱 Remove survey by @afmarcum in #4077
• 🌱 Update Binary-Artifacts and License tests by @seelder in #4079

New Contributors

• @cmwylie19 made their first contribution in #4040
• @seelder made their first contribution in #4079

Full Changelog: v5.0.0-rc1...v5.0.0-rc2

v5.0.0-rc1

Important

This is a v5 prerelease candidate. There may be more breaking changes before the official v5.0.0 release.

What's Changed

Structured Results

We invite users to try out a preview of Structured Results, the main feature from this release candidate. For more details on the feature, please check out the first paragraph of our probes README as well as our blog post.

At a high level, structured results involves breaking the existing 19 Scorecard Checks into individual heuristics so users can pick and choose which ones they care about. You can see a list of all supported probes by checking out the probes/ directory. To run individual probes, use the --probes CLI flag with a comma separated list of names. You must also specify the --format probe option to see the results. Please run scorecard --help if you need more details.

Example:

scorecard --repo github.com/ossf/scorecard --probes archived,fuzzed,hasLicenseFile --format probe

Check Enhancements and Bug Fixes

• Branch-Protection
  • ✨ Branch Protection check now also evaluates if the project requires PRs prior to make changes to the branch. This won't change anything for the users that already require reviews, but will enable score enhancement for those who can't require reviewers. (#3499, @diogoteles08)
• Dependency-Update-Tool
  • ✨ Dependency-Update-Tool now detects Renovate config files in a .gitlab folder. (#3823, @spencerschrock)
  • 🐛 Sonatype Lift is no longer recognized as a Dependency-Update-Tool because it is retired. (#3605, @spencerschrock)
  • 🐛 Dependency-Update-Tool: ignore search commit data for repo clients which dont support it by @spencerschrock in #3756
• Fuzzing
  • ⚠️ Remove OneFuzz from fuzzing checks by @DavidKorczynski in #3666
• Pinned-Dependencies
  • 🐛 Pinned-Dependencies now continues after encountering runtime errors (#3515, @pnacht)
  • 🐛 Scorecard no longer considers unpinned Dockerfiles in vendor and third_party directories. (#3675, @AdamKorcz)
  • 🐛 Files downloaded by Git SHA from GitHub and executed are no longer considered as not pinned by hash. (#3694, @martincostello)
  • 🐛 Shell commands in Dockerfile here-documents are now parsed correctly by the Pinned-Dependencies check (#3774, @jkreileder)
• Signed-Releases
  • 🐛 Fixed a bug which allowed some repos to score higher than 10 in the Signed-Releases check. (#3768, @spencerschrock)
  • ✨ Support .sigstore bundles to check for signed releases (#3772, @edgarrmondragon)
• Vulnerabilities
  • 🐛 Projects without dependencies or packages no longer throw an error for the Vulnerabilities check. (#3803, @spencerschrock)
  • 🐛 Go stdlib vulns are removed Vulnerabilities check output (#3925, @spencerschrock)

RepoClient Improvements

• GitHub

  • 🐛 Scorecard processes commit activity from large GitHub repos in chunks to avoid timeout issues (#3680, @spencerschrock)

• GitLab

  • 🐛 Fix scanning for GitLab private repositories. (#3596, @gabibguti)
  • ✨ Added --commit-depth support for GitLab repos (#3672, @ashearin)
  • 🐛 Parse Gitlab Status fields to align w/Github Status and Conclusion by @ashearin in #3706
  • 🐛 Fix signed release error for empty gitlab repo by @naveensrinivasan in #3753
  • 🐛 Scorecard no longer crashes on GitLab repos with no commits (#3731, @ashearin)
  • 🐛 Fixed a bug which prevented Scorecard from analyzing some self-hosted GitLab repos. (#3819, @spencerschrock)

• Local Directory

  • 🐛 ignore .git folder for localdir by @naveensrinivasan in #3943

Other

• 🐛 Fix nils by @naveensrinivasan in #3750
• ✨ Added logic to ensure check scores are between 0 and 10 (#3769, @spencerschrock)

Breaking Changes

• File access through RepoClient now returns an io.ReadCloser, instead of the full file contents. (#3912, @spencerschrock). This enabled fixing two bugs which affect very large repos.
  • 🐛 Limit Binary Artifact file reads to first 1024 bytes by @spencerschrock in #3923
  • 🐛 Avoid reading every file searching for sonar configs by @spencerschrock in #3929
• ⚠️ refactor: rename fields on Branch Protection Pull Request rules by @diogoteles08 in #3879
• ⚠️ removerule.Remediation and switch users to probe.Remediation by @spencerschrock in #3978

Docs

• 📖 fix typo by @AdamKorcz in #3699
• 📖 Added beginner's guide to scorecard checks docs by @ariathaker in #3617
• 📖 fixup transposition typos in remediation package copy by @daveworth in #3734
• 📖 Update README with zoom meeting info by @leec94 in #3739
• 📖 Clarify lack of 2FA check in README by @raghavkaul in #3784
• 📖 Add documentation about probes and contributing by @AdamKorcz in #3762
• 📖 Spelling by @jsoref in #3804
• 📖 Update contributor ladder to reduce duration requirements by @afmarcum in #3899
• 📖 Update slack image by @afmarcum in #3906
• 📖 Document that .sigstore bundles are part of check for Signed-Releases (#3922, @cpswan)
• 📖 Add survey announcement to readme by @afmarcum in #3942
• 📖 Review and update CONTRIBUTING.md by @spencerschrock in #4002
• 📖 revert PAT scope change and document Go resources by @spencerschrock in #4003

New Contributors

• @ashearin made their first contribution in #3672
• @ariathaker made their first contribution in #3617
• @daveworth made their first contribution in #3734
• @edgarrmondragon made their first contribution in #3772
• @manishtiwari25 made their first contribution in #3732
• @jkreileder made their first contribution in #3774
• @tuminoid made their first contribution in #3783
• @lelia made their first contribution in #3822
• @jsoref made their first contribution in #3804
• @jitsengupta17 made their first contribution in #3302
• @cpswan made their first contribution in #3922
• @adamdmharvey made their first contribution in #3972
• @fhoeborn made their first contribution in #3838

Full Changelog: v4.13.1...v5.0.0-rc1

v4.13.1

What's Changed

New

• Fuzzing

  • ✨ Adds fuzzing probes for C, CPP, Python, Rust and Java by @DavidKorczynski in #3473
  • ✨ Add support for fast-check test runners integrations by @sheerlox in #3568

• Weekly Public Data Cron

  • 🌱 Adding all Intel public GitHub repos by @ware in #3556

Bug Fixes

• SAST
  • 🐛 Fix usage of GitHub CodeQL not being detected correctly by @martincostello in #3591

Docs

• 📖 fix "default" typo by @testwill in #3543

New Contributors

• @testwill made their first contribution in #3543
• @ware made their first contribution in #3556
• @sheerlox made their first contribution in #3568

Full Changelog: v4.13.0...v4.13.1

v4.13.0

What's Changed

New

• Binary Artifacts:

  • ✨ The Binary-Artifacts check supports local repos again by @spencerschrock in #3415
  • ✨ Check for static archives in Binary Artifacts by @DavidKorczynski in #3454

• Branch Protection:

  • ✨ Branch protection now considers repository rulesets by @thepwagner in #3354
  • ✨ Move "EnforcesAdmins" to tier 5 Branch-Protection by @spencerschrock in #3502

• Pinned-Dependencies:

  • ✨ Only score detected ecosystems by @gabibguti in #3436

• Permissions:

  • ✨ broaden job matcher for semantic release by @secustor in #3506

• CLI:

  • ✨ Increase PyPI parsing flexibility for --pypi flag by @joshgc in #3423
  • ✨ Add --output argument to write results to file by @gabibguti in #3482

Bug Fixes

• License:
  • 🐛 Fixed situations where the Licenses folder wasn't being detected. by @spencerschrock in #3412
  • 🐛 Licenses: Get License SPDXId from GitLab API by @raghavkaul in #3413
  • 🐛 License: npe by @raghavkaul in #3500
• Security Policy:
  • 🐛 The Security-Policy check will no longer print to the log if the org's .github repo is empty by @spencerschrock in #3433
• Pinned-Dependencies:
  • 🐛 Add go installs to Pinned-Dependencies score by @gabibguti in #3424
• Fuzzing:
  • 🐛 GitLab: Fix URI() used for OSS-Fuzz detection by @raghavkaul in #3477
  • 🐛 Fix parsing OSSFuzz project repos with subfolders and capitalization. by @spencerschrock in #3364
• Misc:
  • 🐛 Print Info in Empty Repo Scans by @leec94 in #3426
  • 🐛 Set repo commit SHA in results after fetching successfully. by @spencerschrock in #3514
  • 🐛 Fix loop aliasing errors. by @spencerschrock in #3414

Docs

• 📖 Added CDLA data license for the API to the README by @david-a-wheeler in #3404
• 📖 Update bestpractices links by @fredgan in #3448
• 📖 Add webviewer link by @olivekl in #3490
• 📖 Add gitlab links to viewer example by @olivekl in #3494
• 📖 Update docs for Signed-Releases check by @raghavkaul in #3469
• 📖 Fix documentation typos by @omahs in #3505

New Contributors

• @joshgc made their first contribution in #3423
• @AdamKorcz made their first contribution in #3449
• @DavidKorczynski made their first contribution in #3454
• @afmarcum made their first contribution in #3455
• @fredgan made their first contribution in #3448
• @omahs made their first contribution in #3505
• @secustor made their first contribution in #3506

Full Changelog: v4.12.0...v4.13.0

v4.12.0

This version of Scorecard supports GitLab repos by default.

This release also adds preliminary support for the scdiff command which can be used to compare changes in Scorecard scores for a repository between versions of Scorecard, as well as probe support for the Security-Policy check.

Finally, this release fixes scoring issues in the Branch-Protection and Pinned-Dependencies checks.

What's Changed

WIP

• ✨ GitLab: Release by @raghavkaul in #3340
• ✨ [experimental] Probe support for security policy check by @laurentsimon in #3241

Bug Fixes

• 🐛 Fix Branch-Protection scoring by @gabibguti in #3251
• 🐛 Forgive job-level permissions by @pnacht in #3162
• 🐛 Add npm installs to Pinned-Dependencies score by @gabibguti in #2960

Docs

• 📖 Add release process by @spencerschrock in #3322
• 📖 Update GitHub documentation links by @martincostello in #3318
• 📖 Fixed slack badge on README by @eddie-knight in #3311
• 📖 update docs for webhooks documentation by @leec94 in #3299
• 📖 Add contributor ladder by @pnacht in #3246
• 📖 Suggest new score viewer on badge documentation by @diogoteles08 in #3268
• 📖 Update Branch-Protection admin and non-admin requirements by @gabibguti, @pnacht in #2772

New Contributors

• @ajmalab made their first contribution in #3248
• @eustas made their first contribution in #3267
• @martincostello made their first contribution in #3318
• @thepwagner made their first contribution in #3327
• @aaguiarz made their first contribution in #3337

Full Changelog: v4.11.0...v4.12.0

v4.11.0

What's Changed

New

• ✨ Consider haskell-actions/hlint-scan a code scanning action by @chungyc in #2846
• ✨ Detect fuzzing in Haskell by the presence of property tests. by @chungyc in #2843
• ✨ The SAST check will look for workflows with the "github/codeql-action/analyze" action locally instead of the GitHub Search API endpoint by @spencerschrock in #2839
• ✨ Scorecard checks for unpinned dependencies that are retrieved ad-hoc using nuget and dotnet CLIs ("nuget install" and "dotnet add") by @balteravishay in #2779
• ✨ show non-compliant code changes for CI-Tests, Code-Review and SAST checks in --show-details mode by @ashishkurmi in #2835
• ✨ Detect semantic-release as a packaging workflow by @travi in #2964
• ✨ Detect semantic-release as a releasing workflow by @travi in #2989
• ✨ Add support for github GHES by @patelniketm in #2999 and @rajbos in #2788
• ✨ Detect fast-check PBT library for JavaScript Fuzzing by @dubzzz in #3073
• ✨ Run Scorecard on packages hosted at Nuget.org using --nuget=<package>by @balteravishay in #3020

Bug Fixes

• SAST
  • 🐛 Reset stored error when handler is re-inited or setup is re-run. by @spencerschrock in #2893
  • 🐛 Add nil check before accessing a step's uses value. by @spencerschrock in #2935
• Vulnerabilities
  • 🐛 Give inconclusive Vulnerabilities score when osv-scanner panics by @spencerschrock in #2896
  • 🐛 Update osv-scanner dependency to include Vulnerabilities check fixes by @laurentS in #2981
• Pinned-Dependencies
  • 🐛 Pip installs count for Pinned-Dependencies score by @gabibguti in #2922
• Code-Review
  • 🐛 Code Review: Use proportional scoring by @raghavkaul in #2882

Deprecations

• 🌱 Deprecate dependencydiff package by @naveensrinivasan in #3125

GitLab support (WIP)

• ✨ GitLab: Documentation and cleaner errors by @raghavkaul in #2821
• ✨ Gitlab: CI-Tests check by @raghavkaul in #2833
• ✨ Gitlab: Maintained check by @raghavkaul in #2860
• ✨ Enable gitlab Packaging Reporting by @jimrobison in #2941
• ✨ GitLab: Code Review check by @raghavkaul in #2764
• ✨ Gitlab: License check by @raghavkaul in #2834
• 🐛 Gitlab: Commit/Commitor Exceptions by @jimrobison in #3026
• 🐛 Gitlab: test fixes by @raghavkaul in #3027
• ✨ Gitlab: Add projects to cron by @raghavkaul in #2936
• 🐛 GitLab cron: rename by @raghavkaul in #3070
• 🐛 Gitlab status updates by @jimrobison in #3052
• ✨ GitLab: enable more checks in cron by @raghavkaul in #3097
• ✨ GitLab: Add 5000 repos to nightly worker run by @raghavkaul in #3137

Docs

• 📖 Update usage message of the scorecard --verbosity flag by @andrelmbackman in #3190
• 📖 Update checks.md to show the benefit of >=2 reviewers by @joycebrum in #3013
• 📖 Add new frequently asked question to FAQ by @joycebrum in #2923
• 📖 Adds zoom link and agenda link by @hythloda in #3050
• 📖 Tweak Best Practices badge description to clarify things by @david-a-wheeler in #2907
• 📖 Clarify that AI/ML doesn't count as human code review by @david-a-wheeler in #2953
• 📖 Change Facilitators to Maintainers by @jeffmendoza in #3039
• 📖 Make all StepSecurity app endpoint references consistent by @ashishkurmi in #3042
• 📖 Fix broken links in FAQ. by @chungyc in #2858
• 📖 Capitalize proper nouns like Dependabot, Renovate, and GitHub by @leec94 in #2962
• 📖 Fix anchor link to the code review section by @dasfreak in #3058

New Contributors

• @chungyc made their first contribution in #2846
• @ashishkurmi made their first contribution in #2835
• @leec94 made their first contribution in #2962
• @jimrobison made their first contribution in #2941
• @travi made their first contribution in #2964
• @laurentS made their first contribution in #2981
• @patelniketm made their first contribution in #2999
• @rajbos made their first contribution in #2788
• @hythloda made their first contribution in #3050
• @dasfreak made their first contribution in #3058
• @dubzzz made their first contribution in #3073
• @andrelmbackman made their first contribution in #3190

Full Changelog: v4.10.5...v4.11.0

v4.10.5

Changelog

Bug fixes

• Fixed a bug which resulted in increased API usage when running the SAST check with a Personal Access Token
  • 🐛 Pass proper commit depth to github checkrun handler. by @spencerschrock in #2777

GitLab support (WIP)

• ✨ GitLab: Security Policy check by @raghavkaul in #2754

Full Changelog: v4.10.4...v4.10.5

Thanks for all contributors!