Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added Pattern test cases for detectors #3354

Merged
merged 5 commits into from
Oct 2, 2024
Merged

Added Pattern test cases for detectors #3354

merged 5 commits into from
Oct 2, 2024

Conversation

kashifkhan0771
Copy link
Contributor

@kashifkhan0771 kashifkhan0771 commented Oct 1, 2024 •
edited
Loading

Description:

This PR adds pattern test cases for some detectors. It also fix some regex of detectors.

To Test all Pattern Test Cases, run following command:

go test ./pkg/detectors/... -run "^*Pattern$"

Output:
pattern_tests

Checklist:

  • Tests passing (make test-community)?
  • Lint passing (make lint this requires golangci-lint)?

kashifkhan0771
kashifkhan0771 commented Oct 1, 2024
View reviewed changes
pkg/detectors/agora/agora_test.go
"github.com/trufflesecurity/trufflehog/v3/pkg/pb/detectorspb"
)

func TestAgora_Pattern(t *testing.T) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Agora detector key and secret have same pattern and because of that it actually picks same string twice. Is it expected or we need to add some logic to handle this. You can see in test cases that the want is actually a duplicate string.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kashifkhan0771 great catch. Because having a secret and key as the same string should be impossible, we should include some logic to detect if secret==key and skip that case.

@kashifkhan0771
Copy link
Contributor Author

kashifkhan0771 commented Oct 1, 2024 •
edited
Loading

adafruitio detector is not letting me add a Pattern test cases. It says cannot push secrets (Even though these are just placeholders for testing not real secrets) 🥲. Any ideas?

@rgmz
Copy link
Contributor

rgmz commented Oct 1, 2024

@kashifkhan0771 I'd recommend using #3274 as a base. Right now these tests don't run unless the -detectors flag is specified.

@kashifkhan0771
Copy link
Contributor Author

kashifkhan0771 commented Oct 1, 2024 •
edited
Loading

@kashifkhan0771 I'd recommend using #3274 as a base. Right now these tests don't run unless the -detectors flag is specified.

Thanks @rgmz - That is a good approach. I'll use that as a base and structure these unit test cases accordingly.

@CLAassistant
Copy link

CLAassistant commented Oct 1, 2024 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@kashifkhan0771
Copy link
Contributor Author

@rgmz I made the changes according to your PR but I kept the base branch same.

@rgmz
Copy link
Contributor

rgmz commented Oct 1, 2024

Yes, sorry I meant "based on" not "as the PR merge base".

kashifkhan0771
kashifkhan0771 commented Oct 2, 2024
View reviewed changes
pkg/detectors/airship/airship.go
@@ -21,7 +21,7 @@ var (
client = common.SaneHttpClient()

// Make sure that your group is surrounded in boundary characters such as below to reduce false positives.
keyPat = regexp.MustCompile(detectors.PrefixRegex([]string{"airship"}) + `\b([0-9Aa-zA-Z]{91})\b`)
keyPat = regexp.MustCompile(detectors.PrefixRegex([]string{"airship"}) + `\b([0-9a-zA-Z]{91})\b`)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the regex for airship detector 🕵🏻‍♂️

kashifkhan0771
kashifkhan0771 commented Oct 2, 2024
View reviewed changes
pkg/detectors/alegra/alegra.go
@@ -24,7 +24,7 @@ var (

// Make sure that your group is surrounded in boundary characters such as below to reduce false positives.
keyPat = regexp.MustCompile(detectors.PrefixRegex([]string{"alegra"}) + `\b([a-z0-9-]{20})\b`)
idPat = regexp.MustCompile(detectors.PrefixRegex([]string{"alegra"}) + `\b([a-zA-Z0-9.-@]{25,30})\b`)
idPat = regexp.MustCompile(detectors.PrefixRegex([]string{"alegra"}) + `\b([a-zA-Z0-9\.\-\@]{25,30})\b`)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old regex was not accepting special character - because we were not escaping special characters. Fixed it and added a test case for it as well

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's typically safest to put - at the end for this reason. It's an insidious bug.
https://web.archive.org/web/20240525200717/https://pboyd.io/posts/comma-dash-dot/

@kashifkhan0771 kashifkhan0771 marked this pull request as ready for review October 2, 2024 07:39
@kashifkhan0771 kashifkhan0771 changed the title Added Pattern test cases for detectors [WIP] Added Pattern test cases for detectors Oct 2, 2024
ahrav
ahrav approved these changes Oct 2, 2024
View reviewed changes
Copy link
Collaborator

@ahrav ahrav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm 👍 Looks great!

@ahrav ahrav merged commit a4cc5f7 into trufflesecurity:main Oct 2, 2024
12 checks passed
@blsaccess blsaccess mentioned this pull request Oct 8, 2024
Merged
abmussani added a commit to abmussani/trufflehog that referenced this pull request Oct 9, 2024
@abmussani
Merge branch 'main' into fix-privatelink-snowflake-detector
1276dde 
* main: (79 commits)
  Log skipped files on debug level (trufflesecurity#3383)
  build: update retracted bluemonday ver (trufflesecurity#3369)
  Fix git binary handling and add a smoke test (trufflesecurity#3379)
  fix(deps): update module google.golang.org/protobuf to v1.35.1 (trufflesecurity#3382)
  Added Cisco Meraki API Key detector (trufflesecurity#3367)
  improved the agora detector (trufflesecurity#3360)
  fix(deps): update module github.com/xanzy/go-gitlab to v0.110.0 (trufflesecurity#3376)
  fix(deps): update golang.org/x/exp digest to 225e2ab (trufflesecurity#3371)
  fix(deps): update module golang.org/x/net to v0.30.0 (trufflesecurity#3373)
  fix(deps): update module golang.org/x/crypto to v0.28.0 (trufflesecurity#3372)
  chore(deps): update sigstore/cosign-installer action to v3.7.0 (trufflesecurity#3368)
  fix(deps): update module cloud.google.com/go/storage to v1.44.0 (trufflesecurity#3366)
  fix(deps): update module github.com/schollz/progressbar/v3 to v3.16.1 (trufflesecurity#3365)
  [refactor] - Decouple Metrics From Cache Implementation (trufflesecurity#3355)
  fix(deps): update module github.com/snowflakedb/gosnowflake to v1.11.2 (trufflesecurity#3363)
  Updated Cosign Install URL (trufflesecurity#3364)
  fix(deps): update module github.com/jedib0t/go-pretty/v6 to v6.6.0 (trufflesecurity#3361)
  Added Pattern test cases for detectors (trufflesecurity#3354)
  remove size check (trufflesecurity#3351)
  fix(deps): update module go.mongodb.org/mongo-driver to v1.17.1 (trufflesecurity#3357)
  ...

# Conflicts:
#	go.sum
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zricethezav zricethezav zricethezav left review comments

@rgmz rgmz rgmz left review comments

@ahrav ahrav ahrav approved these changes

Assignees
No one assigned
Labels
hacktoberfest-2024-internal
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@kashifkhan0771 @rgmz @CLAassistant @zricethezav @ahrav