Skip to content

Add scoring doc, update readme - #83

Open
omkar-foss wants to merge 1 commit into
chaoss:mainfrom
omkar-foss:update-readme
Open

Add scoring doc, update readme#83
omkar-foss wants to merge 1 commit into
chaoss:mainfrom
omkar-foss:update-readme

Conversation

@omkar-foss

Copy link
Copy Markdown
Contributor

Description
Add scoring doc, update readme

This PR closes #66, #69. By addition of SCORING.md, it potentially also resolves #49.

Notes for Reviewers

Signed commits

  • Yes, I signed my commits.

Generative AI disclosure

Please select one option:

  • This contribution was NOT assisted or created by Generative AI tools.
  • This contribution was assisted or created by Generative AI tools.

If AI tools were used, please provide details below:
- What tools were used?
- How were these tools used?
- Did you review these outputs before submitting this PR?

Signed-off-by: Omkar P <45419097+omkar-foss@users.noreply.github.com>

@andrew andrew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • The confidence ranges in SCORING.md do not match the implementation. A score of 0 has confidence none, not low, and scores above 100 remain high. Please document the ranges as none = 0, low = 1–30, medium = 31–70, and high = 71+. The related issue link on line 3 also points to the old repository and returns 404.

  • Issue #66 asks for CLI examples and this PR says it closes that issue, but SCORING.md only contains a commit example. Please add examples showing the default scoring output and how to use --confidence-levels.

  • The detector summary says both Co-Authored-By and Assisted-By require known AI tool emails. Only Co-Authored-By checks known emails. Assisted-By accepts any non-empty tool name, so please describe these separately.

Comment thread SCORING.md

**Total score: 75 + 0 + 0 + 0 + 20 = 95 pts**

95 pts lies in 71 to 100 range, so it falls in confidence level here is **high**.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have "low/med/high" if we are also providing the numerical value?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've had a filter report by confidence feature (this one) which depends on confidence

Comment thread SCORING.md

## Simple additive scoring

We use simple additive scoring per detector to compute the final score. Steps are as follows:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

additive scoring is subject to drift as new detection methods are added, especially if this is the numeric system we are exposing to users and downstream applications that expect stability

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had added weight average scoring as discussed here, but we settled on additive scoring (SpamAssassin style), see this review.

@MoralCode MoralCode added this to the 1.0 milestone Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

Document numeric scoring in readme with examples Document structure and purpose of detectors in README

3 participants