Add scoring doc, update readme - #83
Conversation
Signed-off-by: Omkar P <45419097+omkar-foss@users.noreply.github.com>
andrew
left a comment
There was a problem hiding this comment.
-
The confidence ranges in
SCORING.mddo not match the implementation. A score of0has confidencenone, notlow, and scores above100remainhigh. Please document the ranges asnone = 0,low = 1–30,medium = 31–70, andhigh = 71+. The related issue link on line 3 also points to the old repository and returns 404. -
Issue #66 asks for CLI examples and this PR says it closes that issue, but
SCORING.mdonly contains a commit example. Please add examples showing the default scoring output and how to use--confidence-levels. -
The detector summary says both
Co-Authored-ByandAssisted-Byrequire known AI tool emails. OnlyCo-Authored-Bychecks known emails.Assisted-Byaccepts any non-empty tool name, so please describe these separately.
|
|
||
| **Total score: 75 + 0 + 0 + 0 + 20 = 95 pts** | ||
|
|
||
| 95 pts lies in 71 to 100 range, so it falls in confidence level here is **high**. |
There was a problem hiding this comment.
Why do we have "low/med/high" if we are also providing the numerical value?
There was a problem hiding this comment.
We've had a filter report by confidence feature (this one) which depends on confidence
|
|
||
| ## Simple additive scoring | ||
|
|
||
| We use simple additive scoring per detector to compute the final score. Steps are as follows: |
There was a problem hiding this comment.
additive scoring is subject to drift as new detection methods are added, especially if this is the numeric system we are exposing to users and downstream applications that expect stability
There was a problem hiding this comment.
I had added weight average scoring as discussed here, but we settled on additive scoring (SpamAssassin style), see this review.
Description
Add scoring doc, update readme
This PR closes #66, #69. By addition of SCORING.md, it potentially also resolves #49.
Notes for Reviewers
Signed commits
Generative AI disclosure
Please select one option:
If AI tools were used, please provide details below:
- What tools were used?
- How were these tools used?
- Did you review these outputs before submitting this PR?