scancode can not find "license agreement" if there is not a comment with "code" in the file

[Stretchdude]

Description

I have a cuple of C-Header files from a third party library. We have a license agreement and I want to use ORT for documentation. Scancode detects some of the header files, as expected with a unknown license. Other files are not detected. In most of the files is exactly the same notice, similar to:

// ---------------------------------------------------------------------------
//               Some produkt - (C) some company
// ---------------------------------------------------------------------------
// This software is supplied under the terms of a license agreement or
// nondisclosure agreement with MYSTERY BRAND and may not be copied or
// disclosed except in accordance with the terms of that agreement.
// ---------------------------------------------------------------------------

The files that were license_detection'ed match "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/license-intro_22.RULE"
The files not detected produce a license_clue for "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/proprietary-license_782.RULE"
I see some words matching the second one. It looks to me, as if 782 is overriding 22 and than it is thrown away because not enought words are matching. Just a guess... It tried to enable logging but I can't find my way to the failing line.
Funny think: If I put a comment line containing "code" somewhere in line 21 of one of the files it will be detected by rule 22 but i can not reproduce this with the example files.

How To Reproduce

I will attach a small example, with some files containing the notice and some dummy code.

scancode -clip --json-pp - ~/dummy/

System configuration

For bug reports, it really helps us to know:

• What OS are you running on? (Windows/MacOS/Linux)
  ubuntu-jammy in a WSL on windows 11
• What version of scancode-toolkit was used to generate the scan file?
  ScanCode version: 32.3.1
  ScanCode Output Format version: 4.0.0
  SPDX License list version: 3.25
• What installation method was used to install/run scancode? (pip/source download/other)
  some custom steps as part of the toolchain we use. It works fine on many packages.

[Stretchdude]

This is a tar archive. I'm not allowed to upload. rename to dummy.tar and unpack :-(

[pombredanne]

@Stretchdude Thanks. I know zip work too, or you can just copy paste the content

[Stretchdude]

Okay, zip then...
I must state, that this is not the exact behavior, as in my real example, which I am not allowed to share. But one can see, that the license-agreement is matches and kind of boosted by a string with "gpl" (=> false positive, on purpose). Then it is a match (for rule-22) and otherwise it's just a clue (for rule-782) and ignored by the ORT toolchain later.
dummy.zip
content is always a variation of somethin like

// ---------------------------------------------------------------------------
// Some produkt - (C) some company
// ---------------------------------------------------------------------------
// This software is supplied under the terms of a license agreement or
// nondisclosure agreement with MYSTERY BRAND and may not be copied or
// disclosed except in accordance with the terms of that agreement.
// ---------------------------------------------------------------------------

int functi(char* func);
int funtio();
void nothing(void);

output snip:

...
"license_detections": [
{
"identifier": "gpl_1_0_plus-2c632f09-4da5-a664-a4a9-c78c55d33466",
"license_expression": "gpl-1.0-plus",
"license_expression_spdx": "GPL-1.0-or-later",
"detection_count": 1,
"reference_matches": [
{
"license_expression": "unknown-license-reference",
"license_expression_spdx": "LicenseRef-scancode-unknown-license-reference",
"from_file": "dummy/file6.h",
"start_line": 4,
"end_line": 4,
"matcher": "2-aho",
"score": 90.0,
"matched_length": 2,
"match_coverage": 100.0,
"rule_relevance": 90,
"rule_identifier": "license-intro_22.RULE",
"rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/license-intro_22.RULE"
},
{
"license_expression": "gpl-1.0-plus",
"license_expression_spdx": "GPL-1.0-or-later",
"from_file": "dummy/file6.h",
"start_line": 16,
"end_line": 16,
"matcher": "2-aho",
"score": 50.0,
"matched_length": 1,
"match_coverage": 100.0,
"rule_relevance": 50,
"rule_identifier": "gpl_bare_word_only.RULE",
"rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/gpl_bare_word_only.RULE"
}
]

...
vs.

...
"license_clues": [
{
"license_expression": "proprietary-license",
"license_expression_spdx": "LicenseRef-scancode-proprietary-license",
"from_file": "dummy/file1.h",
"start_line": 4,
"end_line": 6,
"matcher": "3-seq",
"score": 10.47,
"matched_length": 9,
"match_coverage": 10.47,
"rule_relevance": 100,
"rule_identifier": "proprietary-license_782.RULE",
"rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/proprietary-license_782.RULE"
}
],