MISDETECTION: X11 detected as X11-distribute-modifications-variant

[eclipseo]

Description

Ok I have a misdetection of this header:

-- Simple LÖVE Audio Manager
--
-- Copyright (c) 2011 Matthias Richter
-- Permission is hereby granted, free of charge, to any person obtaining a copy
-- of this software and associated documentation files (the "Software"), to deal
-- in the Software without restriction, including without limitation the rights
-- to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-- copies of the Software, and to permit persons to whom the Software is
-- furnished to do so, subject to the following conditions:
--
-- The above copyright notice and this permission notice shall be included in
-- all copies or substantial portions of the Software.
--
-- Except as contained in this notice, the name(s) of the above copyright holders
-- shall not be used in advertising or otherwise to promote the sale, use or
-- other dealings in this Software without prior written authorization.
--
-- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-- IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-- AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-- OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
-- THE SOFTWARE.
--

coming from https://github.com/SimonLarsen/90-Second-Portraits/blob/master/slam.lua
and the file from this subproject https://github.com/vrld/hump/tree/master

The file are detected as:

X11-distribute-modifications-variant

90-Second-Portraits-1.01b-16.fc38.src.rpm-extract/90-Second-Portraits-1.01b.zip-extract/slam.lua
90-Second-Portraits-1.01b-16.fc38.src.rpm-extract/90-Second-Portraits-1.01b.zip-extract/hump/camera.lua
90-Second-Portraits-1.01b-16.fc38.src.rpm-extract/90-Second-Portraits-1.01b.zip-extract/hump/class.lua
90-Second-Portraits-1.01b-16.fc38.src.rpm-extract/90-Second-Portraits-1.01b.zip-extract/hump/gamestate.lua
90-Second-Portraits-1.01b-16.fc38.src.rpm-extract/90-Second-Portraits-1.01b.zip-extract/hump/README.md
90-Second-Portraits-1.01b-16.fc38.src.rpm-extract/90-Second-Portraits-1.01b.zip-extract/hump/signal.lua
90-Second-Portraits-1.01b-16.fc38.src.rpm-extract/90-Second-Portraits-1.01b.zip-extract/hump/timer.lua
90-Second-Portraits-1.01b-16.fc38.src.rpm-extract/90-Second-Portraits-1.01b.zip-extract/hump/vector-light.lua
90-Second-Portraits-1.01b-16.fc38.src.rpm-extract/90-Second-Portraits-1.01b.zip-extract/hump/vector.lua

with a 99% score:

    {
      "path": "90-Second-Portraits-1.01b-16.fc38.src.rpm-extract/90-Second-Portraits-1.01b.zip-extract/slam.lua",
      "type": "file",
      "name": "slam.lua",
      "base_name": "slam",
      "extension": ".lua",
      "size": 4956,
      "date": "2016-01-02",
      "sha1": "18457ec5bc4b19c6fd9bf1cc5207c5ed6d4a1548",
      "md5": "2e8516f46d7d60542f18c64b09ffe154",
      "sha256": "8322ddffba2f7d13564cba9f9b7178415201ccc3db47117910f31743d53c0cf4",
      "mime_type": "text/plain",
      "file_type": "ISO-8859 text",
      "programming_language": "Lua",
      "is_binary": false,
      "is_text": true,
      "is_archive": false,
      "is_media": false,
      "is_source": true,
      "is_script": false,
      "package_data": [],
      "for_packages": [],
      "detected_license_expression": "x11-fsf",
      "detected_license_expression_spdx": "X11-distribute-modifications-variant",
      "license_detections": [
        {
          "license_expression": "x11-fsf",
          "matches": [
            {
              "score": 99.0,
              "start_line": 4,
              "end_line": 24,
              "matched_length": 198,
              "match_coverage": 100.0,
              "matcher": "2-aho",
              "license_expression": "x11-fsf",
              "rule_identifier": "x11-fsf_2.RULE",
              "rule_relevance": 99,
              "rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/x11-fsf_2.RULE"
            }
          ],
          "identifier": "x11_fsf-02cfe869-98d5-22f3-6b3c-9902b0213ae1"
        }
      ],

output4.txt

If we go to https://spdx.org/licenses/X11-distribute-modifications-variant.html

The test is:

Text

Copyright (c) <year> <copyright holders>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, distribute with modifications, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE ABOVE COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Except as contained in this notice, the name(s) of the above copyright holders shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Software without prior written authorization.

We explicitly don't have the "distribute with modifications" here, so this is instead a standard X11 license:

Copyright (C) 1996 X Consortium

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE X CONSORTIUM BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Except as contained in this notice, the name of the X Consortium shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Software without prior written authorization from the X Consortium.

Environment:

• Python 3.11
• Fedora 38
• scancode-toolkit 32.0.6 from prebuilt archive, extracted

Command:

./extractcode  /home/bob/temp/90-Second-Portraits
./scancode -clpieu --json-pp output4.json /home/bob/temp/90-Second-Portraits/90-Second-Portraits-1.01b-16.fc38.src.rpm-extract/

[eclipseo]

It is correctly detected by the x11-fsf_2.RULE, with "license_expression": "x11-fsf", but for some reason the SPDX identifier is converted to X11-distribute-modifications-variant.

In the rules, https://github.com/search?q=repo%3AnexB%2Fscancode-toolkit+x11-fsf+path%3Asrc%2Flicensedcode%2Fdata%2Frules%2F&type=code

we have both rules that detect with "distribute with modifications" and without, but they both have the same license_expression: x11-fsf. And x11-fsf is always converted to SPDX X11-distribute-modifications-variant.

So I guess we need separate keys for these.

[AyanSinhaMahapatra]

we have both rules that detect with "distribute with modifications" and without

This indeed seems to be an issue, the x11-fsf_2.RULE which is detected here (and others rules of x11-fsf) seems to not be x11-fsf but rather x11-xconsortium since as you rightly pointed out, distribute with modifications is not present in this rule and some others, and this seems to be the primary distinction between these licenses.

I can take a look into all the rules for these two licenses (does not seem to be many of them) and update them based on the conclusions above. But just to be sure that is the correct thing to do, @DennisClark could you take a look too? 😊 Thanks!

[DennisClark]

when the license text being examined has merge, publish, distribute, sublicense, and/or sell... rather than merge, publish, distribute, distribute with modifications, sublicense, and/or sell... we should definitely be returning LicenseDB key x11-consortium (which has SPDX identifier X11).

Note that, especially on really old licenses such as these, the LicenseDB key and the SPDX identifier are often quite different. We are stuck with that situation for historical reasons.

https://scancode-licensedb.aboutcode.org/x11-xconsortium.html

https://scancode-licensedb.aboutcode.org/x11-fsf.html

[AyanSinhaMahapatra]

Thanks @DennisClark , I'll update the rules.

Note that, especially on really old licenses such as these, the LicenseDB key and the SPDX identifier are often quite different. We are stuck with that situation for historical reasons.

Yup, lots of examples like this one, where the SPDX license was added later and we had a license and a (different from the SPDX one) key from much before. Everytime we add support for the a new spdx version we get these 😅

[pombredanne]

@eclipseo thanks for the report!
Just a side note that can help you investigate similar issues in the future.
I use this set of command line options:
scancode -l --license-text --license-text-diagnostics --yaml - <path to file to scan> to investigate license detection issues and get the results out on screen, with a readable YAML formatting, including "highligthing" the differences using [square brackets].

[AyanSinhaMahapatra]

this is fixed, closing! thanks @eclipseo