Question for the new ABCD format

[chinyeungli]

The format and some of the keys for the ABOUT file will need to be changed.
For instance,

Current:

license_name: gpl-2.0
license_file: COPYING

ABCD format:

licenses:
    - key: gpl-2.0
      text_file: COPYING

Following are the current supported keys:

    ('about_resource', AboutResourceField(required=True)),
    ('name', SingleLineField(required=True)),

    ('version', SingleLineField()),
    ('download_url', UrlField()),
    ('description', StringField()),
    ('home_url', UrlField()),
    ('notes', StringField()),

    ('license', ListField()),
    ('license_name', StringField()),
    ('license_file', FileTextField()),
    ('license_url', UrlField()),
    ('copyright', StringField()),
    ('notice_file', FileTextField()),
    ('notice_url', UrlField()),

    ('redistribute', BooleanField()),
    ('attribute', BooleanField()),
    ('track_change', BooleanField()),
    ('modified', BooleanField()),

    ('changelog_file', FileTextField()),

    ('owner', ListField()),
    ('owner_url', UrlField()),
    ('contact', ListField()),
    ('author', ListField()),

    ('vcs_tool', SingleLineField()),
    ('vcs_repository', SingleLineField()),
    ('vcs_path', SingleLineField()),
    ('vcs_tag', SingleLineField()),
    ('vcs_branch', SingleLineField()),
    ('vcs_revision', SingleLineField()),

    ('checksum', ListField()),
    ('spec_version', SingleLineField()),

Which of the above keys should fall into the main top level object types:

• Products
• Components
• Packages
• Files
• Parties
• Licenses

I think the about_resource should fall into the Files.
The 3rd session should fall into the Licenses.
The 6th session should fall into the Parties.

[pombredanne]

You wrote:

I think the about_resource should fall into the Files.

correct.

For the rest we should have a major as component. And support for now a single component per file.
So the yaml would start with:

component:
   <..... data>

Now looking at each field:

• 'about_resource', AboutResourceField required=True , --> would become part of the files: key under component:
  and we can have more than one file listed.
  For instance:

    files:
        - path: lxml-3.6.0-cp27-none-win_amd64.whl
        - path: lxml-3.6.0-cp27-none-win32.whl
        - path: lxml-3.6.0.tar.gz

• 'checksum', ListField would become an attribute of a file, the same this is done in ScanCode "info".
  though we would use not checksums but instead sha1:, md5:, etc as keys

For reference, this is what the JSON looks like in ScanCode:

      "path": "scancode-toolkit-master/NOTICE",
      "type": "file",
      "name": "NOTICE",
      "base_name": "NOTICE",
      "extension": "",
      "date": "2017-02-25",
      "size": 2547,
      "sha1": "f9c28fa2714ad0c2e36d3e5561afb0031fa76831",
      "md5": "8aedb84647f637c585e71f8f2e96e5c8",
      "files_count": null,
      "mime_type": "text/plain",
      "file_type": "ASCII text",
      "programming_language": "reStructuredText",

• 'spec_version', SingleLineField would become top level and would need to be renamed to attributecode_version:
  which is not entirely the same as a spec version though... so we may want to rename it attributecode_spec_version: instead
  and introduce a new attributecode_version: with the software version.

• then these become direct attributes under component:

  • 'name', SingleLineField required=True
  • 'version', SingleLineField
  • 'description', StringField
  • 'home_url', UrlField --> to rename to homepage_url
  • 'notes', StringField
  • 'redistribute', BooleanField
  • 'attribute', BooleanField
  • 'track_change', BooleanField
  • 'modified', BooleanField
  • 'changelog_file', FileTextField ... though this could be deprecated/removed?
  • 'copyright', StringField ... only one text, possibly on multiple lines.
  • 'notice_file', FileTextField ... there can be only one
  • 'notice_url', UrlField... there can be only one

• the licensing would come under component/licenses but we likely need to add
  the top level license expression e.g.

component:
    license_expression: xxxx with zzzz
    licenses:
        - <license1...>

The expression will be all that we need in common cases... though when there
are multiple license files we could also list each license. This licenses: would look this way:

    licenses:
        - 
          key: gpl-2.0
          short_name: GPL 2.0
        

Therefore:

• 'license', ListField becomes either multiple items under licenses: or a license_expression:
• 'license_name', StringField becomes an attribute of a licenses: item, renamed to short_name

These two could be either attributes of the top level component: or items under licenses:... this needs thinking and is TBD.

• 'license_file', FileTextField
• 'license_url', UrlField

These would come under component: packages:

• 'download_url', UrlField

And these too:

• 'vcs_tool', SingleLineField
• 'vcs_repository', SingleLineField
• 'vcs_path', SingleLineField
• 'vcs_tag', SingleLineField
• 'vcs_branch', SingleLineField
• 'vcs_revision', SingleLineField

These come under component: parties:

• 'owner', ListField
• 'owner_url', UrlField
• 'contact', ListField
• 'author', ListField

[chinyeungli]

The previous about.ABOUT file

about_resource: .

name: AboutCode
version: 3.0.0.dev1

owner: nexB Inc.
author: Jillian Daguil, Chin Yeung Li, Philippe Ombredanne, Thomas Druez

home_url: http://dejacode.org

vcs_tool: git
vcs_repository: https://github.com/dejacode/about-code-tool.git

description: AboutCode is a tool to process ABOUT files. An ABOUT file
 provides a simple way to document the provenance (origin and license)
 'about' a software component. This is a small text file stored in the
 codebase side-by-side with the documented software component.


license: apache-2.0
license_name: Apache 2.0
license_file: apache-2.0.LICENSE
copyright: Copyright (c) 2013-2014 nexB Inc.

notice_file: NOTICE

The ABC Data Formatted about.ABOUT

component:
    name: AboutCode
    version: 3.0.0.dev1
    description: AboutCode is a tool to process ABOUT files. An ABOUT file provides a simple way to document the provenance (origin and license) 'about' a software component. This is a small text file stored in the codebase side-by-side with the documented software component.
    home_url: http://dejacode.org
    copyright: Copyright (c) 2013-2014 nexB Inc.
    notice_file: NOTICE

    files:
        - about_resource: .

    licenses:
        -
          license_file: apache-2.0.LICENSE
          license: apache-2.0
          license_name: Apache 2.0

    packages:
        vcs_tool: git
        vcs_repository: https://github.com/dejacode/about-code-tool.git

    parties:
        owner: nexB Inc.
        author: Jillian Daguil, Chin Yeung Li, Philippe Ombredanne, Thomas Druez

@pombredanne
Please see if the above format looks fine and is correct.
(There are still things to be done such as changing the about_resource to Path, license to key etc.)

[chinyeungli]

Some of the tests are expected to fail as the format of the generated ABOUT files are different now.

[chinyeungli]

Based on the recent discussion, the ABOUT file can reference both package and component, and one ABOUT file will only reference one "Resource".
If it's a package, the ABOUT file will look like:

about_resource: test.c
about_resource_path: test.c
license_expression: mit AND bsd-new
licenses:
    -   file: mit.LICENSE
        key: mit
        name: MIT License
        url: https://enterprise.dejacode.com/urn/?urn=urn:dje:license:mit
    -   file: bsd-new.LICENSE
        key: bsd-new
        name: BSD-3-Clause
        url: https://enterprise.dejacode.com/urn/?urn=urn:dje:license:bsd-new
name: AboutCode-toolkit

If it's a component, the ABOUT file will look like:

about_resource: boolean.py-3.6-py2.py3-none-any.whl

md5: da39999eb131b589e84ad935dc4ca642
sha1: d31b55e7ad2ee917232b3213afe3ae9678156a9f

download_url: https://files.pythonhosted.org/boolean.py-3.6-py2.py3-none-any.whl

component:
    name: boolean.py
    version: '3.6'
    description: Implements boolean algebra in one module.
    owner: Sebastian Kraemer
    license_expression: bsd-new

copyright: Copyright (c) 2009-2016 Sebastian Kraemer, basti.kr@gmail.com and others

homepage_url: https://github.com/bastikr/boolean.py

license_expression: bsd-simplified
license_file: bsd-simplified.LICENSE

notice_file: boolean.py-3.6-py2.py3-none-any.whl.NOTICE
notice_url: https://github.com/bastikr/boolean.py/blob/master/LICENSE.txt

attribute: true

as described in #340

Closing this now.