Add support for CycloneDX Output format

[eastmadc]

Similar to the support of exporting to SPDX output format. It would be helpful to add support to export into the increasingly popular CycloneDX format.

Specification
Example Project

[agschrei]

Having support for this output format would be a great help to anybody that wants to ingest ScanCode results into downstream tools like DependencyTrack or enrich them with data from other scanners/sources.

I'm assuming many people have a business need for this, so I'm leaving this comment here in the hopes that it gains traction.

Also, looking at the plugin architecture, it seems to me like implementing this as a post-scan plugin is possible.
Could anybody with more hands-on experience on ScanCode plugins comment on that perhaps?

[pombredanne]

@eastmadc @agschrei yes that would be fairly straightforward to implement especially since CycloneDX embases Package URL that were born here. The approach is to implement an output plugin like in https://github.com/nexB/scancode-toolkit/blob/develop/src/formattedcode/output_yaml.py
You can see the simple commit stream here #2208 covering the whole thing (or this simple single commit 310f450 )

And this library is likely the way to go too https://github.com/CycloneDX/cyclonedx-python

[pombredanne]

@stevespringett FYI ^ :)

BTW in https://github.com/CycloneDX/cyclonedx-python/blob/master/setup.py there are several deps which are pinned which makes it somewhat impractical to reuse this as a library. I will likely provide a patch for your consideration to makes this easier to reuse.

[pombredanne]

@agschrei Are you willing to chip in to help implement this? I can provide hand holding!

[agschrei]

@pombredanne I'll hopefully find some time this weekend to look into it. I can't guarantee anything just yet because of university exams, but will report back on Monday.

Given you know the codebase like the back of your hand, how do you estimate this feature in terms of effort/time required?

[pombredanne]

I'll hopefully find some time this weekend to look into it. I can't guarantee anything just yet because of university exams, but will report back on Monday.

That's very gentle of you!

Given you know the codebase like the back of your hand, how do you estimate this feature in terms of effort/time required?

The easy part is likely to be the writing of the Cyclone file. The harder may be likely how to align the data with the Cyclone data model: for instance what about file-level details vs. packages? ScanCode collects also a lot of details that would likely not map with Cyclone at all.

Now, say as a first pass that this covers only detected packages (with the --package option) I would estimate this to be about 1 to 2 days work with at least one simple test.

[coderpatros]

@agschrei if you want any advice about aligning the data to the CycloneDX spec feel free to hit me up on the CycloneDX Slack

@pombredanne I'm happy to provide feedback on the implementation too.

[coderpatros]

I've just gone back over this thread and the issue @pombredanne raised regarding pinned dependency versions in the CycloneDX Python implementation CycloneDX/cyclonedx-python#211

The Python implementation only supports a subset of the spec. So trying to use it for this might be a hindrance. We of course welcome PRs. Just don't want you to be held up waiting for PRs to be merged when it's so easy to just implement it in Python. Well, JSON format is trivial to implement in Python at least. I haven't implemented XML or Protocol Buffers in Python myself. They should be pretty easy too. But JSON is really easy.

[agschrei]

@coderpatros thanks a ton for the offer, it's helpful to have a point of contact in case I run into trouble.

Concerning the lib, I also noticed that it doesn't provide any support for CycloneDX 1.3 yet, so perhaps also not a good fit in that respect.

I'll have to look into cyclonedx-python to see how much additional work support for revision 1.3 (or rather the parts of it we want/need) requires. Personally, I'd be a lot happier with improving the library rather than rolling our own implementation for the ScanCode plugin, but it's of course also a matter of time and resources.

@pombredanne
what's your preferred flavor of CycloneDX to support here?

I could also introduce two different flags like --cyclonedx for XML and --cyclonedx-json for JSON, similar to what you are offering in the SPDX output plugin.

[pombredanne]

what's your preferred flavor of CycloneDX to support here?

Let's go with JSON for a start. I like its rounded curlies more than XML's pointies.

I could also introduce two different flags like --cyclonedx for XML and --cyclonedx-json for JSON, similar to what you are offering in the SPDX output plugin.

This works fine.

Personally, I'd be a lot happier with improving the library rather than rolling our own implementation for the ScanCode plugin, but it's of course also a matter of time and resources.

Same. I suggest though to start with something concrete here first and this can be extracted and contributed upstream afterwards.

@coderpatros thanks for chiming in !

[coderpatros]

Let's go with JSON for a start. I like its rounded curlies more than XML's pointies.

As good a reason as any XD

[agschrei]

Let's go with JSON for a start. I like its rounded curlies more than XML's pointies.

I can get behind that line of reasoning.

Same. I suggest though to start with something concrete here first and this can be extracted and contributed upstream afterwards.

Sounds good to me, I've already gotten started on some boilerplate, next step is doing the actual mapping to CycloneDX.

[pombredanne]

@agschrei gentle ping :) 'sup on your side? Any help needed?