compute checksum of binary downloads

[hohwille]

With epic #941 we are redesigning our downloads and introducing ide-urls repo.
The idea of this story is to improve security and prevent man-in-the-middle or other attacks that may lead to downloads of evil content and therefore remote-code-execution.
To archive this, we will introduce checksum files (e.g. sha or md5) for all our downloads in ide-urls.
So in addition to downloading the actual package, we would also download the corresponding checksum file.
Then we would compute the checksum locally using an according tool (e.g. sha256sum or md5sum) and compare the result with the content of the downloaded checksum file.
If they match, we continue as before, but otherwise, we would fail with an according error and a link with further instructions (AsciiDoc file with hints and link to report a security issue so we can also check if the vendor has actually changed the binary after releasing on purpose what is an anti-pattern but might also happen in rare cases).

NOTE: even though the feature is released with 2023.04.001 the actual verification of checksums is not yet implemented and will come with a following release. To track all this watch the epic #941.

[hohwille]

For the record: Maven is already following this best practice approach as you can see here:

• There is devonfw-ide-scripts-2023.01.001.tar.gz
• But also there are the checksum files devonfw-ide-scripts-2023.01.001.pom.md5 and devonfw-ide-scripts-2023.01.001.pom.sha1.

I would consider sha1 as obsolete and insecure. sha256 is most probably a perfect choice from security point of view but we would have to consider the following:

• is the checksum tool present on every target environment (MacOS, Windows + GitBash, Linux, Github-Actions aka Linux)? I would assume for md5sum this is given, but we need to check this for sha256sum.
• how long will the signature be and how big will our repository grow just due to bigger signatures? If it is just 1kB+ we should be fine, but if we add megabytes, we should be a little bit more concerned and consider a trade-off between security and sustainability.

[alfeilex]

We can probably also use the command shasum -a 256 <file>. This command should be available for all platforms.

I tested that command on

• Windows
• MacOS (m1)
• Linux

[alfeilex]

Currently, there are ~8000 .urls files in ide-urls. If we use something like this as checksum file

ca78815afd657f887de7f9b74014dc4bddffe80fd28028179b271a3c4f64f29a terraform_1.3.9_darwin_amd64.zip

then each of these files has a size of 100bytes. This would increase the size of repository to 781.25KB. That should be fine.