Add Librem boards inside of public CIs

[tlaurion]

@MrChromebox

Any reason the Librem boards (which need to stay on 4.8.1 for now) aren't being built (without blobs) as part of CI testing?

Added librems for testing under CircleCI tlaurion@d643838

But the build will obviously fail because blobs are not provided in Heads's tree:
https://github.com/osresearch/heads/blob/master/config/coreboot-librem13v2.config#L13-L23
https://github.com/osresearch/heads/blob/master/config/coreboot-librem13v4.config#L13-L23
https://github.com/osresearch/heads/blob/master/config/coreboot-librem15v3.config#L13-L23
https://github.com/osresearch/heads/blob/master/config/coreboot-librem15v4.config#L13-L23

So I will remove those boards in next commit once that point is proven with current board+coreboot configs, which is why the x220 and t420 (which have blobs/* corresponding directories) are not built from CIs since not blob free as of now.
https://github.com/osresearch/heads/blob/master/config/coreboot-x220.config#L13-L18
https://github.com/osresearch/heads/blob/master/config/coreboot-t420.config#L13-L18

You can join forces #307, that ticket would feel less lonely.

Originally posted by @tlaurion in #721 (comment)

[tlaurion]

One idea here would be to duplicate boards similarly to the x230-flash board, leaving ME GBE and IFD and other blobs regions untouched and as the responsibility of the external reprogrammer to extract the blobs from backup (following blobs/$BOARD/instructions) for the initial external flashing, while CIs builds a fake IFD based full image (x230) matching combined SPI full size flash size for flashrom to not complain and for flashrom to only care about the BIOS region inside of it, flashed internally per $BOARD flashrom statement.

Else #307 is the way...

[MrChromebox]

what are your thoughts on building blob-less for CI, simply to ensure all paths are build-tested? Would require separate configs obviously but not really an issue

[tlaurion]

@MrChromebox concern is to have blobs in path and legal issues linked to hosting them in github, hence #307

[MrChromebox]

I feel like we shouldn't be bothering with non-publicly available blobs for CI purposes. We've spent so much effort on trying to come up with a reproducible procedure for the Thinkpads when we could have just documented / scripted the process for users to read their own firmware, clean the ME, modify the IFD, etc.

[tlaurion]

@MrChromebox : the end result being a flashable BIOS region in that case (as opposed to a full ROM that could be updated per fwupd), just like the x230 board is currently (with faked IFD, without GBE nor ME, considering its there on SPI else #700 happens, without the capability of using all ME freed space cause IFD is not unlocked, BIOS region is not extended and ME is not reduced.

Else I do not understand what would be your plan here, which debate should be under #307 not here, else I do not understand what would be CI reproducible, considering all roms being at full capacity, which is what #307 is all about. So coreboot configs need to specify maximum BIOS flashable region for each board, which requires prior end user knowledge, tools, precompilation of ifd or static tools deployement... We expect the user to be knowledgeable.

The proof here being for the x230 is need to augment CBFS region to 7100000 to buils coreboot 4.12, which is the absolute limit without taking changing our approach. Addition of other tools is now impossible, where x220 and t420 are currently (untested) left behind without #307 unless #590 or #730 or better ideas.

Else what you propose is actually to have, per board, maximized CBFS size under coreboot configs (considering ME cleaning and region reduction has already been applied and IFD unlocked+expended for BIOS region), while those coreboot configurations not including any blobs in coreboot configs are required to be CI's buildable.

So basically, two board configs per physical boards:

• boards/$BOARD-external. With prior blobs/$BOARD user applied instructions/automated script for end user to backup, unlock ifd, extract blobs and reflash externally produced ROMs, resulting in boards/$BOARD-initial ROM local build with blobs injected in ROM per previous instructions to be flashed once/reflashed on ME update by external reprogrammer, the automated script taking into consideration a single programmer to use (ch341a?) doing the whole process automatically with proper failsafes since the user cannot simply flash top.rom and bottom rom images.
• boards/$BOARD-internal, not including any blob references in coreboot config and respecting SPI IFD definition to flash only its defined BIOS region (previously defined by $BOARD-initial board) and CIs reproducible.

That would be the only other alternative I see. Other ideas?

[MrChromebox]

So basically, two board configs per physical boards:

I think that's a perfect solution for boards without publicly available blobs. Wouldn't be required for Librems though since all blobs are available publicly via our purism-blobs repo

[tlaurion]

@MrChromebox

I think that's a perfect solution for boards without publicly available blobs. Wouldn't be required for Librems though since all blobs are available publicly via our purism-blobs repo

Then if it the case, I do not understand why they could not be automatically downloaded to have those boards built directly by CIs and be totally reproducible... And you are actually agreeing with #307! Download a rom/backup a rom to extract blobs? Ask the user to apply a script manually when the blobs are hosted instead of downloading it per build process? I'm sorry I do not understand what we are doing/asking users to do if the goal is to make this security accessible.

If the blobs can be downloaded (ME is not a problem for neither xx20 xx30 nor Librems, IFD is ok for all platforms to be hosted on osresearch's repo, and GBE could be untouched/generated per bincfg while being hosted until reprimanded per fair use... ) then Heads should provide them by downloading them/hosting them and produce full ROM images to be fwupd updateable directly from CIs.

[MrChromebox]

Then if it the case, I do not understand why they could not be automatically downloaded to have those boards built directly by CIs and be totally reproducible

once the boards move to coreboot 4.12 and the librem-blobs module is added, then sure. Until then, it would still be useful to have blob-free versions build tested, no?

If the blobs can be downloaded (ME is not a problem for neither xx20 xx30 nor Librems, IFD is ok and GBE could be untouched/generated per bincfg) then Heads should provide them by downloading them/hosting them and produce full ROM images to be fwupd updateable directly from CIs.

I thought the whole point of this was to avoid hosting the ME for 20/30-series boards. If you're going to host it, then just host the cleaned blob and modified IFD and call it a day. Reuse the user's existing GBE so they don't lose their MAC address, easy to automate that

[tlaurion]

I thought the whole point of this was to avoid hosting the ME for 20/30-series boards. If you're going to host it, then just host the cleaned blob and modified IFD and call it a day. Reuse the user's existing GBE so they don't lose their MAC address, easy to automate that

This is #797 (downloading ME update capsule, extracting exe with innoextract, and then applying me_cleaner on top of it)

I have no idea how to automate calling external tools for archives downloaded by Make scripts which cannot be extracted per normal tools (not tar etc) which needs to be processed by cabextract/innoextract). Once Lenovo downloaded ME blobs are downloaded for xx20 and xx30 platform, if we host the ifd on heads repository (@osresearch has nothing against it) we can have maximal usable space with ME in ROM. Best would be to generate GBE with static MAC #796 (so bit by bit reproducible) but could be postponed and gbe also included with sacrificed mac address, or as you noted, not include GBE and expect user to have it still on (while rom not totally flashable: so BOARD config would need to specify which regions to flash: ME, BIOS..) Best would be the whole flash, once again, with the OS responsible to randomize mac address.

Note that those issues have bounties tag on them. If you have time are interested in those, feel free to propose a quote on those issues directly and you will receive funds upon PoW.