GPU (and PCIe) Support in Firecracker

[xmarcalx]

Dear All,

Thanks to everyone who attended the first community meeting to discuss GPU support in Firecracker on 2024-10-09.
This meeting was scheduled to kick the project as part of the engagement we had in #1179.

Moving forward, we will track the notes of our community meeting around the topic in this discussion instead of the issue.

Starting from 2024-10-09, we will meet every 4 weeks, Wednesday at 18:00 GMT for up to one hour at:

https://chime.aws/4350163809
Meeting ID: 4350 16 3809
United States Toll-Free (1): +1 855-552-4463
United Kingdom Toll-Free (1): +44 800 085 5175
International: https://chime.aws/dialinnumbers/

The meeting is open to everyone who is interested in this initiative.

On 2024-10-09, we discussed the following points:

1. We are aligned to collaborate together with the community to support PCIe devices and consequentially PCIe-enabled GPUs into Firecracker microVMs.
2. Supporting PCIe passthrough may impact the capability to overcommit guest memory, and may slow down the boot of microVMs. However, those constrains are acceptable for the community especially for a first iteration. We will evaluate SLO and define what looks like acceptable from a performance point of view at a later stage of the project.
3. In the initial iteration, we will not aim to be able to snapshot GPU devices.
4. Firecracker Team will help the community to build PCIe and GPU support sheparding the project. The team will be responsible to project manage this initiative, set up roadmap, milestones and clear KPI in order to evaluate/gate progresses. The team will also drive security reviews, provide technical guidance/support and code reviews, and last but not the least help to build the infrastructure required to enable the community to run tests during the development of this feature. @Manciukic is going to be the tech leader of the Firecracker team who will lead this initiative.
5. The community will be responsible for the actual co-design and implementation of the feature.
6. Considering the implication of supporting PCIe passthrough, we will perform our day by day development task in a feature branch to do not impact the normal Firecracker release lifecycle. We will aim to merge the change back in main at each milestone if they do meet KPI, functional and security reviews.
7. We will held regular Community Meeting, every 4 Wednesday at same time 18:00 BST for up to 1 hour to sync and share progresses. Once the project will be rolling, we will evaluate if there is there is the need of a bi-weekly sync too.

Next steps :

1. Firecracker team will provide to the community a roadmap to support PCIe devices and GPU to be reviewed. This will milestones, and a proposed set of tasks for the community by 2024-11-06.
2. Firecracker Team will setup the initial feature branch to kick off the deployment activities.

Kind Regards,
Marco

[Manciukic]

Hey folks,

ahead of today meeting, I have resurrected and extended the old PCIe PoC (sandreim/pci_experiment) into a dedicated branch in the main repository (poc/pcie). The implementation is scrappy, ugly, likely not free of bugs, and doesn't pass style and unit tests, but:

• it's possible to enable PCI for all virtio devices with a new vm config json parameter (I haven't had to add the API endpoint yet): "pci": { "enabled": true }
• it's possible to attach multiple PCIe devices through vfio (I got up to 8 GPUs working on a single FC VM): "pci": { "enabled": true, "vfio_devices": [ { "path": "/sys/bus/pci/devices/0000:18:00.0/" } ] }
• VM memory is pre-allocated if there are VFIO devices, so oversubscription with device pass-through is not possible
• snapshot/resume of PCI devices is not supported (not even for virtio devices yet, as I've taken some shortcuts).

Initial performance results on the PoC look promising, but I need more time to prepare the testing artifacts and infrastructure, refine the PoC, and gather more data. You're welcome to play with it as well.

In today's meeting we'll be reviewing the draft of the roadmap, which I have published as a draft PR to make it easy to leave comments on it, discuss it, and iterate on it: #4894 . Feel free to take a read, leave comments, and share your opinion!

Finally, I have also created a new (but empty) feature branch for PCIe development: feature/pcie. I'm looking forwards to the contributions!

Thanks,
Riccardo