Request: add arm64 machine to CI (jenkins)

[afbjorklund]

In order to test multi-arch support, we need a non-amd64 worker for CI testing.
I'm thinking s390x might be hard to find, easier to set up an arm64 server ?

The qemu-system-aarch64 support is a little buggy, so has to be hardware.
It needs to have enough memory (4G) and needs to have a 64-bit processor.

This desktop setup works, if on a budget:

• Raspberry Pi 4b (4G), running Ubuntu 19.10 Server (arm64+raspi3)
  https://ubuntu.com/blog/roadmap-for-official-support-for-the-raspberry-pi-4
  • https://ubuntu.com/download/raspberry-pi
  • https://www.raspberrypi.org/products/raspberry-pi-4-model-b/

But it runs really hot (85°C), so it needs to have a passive / active cooling.
So probably better off with a server model, several providers offer ARMv8...

• https://pimodules.com/product/picoolfan4
• https://blog.alexellis.io/cooling-off-your-rpi4/

[afbjorklund]

Related issues:

• Support for multiple architectures (beyond amd64) #3890
• Add support for running on arm64 (aarch64) #5667

[afbjorklund]

Here is my new Raspberry Pi 4:

Open, showing the PiCoolFAN 4:

On my RIGGAD desktop lamp, for scale.

Shopping list, https://www.pishop.us/

• Raspberry Pi 4 Model B/4GB ($55) - I have the Raspberry Pi 4 Model B/2GB $45
• Raspberry Pi 15W Power Supply, US, White ($8) - I have the matching EU model
• Aluminum Heatsink for Raspberry Pi 4B ($2)
• MicroSD Card - 16/32 GB - Class 10 ($8)
• Some kind of regular Ethernet Cable ($2)
  Case and extra cooling if required:
• Raspberry Pi 4 Case, Red/White ($5)
• I instead got the excellent PiCoolFAN4 (£11 = $15)

Should come in well under $100 in total, like $75+case ?

[afbjorklund]

Eventually it will also be possible to run Fedora on the RPi4, but currently it has some issues.

https://fedoraproject.org/wiki/Architectures/ARM/Raspberry_Pi#Raspberry_Pi_4

There was some details about running Fedora on the RPi3, in the other issue created earlier.

#5667 (comment)

[medyagh]

we could probably add a jenkins agent, if we have a remote arm64 server ... or I wonder if we could use a public cloud for that?

[edmondsw]

Travis CI now supports multiple architectures, including not only arm64 but also ppc64le and s390x: https://blog.travis-ci.com/2019-11-12-multi-cpu-architecture-ibm-power-ibm-z

[afbjorklund]

Travis only tests the basics (unit tests and such) for us, we need a Jenkins slave to do the full integration testing...

[tstromberg]

I have a rockpro64 that I can see about donating to the effort.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[medyagh]

now we have AWS account ( I mailed the creds to all minikube maintainers), please feel free to add setup a arm64 machine and I will hook it up to github actions or jenkins

[kevinzs2048]

Hi Guys, this is Kevin from Linaro and we can offer the Arm64 bare metal machines as the builder. Could you tell me how to proceed with this topic?

[afbjorklund]

@kevinzs2048 : hello, I think we are past the jenkins-under-the-desk stage for the project that this post originally described (and that I have at home), and we are now using GitHub Actions with custom runners. hopefully @medyagh can tell you more about it?

The actually build is done with cross-compilation, so that part works fine. The main need is for doing regression testing, with the various drivers (mostly one VM and one container, but we have various hypervisors and various engines if there are resources...)

[kevinzs2048]

@afbjorklund Thanks for that info. Looks that on Arm64 we can enable none/docker/kvm2 drivers. And for KVM2 driver, the testing machine should be a bare metal, since on Arm64 there is no nested virtualization support.

[afbjorklund]

Currently we don't have any OS images for testing the docker (#9227) or kvm2 (#9228) driver on arm64.
There are partial proof-of-concept implentations, but nothing is available for public testing at the moment.

The basic operating systems (ubuntu for KIC, buildroot for ISO) does have ARM (arm32/arm64) support.
But we need to complete the packaging for minikube. The main reason for supporting armv7 is Raspbian.

[kevinzs2048]

@afbjorklund Thanks, I see.
Is there anything I can help to accelerate the process? I'm from Linaro so that contributing anything for Arm64 is my mission.

[spowelljr]

/assign @ilya-zuyev

[ilya-zuyev]

aws linux-arm64 runner added to mk Jenkins CI

[medyagh]

done