Skip to content

Commit 5484b0c

Browse files
gubertiylc
guberti
authored and
ylc
committed
[microTVM][RVM] Add Arduino RVM (apache#8748)
* Functioning Arduino Vagrant VM Begin building Arduino Vagrant VM Mostly working Vagrant VM Changes for debugging Add ignored json file Fix venv path * Generalize parts of RVM for multiple platforms cwd hack Add unit tests from apps directory to task_python_microtvm.sh Generalize parts of RVM for multiple platforms * Add Vagrantfile lint exceptions * Address PR comments Address Mehrdad's PR comments More PR comments Documentation tweaks Add dialout group to user * Rerun tests * Spresense fix * Rerun CI tests * Rerun tests
1 parent 7eed325 commit 5484b0c

File tree

17 files changed

+561
-35
lines changed

17 files changed

+561
-35
lines changed

‎apps/microtvm/arduino/template_project/microtvm_api_server.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,11 @@ class BoardAutodetectFailed(Exception):
6565
"architecture": "esp32",
6666
"board": "feathers2",
6767
},
68+
"metrom4": {
69+
"package": "adafruit",
70+
"architecture": "samd",
71+
"board": "adafruit_metro_m4",
72+
},
6873
# Spresense only works as of its v2.3.0 sdk
6974
"spresense": {
7075
"package": "SPRESENSE",

‎apps/microtvm/reference-vm/arduino/.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
/.vagrant

‎apps/microtvm/reference-vm/arduino/README.md‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
<!--- Licensed to the Apache Software Foundation (ASF) under one -->
2+
<!--- or more contributor license agreements. See the NOTICE file -->
3+
<!--- distributed with this work for additional information -->
4+
<!--- regarding copyright ownership. The ASF licenses this file -->
5+
<!--- to you under the Apache License, Version 2.0 (the -->
6+
<!--- "License"); you may not use this file except in compliance -->
7+
<!--- with the License. You may obtain a copy of the License at -->
8+
9+
<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
10+
11+
<!--- Unless required by applicable law or agreed to in writing, -->
12+
<!--- software distributed under the License is distributed on an -->
13+
<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
14+
<!--- KIND, either express or implied. See the License for the -->
15+
<!--- specific language governing permissions and limitations -->
16+
<!--- under the License. -->
17+
18+
# microTVM Arduino Reference Virtual Machine
19+
20+
This directory contains setup files for Arduino virtual machine used for testing
21+
microTVM platforms that are supported by [Arduino](https://www.arduino.cc/).
22+
23+
## VM Information for Developers
24+
Arduino VM is published under [tlcpack](https://app.vagrantup.com/tlcpack).
25+
Here is a list of different release versions and their tools.
26+
27+
(none currently)
28+
29+
## Supported Arduino Boards
30+
This RVM has been tested and is known to work with these boards:
31+
- Adafruit Metro M4
32+
- Adafruit Pybadge
33+
- Arduino Due
34+
- Arduino Nano 33 BLE
35+
- Feather S2
36+
- Sony Spresense
37+
- Wio Terminal
38+
39+
However, the RVM *should* work with any Arduino with sufficient memory, provided
40+
its core is installed in `base-box/base_box_provision.sh`.
41+
42+
Note that this RVM does not work with the Teensy boards, even though they are
43+
supported by microTVM. This is because arduino-cli does not support Teensy
44+
boards (https://github.com/arduino/arduino-cli/issues/700)/).

‎apps/microtvm/reference-vm/arduino/Vagrantfile‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
Vagrant.configure("2") do |config|
19+
config.vm.box = "tlcpack/microtvm-arduino-0.18.3"
20+
21+
if ENV.has_key?("TVM_RVM_NUM_CORES")
22+
num_cores = ENV["TVM_RVM_NUM_CORES"]
23+
else
24+
num_cores = 2
25+
end
26+
27+
if ENV.has_key?("TVM_RVM_RAM_BYTES")
28+
ram_bytes = ENV["TVM_RVM_RAM_BYTES"]
29+
else
30+
ram_bytes = 2048
31+
end
32+
33+
tvm_home = "../../../.."
34+
dirs_to_mount = [Pathname.new(Pathname.new(tvm_home).expand_path())]
35+
if ENV.has_key?("TVM_PROJECT_DIR") then
36+
dirs_to_mount.append(ENV["TVM_PROJECT_DIR"])
37+
puts "NOTE: also configuring project dir: %s" % [dirs_to_mount[-1]]
38+
end
39+
40+
git_file = Pathname.new(tvm_home + "/.git")
41+
if git_file.ftype() == "file" then
42+
gitdir_match = Regexp.new('^gitdir: (?<gitdir>.*/.git).*\n$', Regexp::MULTILINE).match(git_file.read())
43+
if !gitdir_match.nil? then
44+
dirs_to_mount.append(Pathname.new(tvm_home).realpath.join(gitdir_match.named_captures["gitdir"]))
45+
puts "NOTE: also configuring git-worktree gitdir: %s" % [dirs_to_mount[-1]]
46+
end
47+
end
48+
49+
config.vm.provision "shell", path: "provision_setup.sh", env: {"TVM_HOME": dirs_to_mount[0]}, privileged: false
50+
51+
# Enable USB Controller on VirtualBox
52+
vm_name = "microtvm-arduino-#{Time.now.tv_sec}"
53+
config.vm.provider "virtualbox" do |vb, overrides|
54+
vb.name = vm_name
55+
vb.cpus = num_cores
56+
vb.memory = ram_bytes
57+
vb.customize ["modifyvm", :id, "--usb", "on"]
58+
vb.customize ["modifyvm", :id, "--usbehci", "on"]
59+
vb.customize ["modifyvm", :id, "--usbxhci", "on"]
60+
vb.customize [ "guestproperty", "set", :id, "/VirtualBox/GuestAdd/VBoxService/--timesync-set-threshold", 10000]
61+
dirs_to_mount.each do |d|
62+
overrides.vm.synced_folder d.to_s, d.to_s
63+
end
64+
end
65+
66+
end

‎apps/microtvm/reference-vm/arduino/base-box/.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
*.box
2+
.vagrant
3+
/output-packer-*
4+
/packer.json

‎apps/microtvm/reference-vm/arduino/base-box/Vagrantfile.packer-template‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
Vagrant.configure("2") do |config|
19+
# From hashicorp default template:
20+
# https://github.com/hashicorp/packer/blob/master/builder/vagrant/step_create_vagrantfile.go#L23-L37
21+
22+
config.vm.define "source" do |source|
23+
source.vm.box = "{{.SourceBox}}"
24+
config.ssh.insert_key = {{.InsertKey}}
25+
end
26+
27+
config.vm.define "output" do |output|
28+
output.vm.box = "{{.BoxName}}"
29+
output.vm.box_url = "file://package.box"
30+
config.ssh.insert_key = {{.InsertKey}}
31+
end
32+
33+
{{ if ne .SyncedFolder "" -}}
34+
config.vm.synced_folder "{{.SyncedFolder}}", "/vagrant"
35+
{{- else -}}
36+
config.vm.synced_folder ".", "/vagrant", disabled: true
37+
{{- end}}
38+
39+
40+
{{ if eq .BoxName "microtvm-base-vmware_desktop" -}}
41+
config.vm.provision "shell", inline: "touch ~/skip_zeroing_disk", privileged: false
42+
{{- end}}
43+
44+
# NOTE: base_box_setup.sh resides in the parent directory (../) because this template is expanded into a
45+
# sub-directory of base-box (output-packer-*).
46+
config.vm.provision "shell", path: "../base_box_setup.sh", privileged: false
47+
end

‎apps/microtvm/reference-vm/arduino/base-box/base_box_provision.sh‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
#!/bin/bash -e
2+
# Licensed to the Apache Software Foundation (ASF) under one
3+
# or more contributor license agreements. See the NOTICE file
4+
# distributed with this work for additional information
5+
# regarding copyright ownership. The ASF licenses this file
6+
# to you under the Apache License, Version 2.0 (the
7+
# "License"); you may not use this file except in compliance
8+
# with the License. You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing,
13+
# software distributed under the License is distributed on an
14+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
# KIND, either express or implied. See the License for the
16+
# specific language governing permissions and limitations
17+
# under the License.
18+
#
19+
# Using this script we can reuse docker/install scripts to configure the reference
20+
# virtual machine similar to CI QEMU setup.
21+
#
22+
23+
set -e
24+
set -x
25+
26+
source ~/.profile
27+
28+
# Init Arduino
29+
cd ~
30+
31+
sudo apt-get install -y ca-certificates
32+
33+
# Install Arduino-CLI (latest version)
34+
export PATH="/home/vagrant/bin:$PATH"
35+
wget -O - https://raw.githubusercontent.com/arduino/arduino-cli/master/install.sh | sh -s
36+
37+
# Arduino (the CLI and GUI) require the dialout permission for uploading
38+
sudo usermod -a -G dialout $USER
39+
40+
# ubuntu_init_arduino.sh only installs a few officially
41+
# supported architectures, so we don't use it here
42+
43+
# 3rd party board URLs
44+
ADAFRUIT_BOARDS_URL="https://adafruit.github.io/arduino-board-index/package_adafruit_index.json"
45+
ESP32_BOARDS_URL="https://raw.githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_dev_index.json"
46+
SPARKFUN_BOARDS_URL="https://raw.githubusercontent.com/sparkfun/Arduino_Boards/master/IDE_Board_Manager/package_sparkfun_index.json"
47+
SEEED_BOARDS_URL="https://files.seeedstudio.com/arduino/package_seeeduino_boards_index.json"
48+
SPRESENSE_BOARDS_URL="https://github.com/sonydevworld/spresense-arduino-compatible/releases/download/generic/package_spresense_index.json"
49+
arduino-cli core update-index --additional-urls $ADAFRUIT_BOARDS_URL,$ESP32_BOARDS_URL,$SPARKFUN_BOARDS_URL,$SEEED_BOARDS_URL,$SPRESENSE_BOARDS_URL
50+
51+
# Install supported cores from those URLS
52+
arduino-cli core install arduino:mbed_nano
53+
arduino-cli core install arduino:sam
54+
arduino-cli core install adafruit:samd --additional-urls $ADAFRUIT_BOARDS_URL
55+
arduino-cli core install esp32:esp32 --additional-urls $ESP32_BOARDS_URL
56+
arduino-cli core install Seeeduino:samd --additional-urls $SEEED_BOARDS_URL
57+
arduino-cli core install SPRESENSE:spresense --additional-urls $SPRESENSE_BOARDS_URL
58+
59+
# The Sony Spresense SDK has a major bug that breaks TVM. It's scheduled to be fixed in
60+
# release 2.3.0, but until that's published we need to use the below hack. This ONLY
61+
# fixes the bug in the main core release SDK - the subcore release SDK and both
62+
# the main and subcore debug SDKs will continue to fail until an official fix is made.
63+
# https://github.com/sonydevworld/spresense/issues/200
64+
SPRESENSE_NUTTX_BUGFIX_PATH=~/.arduino15/packages/SPRESENSE/tools/spresense-sdk/2.2.1/spresense/release/nuttx/include/sys/types.h
65+
sed -i 's/#ifndef CONFIG_WCHAR_BUILTIN/#if !defined(__cplusplus)/g' $SPRESENSE_NUTTX_BUGFIX_PATH
66+
67+
# There's also a bug in arduino-cli where {runtime.os} is not properly templated in
68+
# platform.txt. This bug only seems to appear with the SPRESENSE SDK. A fix has been
69+
# merged and will be part of arduino-cli 0.18.4, but that has yet to be published.
70+
# This change is only needed to upload code (not compile) for the Spresense.
71+
# https://github.com/arduino/arduino-cli/issues/1198
72+
SPRESENSE_FLASH_WRITER_BUGFIX_PATH=~/.arduino15/packages/SPRESENSE/hardware/spresense/2.2.1/platform.txt
73+
sed -i 's/tools.spresense-tools.cmd.path={path}\/flash_writer\/{runtime.os}\/flash_writer/tools.spresense-tools.cmd.path={path}\/flash_writer\/linux\/flash_writer/g' $SPRESENSE_FLASH_WRITER_BUGFIX_PATH
74+
sed -i 's/tools.spresense-tools.cmd.path.linux={path}\/flash_writer\/{runtime.os}\/flash_writer/tools.spresense-tools.cmd.path.linux={path}\/flash_writer\/linux\/flash_writer/g' $SPRESENSE_FLASH_WRITER_BUGFIX_PATH
75+
76+
# Cleanup
77+
rm -f *.sh

‎apps/microtvm/reference-vm/arduino/base-box/base_box_setup.sh‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
#!/bin/bash -e
2+
# Licensed to the Apache Software Foundation (ASF) under one
3+
# or more contributor license agreements. See the NOTICE file
4+
# distributed with this work for additional information
5+
# regarding copyright ownership. The ASF licenses this file
6+
# to you under the Apache License, Version 2.0 (the
7+
# "License"); you may not use this file except in compliance
8+
# with the License. You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing,
13+
# software distributed under the License is distributed on an
14+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
# KIND, either express or implied. See the License for the
16+
# specific language governing permissions and limitations
17+
# under the License.
18+
19+
set -e
20+
set -x
21+
22+
skip_zeroing_disk=0
23+
if [ -e "$HOME/skip_zeroing_disk" ]; then
24+
echo "NOTE: will not zero disk at the end due to VMWare Fusion bug"
25+
echo "See: https://communities.vmware.com/t5/VMware-Fusion-Discussions/VMWare-Fusion-Pro-11-15-6-16696540-causes-macOS-crash-during/m-p/2284011#M139190"
26+
skip_zeroing_disk=1
27+
fi
28+
29+
sudo apt update
30+
sudo apt install -y build-essential
31+
sudo apt-get --purge remove modemmanager # required to access serial ports.
32+
33+
sudo apt install -y --no-install-recommends git \
34+
cmake cmake-data \
35+
ninja-build gperf ccache dfu-util device-tree-compiler wget \
36+
python3-dev python3-pip python3-setuptools python3-tk python3-wheel xz-utils file \
37+
make gcc gcc-multilib g++-multilib libsdl2-dev
38+
39+
OLD_HOSTNAME=$(hostname)
40+
sudo hostnamectl set-hostname microtvm
41+
sudo sed -i.bak "s/${OLD_HOSTNAME}/microtvm.localdomain/g" /etc/hosts
42+
43+
# Poetry deps
44+
sudo apt install -y python3-venv
45+
46+
# TVM deps
47+
sudo apt install -y llvm
48+
49+
# ONNX deps
50+
sudo apt install -y protobuf-compiler libprotoc-dev
51+
52+
# TODO do we need this?
53+
echo 'export PATH=$HOME/vagrant/bin:"$PATH"' >> ~/.profile
54+
source ~/.profile
55+
echo PATH=$PATH
56+
57+
# Poetry
58+
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3
59+
sed -i "/^# If not running interactively,/ i source \$HOME/.poetry/env" ~/.bashrc
60+
sed -i "/^# If not running interactively,/ i\\ " ~/.bashrc
61+
62+
# Clean box for packaging as a base box
63+
sudo apt-get clean
64+
if [ $skip_zeroing_disk -eq 0 ]; then
65+
echo "Zeroing disk..."
66+
EMPTY_FILE="$HOME/EMPTY"
67+
dd if=/dev/zero "of=${EMPTY_FILE}" bs=1M || /bin/true
68+
if [ ! -e "${EMPTY_FILE}" ]; then
69+
echo "failed to zero empty sectors on disk"
70+
exit 2
71+
fi
72+
rm -f "${EMPTY_FILE}"
73+
else
74+
echo "NOTE: skipping zeroing disk due to command-line argument."
75+
fi

‎apps/microtvm/reference-vm/arduino/base-box/base_box_test.sh‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
#!/bin/bash -e
2+
# Licensed to the Apache Software Foundation (ASF) under one
3+
# or more contributor license agreements. See the NOTICE file
4+
# distributed with this work for additional information
5+
# regarding copyright ownership. The ASF licenses this file
6+
# to you under the Apache License, Version 2.0 (the
7+
# "License"); you may not use this file except in compliance
8+
# with the License. You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing,
13+
# software distributed under the License is distributed on an
14+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
# KIND, either express or implied. See the License for the
16+
# specific language governing permissions and limitations
17+
# under the License.
18+
#
19+
# Usage: base_box_test.sh <MICROTVM_PLATFORM>
20+
# Execute microTVM Arduino tests.
21+
#
22+
23+
set -e
24+
set -x
25+
26+
if [ "$#" -lt 1 ]; then
27+
echo "Usage: base_box_test.sh <MICROTVM_PLATFORM>"
28+
exit -1
29+
fi
30+
31+
microtvm_platform=$1
32+
33+
pytest tests/micro/arduino/test_arduino_workflow.py --microtvm-platforms=${microtvm_platform}
34+
35+
if [ $microtvm_platform == "nano33ble" ]; then
36+
# https://github.com/apache/tvm/issues/8730
37+
echo "NOTE: skipped test_arduino_rpc_server.py on $microtvm_platform -- known failure"
38+
else
39+
pytest tests/micro/arduino/test_arduino_rpc_server.py --microtvm-platforms=${microtvm_platform}
40+
fi

0 commit comments

Comments
 (0)