Dynamic App Loader Helper #387

viswajith-g · 2024-04-01T06:28:50Z

This PR introduces the helper app to validate the kernel's dynamic process loading functionality (tock/tock#3941). The helper app triggers the dynamic process loading with a button press.

This application is located in /examples/tests/app_loader and was tested on the nRF52840DK. A previous version of the dynamic process loader was tested with the same application on an Imix board as well.

examples/ota_app_wip/main.c

libtock/internal/app_loader.h

examples/ota_app_wip/main.c

libtock/internal/app_loader.h

libtock/internal/app_loader.c

alistair23 · 2024-04-02T10:51:47Z

examples/tests/app_loader/main.c

+uint32_t app_size = 0;        // variable to store app size
+
+// Tock Application Binary to be flashed.
+const uint8_t app_binary[] = {0x2, 0x0, 0x34, 0x0, 0x0, 0x8, 0x0, 0x0, 0x1, 0x0, 0x0, 0x0, 0xD7, 0x75, 0x50, 0x6E,


This application can build for any architecture. How does this data then work?

examples/tests/app_loader/main.c

alistair23 · 2024-04-02T23:46:25Z

examples/tests/app_loader/main.c

+  }
+
+  for (uint32_t offset = 0; offset < write_count; offset++) {
+    memcpy(write_buffer, &app_binary[FLASH_BUFFER_SIZE * offset], FLASH_BUFFER_SIZE);     // copy binary to write buffer


What if app_binary isn't a multiple of FLASH_BUFFER_SIZE?

examples/tests/app_loader/main.c

libtock/internal/app_loader.c

bradjc · 2025-02-12T17:50:42Z

Can you rebase this on master and

move the syscall wrappers to their own file
move app_binaries and the python script to the test directory

viswajith-g · 2025-02-12T21:53:08Z

Can you rebase this on master and

I believe the repo was already rebased on master last time i pushed it.

move the syscall wrappers to their own file

Done. I also renamed the functions to start with libtock_ to keep in line with the other the new libtock format.

move app_binaries and the python script to the test directory

Done

bradjc · 2025-02-13T14:35:18Z

examples/tests/app_loader/button-press-loading/app_binaries.c

@@ -0,0 +1,1304 @@
+#include <examples/tests/app_loader/button-press-loading/app_binaries.h>


Suggested change

#include <examples/tests/app_loader/button-press-loading/app_binaries.h>

#include "app_binaries.h"

bradjc · 2025-02-13T14:39:37Z

libtock/internal/app_loader/app_loader.c

Move these to libtock/kernel

bradjc · 2025-02-13T14:40:11Z

libtock/internal/app_loader/syscalls/app_loader_syscalls.c

+// #include "libtock/tock.h"
+
+// bool libtock_app_loader_exists(void) {
+//   syscall_return_t res = command(DRIVER_NUM_APP_LOADER, 0, 0, 0);
+//   return tock_command_return_novalue_to_returncode(res);
+// }


Suggested change

// #include "libtock/tock.h"

// bool libtock_app_loader_exists(void) {

// syscall_return_t res = command(DRIVER_NUM_APP_LOADER, 0, 0, 0);

// return tock_command_return_novalue_to_returncode(res);

// }

bradjc · 2025-02-13T14:40:35Z

lvgl/lvgl

Revert this change.

i did not touch this though?

nvm i see it now

this app sends the tab file of an app that needs to be flashed to the capsule. once the app binary is written to flash, the userland app requests kernel to load the app.

… in internal/binaries.c. cleaned up and formatted removed old code removed userspace code from a previous implementation that was giving rise to confusion. added comments to app_loader.h added binaries headers and updated code added binaries in external files to make the app file more readable. Added a new subscribe and callback to support the async setup update readme

added tock-dpl-hello and tock-welcomes-dpl. The first app is a simple c_hello analog that prints a message and terminates on boot. The second app is a helper app that installs tock-dpl-hello on the board during runtime. These apps were made to test the dynamic process loading using hw-ci.

…hon tool to tests.

…s alarm upcall panic issue.

viswajith-g · 2025-05-01T04:43:55Z

ok, given the kernel part is now merged, i will leave this comment here:

I am all for removing button-press-loading and renaming abort-test to dynamic-process-loading-test because abort test is an extension of button press. We add the abort function and test it is all. I don't think it is worth two separate examples and a whole lot of code repeat for that reason. Thoughts @bradjc?

bradjc

I think we should have one interactive app meant for trying out DPL, eg button-press-loading. The others should be more friendly for CI (ie not require user input). I think the abort test should be able to run and test abort without requiring a user to get the button timing right.

I also think we should come up with a folder and more general structure for storing the .tbf binaries. We should also make it clear the binaries are cortex-m4 binaries. They should match what tockloader generates.

bradjc · 2025-05-02T15:51:19Z

libtock/kernel/app_loader.h

+ * This function takes in the function that will be executed
+ * when the callback is triggered.
+ */
+returncode_t libtock_app_loader_set_setup_upcall(subscribe_upcall cb, void* userdata);


These need to be updated to match https://github.com/tock/libtock-c/blob/3ff91f69e51ee305ab37e0878c88ba8a3a2b4ac2/doc/guide.md

bradjc · 2025-05-02T15:51:47Z

examples/tests/app_loader/tock-dpl-hello/main.c

What is this app for? It seems the same as c_hello.

This is the app that's compiled and fed to tock-welcomes-dpl.

viswajith-g · 2025-05-02T16:01:46Z

Okay, I agree with that

viswajith-g · 2025-05-02T23:06:11Z

I also think we should come up with a folder and more general structure for storing the .tbf binaries. We should also make it clear the binaries are cortex-m4 binaries. They should match what tockloader generates.

Okay, so I gave it some thought and the way I have it working right now is to move the binaries to a different directory and create a symlink so the Makefile can find and compile the binaries for each example. I still have them stored in a single app_binaries.c file but I am considering splitting each binary into its own file. This should hopefully reduce the size of the compiled test app, with people getting to choose and test their own binaries easily downstream. The downside to this would be multiple symlinks.

The structure looks like this right now:

│ 
├─ app-binaries/
│   ├─ cortex-m4/
│   	   ├─ app_binaries.h
│	   ├─ app_binaries.c
│
├─ abort-test/
│   ├─ app_binaries.c -> symlink
│   ├─ main.c
│   └─ Makefile
│
├─ button-press-loading/
│   ├─ app_binaries.c -> symlink
│   ├─ main.c
│   └─ Makefile

…ls to match naming conventions

viswajith-g · 2025-05-03T00:36:22Z

These pushes only address the directory structure changes and not the ci-ease concern that was raised. That will come with a separate commit

viswajith-g · 2025-05-04T07:03:09Z

The current directory structure looks like this:

│ 
├─ app-binaries/
│   ├─ cortex-m4/
│   	   ├─ tock-apps.h
│	   ├─ tock-dpl-hello.c
│          ├─ blink.c
│          ├─ adc.c
│
├─ abort-test/
│   ├─ tock-dpl-hello.c  -> symlink
│   ├─ blink.c           -> symlink
│   ├─ adc.c             -> symlink
│   ├─ main.c
│   ├─ Readme
│   └─ Makefile
│
├─ button-press-loading/
│   ├─ blink.c  -> symlink
│   ├─ adc.c    -> symlink
│   ├─ main.c
│   ├─ Readme
│   └─ Makefile

The binaries match the ones generated by tockloader. I pulled from the examples in the repo for blink and adc. I wrote my own example for the third, but it is a simple variation of c-hello.

For the abort-test, the user can press the button whenever, and it'll still abort. I've tested it multiple times, the NonvolatileStorage driver is just fast.

However, I've changed the interface so that the app is listening for a console command instead of a button press. Essentially, the characters '0', '1', or '2' load a different app based on the command. This should eliminate those pesky debounce issues and whatnot. Plus it is easy to expand this anyway regardless of the board button support capabilities, while introducing the undefined behavior when multiple apps are listening however.
'2' installs adc, but the app is programmed so that at the 50% mark, the app sends an abort signal internally. One could write a script to automate this.

I have absolutely no idea why the ci-format failed on print statements labeled error only in abort-test and not in button_press_loading. Plus, this failure did not happen before either. (edit: fixed now)

viswajith-g · 2025-05-05T19:40:25Z

I wonder, should I get rid of the other two apps and just have it run the adc + abort alone? While still requiring console input to make sure it does not just trigger on boot.

bradjc · 2025-05-05T22:24:21Z

Why does the abort test need to be interactive at all? It is simpler and easier to run if all you have to do is install it and see if it prints success or not.

You could add a timer (potentially with some random delay) if you want to not run immediately at boot or randomize when the abort signal happens.

bradjc · 2025-05-05T22:25:32Z

Okay, so I gave it some thought and the way I have it working right now is to move the binaries to a different directory and create a symlink so the Makefile can find and compile the binaries for each example. I still have them stored in a single app_binaries.c file but I am considering splitting each binary into its own file. This should hopefully reduce the size of the compiled test app, with people getting to choose and test their own binaries easily downstream. The downside to this would be multiple symlinks.

Why can't we just include in the makefile which binary apps to include?

viswajith-g · 2025-05-05T23:08:04Z

I don't remember exactly why but the compiler kept complaining when I included the binaries in the make file. Something about a path issue. So I ended up making symlinks. I can try again.

alistair23 suggested changes Apr 2, 2024

View reviewed changes