Flexible OSD

[seriyps]

Right now the OSD in pixelpilot is quite rudimentary and mostly hardcoded. I think we should make it possible to define it declaratively in some config file. Later external programs can be used to generate such config.

Also I think the values to display in OSD should not be calculated ad-hoc, but rather being "emitted" in raw form to some kind of bus and OSD elements can "subscribe" to (one or many) such value updates.

Let's discuss the ways we can achieve that in this ticket.

[seriyps]

I have a prototype that was based on some ideas from QGroundControl and my experience working with monitoring dashboards (eg, Grafana/Prometheus etc). OSD is more or less indeed a monitoring dashboard overlayed over the video, so it makes sense to borrow ideas from that area =).
It has the following structure:

Presentation layer:

• Base Widget class that stores the coordinate where it should be placed and it has a draw() function
• Several subclasses of Widget including:
  • Subclass that can draw a static string
  • Subclass that can draw templatized string
  • Subclass that can draw an icon followed by templatized string
  • more widgets to come; I really would like to play with some ideas of vidgets which will manipulate the images
  • one of the ideas is to have a "msposd" widget - there is a feature in MSPOSD where it forwards the MSP to UDP port, and we can make pixelpilot to read those messages and render the OSD on the ground instead of embedding it into the video. So this widget can subscribe to MSP messages and use them to draw BF/iNav OSD
  • maybe we can later add widgets wich will execute a LUA code and can be "injected" as plug-ins. I don't have better idea of how can we make it flexible enough just using C++ - we will end up inventing our own scripting language in the end if we go this way, so better to use already existing one.

Data layer:

• A Fact class which basically is a pair of "name" and dynamically typed "value" (plus a few more things). Thinking about adding a creation timestamp as well. Would be an equivalent of individual metric in monitoring system. For example, we received X GPS coordinate from Mavlink - we create a Fact("mavlink.gps_x", coordinate_x) and we "publish" this fact/metric to a centralized bus. Ideally the fact itself should be as "raw" as possible, so, no aggregation, no preprocessing (min/max/avg etc) should be done. So the code that publishes the fact won't need to make assumptions how this fact will be later used for display.
• Widgets can subscribe to those fact updates and use the values from them to decide how it should be drawn. The simplest example is that templatized widget may insert the value into template string

---

Widgets (their coordinates, type + arguments, facts they want to subscribe to) can be declaratively described in JSON file (or .yaml, or .ini, need to decide). And maybe later someone will create a GUI to generate such file.
Smth like:

format_vsn: "0.0.1"
widgets:
  - type: PlainText
    x: 10
    y: 10
    text: "Hello world"
 - type: Template
   x: 10
   y: 30
   template: "GPS %f:%f"
   facts:
     - name: "mavlink.gps_x"
     - name: "mavlink.gps_y"
- type: IconAndTemplate
  x: -30   # bound to the left side of the window
  y: 50
  icon: "/usr/lib/pixelpilot/wfb.png"
  template: "RSSI: %i, SNR: %i, FEC: %i"
  facts:
    - name: "wfb.rssi"
    - name "wfb.snr"
    - name: "wfb.fec_corrected"
- type: Lua
  x: 10
  y: -10  # bound to the bottom of the window
  lua: "/config/my_widget.lua"
  facts:
    - name: "my-fact-1"
    - name: "my-fact-2"
    - "..."

That is all for now, but I think it might be nice to add some map and reduce operations on top of the Fact system because sometimes the raw fact might be not really useful. Say, if we periodically publish the fact that "we received yet another video frame". But how do we display it in OSD? In OSD we want to display FPS instead. So we can add some statefull aggregator that will consume "video.frame_received" facts and will calculate the avg FPS over period of time and feed aggregated data to the widget. This work can be done directly by the widget, but then we risk to repeat the same patterns in each widget. Maybe it's fine if we have LUA widgets though.

[seriyps]

I'm working on it in this branch, if someone is interested in early preview master...seriyps:PixelPilot_rk:flexible-osd