How to accommodate for different means of measurement?

[simonduq]

For power, some testbeds can directly measure, others can't. In the latter case, a reasonable choice is to measure duty cycle in software. Even in testbeds that can measure power, some metrics might require to have the UART active (e.g., to measure channel utilization, queue statistics etc.), in which case power measurement is no longer an option.

Would be nice to make it possible for a profile to have "power or duty cycle" as an output metric. Or maybe not? What is a better way to approach this?

[romain-jacob]

I would maybe approach the problem the other way around:

1. What are the metrics that we want to have available for a given profile?
2. Can we measure this? If yes, on which test environment? If no, what would it cost to get this measurement?

In a sense, the definition of the benchmark will be a way of driving the future development of both testbeds and simulators (as we described in the whitepaper).

Another way to phrase this: the current capabilities of the testbeds should not restrict us in the formulation of the profiles. At least not in the first phase.

Let's first define what we want. Then we will see what we can reasonably get.