gnrc_gomach: a traffic adaptive MAC protocol for IEEE 802.15.4 networks (GoMacH). #5618

zhuoshuguo · 2016-07-11T00:55:06Z

This PR is an implementation of GoMacH, (we believe) "a General, nearly Optimal MAC protocol for multi-Hop communications", for IEEE 802.15.4 IoT/WSNs networks.

Its main features are:

an asynchronous solution which doesn't rely on global synchronization;
supports multi-hop and mesh network;
adopts duty-cycle scheme for conserving power;
provides high traffic adaptation for handling burst or dynamic traffics. It achieves this by dynamically allocating transmission slots for intensive senders (those that have pending packets); it enables a (or several) sender to burst transmit all of its buffered packets (for the same destination) to the receiver in one shot (or during a shot notice); The burst slotted-transmissions (of multi senders) will be ordered by the receiver device in a TDMA period;
adopts multi-channel scheme for avoiding/reducing wireless interference jam.

Note: source of all the following figures is from our IEEE-LCN-2017 conference paper (GoMacH: A Traffic Adaptive Multi-channel MAC Protocol for IoT).

TODO

Port to gnrc_netif2
Add radio duty-cycle display (debug)

Display achieved radio duty-cyle of the GoMacH

You can print out the achieved radio duty-cyle (a rough one) of GoMacH by setting the GNRC_GOMACH_ENABLE_DUTYCYLE_RECORD flag to "1".

By doing so, each time when a device sends or receives a packet, it will print out its achieved duty-cycle (start counting from power-up or reboot).

For instance:
On the sender side:

txtsnd 4 79:67:27:72:f4:57:9f:e6  helloworld
2017-11-07 14:17:33,174 - INFO # txtsnd 4 79:67:27:72:f4:57:9f:e6  helloworld
> 2017-11-07 14:17:33,828 - INFO #  [GoMacH]: achieved radio duty-cycle: 1 %

On the receiver side, you will see:

2017-11-07 14:17:33,826 - INFO # [GoMacH]: achieved radio duty-cycle: 1 % 
2017-11-07 14:17:33,829 - INFO # PKTDUMP: data received:
2017-11-07 14:17:33,834 - INFO # ~~ SNIP  0 - size:  10 byte, type: NETTYPE_UNDEF (0)
2017-11-07 14:17:33,838 - INFO # 00000000  68  65  6C  6C  6F  77  6F  72  6C  64
2017-11-07 14:17:33,843 - INFO # ~~ SNIP  1 - size:  24 byte, type: NETTYPE_NETIF (-1)
2017-11-07 14:17:33,846 - INFO # if_pid: 4  rssi: 42  lqi: 255
2017-11-07 14:17:33,847 - INFO # flags: 0x0
2017-11-07 14:17:33,850 - INFO # src_l2addr: 79:67:08:4a:13:d3:2c:02
2017-11-07 14:17:33,853 - INFO # dst_l2addr: 79:67:27:72:f4:57:9f:e6
2017-11-07 14:17:33,858 - INFO # ~~ PKT    -  2 snips, total size:  34 byte

The following figure shows the energy consumption rate measured on iotlab-m3 nodes when applying GoMacH as the MAC layer protocol, given an example duty-cycle of 30% (the duty-cycle can be definitely set to be much lower, this is just to show more cleanly the power consumption rate when the radio is on).

Generally, the power consumption rate can be brought down to 0.08 W (when the radio is turned off) from 0.12 W (when the radio is on) on iotlab-m3 nodes. In other words, in a nearly optimal case (with normal light traffic), currently, the duty-cycle scheme can manage to reduce about 1/3 of the power consumption on iotlab-m3 nodes. Notably, when the MAC is with the deep-sleep mode of RIOT in the future, the power saving feature will be more obvious.

Big picture

GoMacH is a completely asynchronous and distributed solution for IoT IEEE 802.15.4 networks, which doesn't require global synchronization to maintain its manipulations. The backbone of GoMacH is a typical low duty-cycling scheme for supporting basic transmissions. To tackle traffic variations and make GoMacH suitable for a larger range of applications (e.g., with bursty traffic), GoMacH adopts an efficient dynamic slots allocation scheme which utilizes the sender's MAC queue-length information to accurately allocate transmission slots with nearly no extra overheads. Furthermore, GoMacH adopts a novel multi-channel communication scheme that spreads its communications onto different channels. Packet transmissions that fail on one channel due to interferences will be soon recovered on another channel (like in TSCH).

GoMacH’s low duty-cycle scheme

In GoMacH, each node locally maintains its own superframe. The superframe starts with a short wake-up period (WP). Each node periodically wakes up in its own WP and listens for potential incoming packets. The WP period has an initially small duration, but is variable as in T-MAC, i.e., right after each packet reception in WP, the receiver automatically extends its WP to another basic WP duration to receive more potential incoming packets. On the other hand, after phase-locked with the receiver (the phase-lock scheme will be illustrated later), each sender wakes up right at the beginning of the receiver's WP and uses CSMA/CA to send packets, and turns off right after the transmission if it has no pending packets for the receiver, as illustrated above. With CSMA/CA, GoMacH allows multi senders to transmit packets to the same receiver in the latter's WP. Notably, in WP, it is restricted that a sender can send only one single packet to the receiver (even if it has multi pending packets). Buffered packets for the receiver will be transmitted in the following slots period.

Provide instantaneous high throughput: dynamic slots allocation

To truly provide sufficient throughput for varying network situations, the protocol adopts an efficient queue-length based slots allocation scheme, which has been previously proposed in iQueue-MAC.

In short, right before triggering each data packet transmission, each sender embeds its current queue-length value (the number of buffered packets for the targeted receiver, also called queue-length indicator) into the packet's MAC header. In WP (actually, in each packet reception), the receiver collects these queue-length indicators from all the senders. Then, right after WP, if the collected queue-length indicators are not all zero (indicating pending packets), the receiver immediately generates a slots-allocation scheduling beacon and broadcasts it out. The dynamically allocated transmission slots compose the variable TDMA period (vTDMA) in GoMacH.

Slots application is through data piggybacking, and the beacon only exists when there are slots to allocate. In light traffic scenarios with no packet pendings, GoMacH resumes to a typical low duty-cycle MAC, as shown in the first figure.

The above figure shows a typical WP+vTDMA transmission procedure in GoMacH (after phase-locked with the receiver). Sender-1 and sender-2 both have pending packets for the same receiver. After phase-locked, they wait for the wake-up period (WP) and use CSMA to send their first packets, while at the same time telling the receiver the exact number of pending packets. Right after the WP, the receiver immediately broadcasts a beacon that carries the vTDMA slots allocation list that schedules the following transmissions in the well-order TDMA manner on the receiver's sub-channel. Note that the vTDMA procedure is not shown in this figure since it is carried out on another channel.

High Robustness: multi-channel operation

--------------------------Channel usage in GoMacH

GoMacH is robust against external interferences. The protocol adopts multi-channel technique to spread its transmissions onto different channels. In short, GoMacH adopts:
1 ) a dual public channel scheme for transmissions in WP,
2 ) carries all the vTDMA slotted transmissions onto different sub-channels of different receivers.

By default, IEEE 802.15.4 channel 11 and 26 are selected as the dual public channels (public channel 1 and 2) for WP communications, while the rest of the channels are used as sub-channels for vTDMA transmissions. Of cause, the channel sequences of public channel 1 and 2 can be reconfigured according to users.

1. Dual public channel communication in WP

To guarantee the reliability of WP communication, GoMacH adopts a dual public channel scheme. All nodes in GoMacH adopt the same dual public channel sequences. Each node in GoMacH alternatively switches its radio between public channel 1 and 2 during consecutive WP periods, with increase of cycle counts. Senders that track the WP phase of the receiver also track the latter's public channel phase. The idea of the dual public channel scheme is simple: as depicted in the figure above, once transmissions in WP on one public channel fail due to external interferences in receiver's cycle-N, they can soon be recovered (retransmitted) right in the next cycle's cycle-(N+1) WP on the other public channel, like in TSCH.

2. vTDMA on sub-channels

When a receiver initiates vTDMA transmissions after WP (i.e., allocates transmission slots to senders), it carries out all the slotted transmissions onto its locally unique sub-channel. Through a sub-channel selection procedure right after powered up (GoMacH initialization), each node manages to maintain a locally unique sub-channel sequence among its one-hop neighbors, e.g., in the figure above, receiver-1's sub-channel is channel-15 and receiver-2's sub-channel is channel-21. Each time when a receiver initiates vTDMA communications, it embeds its sub-channel sequence onto its beacon and carries out the vTDMA procedure onto its sub-channel. After extracting the sub-channel sequence from the received beacon, the senders turn to the defined sub-channel and send their packets. Since different nodes have locally different sub-channels, vTDMA transmissions of nearby nodes can be carried out in parallel without collisions.

GoMacH’s phase-lock and broadcast schemes

phase-lock scheme

To save power from the sender side, GoMacH adopts phase-lock scheme to allow senders to track the receiver's wake-up phase and the related public channel phase of WP. Since each node's WP is alternatively located on public channel 1 or public channel 2, to catch one of the receiver's WP during the preamble period (which is slightly longer than the superframe cycle), the preamble stream is actually composed of two parallel sub-streams on the two public channels.

The above figures shows the "preamble stream + preamble-ACK + data +ACK" Phase-lock procedure in GoMacH. Notably, this figure only shows the communication procedure on one public-channel, there is another preamble stream broadcasted on the other public-channel, simultaneously, before the sender gets the preamble-ACK from the receiver.

broadcast scheme

The broadcast scheme is exactly the same as the preamble (phase-lock) scheme, except that each broadcast will fully last for one cycle duration.

How to test:

Currently,GoMacH requires RTT to run it, which provides the underlying timer source. But, as a future plan, to make GoMacH available for more devices, I will replace RTT with a more general timer source like xtimer-based gnrc_mac's timeout module.

Now, GoMacH fully supports samr21-xpro and iotlab-m3 boards (that have RTT). If you don't have those nodes with you, you can still try GoMacH with iotlab-m3 nodes in the FIT IoTLAB remotely for free.
Of cause, you also can try it with other nodes that have RTT module (but, remember to add the board's name into the white-list of the test makefile).

Test by using the gomach test-example:
I have simply copied the default example from RIOT/examples to build a text example in RIOT/tests/gomach for testing this protocol.

$ cd tests/gomach
$ BOARD=samr21-xpro make flash term

Manually send packet from one board to the other via interactive shell:

txtsnd 4 79:67:08:77:01:9f:33:1e  testmessage

you can also broadcast a packet by typing:

txtsnd 4 bcast helloworld

Some evaluation results

For performance comparison, I compared GoMacH to an adapted X-MAC. A test-bed of SAMR21-xpro nodes was established in my office room environment, and several real-world experiments have been carried out. The above table shows the key parameters of GoMacH and X-MAC and some experimental settings we adopt through all the experiments.

1. Impact of transmission hops

We tested GoMacH and X-MAC on a multi-hop test-bed to evaluate the impact of transmission hops on their performances. The above figure shows the linear multi-hop test-bed that has a maximum of 6-hops transmissions. Packets are generated at the left-end node (sender) and relayed to the right-end node (receiver). Relay nodes don't generate packets. We fix the data rate of the sender to 2 packets per second. The two protocols adopt the same channel check rate of 2Hz (i.e., cycle duration is 500ms).
We vary the transmission hops (number of relay nodes) between the sender and receiver from 1 to 6 hops to generate different test scenarios. Each test scenario lasts for 300 seconds. Each test scenario has been repeated for three times and we average the results.

2. Impact of parallel sender number

This test experiment investigated the impact of parallel sender number on the performance of GoMacH and X-MAC. In this experiment, there is one receiver and several senders that all generate packets for the receiver. The data rate of all senders is fixed to 2 packets per second and both GoMacH and X-MAC adopt a cycle duration of 500ms. We vary the number of the senders to generate different test scenarios. Each test scenario lasts for 300 seconds.
PS: this experiment is based on the RIOT/examples/default application.

3. Impact of data rate

This test experiment investigated the effectiveness of GoMacH's traffic adaptation capability (i.e., the dynamic slots allocation scheme). A tree network shown above was builded. The above figure shows the tree test-bed in which data packets are generated at the senders and relayed to the sink. Relay nodes don't generate packets. In this experiment, both GoMacH and X-MAC adopt a cycle duration of 500ms. We varied the data rate of senders from one packet per 10 seconds to an intensive value of 5 packets per second, to generate different test scenarios. Each test scenario lasts for 300 seconds.
PS: this experiment is based on the RIOT/examples/default application.

4. Robustness against external interference

GoMacH adopts multi-channel technique to enhance its robustness against external interferences.
This test experiment investigated the effectiveness of GoMacH's multi-channel scheme, by using the tree test-bed shown above, with the exist of wireless interferences.

In this experiment, all the senders adopt a fixed data rate of 1 packet per second. Relay nodes don't generate packets. Both GoMacH and X-MAC adopt a cycle duration of 250ms. We use one SAMR21-xpro node as the jamming source which continuously generates busytone that covers the whole network. The busytone is generated on channel-26 which is used as the communication channel of X-MAC and one of the public channels of GoMacH. The busytone has a cycle of 1 second and we vary the active ratio of the jamming signal (from 0% to 100%) to generate different test scenarios.

PS: this experiment is based on the RIOT/examples/default application.

5. Bursty experiment with RPL

This test experiment intends to evaluate/verify that GoMacH can cooperate well with popular upper layer protocols like RPL and UDP in bursty traffic conditions. Two test-beds of 11 SAMR21-xpro nodes were deployed over one layer of our office building. The above figure shows the test-beds and the communication stack we applied to all the nodes which includes UDP, RPL, IPv6 and 6LoWPAN (PS: this experiment is based on RIOT/examples/gnrc_networking application).
We deploy two different test-beds of "Mesh" and "Local", with different topologies as shown above.
In the "Local" test scenario, the test-bed is deployed in one office room that all nodes are in each other's communication range, simulating a dense network. While in the "Mesh" test scenario, nodes are scattered over the corridors, simulating a multi-hop network. In each node, an application layer generates data packets and uses UDP to send the packets to the sink. RPL is used to automatically build the routing paths. In all the tests, each node generates bursty data packets with an interval of 30 seconds. In each bursty data period, each node generates a bunch of 6 packets simultaneously. Each node adopts a cycle duration of 800ms and generates a total number of 300 packets. Notably, in each test, we extract our results from a recording period of the last 33 minutes of the experiment (started from 3 minutes ahead of the bursty data period, as shown in the following figure) where the topology is stable. We run both "Mesh" and "Local" scenarios for three times and average the results.

This figure shows the dynamic slots allocation procedure in one "Mesh" test scenario, i.e., GoMacH dynamically allocates transmission slots to tackle burst traffic loads.

6. Stability evaluation

Stability evaluation 1

Finished a first long-time experiment to test GoMacH's stability.
Settings:

Topology: one sink (receiver) and 5 senders:
This experiment is based on examples/default;
5 senders adopt a data rate of 1 packet/s (each packet is with a raw payload of 80 bytes) and transmit generated data to the sink;
MAC cycle duration for all nodes: 200ms;
MAC Tx-Queue size for each node: 10 packets;

Experiment duration: 64 hours (> 2 and a half days);

Results:

2017-08-01 12:02:53,377 - INFO # 2c02, 228773, 228720, 1146113. 
2017-08-01 12:02:53,572 - INFO # 71f2, 228856, 228843, 1146114. 
2017-08-01 12:02:53,581 - INFO # c13a, 229509, 229509, 1146115. 
2017-08-01 12:02:54,173 - INFO # 8032, 229523, 229480, 1146116. 
2017-08-01 12:02:54,371 - INFO # 79f6, 229568, 229565, 1146117.

Packet delivery ration of the network: 1146117 / 1146229 = 99.99022883%

Stability evaluation 2

Evaluated GoMacH through a second long-time stability experiment:
A multi-hop test-bed of SAMR21-xpro with at most 3 hops transmission. All the nodes were deployed in my office room environment (all nodes are in each other's radio/interference range). All nodes (except sink) generate data packet and send/relay the packets to the sink.

Application: based on the default application in RIOT/examples/default
Cycle duration of all nodes: 200ms;
data rate: 1 packet / 1 seconds per node.
Maximum link layer packet retransmission: 6 times.
Experiment duration: 48 hours (two days).

Results:

2017-09-14 12:36:31,559 - INFO # 4c66, 179984, 179982, 898812. 
2017-09-14 12:36:31,765 - INFO # 71f2, 179550, 179549, 898813. 
2017-09-14 12:36:31,781 - INFO # 383a, 179546, 179537, 898814. 
2017-09-14 12:36:31,966 - INFO # 2c02, 179684, 179682, 898815. 
2017-09-14 12:36:31,981 - INFO # 8032, 180066, 180066, 898816.

Packet deliver ratio: 898816/898830 = 99.9984424195899%. Only 14 packets dropped.

Stability evaluation 3

Evaluated GoMacH through a third long-time stability experiment, settings:

This experiment is based on the examples/gnrc_networking which includes UDP and RPL.
There are one sink and 5 senders.
The sink initiates the RPL network and sets itself as the root. All the senders send packets to sink using UDP.
In the experiment, there are simultaneously two types of traffics:
1, upward traffic generated by son noders (senders) heading to the sink,
2, downward traffic generated by the sink heading to all senders.

For upward traffic, all senders use UDP to send packets to the link local address of the sink;
For downward traffic, the sink uses UDP to send packets to the configured ipv6 addresses of all the senders which are informed to the sink through RPL networking, i.e., downward traffic is based on RPL routing.
PS: In this experiment, the application layer didn't have a packet retransmission scheme to recover packet drops.

Other key MAC settings:
Cycle duration of GoMacH for all nodes: 200ms;
Maximum link layer packet retransmission: 6 times.

Data rate:

for upward traffic of senders: 1 packet/2 seconds per sender;
for downward traffic of sink: 1 packet/5 seconds (i.e., the sink sends a data to each specific sender every 25 seconds).

The experiment lasted about 48 hours (two days).

Results:
For upward traffic: 431425 (received) / 431528 (generated) =99.976%
For downward traffic : 34594 (received) / 34604 (generated) = 99.971%

Stability evaluation 4

Finished a fourth long-time experiment (the final one) for evaluating GoMacH's stability:
Settings:

Topology: one sink (receiver) and 5 senders:
This experiment is based on examples/default
5 senders adopt an intensive data rate for continuing generating data packets and sending to the sink;
Data rate for each sender: 10packets/1second;
Data size for each packet: 80 bytes (raw payload);
MAC cycle duration for all nodes: 200ms;
MAC Tx-Queue size for each node: 8 packets;
Experiment duration: 49 hours (> two days);

Notably, the taffic of the network (taffic loads from all the senders) is currently beyong the offered throughput of GoMacH. And by applying such an overwhelming traffic, the goal of this experiment is to check that:

Given the intensive data rate, and thus, intensive memory/buffer allocation and release manipulations, will GoMacH lead to memory leak or stack overflow, i.e., to see whether the protocol still has critical bugs of improper memory/buffer manipulation;
to test the robustness of the protocol that, given an overwhelming traffic rate, whether the protocol can still maintain basic communication functionality, i.e., normal sending/receiving. We don't want GoMacH dead (not functioning) in case of overwhelming traffic situation.

Results:

2017-09-30 18:19:06,656 - INFO # 71f2, 1721548, 1684624, 8475545. 
2017-09-30 18:19:06,661 - INFO # 2c02, 1722083, 1705502, 8475546. 
2017-09-30 18:19:06,666 - INFO # c13a, 1726374, 1643696, 8475547. 
2017-09-30 18:19:06,671 - INFO # 675e, 1729352, 1724512, 8475548. 
2017-09-30 18:19:06,678 - INFO # 4262, 1723393, 1717215, 8475549.

Packet deliver ratio: 8475549 (received) / 8622750 (generated) = 98.2928%;
After 49 hours running time, no devices go dead, or be halted.
GoMacH successfully maintains normal sending/receiving capability; To be more specific, all the nodes keep successfully delivering their packets to the receiver, although some of the generated packet will be dropped due to Tx-queue full or meeting the maximum Tx-retry limit.

Actually, with some adaptation/optimization (on the WP period of GoMacH, or having larger TX-queue size), GoMacH's throughput can be further boosted/improved, i.e., achieving higher traffic adaptation. That will be done in the future (maybe in following PRs.) ;-)

daniel-k · 2016-07-11T10:04:41Z

Cool! I just skimmed over the code and found some parts that really look familiar to me 😄 I think it would be a good idea to refactor the commonalities of our code into more generic modules or MAC helpers. It looks like you also need to track neighbours and timeouts. Regarding the packet queue, there is gnrc/pktqueue.h, which is a lot simpler. I only found after I already had implemented mine.

zhuoshuguo · 2016-07-11T10:15:29Z

@daniel-k Exactly, I also agree that 😄 :

it would be a good idea to refactor the commonalities of our code into more generic modules or MAC helpers

Actually, as I wrote above, after I have read your codes (Lw-MAC), I found many modules and functionalities of Lw-MAC are very useful and helpful, so why not build my own work on them 😄. So, to facilitate the iQueue-MAC implementation, I taken part of your work (like timeouts and neigbour list, etc.) for helping! Thanks for the nice work!

zhuoshuguo · 2016-07-11T10:21:12Z

Yes, iQueue-MAC also track neighbours and timeouts. But, during my experiments, I found that RTT has strong time drift. In my experiments, the time drift between two node can be 1ms per 20mins. So, as stated in Lw-MAC , it will be good to:

use xtimer instead of RTT peripheral when low power timer source is implemented (#3990 et al.)

in the future.

daniel-k · 2016-07-11T10:33:14Z

@zhuoshuguo You're very welcome to as much of my code as you need! It's just unfortunate to have redundant code.

Yes, iQueue-MAC also track neighbours and timeouts. But, during my experiments, I found that RTT has strong time drift. In my experiments, the time drift between two node can be 1ms per 20mins. So, as stated in Lw-MAC , it will be good to:

use xtimer instead of RTT peripheral when low power timer source is implemented (#3990 et al.)

in the future.

I guess that this won't help at all. It's in inherent problem to drifting clocks on distributed systems. 1ms in 20min is somehow in the range of what I would expect (I did some calculations back then). Therefore, this was an upper bound to my sleep duration / time that nodes don't communicate.

The comment about using xtimer instead of RTT was to simplify the code as having two timer systems makes it unecessarily complex.

OlegHahm · 2016-07-11T11:10:39Z

I guess that this won't help at all. It's in inherent problem to drifting clocks on distributed systems. 1ms in 20min is somehow in the range of what I would expect (I did some calculations back then).

That looks rather big (~830 ppm). A typical cheap crystal on these systems have a drift of about ±30 or ±50ppm.

zhuoshuguo · 2016-07-11T12:54:13Z

That looks rather big (~830 ppm). A typical cheap crystal on these systems have a drift of about ±30 or ±50ppm.

Yes, that looks big. I have tested this drift effect in a simple experiment: use xtimer (xtimer_set_msg() function) to set a alarm (for example: 9 seconds), then, at the end of this 9 seconds period (when timeout message arrive), ask two nodes to report how many RTT ticks they have experienced during this 9 seconds.
Here are the results:

Node-A:

Welcome to pyterm!
Type '/exit' to exit.
2016-07-11 14:33:42,595 - INFO # 294925 .
2016-07-11 14:33:52,595 - INFO # 294931 .
2016-07-11 14:34:02,595 - INFO # 294930 .
2016-07-11 14:34:12,595 - INFO # 294922 .
2016-07-11 14:34:22,595 - INFO # 294927 .
2016-07-11 14:34:32,594 - INFO # 294925 .
2016-07-11 14:34:42,593 - INFO # 294889 .
2016-07-11 14:34:52,593 - INFO # 294886 .
2016-07-11 14:35:02,594 - INFO # 294915 .
2016-07-11 14:35:12,594 - INFO # 294931 .
2016-07-11 14:35:22,594 - INFO # 294934 .
2016-07-11 14:35:32,593 - INFO # 294912 .
2016-07-11 14:35:42,593 - INFO # 294929 .
2016-07-11 14:35:52,593 - INFO # 294936 .
2016-07-11 14:36:02,593 - INFO # 294929 .
2016-07-11 14:36:12,592 - INFO # 294901 .
2016-07-11 14:36:22,592 - INFO # 294916 .

Node-B:

Welcome to pyterm!
Type '/exit' to exit.
2016-07-11 14:33:52,511 - INFO # 295616 .
2016-07-11 14:34:02,511 - INFO # 295623 .
2016-07-11 14:34:12,511 - INFO # 295611 .
2016-07-11 14:34:22,510 - INFO # 295610 .
2016-07-11 14:34:32,511 - INFO # 295626 .
2016-07-11 14:34:42,509 - INFO # 295583 .
2016-07-11 14:34:52,508 - INFO # 295554 .
2016-07-11 14:35:02,508 - INFO # 295539 .
2016-07-11 14:35:12,509 - INFO # 295581 .
2016-07-11 14:35:22,508 - INFO # 295557 .
2016-07-11 14:35:32,508 - INFO # 295572 .
2016-07-11 14:35:42,509 - INFO # 295606 .
2016-07-11 14:35:52,509 - INFO # 295612 .
2016-07-11 14:36:02,509 - INFO # 295605 .
2016-07-11 14:36:12,510 - INFO # 295622 .
2016-07-11 14:36:22,510 - INFO # 295630 .

You can see the two nodes have different RTT ticks for 9 seconds (setted from xtimer), and the gap is not small. I am not sure which module (xtimer or RTT) is more accurate (maybe xtimer is more accurate than RTT), but I guess at least one of them has relatively big drift effect. Also, I am not sure whether the OS has also introduce some effect on this.

And, yes, distributed system has to face timer drift effect. In iQueue-MAC, there is also a re-phase-lock scheme to fix that if the sender get lost with its receiver's phase (wake-up period) due to timer drift.

jnohlgard · 2016-07-13T12:10:43Z

What are you using for the RTT test? The 29.4 kHz frequency seems odd.

zhuoshuguo · 2016-07-13T12:43:50Z

I used the samr21-xpro board for the RTT test. By the way, the the ticks shown above is not for 10 sends, it is for 9secons, e.g., 294925 tickes for 9s (in this case the frequency could be about 32.769kHz).

zhuoshuguo · 2017-08-01T17:47:24Z

Finished a first long-time experiment to test GoMacH's stability.
Settings:

Topology: one sink (receiver) and 5 senders:
This experiment is based on examples/default;
5 senders adopt a data rate of 1 packet/s (each packet is with a raw payload of 80 bytes) and transmit generated data to the sink;
MAC cycle duration for all nodes: 200ms;
MAC Tx-Queue size for each node: 10 packets;

Experiment duration: 64 hours (> 2 and a half days);

Results:

2017-08-01 12:02:53,377 - INFO # 2c02, 228773, 228720, 1146113. 
2017-08-01 12:02:53,572 - INFO # 71f2, 228856, 228843, 1146114. 
2017-08-01 12:02:53,581 - INFO # c13a, 229509, 229509, 1146115. 
2017-08-01 12:02:54,173 - INFO # 8032, 229523, 229480, 1146116. 
2017-08-01 12:02:54,371 - INFO # 79f6, 229568, 229565, 1146117.

Packet delivery ration of the network: 1146117 / 1146229 = 99.99022883%

zhuoshuguo · 2017-09-14T16:29:54Z

Updated GoMacH again and tackled a series of bugs and issues, which introduced memory leak and timing chaos in transmissions (some were introduced when rebasing).

After that, evaluated GoMacH through a second long-time stability experiment:
A multi-hop test-bed of SAMR21-xpro with at most 3 hops transmission. All the nodes were deployed in my office room environment (all nodes are in each other's radio/interference range). All nodes (except sink) generate data packet and send/relay the packets to the sink.

Application: based on the default application in RIOT/examples/default
Cycle duration of all nodes: 200ms;
data rate: 1 packet / 1 seconds per node.
Maximum link layer packet retransmission: 6 times.
Experiment duration: 48 hours (two days).

Results:

2017-09-14 12:36:31,559 - INFO # 4c66, 179984, 179982, 898812. 
2017-09-14 12:36:31,765 - INFO # 71f2, 179550, 179549, 898813. 
2017-09-14 12:36:31,781 - INFO # 383a, 179546, 179537, 898814. 
2017-09-14 12:36:31,966 - INFO # 2c02, 179684, 179682, 898815. 
2017-09-14 12:36:31,981 - INFO # 8032, 180066, 180066, 898816.

Packet deliver ratio: 898816/898830 = 99.9984424195899%. Only 14 packets dropped (under none application layer retransmission scheme).

Going to run a third long-time stability evaluation experiment.

jnohlgard · 2017-09-20T16:49:24Z

A curious question: is this part of a PhD thesis work or similar? The level of detail in the description of the algorithm is very good.

zhuoshuguo · 2017-09-20T18:02:17Z

Thanks. ;-) Actually, the origin idea of GoMacH comes from this paper, iQueue-MAC, which was indeed part of my PhD thesis when I was in China. Well, now (design and implementation of) GoMacH (which is very different from iQueue-MAC) is part of my RIOT-ADT project in INRIA.

zhuoshuguo · 2017-09-23T15:33:27Z

Finished a third time experiment for evaluating GoMacH's stability:
settings:

This experiment is based on the examples/gnrc_networking which includes UDP and RPL.
There are one sink and 5 senders.
The sink initiates the RPL network and sets itself as the root. All the senders send packets to sink using UDP.
In the experiment, there are simultaneously two types of traffics:
1, upward traffic generated by son noders (senders) heading to the sink,
2, downward traffic generated by the sink heading to all senders (one by one).

For upward traffic, all senders use UDP to send packets to the link local address of the sink;
For downward traffic, the sink uses UDP to send packets to the configured ipv6 addresses of all the senders (one by one) which are informed to the sink through RPL networking, i.e., downward traffic is based on RPL routing.
PS: In this experiment, the application layer didn't have a packet retransmission scheme to recover packet drops.

Other key MAC settings:
Cycle duration of GoMacH for all nodes: 200ms;
Maximum link layer packet retransmission: 6 times.

Data rate:

for upward traffic of senders: 1 packet/2 seconds per sender;
for downward traffic of sink: 1 packet/5 seconds (i.e., the sink sends a data to each specific sender every 25 seconds).

The experiment lasted about 48 hours (two days).

Results:
For upward traffic: 431425 (received) / 431528 (generated) =99.976%
For downward traffic : 34594 (received) / 34604 (generated) = 99.971%

zhuoshuguo · 2017-09-26T09:35:12Z

With ICN conference CCN-lite tutorial session going on, apply and play GoMacH on examples/ccn-lite/relayapplication, 😄 : it works~

2017-09-26 17:24:30,697 - INFO # 	pid | name                 | state    Q | pri | stack ( used) | base       | current     
2017-09-26 17:24:30,706 - INFO # 	  - | isr_stack            | -        - |   - |   512 (  140) | 0x20000000 | 0x200001c0
2017-09-26 17:24:30,713 - INFO # 	  1 | idle                 | pending  Q |  15 |   256 (  128) | 0x200025b4 | 0x20002634 
2017-09-26 17:24:30,721 - INFO # 	  2 | main                 | running  Q |   7 |  1536 (  676) | 0x200026b4 | 0x20002b04 
2017-09-26 17:24:30,729 - INFO # 	  3 | at86rf2xx-gomach     | bl rx    _ |   2 |  1024 (  556) | 0x2000559c | 0x2000586c 
2017-09-26 17:24:30,737 - INFO # 	  4 | ccnl                 | bl rx    _ |   6 |  1536 (  200) | 0x200005e4 | 0x20000b1c 
2017-09-26 17:24:30,743 - INFO # 	    | SUM                  |            |     |  4864 ( 1700)

On samr-21 board-1:

ccnl_cont /riot/shuguo/1  Hello! Free university! from Bob
2017-09-26 17:25:31,180 - INFO #  ccnl_cont /riot/shuguo/1  Hello! Free university! from Bob
> ccnl_cont /riot/shuguo/2  Hello! Free university! from Alice
2017-09-26 17:25:51,321 - INFO #  ccnl_cont /riot/shuguo/2  Hello! Free university! from Alice

On samr-21 board-2:

ccnl_int /riot/shuguo/1
2017-09-26 17:25:56,801 - INFO #  ccnl_int /riot/shuguo/1
2017-09-26 17:25:57,380 - INFO # Content received: Hello! Free university! from Bob
> ccnl_int /riot/shuguo/2
2017-09-26 17:26:01,268 - INFO #  ccnl_int /riot/shuguo/2
2017-09-26 17:26:01,776 - INFO # Content received: Hello! Free university! from Alice

zhuoshuguo · 2017-09-30T16:33:47Z

Stability evaluation 4

Finished a fourth long-time experiment (the final one) for evaluating GoMacH's stability:
Settings:

Topology: one sink (receiver) and 5 senders:
This experiment is based on examples/default
5 senders adopt an intensive data rate for continuing generating data packets and sending to the sink;
Data rate for each sender: 10packets/1second;
Data size for each packet: 80 bytes (raw payload);
MAC cycle duration for all nodes: 200ms;
MAC Tx-Queue size for each node: 8 packets;
Experiment duration: 49 hours (> two days);

Notably, the taffic of the network (taffic loads from all the senders) is currently and slightly beyong the offered throughput of GoMacH (Actually, I firstly assumed that 10packets/s data rate should be already very intensive, but it turns out that GoMacH can still handle most of the generated traffic. See below the final PDR ;-)). And by applying such an overwhelming traffic, the goal of this experiment is to check that:

Given the intensive data rate, and thus, intensive memory/buffer allocation and release manipulations, will GoMacH lead to memory leak or stack overflow, i.e., to see whether the protocol still has critical bugs of improper memory/buffer manipulation;
to test the robustness of the protocol that, given an overwhelming traffic rate, whether the protocol can still maintain basic communication functionality, i.e., normal sending/receiving. We don't want GoMacH dead (not functioning) in case of overwhelming traffic situation.

Results:

2017-09-30 18:19:06,656 - INFO # 71f2, 1721548, 1684624, 8475545. 
2017-09-30 18:19:06,661 - INFO # 2c02, 1722083, 1705502, 8475546. 
2017-09-30 18:19:06,666 - INFO # c13a, 1726374, 1643696, 8475547. 
2017-09-30 18:19:06,671 - INFO # 675e, 1729352, 1724512, 8475548. 
2017-09-30 18:19:06,678 - INFO # 4262, 1723393, 1717215, 8475549.

Packet deliver ratio: 8475549 (generated) / 8622750 (received) = 98.2928%;
After 49 hours running time, no devices go dead, or be halted.
GoMacH successfully maintains normal sending/receiving capability; To be more specific, all the nodes keep successfully delivering their packets to the receiver, although some of the generated packet will be dropped due to Tx-queue full or meeting the maximum Tx-retry limit.

Actually, with some adaptation/optimization (on the WP period of GoMacH, or having larger TX-queue size), GoMacH's throughput can be further boosted/improved, i.e., achieving higher traffic adaptation. That will be done in the future (maybe in following PRs.) ;-)

miri64 · 2017-11-30T17:16:05Z

Tested successfully on IoT-lab with tests/gnrc_gomach and examples/gnrc_networking_mac (though I noticed that the latency is quite high so I wasn't able to ping with a timeout of 1s; is it supposed to be like this?). Let me have a final look at the code, then we might give this a go.

miri64 · 2017-11-30T17:23:30Z

(okay, after some warm-up period the latency gets a little better)

zhuoshuguo · 2017-11-30T17:27:16Z

Yap, that's great!~

miri64

Some minor nits (the static inlines are internal, so their mis-characterization shouldn't be reflected in size, but it is weird to have >2 line (or 5 lines if they contain an if-else or variables) functions be inline). Please squash changes immediately.

miri64 · 2017-11-30T17:18:05Z

sys/include/net/gnrc/gomach/gomach.h

@@ -0,0 +1,353 @@
+/*
+ * Copyright (C) 2016 INRIA


miri64 · 2017-11-30T17:18:13Z

sys/include/net/gnrc/gomach/hdr.h

@@ -0,0 +1,143 @@
+/*
+ * Copyright (C) 2016 INRIA


miri64 · 2017-11-30T17:18:21Z

sys/include/net/gnrc/gomach/timeout.h

@@ -0,0 +1,100 @@
+/*
+ * Copyright (C) 2016 INRIA


miri64 · 2017-11-30T17:18:30Z