Skip to content

Packet Parsing API

Jump to bottom
Andrew Madsen edited this page Jul 2, 2018 · 7 revisions

ORSSerialPort Packet Parsing Programming Guide

Very commonly, data received through a serial port is structured in the form of packets. Packets are discrete units of data, with some structured format. Because the underlying serial APIs can’t know the specifics of any given packet format, they deliver data as it is received. Responsibility for buffering incoming data, and parsing and validating packets is left up to the application programmer.

ORSSerialPort includes an API to greatly simplify implementing this common scenario. The primary API for this class is ORSSerialPacketDescriptor and associated methods on ORSSerialPort. Use of this API is entirely optional, but it can be very useful for many applications.

This document describes the packet parsing API in ORSSerialPort including an explanation of its usefulness, an overview of how it works, and sample code. As always, if you have questions, please don't hesitate to contact me.

Quick note: This document uses Swift in all code examples. However, everything described here works in Objective-C as well (afterall, ORSSerialPort itself is written in Objective-C). The Objective-C version of the PacketParsingDemo example project is useful for seeing how the things described here work in Objective-C.

Brief Overview

The ORSSerialPort packet parsing API provides the following:

  • An easy way to represent a particular packet format using ORSSerialPacketDescriptor
  • Buffering and packetization of incoming data.
  • Support for multiple simultaneous packet formats.

Detailed Overview

The Setup

For the sake of discussion, imagine a piece of hardware with a knob or slider (aka. a potentiometer). When the knob or slider’s position is changed, the device sends a serial packet:

!pos<x>; (where <x> is the knob position, from 1 to 100.)

Code is available for an Arduino Esplora to implement exactly this system.

The Problem

For the device above, one might write a program containing the following:

func serialPort(serialPort: ORSSerialPort, didReceiveData data: Data) {
    let packet = String(data: data, encoding: .ascii)
    print("packet = \(packet)")
}

However, this won't produce the desired results. As raw incoming data is received by a computer via its serial port, the operating system delivers the data as it arrives. Often this is one or two bytes at a time. This program would likely output something like:

$> packet = !p
$> packet = o
$> packet = s4
$> packet = 2
$> packet = ;

Of course, we'd much prefer to simply receive a complete packet consisting of !pos42;!

In the absence of an API like the one provided by ORSSerialPort, an application would have to implement a buffer, add incoming data to that buffer, and periodically check to see if a complete packet had been received:

func serialPort(serialPort: ORSSerialPort, didReceiveData data: Data) {
    buffer.appendData(data)
    if bufferContainsCompletePacket(self.buffer) {
        let packet = String(data: data, encoding: .ascii)
        print("packet = \(packet)")
        // Do whatever's next
    }
}

This very simple approach will often be acceptable. However, it suffers from a number of possible problems. It is actually deceptively difficult to implement in a robust, flexible way, and there are some very common mistakes to be made when implementing an incoming packet buffering and parsing system. These problems are compounded when multiple different packets are to be processed, the connection can be intermittent, etc.

The Solution

ORSSerialPort's packet parsing API makes solving these problems much easier. Using that API, the above program would look like:

func setupSerialPort() {
    ... // Set up serial port
    let descriptor = ORSSerialPacketDescriptor(prefixString: "!pos", suffixString: ";", maximumPacketLength: 8 userInfo: nil)
    serialPort.startListeningForPacketsMatchingDescriptor(descriptor)
}

func serialPort(serialPort: ORSSerialPort, didReceivePacket packetData: NSData, matchingDescriptor descriptor: ORSSerialPacketDescriptor) {
    self.sliderPosition = self.positionValueFromPacket(packetData)
}

Here, ORSSerialPort will handle buffering incoming data, parsing that data, and notifying the delegate when a packet is received. You can tell the port to listen for multiple packet types, and it will deliver them individually. It even supports nested packets, as well as packets that match more than one descriptor.

The API in Detail

You tell ORSSerialPort about the details of packets you're interested in using instances of ORSSerialPacketDescriptor. There are four ways to create an ORSSerialPacketDescriptor, in order from simplest to most sophisticated:

  • Using a fixed sequence of data
  • Using a prefix and suffix
  • Using a regular expression
  • Using a custom 'packet evaluator block'

If the packets you are interested in contain a simple, fixed sequence of bytes (ie. don't change at all from packet to packet), use:

init(packetData:, userInfo:)

If the packets you are interested can be validated using a simple prefix and suffix, with variable data in the middle, use one of:

init(prefix:, suffix:, userInfo:)
init(prefixString:, suffixString:, userInfo:)

If your packets can't be validated using a simple prefix and suffix, but are text (ASCII or UTF8), you can provide a regular expression to use to match valid packets:

init(regularExpression:, userInfo:)

Finally, if your packet format is too complex to validate using one of the methods above, you can provide a block that takes a chunk of data, and returns YES or NO depending on whether the data consists of a complete, valid packet. For example, you could use this approach if your packets contain a checksum that must be checked to determine if the packet is valid. Use:

init(userInfo:, responseEvaluator:)

Important note about packet length: All of the init methods for ORSSerialPacketDescriptor include a maximumPacketLength argument. You must provide a valid value for this argument. The value you provide must be equal to or greater than the maximum length of packets matching the descriptor. However, you should not pass in a value that is too big, or performance will suffer.

The Response Evaluator

In your implementation of the response evaluator block, you should only return true if the passed in data contains a valid packet with no additional data at the beginning or end. In other words, be as strict and conservative as possible in validating the passed-in data. You should also be sure to gracefully handle invalid or incomplete data by returning NO.

The implementation of the response evaluator block will depend entirely on the specifics of your data protocol.

Since you will usually need to parse a response after it has been received, to avoid duplicating very similar code, it often makes sense to factor response parsing code out into a function or method. Then, you can call this function/method in your response evaluator block as well as using it to parse successfully received responses:

func temperature(from responsePacket: Data) -> Int? {
    guard let dataAsString = String(data: responsePacket, encoding: .ascii) else { return nil }
    if dataAsString.count < 6 || !dataAsString.hasPrefix("!TEMP") || !dataAsString.hasSuffix(";") {
        return nil
    }

    let startIndex = dataAsString.index(dataAsString.startIndex, offsetBy: 5)
    let endIndex = dataAsString.index(before: dataAsString.endIndex)
    let temperatureString = dataAsString[startIndex..<endIndex]
    return Int(temperatureString)
}

Then use it in both your response evaluator block, and your response handling code:

func serialPortWasOpened(serialPort: ORSSerialPort) {
    let descriptor = ORSSerialPacketDescriptor(maximumPacketLength: 8, userInfo: nil) { (data) -> Bool in
        return self.temperatureFromResponsePacket(data) != nil
    }
    serialPort.startListeningForPacketsMatchingDescriptor(descriptor)
}

func serialPort(serialPort: ORSSerialPort, didReceivePacket packetData: NSData, matchingDescriptor descriptor: ORSSerialPacketDescriptor) {
    self.temperature = self.temperatureFromResponsePacket(packetData)
}

Example App

A simple example app showing how the packet parsing API can be used can be found in ORSSerialPort's Examples folder. It is called PacketParsingDemo, and both Objective-C and Swift versions are available. It expects to be connected to an Arduino Esplora board running the firmware found in this repository.

The Arduino firmware sends a serial packet any time the onboard slider's position is changed. The app simply listens for packets matching the format !pos<x>; and uses the value in received packets to update the value of an NSSlider.

The serial communications are implemented essentially entirely in SerialCommunicator.swift.

Clone this wiki locally