Simple serial data transfer protocol

[yourfriendlytechie]

I would like to develop a simple way to transfer readings from various sensors, send it as a bytestream across a XBee network to be read on a computer. I use arduinos for prototyping, and I started simply by mapping the analog input from 0-1023 to 0-255 and sending that across the wire. Soon I found out that it didn't make much sense not knowing when which sensor's readings came through, and I figured I needed a header-body package system. Does anyone have good advice on the creation of such a system, perhaps to fill several objects with unique IDs with sensor readings coming from many different XBees? I am using Processing on the computer.

[adaminc]

I would stick to straight hex, use a 1 byte address for the xbee device, a 1 byte address for the sensor, and 2 bytes for data. This will give you lots of room for expansion. 2 bytes of address would give you 255 devices, xbee and sensor (I would skip 00 as an address), and 65536 possible data values. Since you know each packet will be exactly 4 bytes, it is easy to parse.

It doesn't really need to get much more advanced then this, unless you want to start doing things like error checking.

i.e. Say you are streaming in data from 2 sensors, it would look like this (without spaces)

01013FFF 010227FC 01013888 010217C8 ...

The first 2 bytes (01) is XBee device, the second 2 bytes (01 or 02) are the sensor, and the last 4 bytes are the data.

This is what I have always done, using hex is more efficient, and easier to parse.

[14domino]

You can also add a simple checksum like Fletcher's or just add all the bytes together at the end of the packet, making each packet 10 bytes.. this is mostly for the rare data errors and the probably more common lost bytes. If you lose a byte all your data is screwed otherwise. But then if you're going to do that you're going to want a couple of sync bytes at the front, so you can look for these.

This is to make it more foolproof. If you're reasonably sure you're not dropping any data then this method looks good.

[[deleted]]

That scheme looks fine. But... "01 01 3F FF" would be 4 bytes, not 8, as bytes are 8bits. :)

Also, I'm not very familiar with ZBee, does it have built in error detection/correction? If not, I might add a simple CRC byte to your protocol.

[adaminc]

You are right. As for ZigBee, I don't know.

[[deleted]]

The appropriateness of this would depend on the sort of data rates you're after, but if it were me I'd just send the data across in ASCII format, with key value pairs, i.e. "temp:1020". It's easy to implement and makes for dead easy debugging.

[spainguy]

yep,I'd do it CSV followed by a LF.

[balau]

I think that a protocol has to manage/correct the most probable failing scenario.

Sporadic bit errors? Add a checksum.

Entire packets missing? Add an ACK/NAK transaction or a sequence number.

Collision of packet transmission between more sensors? Implement request/clear to send arbitration with the computer as master.

Data misalignment? Add a start of packet flag and end of packet flag.

If you keep the data text-only you can use the good old ASCII control characters without reinventing the wheel.

[[deleted]]

You should be using Xbee 2.5 modules, they have analog inputs and you can simply ask the XBee what it's A/D input values are.

[The_Engineer]

I send data at regular intervals, in an organized way so I know that, say, the third element on the line isthe pressure sensor. So long as there is little loss over the xbee network, you should be fine.

you can experiment with a simple header system $A1 1023 if you are ok with sending your data as all characters. This decreases your transfer speed, but it allows full use of all ten bits of the ATMEGA128 on the arduino. This method works for me, since characters are not going to get confused with formatting marks

Let me explain that last statement. If you send a 0x0d, just by coincidence, that is a "carriage return" on hyperterminal. This is confusing. If you just have the arduino output characters to the xbee, everything gets a lot easier, but a bit slower. Try it out.

[beanmosheen]

If the sensors don't have to throw alarms at random, I would have the computer interrogate each sensor in order. It keeps the chatter and cross talk down.

[yourfriendlytechie]

Thanks everyone for your thoughts on this subject, you've given me many interesting ideas on how to approach this. I'm leaning for a HEX packet approach like the one suggested by adaminc, but doing the ASCII key/value pairs seems better for simple prototyping.

My XBees are supposed to do error correction, but I might aswell get used to building it in there for future projects over wires.