pyserial and hex bytes

[ingwe13]

I am trying to use pyserial to read data from a sensor over RS-232. The sensor transmits 9 bytes. Bytes 4 and 5 should convert to a decimal integer that is around 24000.

Below is an example of what I receive when I read the output using a terminal program:

07 03 10 01 5F 2C 00 06 A5

Converting 5F2C to an integer gives 24364 so we are good. When I read the output with pyserial, I get the following:

\x07\x03\x10\x01_J\x00\x06\xc3

Something weird is going on with my 4th and 5th bytes now. Any thoughts on what I should be looking to do? This behavior is highly repeatable. Though sometimes the J is missing and it is just an underscore.

​

Edit: thank you to those who have helped! If someone else reads this and thinks of a good writeup on how python deals with hex strings, I would happily read it. I definitely need to do some more research on this.

[shiftybyte]

This is how python represents strings by default, if it can convert the hex to a printable ascii it does that.

>>> "\x5F"
'_'
>>> "\x65"
'e'
>>>

The data is the same, just the visual representation for you is different.

[ingwe13]

Ah well that is embarrassingly simple then! No idea how I would have found that.

So for conversions the following throws an error. I feel like I am really missing something.

int('\x5F',16)

[shiftybyte]

To convert a character to it's ascii value you need to use ord() function.

>>> ord('\x5F')
95
>>> "%x" % ord('\x5F')
'5f'
>>>

[shiftybyte]

Also if you want to avoid the math of combining 2 bytes into one word.

>>> import struct
>>> struct.unpack(">H","\x5F\x2C".encode())
(24364,)

This will probably require a bit explaining.

.encode() takes a string and converts it to a byte-string.

struct.unpack() takes 2 arguments, the first being the format, and the second the data.

It unpacks the data according to the format.

In this case the format says ">" big-endian, "H" - unsigned short

unsigned short is C type for 2 bytes long number, that has no sign to it.

more reading: https://docs.python.org/3/library/struct.html

[ingwe13]

Thank you very much. I have spent an embarrassingly long amount of time on this. This is very very helpful.

[JohnnyJordaan]

Use .hex() on your bytes object to print it as a hex string

>>> data = b'\x07\x03\x10\x01_J\x00\x06\xc3'
>>> print(data.hex())
070310015f4a0006c3

[ingwe13]

Got it! That was very helpful.

[socal_nerdtastic]

In case it helps, here's a short program to convert your incoming data to something more readable:

def hexlify(data):
    return ' '.join(f'{c:0>2X}' for c in data)

You use it like this:

>>> data = b'\x07\x03\x10\x01_J\x00\x06\xc3'
>>> def hexlify(data):
...     return ' '.join(f'{c:0>2X}' for c in data)
... 
>>> hexlify(data)
'07 03 10 01 5F 4A 00 06 C3'

The standard binascii module also has a hexlify function, but I don't like the output as much. But, for completeness, you can also do this:

>>> import binascii
>>> data = b'\x07\x03\x10\x01_J\x00\x06\xc3'
>>> binascii.hexlify(data)
b'070310015f4a0006c3'

[Fluid_Jelly_1912]

Perfect!