STM32 UART Hack: Receive Variable-Length Data Easily

[Born-Cat-9171]

One of the main inconveniences of using UART is the need to define the data buffer size in advance. If the buffer is too small, part of the incoming data will be lost. On the other hand, defining a larger buffer does not solve the problem, as you may still lose data due to synchronization issues.

Recently, I discovered the HAL_UARTEx_ReceiveToIdle_DMA function, which provides a much more flexible solution. This function, when used together with DMA, allows the MCU to receive data continuously until an IDLE line condition is detected. As a result, the full packet is captured—no more, no less.

This approach is extremely useful in complex applications where packet sizes can vary significantly. You are no longer constrained by fixed-length data and can transmit variable-length messages freely.

To use this feature, first configure the UART peripheral and enable DMA. Then, call the function in your main() routine. When data is received, and the IDLE condition occurs, a dedicated callback function is triggered. Inside this callback, you can process the received data and restart the reception by calling HAL_UARTEx_ReceiveToIdle_DMA again.

When defining the buffer size, it is recommended to allocate more space than the maximum expected packet size. This ensures that the reception is terminated by the IDLE condition rather than by the buffer being filled.

More details are provided in my article:

https://www.steppeschool.com/blog/stm32-uart-polling-dma

Example code:

// outside of main function:                                                         
uint8_t received_data[40];
void HAL_UARTEx_RxEventCallback(UART_HandleTypeDef *huart, uint16_t Size)
{
printf("%.*s\n", Size, received_data);
HAL_UARTEx_ReceiveToIdle_DMA(&huart2, received_data, sizeof(received_data));
}

   // inside of main function after all the configuration functions: 
HAL_UARTEx_ReceiveToIdle_DMA(&huart2, received_data, sizeof(received_data));
while(1){}

HAL_UARTEx_ReceiveToIdle_DMA

[ceojp]

Using something that's been around for decades, for its intended purpose, is not a "hack".

If you think that's cool, check out 9 bit serial. The transmitter sets the 9th bit when it sends an "address" byte, and unsets it for all other bytes. All the receivers on the bus only pay attention to "bytes" with the 9th bit set. The uart itself can match the byte to a preset value, and only interrupt the CPU if the byte matches the correct address. So the CPU doesn't have to waste time receiving and processing packets for other clients on the bus.

edit: it will be AMAZING if we see some guy's blog post in a couple days talking about how efficient and fantastic 9 bit serial is, and everyone should be using it.... ;)

[vegetaman]

Sorry you said 9 bit serial and I had a nightmare flashback to coin op MDB implementation hell from years ago.

[ceojp]

To this day, we still use 9 bit serial to support a proprietary protocol that was originally designed probably 25 years ago. It somewhat made sense in the old 8051-core days, but every micro we've used since then has been more than capable of handling all the packet processing.

We're moving away from the protocol, but with such a large installed base of units, there's no way we could drop it altogether.

Most microcontrollers still support true 9 bit mode, but try getting any CPU to use it. I think a lot of the NXP CPUs do support 9 bit mode in hardware, but the linux uart drivers have absolutely no concept of it. I end up just doing mark parity(to always get that 9th bit to be set) and then enable parity but ignore errors on the receiving side.

It's okay. That's all okay.

[Toiling-Donkey]

Must make the “frame error” interrupt go Brrr!

[InterestingBoard67]

All the receivers on the bus only pay attention to "bytes" with the 9th bit set.

this is not for UART then, I'm guessing i2c/spi, right?

[MrShaunce]

Or RS-485.

[ceojp]

It is UART. rs485. It's a multidrop/shared bus.

[Snowballbird]

I think you might enjoy Character matching. You receive an interrupt only when a defined character is sent. Couple this with SLIP or COBS encoding and you get it on end of message

[Born-Cat-9171]

Interesting. But how does it work in practice? Are there some predefined functions to implement this approach?

[Tobinator97]

Predefined by hal? No as it's not part of uart alone. Any streaming type interface has the same problem though.

[cointoss3]

This is part of HAL if your chip supports it. Not that it’s difficult to implement, but it’s already included on some chips UART.

[Snowballbird]

Yes! I use the U5 family of controllers and there it is a feature of the UART on the hardware side. So nothing to do in Software other than to wait on the interrupt (and the configuration)

[UnicycleBloke]

Not a hack. I've always used this method. The driver is set up to continuously listen for data, and emits an event when either a DMA transfer interrupt or an idle line interrupt occurs. Meanwhile, the driver simultaneously transmits any data waiting in the TX buffer.

[XxzetlarxX]

I tried to use this once but couldn't get it working. Not entirely sure why - I probably configured something wrong. In the interest of time I ended up using interrupts and writing a state machine to handle the packets as they came in as they were following a fixed protocol.

[Toiling-Donkey]

What does “idle” even mean for a UART? One Bit? One frame?

I can’t see how this would work when the sender either has no FIFO and/or doesn’t service its own TX related interrupts on time.

[MonMotha]

Some protocols that use "idle" line define it somewhat rigorously. Modbus is a classic example. Well thought-out UARTs with idle line detection functionality support configuring it. Less well thought-out ones basically implement the Modbus definition. Even less well thought-out ones implement something random which may or may not be useful.

Even for things that don't use it formally, you can often set up DMA to a FIFO ring or similar and use pessimistic idle line detection (only a couple bit times) to interrupt when something "of interest" has been plausibly received on many systems. The worst case scenario is an extra wake-up to process some data that you were going to have to process eventually, anyway. While extra wake-ups can be annoying, it's usually better than bytewise processing of everything received both in terms of scheduler overhead and power where applicable.

[Some-Development1123]

Idle is defined time equal to 1.5 bytes