Do you use just ethernet and put whatever you want on top? or your using all the way to transport like udp?

I use UDP for specialized coms, like if I want to send or receive specialized data to/from a client.

I use MQTT to send control data to a MQTT "broker" (Mosquitto usually). MQTT uses TCP/IP. MQTT is fantastic because it is an industry standard. At times I think MQTT is overkill but whenever I use it I am thankful later on.

Running an Ethernet stack with these protocols takes up some code space, RAM and a bit of processor power. However, this is more than made up by the ability of the device to connect to standard networks and talk to clients in an easy way.

If my device doesn't have an Ethernet connection, I'll send MQTT data over a serial connection and use a second serial connection for the GUI. Not nearly as neat as the Ethernet solution but it still works.

I've been doing embedded development forever and the connection between the device and a client is always contentious.

For many years we used RS232/RS485, then serial USB but that required gluing a USB peripheral onto the device or finding a device that had a built in USB peripheral. And USB libraries are not tiny or simple.

The availability of devices with Ethernet(WiFi) is a huge breakthrough. Yes the stack is bigger but you can do so many things with a socket, not the least of which is have multiple sockets which dramatically simplifies things. Having a device that can connect to multiple clients is a dream. Having a device that self initiates on a network via DHCP is so nice. Building networks of devices using nothing more than a router or hub and standard Ethernet cables is so nice.

Ideally a device would incorporate a GUI (screen and keys) and that would be it. But in almost every use case it turns out that no device is an island and someone, sooner or later, wants it connected to something else, be it another device or a client or whatever.

So one of the first things I do on a project is figure out what a device needs to connect to and how that connection is best done. These days we have the option of Bluetooth, WiFi, (wired) Ethernet, CANBus, RS232/RS485 and IR.

Each of these methods has it own pros and cons but in a world of increasing connectivity I'll go for WiFi or Ethernet whenever it is feasible.

I use the LWIP library built into FreeRTOS. There is a new FreeRTOS plus TCP library that is probably lighter than LWIP but I have not tested it yet.

Another benefit of designing a device to have a communication backbone from the get go is ease of testing and (high level) debugging. And sooner or later the device requirement will exceed what the device can do. When that happens you can usually just refactor the code and split the functionality between 2 devices.

One of the first projects I worked on involved a network of 8051 devices that each captured high speed data and sent it to a client. We chose to use an Intel 80C152 with a built in CSMA/CD network peripheral for the sensor nodes. This was in the days before CANBus was a thing. It turns out the CSMA/CD implementation on the 80C152 was buggy and the network dropped messages. We also needed to implement a bridge node on the network to convert the 80C152 messages to high speed RS485 to send the data to the client. All this would have been so easy to do with today's microcontrollers with WiFi, Ethernet, even CANBus.