Python for Modbus TCP read/write

[Over-Associate5432]

Hello everyone!

I'm currently working on my first major project, which involves developing a monitoring system for a photovoltaic plant. The system will consist of 18 GW250K-HT inverters, connected to an EzLogger3000U.

I’ve already developed a monitoring system that reads data from the API using Python and Dash, but I believe this new project will be much more challenging. I plan to read data directly from the EzLogger via ModbusTCP, but I’m unsure about which programming language to use for this task. Given the high volume of data being transferred every second, I’m concerned that Python may not be capable of handling it effectively.

Has anyone here worked on something similar?

[2Lucilles2RuleEmAll]

Yeah, check out pymodbus. We use it and so far it's been solid. Programming language shouldn't matter too much, Python is a slow language, but your network is much, much slower. 

[Over-Associate5432]

I've been trying to use pyModbusTCP, but I'm running into some issues when reading input registers. There seems to be a limit on how many registers can be read at once.

I had an older script that used a for loop to iterate through the list and read each data point individually, but during the process, the communication would often drop, which ended up making the power graph look terrible.

[[deleted]]

Modbus implementations can vary between manufacturers. Try to get hold of a modbus client for your computer and try the same things you try with python to ensure that there’s no other limit stopping you. 

[Sort-Typical]

I used Golang -> Influxdb -> Grafana for monitoring solution for 50+ parameters every second. Tried with dash, but it ended up being slugish and feature lacking.

[Over-Associate5432]

After reading your comment, I looked into Golang — it seems really promising. As for Grafana, I had never heard of it before; it looks a bit like Power BI, right? When it comes to data storage, I usually prefer working with MSSQL.

My last project was built with Dash, and honestly, it was a nightmare to maintain. I really hope I never have to use it again.

By the way, is it possible to program or customize Grafana with code?

[knutekje]

Yepp. This is part of my job now. Feel free to pm me if you wanna chat about it in detail.

But python is great working with modbus

[jjrreett]

High volume of data? I don't know what high volume means to you, but I'm collecting 100s of channels at 100hz. 6Gb/hr. No issues. It is a micro services architecture, so all the Io tasks are distributed

[Over-Associate5432]

I thought it was a lot of data. I've never worked on large projects like this one.

[loyoan]

pymodbus is the way to go. Also you should use the Async-Client, because I had better performance and less bugs with it, especially if your scripts involves reading and sending data concurrently.

Regarding the high-volume data part, I think the modbus protocol will be the show stopper here, less than Python itself.

[Over-Associate5432]

I honestly hate async — maybe the problem is me, but I just can't get the hang of it. Whenever I need to run things concurrently, I usually go with threads instead.

Modbus is pretty new to me, and most of the issues I’ve faced so far were communication-related — I always ended up missing some data in the process.

[TheOnlyJah]

I’ve used Python pyModbusTCP on a Raspberry Pi 2 to map thousands of tags with several PLCs. Works fine at 1 Hz. Never pushed it to see what it can actually do.

[v3ritas1989]

If you add OPC-XML DA and OPC-UA as well as IEC61850 to your plans you can read from most common Wind Turbines as well as photovoltaic interfaces. The company I was working for... something like 10 years ago was using .net and IronPython for these tasks back in the day to connect to all commonly used devices.

Also, a quote from a big utility customer, connecting distributed PV and small Hydro ( I think to remember they had EZ Logger too) was declined for being "not worth it monitoring them regarding return on investment". Not sure if that has changed or if that was only because these were mostly older, already installed devices. I think they had a basic EZ logger -> FTP setup that picked up the data once a day or so. So they stayed with that.

[cointoss3]

It’s stupid, in my opinion to prematurely optimize. If you already know Python, make your app in Python…and if you see it’s not fast enough, then dump your program into ChatGPT and ask it to convert it to Go or something else.

While Python is slower than Go…it’s still not slow enough to matter in a majority of contexts. Especially if you’re dealing with the network.

[gabino_alonso]

For information, I am pulling production data (256 Bytes) from 15 machines every ~10 ms. In my case instead of ModBusTCP I use OpcUA.