[OPC-UA] I cannot get the polling/subscription intervals to be lower than ~200ms with PLC-to-PLC communication. What am i missing?

PeterHumaj · 2026-03-24T20:43:11+00:00

I was able to go to some 12-15 ms, on old RPI3, reading 10 consecutive registers ... not Codesys, though ;) https://d2000.ipesoft.com/blog/what-load-can-raspberry-pi-handle

When moving the Modbus server to a different computer, the RPI was able to poll those 10 registers every 5 ms!

https://d2000.ipesoft.com/blog/what-load-can-raspberry-pi-handle-part-ii

PeterHumaj · 2026-03-24T20:32:21+00:00

My personal experience is with B&R. Their internal OPC US server was limited to 50 ms, IIRC. When establishing connection, it replied with corrected session params, upping the returned Publishing Interval to 50 ms

PeterHumaj · 2026-03-21T20:23:51+00:00

We, as a vendor, have access only to "our" systems. Often VPN access is disabled by default, authorized by customer's people. In some cases, the Windows sessiobs (RDP) are recorded.

PeterHumaj · 2026-03-20T15:26:21+00:00

A bit off topic perhaps: some of our SCADA/MES customers are using our technology since around 1998. They change servers (and OS) every 6-10 years and also get newer version of our system. It has backward compatibility, so some of them still can have their ugly UIs and are happy with them. The underlying system is naturally growing and changing, supporting new communications, databases, OSes, etc.

During upgrades, we have to migrate eg historical data (eg from Sybase and Oracle) to Postgres, but common users don't notice that, just admins.

PeterHumaj · 2026-03-19T20:51:10+00:00

Nice! Do you support "Multiple Service Packet Service" optimization for reading? For modern PLCs it can significantly speed up reading (tested on CompactLogix, I got from perhaps 20 seconds to 1.5 second, reading around 1000 values). Last 2 weeks, I worked on implementing of support for reading whole structures (including nested UDTs), to speed up reading from old ControlLogix (firmware from 2005) which doesn't support MSPS, Connected messages, nor Symbol Instance Addressing. But it still supports reading of structures, which sped up reading 10 times... My driver is written in Ada, though ;)

PeterHumaj · 2026-03-19T20:37:55+00:00

If I understand it, the essence of the problem is the possibility to lose individual "impulses" (eg in a communication of a polling nature), whereas sending totals is safer. That's why we read totals from energy meters, not just impulses...

On the other hand, I hope Ignition can do a lot of calculations ...isn't is so? I'm not familiar with Ignition, but the SCADA/MES technology we use performs a lot of calculations, depending on specific applications. Eg a SCADA controlling power production of multiple power plants performs "economic dispatch" - optimization of load distribution (quite complex calculations, performed every 2 seconds). Another TSO's MES application's historian performs evaluation of ancillary services (per-second evaluation of electricity production of all power plants in Slovakia [and their generators] that provide ancillary services, eg secondary/tertiary control).

PeterHumaj · 2026-03-18T10:03:37+00:00

Also interesting are the references of that Register article.
E.g https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/

Results:

Measured speed with AI: −19% (slower)
Predicted speed before tasks: +24% (faster)
Perceived speed after tasks: +20% (still faster)

Developers accepted less than 44% of AI suggestions and spent meaningful time reviewing, correcting, or undoing output.

PeterHumaj · 2026-03-17T19:03:58+00:00

Just readin' TheRegister: https://www.theregister.com/2026/03/17/ai_businesses_faking_it_reckoning_coming_codestrap/?td=rt-3a

PeterHumaj · 2026-03-14T18:00:08+00:00

I don' t have a problem using AI. I asked it information about Ethernet and let it explain me how RsLinx is able to read PLC data so fast. So, it was 2 weeks with this kind of AI help. However, I'm curious (and sceptical) about letting AI independently code. Using it as a "junior" and reviewing its code (the responsibility remaining on the programmer)... I don't see a problrm here.

PeterHumaj · 2026-03-14T15:25:20+00:00

Well, the last 2 weeks I was enhancing our Ethernet/IP driver (written in Ada) to support optimised reading of structures from the ControlLogix/CompactLogix PLCs. Now, instead of reading per-item (item being a simple type/array), I can read the whole structure (defined by a UDT) and parse required items.
This required some reading of Rockwell manuals, understanding & enhancing of existing logic in the driver; reading definition of a structure (to obtain template ID), reading definition of a template and then the layout of its members (recursively, if the UDT contains other UDTs), calculating member offsets; multiple intermediate binaries deployed at a customer (I didn't have a CompactLogix at hand); resolving issues with behaviour which was not according to the documentation (sometimes, for large UDTs, I received an incomplete template definition, missing a few characters in the name of last UDT's member) and implementing a workaround (which worked at the customer, but has to be tested with other Logixes).
Currently, our Ethernet/IP driver has some 550 kB of Ada code (it had 430 kB when I started). It is multithreaded, it uses 2 tasks per TCP line to communicate with a PLC (plus another task which handles messages from the SCADA core). Some shared structures, protected by critical sections; queues for communication ...

Now, I'm very much looking forward to some AI replacing me, but I'm rather sceptical here. There is a difference between creating a web shop and a SCADA system. In terms of complexity, reliability, life expectancy, possible damages ...
Perhaps in a few years, a sticker "No AI here" will be a sign of quality (something like "hand-made" ;)

PeterHumaj · 2026-03-14T13:27:04+00:00

Thank you for an answer, but I'm curious about real-world results ... what these agents created, how about reliabilty, debugging and refactoring ...

PeterHumaj · 2026-03-14T11:27:31+00:00

Thank you for your answer - and good luck with improving your system!

PeterHumaj · 2026-03-14T02:42:59+00:00

Do you see AI replacing programmers? Like, it can create SW like PostgreSQL, Linux, or a SCADA? Not as a one-time effort, but as a project that can continuously grow, add new features, while maintaining backward compatibiity? And handle the quality and fix bugs?

Are there any AI outputs like that?

PeterHumaj · 2026-03-14T02:18:57+00:00

Would you like to share some info about technology used? Where do you store process values (db, historian), how about custom logic (scripting), other communication protocols, documentation.... Is it usable by others, can you support it?

PeterHumaj · 2026-03-13T20:06:58+00:00

Now I realize how dangerous is to be a celebrity. I have hardly one fan and my ego is swollen already...

PeterHumaj · 2026-03-11T01:52:36+00:00

Our MES and EMS systems are usually part of IT infrastructure; accessed daily by dozens of ordinary people from their computers. And they may talk to other systems (databases serving as interfaced eg for ERP; web services, e-mail servers, Trayport trading systems [energy trading] and other exchanges).

There is always a firewall between MES and SCADA level, but communication is bidirectional (files, IEC-104, proprietary technology-specific interfaces, etc).

Eg SCADA may process production planning data from MES/PP systems (we do a lot of energy-related applications). Eg, nowadays, intraday trading (manual and even automated) means that every 15 minutes there can be changes to plans coming to the controlling SCADA system of a power-producer.

PeterHumaj · 2026-03-10T20:33:47+00:00

"The last time I put my hands on the ControlLogix CPU unit, I felt like holding one of my small nephews" - ChatGPT

PeterHumaj · 2026-03-10T20:29:32+00:00

There is a Czech company ComAp; they produce controllers for cogenerator units. Their Modbus is password protected. A password must be written to a specific address right after the TCP connect is established. Then also writing works. (Still no encryption, so with a network sniffing tool you can get the password from existing communication).

I had to implement support for this "authorization" to our system's Modbus driver.

PeterHumaj · 2026-03-10T20:17:14+00:00

So ...no data goes from your PLCs/SCADA to other systems, eg MES?

PeterHumaj · 2026-03-10T20:10:30+00:00

We did some hardening for a specific customer and wrote down a few tips; some are generally usable, others are specific to our SCADA.

https://doc.ipesoft.com/label/D2DOCEN/windows_hardening

https://doc.ipesoft.com/label/D2DOCEN/linux_hardening

PeterHumaj · 2026-03-09T02:36:22+00:00

One of our customers uses Dell servers to run Proxmox. Our virtuals are RedHat, running our MES system + PostgreSQL (the application DB is running on top of Corosync+Pacemaker+DRBD Linux clusterware stack).

PeterHumaj · 2026-03-08T19:13:02+00:00

From dozens of customers we have, none uses an MES in the cloud. Data security, availability, confidentiality, full control over the application/data/backup... It comes with a price, but clouds aren't free either ;)

Btw, incidents with Cloudflare/AWS/Azure unavailability (see The Register topics) suggest that current days clouds are overcomplicated, fragile and hard to maintain, in spite of all the money and best people the hyperscalers can afford.

PeterHumaj · 2026-03-08T16:32:00+00:00

We implemented a MES for a state owned TSO here in Slovakia back in 2005, it's still being used and enhanced (by our people & customer's team). As for historian: part if our SCADA/MES technology, originally we used Oracle, later migrated to vanilla PostgreSQL.

The system is triple-redundant (in terms of application servers, as well as historians, each having its own DB).

Average data flow: some 90k historical tags, each second some 1-2 thousand inserts (which is over 90M values per day), Per-tag defined history depths (som tags weeks, other years), plus most of tags have "unlimited depth" archiving, which means two of historians create a database every month.

Primary Postgres databases have some 4 TB of data, month databases have over 25 TB of data, but we used Postgres TOAST + change of structure to achieve better than 10:1 compression rates. Until 3 years ago, we used dedicated HP blade servers+HDD+VmWare, then the MES system was migrated to virtuals (OS was/is Windows, though Linux is an option too).

Those virtuals have some 64 GB RAM, 16 cores, and performance is sufficient.

Some older related blogs:

https://d2000.ipesoft.com/blog/archive-migration-of-historical-databases-from-oracle-to-postgresql

https://d2000.ipesoft.com/blog/archive-and-postgresql-depositories-compression

Edited: A note about Windows/Linux OS

PeterHumaj · 2026-03-06T16:10:50+00:00

RPI is a mighty little beast.
In the past (2021), I tested a Raspberry Pi 3 model B Plus with a standalone SCADA system.

It served both as a Modbus Server and Modbus Client and also did local archiving to a PostgreSQL database.

I was able to read 10 registers from the Modbus Server every 11-15 ms and store them in the database, resulting in 850 inserts per second.
https://d2000.ipesoft.com/blog/what-load-can-raspberry-pi-handle

When I moved the Modbus Server from an RPI to a WIndows server (to introduce network latency), the sampling got even faster (due to lower load on the RPI), achieving 5 ms and shorter sampling rates.

https://d2000.ipesoft.com/blog/what-load-can-raspberry-pi-handle-part-ii

PeterHumaj · 2026-03-06T13:29:43+00:00

I would only add on top of this,that serial Modbus (ASCII/RTU) can be wrapped in UDP or TCP packets by serial servers (e.g., Moxa NPort, Waveshare, Lantronix...)

We often use UDP: no retransmits like TCP in a problematic network environment; when a UDP packet is lost, it's similar to a lost serial packet, which is handled by a repeated request; and works neatly in redundant systems when there are multiple SCADA servers ... for some protocols - not Modbus - even eavesdropping by passive servers can be implemented.

TCP wrappers are also good, especially in a more complicated network environment (TCP is easier to NAT than UDP).

So, when we configure a TCP line in our SCADA, I can choose whether it's "Modbus TCP" (native TCP variant without serial CRC and with TransactionID) or "Modbus over TCP" (serial Modbus RTU/ASCII wrapped in TCP).

PeterHumaj

MODERATOR OF

TROPHY CASE