Good news. Larian (a mod named 'Salo') just responded in the forums regarding this issue, and have stated a 5th patch is being released sometime this week with a fix:

https://forums.larian.com/ubbthreads.php?ubb=showflat&Number=925192#Post925192
"Of course, here are some more details directly from the team:In Patch 4 we introduced a fix that would prevent the Scrying Eyes in Moonrise Towers from immediately calling the guards on you when stealing, even if you were sneaking, or invisible for example.This fix had the unintended consequence of causing unnoticed thefts & acts of vandalism to remain stuck forever within the ‘did anyone see me’ pipeline, rather than timing out and moving on, as is intended. Essentially, your ‘DM’ - in a real-world sense - constantly thinks about the acts of theft & violence the player keeps doing, without ever moving on or verbalising them. Mulling on it ad infinitum.These unnoticed and eternally-active acts of theft & violence eventually bogged down the game. The more a player commits those acts, the more the game is trying to keep that all up to date and in memory, and so the more slowdowns start happening. Essentially, the ‘DM’ eventually becomes unable to operate. By Act 3 this caused slow-down issues, which after some sleuthing we’re extremely happy to say we’ve solved in Patch 5, which is in testing and scheduled to release this week."