I think I found the reason for the Barbican slowdown. When it has no target the ballista enemy keeps doing expensive tracelines for the player to see if he is visible, even when it's not even in the PVS. I'm pretty sure this is an oversight, because in other places there are checks for the PVS or it only checks the visibility of entities in a certain radius.
I'll make a new progs.dat and test if it fixes the problem without breaking the ballista AI