Who says?
I am not sure that is right, if you mean because the EXE is bigger. At least on the PC, because it only caches what is run... On miggy it could be different though. But if you mean because the added features slow it down, then that is sensible I guess.
But I disagree (slightly) they do continually looks for speed improvements, though as it is not the main focus.... probably not a huge effort goes into this.
An example though, would be Aaron adding a dynarec core. Not very accurate to the hardware that!