I've released a few AGA demos that didn't use the AGA fetch modes. They had mainly c2p effects, no sprites or hw scrolling, so there was no technical reason I couldn't use 4x fetch.
For example this demo:
https://www.pouet.net/prod.php?which=69702 runs about 10% faster with this fixed.
In other cases like when using blitter, on slow CPUs etc the difference can be much bigger.
So, on AGA, always use 4x if you can.