From looking at the Fusion 40 faq
http://www.lundblad.net/mattias/F40-FAQ.txt it seems that with no ZII RAM, the A2091 doesn't do PIO to fast, but uses DMA to chip, which is then copied to fast, which is slower than either PIO or having a suitable piece of fast RAM to DMA to.
Anyway, potential sources of problems could be not using the version of the 68040.library that came with the F40, or possibly using old versions of the A2091 ROM. I know from experience that the F40 is somewhat picky about which ZII cards it would work with, my Trumpcard grand slam was an absolute no-no, but I think I did have the A2091 working with it for a while, until I upgraded to a GVP series II. However, the card was never entirely stable, I put it down to the poor condition mine was in when I got it (non working and covered in some kind of oil, which had reacted with the IC sockets and turned the metal parts green), however, it was the first 68040 accelerator on the market, and as such probably lacks the refinements of later alternatives. The rev 2.0 ones are probably better, and certainly faster with their 35MHz cpu and interleaved RAM for better memory performance.