sometimes fast memory is only used to save some loading times... and sometimes fast memory is used to load the code: here you get massive speedup.
Most system-compliant games will naturally use fastmem (unless the exec hunks are forced to CHIP like Chaos Engine CD32 or Ultimate Body Blows CD32 grrrrr).
So I'd rather make a list of games which WON'T use fastmem (add the 2 above in the list, unless you're using the whdload patch
)