Strange.... I thought I've tried this before, but it seems to be stable this evening with the metal rf shield taken out. I also have the case of the A1200 open so I will try to close it and leave the metal shield out to see what it does.
This could mean 2 things I guess? Maybe the metal shield touched the card or the amiga is getting too hot. I have some heatsinks which I can try so maybe this will fix my problems?