Hi guest.r,
When it comes to plain filtered output I'm quite charmed by the MAME solution, which allows a point prescale factor to be set before filtering is done. Usually with "prescale" factor at 2 or 3, the filtered (bilinear) output is quite sharp (sharp enough to be alike real pixel output, but without the pixel wobbles of straight non-integer linear scaling).
The idea is that we just have a "simple" shader that applies does 2x or 3x of point scaling before the bilinear pass.
I wondered if you could take a look at the prescale.fx from MAME and see whether it can be ported to WinUAE?
The source of "prescale.fx" is here:
https://github.com/mamedev/mame/tree/master/hlsl