Latency tests - No difference between "No buffering" and "Double buffering"

[Dr.Venom]

Hi Toni,

I've finally came around to doing some objective (input) latency tests using a LED wired to a digital joystick and 240 fps camera recording, comparing a real Amiga versus WinUAE.

The results show the latency in frames from a button press to the display changing. I won't bore you with all the data, but only summarize with the Average, Minimum, Maximum (Avg / Low / High). At the end of this post is a short explanation of the testing methodology.

Important finding

An important finding during the various tests is that there's no difference in latency between "no buffering" and "double buffer" mode, both with low latency vsync enabled and with vsync disabled. Both on my AMD setup and my NVidia setup. I.e. we always seem to be getting double buffered latency, regardless of setting.

Hopefully you can investigate what's causing this?


=====================================

Results (latency in Amiga frames, i.e. 1 frame is ~20 milliseconds)

Real Amiga 1200: 0.8 / 0,4 / 1,5
WinUAE 3.5.0, low latency vsync - no buffering: 2,6 / 2,3 / 3,1
WinUAE 3.5.0, low latency vsync - double buffer: 2,8 / 2,5 / 3,1
WinUAE 3.5.0, vsync disabled, no buffering: 2,0 / 1,5 / 2,5
WinUAE 3.5.0, vsync disabled, double buffer: 2,0 / 1,5 / 2,3

Conclusion 1: WinUAE versus real Amiga: additional input latency on average 1,8 frames for low latency vsync and 1,2 frames with vsync disabled.

Conclusion 2: There is no real difference between "no buffering" and "double buffer". One would expect 1 full frame of video latency difference between the two, but the difference is actually ~0:

• low latency vsync "no buffering" vs "double buffer": on average 0,2 frames. I.e. no real difference (0,2 frames is also marging of error at 240 fps)
• with vsync disabled, "no buffering" vs. "double buffer" on average 0,0 frames. I.e. no difference.

To make sure this isn't a driver issue, I checked whether "no vsync" shows tearing and it does.

To further make sure I did the same tests on a completely different PC with an NVidia GTX 1070 (the one attached to CRT has an AMD HD 6850) and again: no difference between "no buffering" and "double buffer", both in low latency vsync mode and with vsync off. (NB. I needed to revert the NVidia driver for the GTX 1070 to one from March 2017, as with the newer NVidia drivers low latency vsync will not work.)

==========================================================

Testing setup and methodology:

• Real Amiga vs WinUAE on Windows 10 x64 with Intel i7-7700K
• Both setups use the same DB-9 joystick and the same display (CRT)
• The PC uses a lagless converter for the joystick (I-PAC 2)
• The joystick has a LED wired to the button, such that the camera recording shows when the button is pressed (LED goes off)
• For the tests I used the "Button" program (copyright Mr. Crystal Ball, thanks Toni ), also attached.
• The Button test with a value of 260 (AmigaDOS -> "Button 260") was used, which basicly checks at line 260 of the raster position whether the mouse button is pressed, and if so waits for vsync and then turns the screen yellow.
• This was filmed with the iPhone 6 720p slow-motion mode, which captures at 240fps
• For each run I recorded ten button presses
• The camera recording was stepped through frame by frame, using SMPlayer for Windows, counting the number of frames from button press (LED off) to on screen action (screen turning yellow). The results for all button presses were averaged and then converted to Amiga frames (1 Amiga frame equals 4,8 camera frames)

EDIT: Some additional questions on the testing methodology have been answered in the thread Input latency measurements. Please take a look there if you're looking for more in-depth information.

[mark_k]

Interesting.

Was your testing with WinUAE in full-screen mode?
Are you able to test on Windows 7 with DWM disabled to see whether that makes any difference?
You could try adding -extraframewait option too, if you carry out the tests again.

[Dr.Venom]

Yes, all tests were done with full-screen mode.

I wish I still had a Windows 7 setup running, for easy testing, but unfortunately I do not. But then again, as far as I understand disabling DWM in Windows (7) is only helpful for full-window applications, like the standalone BSNES / Higan emulator etc. It should have no impact on real full-screen applications.

The extraframewait option in WinUAE is currently unsupported for a reason I think, as it does not work reliably with the Wasapi audio modes (the framewait loop gets aborted as soon as a wasapi pull event happens, making extraframewait only work partially). In my experience it doesn't really help a lot currently, but using it makes the emulation / sound buffer less stable. It has the status of a patch-on / hack (therefore unsupported), and Toni mentioned a solid extraframewait implementation needs a rewrite of various parts.

[Dr.Venom]

I did an additional test, changing the rasterline where the button test checks whether a button is pressed.

This is meant to illustrate potential benefit of a frame delay.

When measuring the input response with the button test set to rasterline 260 we got the following latency in Amiga frames (Avg / Low / High):

Real Amiga 1200 ("Button 260"): 0.8 / 0,4 / 1,5

Now I did a test with "Button 25" which moves the check for a button to the start of the frame (about first line where visible display starts)

This closes the "window" where input can be read by 260-25 = 235 lines. One would thus expect for the "button 25" test to have a 235 / 313 = 0,75 frame of higher input latency (in other words 15 milliseconds slower input response). Running this test on a real A1200 gave the following:

Real Amiga 1200 ("button 25"): 1,7 / 1,5 / 2,1

The difference being: 0,9 / 1,1 / 0,6 which comes pretty close to the expected 0,75 frames of additional latency. Of course sample size is small on both button 260 and button 25 tests (only ten presses each), so probably if both button tests were repeated 1000 times, it would sort of start to match the 0,75 frame of latency difference.

So how does the same test compare for WinUAE? The expectation would be that WinUAE would show limited to no difference between the two, because the emulation is run at the same point of the host PC frame, mostly the beginning of the frame (?). Results for low latency vsync:

WinUAE 3.5.0 ("button 260"): 2,6 / 2,3 / 3,1
WinUAE 3.5.0 ("button 25"): 2,9 / 2,5 / 3,1

The difference being: 0,3 / 0,2 / 0,0. Close to zero difference.

A proper framewait implementation could thus, for this specific testcase (on a fast enough PC), respectively gain 0,6 / 0,9 / 0,6 frames.

This is of course just an illustration of the potential benefit of a proper frame delay implementation.The theoretical benefit could be close to a whole frame in certain situations.

Bottomline

Toni, if and when you have some time / energy for working on a robust extraframewait implementation, I think we now have at least an objective testing methodology we can use to see whether or not a frame delay function works as expected (no more subjective judgement /guessing!)

It would of course be a pleasure for me to help / do those tests .

[Dr.Venom]

Hi Toni,

Trying to investigate further into why the latency is the same for no buffering and double buffer mode, both with vsync disabled and enabled (see first post).

I checked no-buffering and double buffer visually both with vsync disabled, and I see tearing in both no-buffering (expected) but I also seen tearing with double buffer enabled. Is that supposed to be when vsync is disabled? (triple buffer is only mode that does not show tearing with vsync disabled).

What do you think? It would be good to know your opinion before I try to do any additional tests / investigation / keep bugging you with it...

[Toni Wilen]

I remember seeing difference between no and double when I had my triple monitor nvidia surround setup (different tearing effect) but I can't see any obvious different now. (single 3440x1440 100Hz ultra-wide monitor)

There are too many unknown variables in display chain..

[Dr.Venom]

Originally Posted by Toni Wilen

I remember seeing difference between no and double when I had my triple monitor nvidia surround setup (different tearing effect) but I can't see any obvious different now. (single 3440x1440 100Hz ultra-wide monitor)

There are too many unknown variables in display chain..

Yeah, it's not trivial at all, unfortunately..

I did a little test with "-scanlineadjust", to purposedly create tearing (when normally there is not), to see if no and double would behave differently, and they do.

If I use "-scanlineadjust 150" with low latency sync and no buffer I get a tearing line around the middle of the screen. If I enable double buffering, the tearing goes away. Enable no buffer again and the tearing is back.

Could you shortly explain a little what the "-scanlineadjust" does technically? (E.g. Is the moment where the emulation in the frame runs also affected?)

I noticed that with a scanlineadjust value of 180 the video display speeds up a lot in no buffer mode but not with double buffer. But for both no buffer and double buffer the sound will completely be distorted. Would that be expected behaviour?

I intend to do some more latency tests, so would be great if you could give some color to the above, just that I understand what is technically happening when using the scanlineadjust parameter..