Input latency measurements (and D3D11)

[Dr.Venom]

Hi,

I did some additional input latency measurements and thought I'd collect them in this thread for future reference. These are objective input latency measurements comparing WinUAE to a real Amiga. For people interested in the methodology, it's summarized at the end of this post: Latency tests - No difference between "No buffering" and "Double buffering". That post also explains the button test.

The purpose of this thread is that we get a reference of the actual input latency between a real Amiga and WinUAE on a fast PC, both attached to the same joystick and display hardware. With this:

• any potential future input latency improvements in WinUAE can be objectively tested
• potentially a comparison can be made between the latency of different Amiga Emulator implementations (WinUAE, FS-UAE, GroovyMAME's Amiga core, Retroarch core, etc..)
• Any potential regressions between WinUAE versions could easily be tested for

There are three tests:

1. "Button 260" - checks for a button press near the end of a frame then waits for vsync and then turns screen yellow
2. "Button 25" - checks for a button press at the beginning of a frame then waits for vsync and then turns screen yellow
3. Turricun II "jump" - this measures the latency of a jump of Bren McGuire (yep that's his name ) at the start of Turrican II. This may be interesting as it's a real game example.

The results are the number of Amiga frames of latency from button press to on-screen change. It's the measured average for a number of button presses.

The results:

"Button 260" :
Real Amiga 1200: 0,8 frames
WinUAE 3.5.0, low latency vsync - no buffering: 2,6 frames
WinUAE 3.5.0, low latency vsync - double buffer: 2,8 frames
WinUAE 3.5.0, vsync disabled, no buffering: 2,0 frames
WinUAE 3.5.0, vsync disabled, double buffer: 2,0 frames

"Button 25" :
Real Amiga 1200: 1,7 frames
WinUAE 3.5.0, low latency vsync - no buffering: 2,9 frames
WinUAE 3.5.0, vsync disabled, no buffering: 2,7 frames

Turrican II jump
Real Amiga 500: 2,2 frames
WinUAE 3.5.0, low latency vsync - no buffering: 3,7 frames

Using the debugger ("dj 6") with logging enabled it shows that Turrican II at the start reads input at line 273, which is late in the frame. Interestingly, when you move right towards the waterfall, input reading gets pushed even later in the frame, like at 312 and wrapping to the next frame.

Checking some other games (Hybris at line 255, Addams Family at line 256, Leander in-game at line 255, Jim Power at line 294), it seems safe to say that most games check for input late in the frame, which would suggest after the game loop.

Conclusion:
WinUAE's input latency in vsynced low latency no buffer mode is between 1,2 and 1,8 frames behind a real Amiga, depending on where the (emulated) Amiga reads its input. If it's late in the frame then latency tends more towards 1,8 frames of latency, if it's early it tends more to 1,2 frames of latency. Since most games seem to check input late in the frame, the additional input latency versus a real Amiga with games will tend more towards 1,8 frames.

Can this be lowered further? Not much. WinUAE is pretty much optimized already regarding video buffering and how it polls input (using rawinput and acquiring input at the moment the emulated Amiga reads it).

The only remaining option is the so called frame delay implementation. This could lower the latency versus a real Amiga by about 0,8 frames on a fairly fast PC, so to between 0,4 (1,2-0,8) and 1,0 (1,8-1,0) frames depending on whether the game reads input early or late in the frame.

A total input latency of 0,4 to 1,0 frames would practically be the holy grail. If you also feel strongly about that being a great addition to WinUAE, please let Toni now.

[Retro-Nerd]

Interesting. Turrican II with 2,2 Frames on real Amiga. Thought this would be nearly zero, using a normal Stick (no USB converter) and a CRT. Or did you use a LED/LCD with your real Amiga?

For Frame Delay: Retroarch has this, also a GPU hardsync option. Haven't tested WinUAE/FS-UAE but the 240p Testsuite for e.g. Sega Genesis gave pretty good results. Not sure if this is the real latency of Windows 10+the Emu Cores, with optimized settings i get 13-16ms lag.

edit: Most of the Amiga games are optimized for a Joystick. It's pretty normal that the "jumps" are late due to the stick controls. Have you tried Turrican III? This one has a real jump button.

[Dr.Venom]

Originally Posted by Retro-Nerd

Interesting. Turrican II with 2,2 Frames on real Amiga. Thought this would be nearly zero, using a normal Stick (no USB converter) and a CRT. Or did you use a LED/LCD with your real Amiga?

For both CRT is used (please read testing methodology, link at the top of the post).

But yeah, I think most people are under the impression that real hardware has next frame response on most games, while they do not. I was under the same impression until some time ago. "Unfortunately" most Amiga games have similar latency to Turrican II.

The user Brunnis did some nice tests with a real SNES (for which most people also expect next frame response). His test shows that Super Mario World on a real SNES, connected to CRT has 3,6 frames of latency... See his post here An input lag investigation

With regards to Turrican II, the explanation for the latency is somewhat straightforward. It reads input late in the frame, after the game loop (at about line 273). Since a full frame takes 20 milliseconds, you will start with on average ~10ms delay for input to be registered by the game. This input is then applied in the subsequent game loop that runs in the next frame, which adds another 20ms. Then it is scanned out by the raster. A full scan out takes 20ms (i.e. for the beam to reach bottom position). Since the Turrican character stands abouth 3/4 towards the bottom, about 3/4ths of the 20ms scanout is added. I.e. you have 10+20+15 = ~45 ms of latency on average, which equals about 2,2 frames. Of course how close you push your button to the rasterline where Turrican reads input, and where the main sprite is standing will affect the input latency .

[Retro-Nerd]

Mmh, i've read that a few days ago. Obviously missed the part where he uses a CRT. Somehow i'm still not convinced, 3,6 Frames on real hardware would be a huge delay.

[vagrant]

Groovymame's frame delay modifier makes a world of difference.. I simply cannot use mame without it anymore. Low latency vsync implemented into winuae was a really nice improvement and it's pretty tough to notice any lag with digital inputs, though I still notice it with mouse, especially after switching between real hardware. If it could be further improved would be awesome

[mark_k]

Here's a tiny program (104 bytes long) with source which could possibly be used for display/input latency comparison.

https://www.media!fire.com/file/97fgadd6go7ka5j/LatencyTest.lha

It disables interrupts. Sets background colour to blue then to yellow immediately (or as immediately as is possible) on pressing left mouse button or joystick fire button. Press right mouse button to reset back to blue. Press both left and right buttons to exit.

If the raster beam is in the visible display area, where the blue->yellow transition is corresponds to the moment the button press was detected.

[Dr.Venom]

Originally Posted by mark_k

It disables interrupts. Sets background colour to blue then to yellow immediately (or as immediately as is possible) on pressing left mouse button or joystick fire button. Press right mouse button to reset back to blue. Press both left and right buttons to exit.

I don't (yet) see what this would add?

If you're after non-vsynced response times, then the "Button" program that I attached in the other post already has this functionality. If you use it, you'll see the screen turn red immediately as soon as a button is pressed. Then it waits for vsync and turns screen yellow.

I have focused on the vsynced response as it is the most relevant to us (most games are vsynced).

[Retro-Nerd]

Just read your test equipment. I still think there could be a weak link in your technics chain to measure it 100% accurately. Would love to hear more opinons from the usual EAB tech gurus (and still can't believe the 3,6 frames for a real SNES). And if Turrican II is programmed to be that slow/late than my above comment (the edited part) would be another explanation for it.

[modrobert]

Originally Posted by Dr.Venom

The purpose of this thread is that we get a reference of the actual input latency between a real Amiga and WinUAE on a fast PC, both attached to the same joystick and display hardware.

I have some doubts and questions about the red part there regarding latency.

1) What exactly are the electronic interfaces used on when testing the "same" joystick on a PC?

2) Also, in the case WinUAE using the same CRT, how is that achieved exactly?

The Amiga uses digital joysticks, so assuming there is some kind digital <-> USB to adapter when this is connected to a PC, or if USB joystick is used via adapter to the Amiga. In either of these cases you will most like have a weak (low power) MCU adding latency, not to mention serial to parallel protocol conversions.

The same goes for the CRT connected to PC, it will require some kind of display adapter handling the HDMI or display port signal to process the analog CRT video (RGB or composite) signal adding to latency.

When dealing with latency there are so many factors, even the speed of light when considering low latency solutions. The speed of light (photons) travel roughly 30cm (~299.792458mm) in 1ns, and electrons move a bit slower, which means you introduce over 3ns lag by just having a 1 meter cable. This might seem as a joke, but if you are going to accurately measure latency all factors need to be taken into account, especially any technology converting parallel to serial and analog to digital.

EDIT: Found the joystick adapter used in the other thread, "lagless" I-PAC 2. Unfortunately nothing is "lagfree", so my doubts still stand. Out of curiosity, what MCU is I-PAC 2 using? Can't find any official latency/lag specifications for I-PAC 2 when searching either. Still, there is no mention of the display adapter used for the CRT in the PC/WinUAE case.

[Dr.Venom]

Originally Posted by modrobert

EDIT: Found the joystick adapter used in the other thread, "lagless" I-PAC 2. Unfortunately nothing is "lagfree", so my doubts still stand. Out of curiosity, what MCU is I-PAC 2 using? Can't find any official latency/lag specifications for I-PAC 2 when searching either. Also, there is no mention of the display adapter used for the CRT in the PC/WinUAE case.

I understand your interest in this topic, there's a lot unknown about USB joystick adapters (most are relatively slow), or the things needed to run low resolution screenmodes natively on a CRT connected to a PC. I've been down that rabbithole myself in the past. I'll take your questions one by one.

I-PAC 2
A standard I-PAC 2 has a maximum latency of 4 milliseconds, thus on average 2ms. My tests were done with both the standard I-PAC 2 and also the new "2015" version (it allows for firmware updates) and a firmware that has the controller interface at 1 ms, thus on average 0,5ms latency. Andy Warne from Ultimarc was so kind to supply me the custom firmware. For practical uses 0,5 ms average latency is about zero latency or "lagless"*. If you're interested into looking this up yourself, download the tool USBView from Microsoft. This tool will show you the "Device Bus Speed" and "bInterval" of all connected devices. You can then look up the bInterval value on the USB_ENDPOINT_DESCRIPTOR structure page on MSDN, and it will show you the actual polling "period" / speed the device is configured at.

*Any lags still present would have to do with the USB Stack in Windows.

If you're interested in the hardware side of the I-PAC 2, please contact Andy Warne at Ultimarc.com, he's a very knowlegdeable guy. According to him the MCU of the I-PAC2 is easily capable of servicing 1000hz (1ms) polling interval.

Native RGB output on PC
With regards to the CRT connected to the PC. There is no adapter or conversion. There's a project called "CRT_Emudriver". The name doesn't tell it, but it is "simply" a patched driver for AMD Radeon Cards that (re-)enables low resolution video modes in the driver. It's by the same guy who is responsible for the GroovyMAME patch. Look it up here: New CRT Emudriver/VMMaker/Arcade OSD download, documentation and discussion site. All you need is an AMD card with analog output, i.e. VGA or DVI-I on the back and a cable to connect the RGB lines to SCART. The cable you can solder yourself, or buy one, see my post in another thread here.

[Retro-Nerd]

Sorry, but this are too many unknown variables. And the measurement method filming via iPhone cam sounds adventurous too.

Is there really a method to get accurate (or nearly accurate) Input lag values? I could image that would work for real hardware somehow. In PC emulation the software/hardware chain is too variable to get prevailing results.

[Dr.Venom]

Originally Posted by ED-209

Groovymame's frame delay modifier makes a world of difference.. I simply cannot use mame without it anymore. Low latency vsync implemented into winuae was a really nice improvement and it's pretty tough to notice any lag with digital inputs, though I still notice it with mouse, especially after switching between real hardware. If it could be further improved would be awesome

I have a similar experience, on its own low latency vsync is really great already, but switching between real hardware makes the remaining lag noticable. (I notice it mostly on pinball games and fast shoot 'm ups.)

The key remains after switching between real hardware. Maybe that's why it may go unnoticed by many, they simply don't have access to the real hardware anymore..

[modrobert]

Originally Posted by Dr.Venom

I-PAC 2
A standard I-PAC 2 has a maximum latency of 4 milliseconds, thus on average 2ms. My tests were done with both the standard I-PAC 2 and also the new "2015" version (it allows for firmware updates) and a firmware that has the controller interface at 1 ms, thus on average 0,5ms latency. Andy Warne from Ultimarc was so kind to supply me the custom firmware. For practical uses 0,5 ms average latency is about zero latency or "lagless"*. If you're interested into looking this up yourself, download the tool USBView from Microsoft. This tool will show you the "Device Bus Speed" and "bInterval" of all connected devices. You can then look up the bInterval value on the USB_ENDPOINT_DESCRIPTOR structure page on MSDN, and it will show you the actual polling "period" / speed the device is configured at.

OK, did you take those 4ms (worst case) into account in the first post (which is 0.20 frames then)?

*Any lags still present would have to do with the USB Stack in Windows.

Yes, I can imagine that will add as well.

Native RGB output on PC
With regards to the CRT connected to the PC. There is no adapter or conversion. There's a project called "CRT_Emudriver". The name doesn't tell it, but it is "simply" a patched driver for AMD Radeon Cards that (re-)enables low resolution video modes in the driver. It's by the same guy who is responsible for the GroovyMAME patch. Look it up here: New CRT Emudriver/VMMaker/Arcade OSD download, documentation and discussion site. All you need is an AMD card with analog output, i.e. VGA or DVI-I on the back and a cable to connect the RGB lines to SCART. The cable you can solder yourself, or buy one, see my post in another thread here.

How does the CRT_Emudriver deal with the fact that PAL CRT is ~40ms per full frame (25 Hz), half of that ~20ms (50 Hz) for each interlaced/interleaved sweep? This can be important, if the video framebuffer waits for full CRT sweep (~40ms) before updating with next frame, then your PC comparisons could potentially add one whole frame to the result compared to real Amiga.

On the same subject, how do you deal with the full frame when using your 240fps camera? The other post mentioned ~20ms, so guess you don't wait for complete (two sweeps) CRT electron beam scan, right?

[modrobert]

Originally Posted by Retro-Nerd

and still can't believe the 3,6 frames for a real SNES.

The SNES controller protocol is pretty slow, IIRC it sends a packet with button updates every 16ms, so there is lag there even before the data reaches the slow 65816 CPU @ ~3.6MHz.

[Dr.Venom]

Originally Posted by modrobert

OK, did you take those 4ms (worst case) into account in the first post (which is 0.20 frames then)?

I used the new "2015" I-PAC 2 with the 1ms firmware for the tests. If you repeat the button test enough times it may err in 1 out of 20 button presses versus real hardware. In the averages it may thus show up as 1/20th * 1 frame = 0,05 frames of additional latency versus real hardware because of the I-PAC 2 interface latency. It's negligible.

How does the CRT_Emudriver deal with the fact that PAL CRT is ~40ms per full frame (25 Hz), half of that ~20ms (50 Hz) for each interlaced/interleaved sweep? This can be important, if the video framebuffer waits for full CRT sweep (~40ms) before changing dealing with next frame, then your PC comparisons could potentially add one whole frame to the result compared to real Amiga.

None of that plays a part with these tests.

You have to realize Amiga games run in PAL non-interlaced mode (50hz). This is what is called "progressive field mode". Do not confuse this with a PAL TV broadcast signal. It sweeps the electron beam 50 times per second in single field mode and does no alternating between long/odd and short/even field. Interlace / 25hz has no part in this.

As I wrote CRT_Emudriver allows for low resolution screen modes. You can simply add a "modeline", i.e. screenmode in video driver jargon, that exactly matches the Amiga progressive field mode.

I.e. if you want 640x256@50hz or 320x256@50hz or 744x287@50hz, you can add these and have them displayed natively in 15Khz via the VGA out of the video card when using the VGA to SCART cable. It's -identical- to the video output a real Amiga has

[Toni Wilen]

I probably won't bother with this until DX11+ renderer exists, it seems to allow much more flexibility (both latency and variable refresh rate stuff), instead of letting the driver decide too many things..

[Retro-Nerd]

Interesting. Could you go into more detail what a DX 11+ renderer could do?

[modrobert]

Originally Posted by Dr.Venom

You have to realize Amiga games run in PAL non-interlaced mode (50hz). This is what is called "progressive field mode". Do not confuse this with a PAL TV broadcast signal. It sweeps the electron beam 50 times per second in single field mode and does no alternating between long/odd and short/even field. Interlace / 25hz has no part in this.

Yes, my mistake, got confused when I searched for the PAL specs to find out how many milliseconds each frame take.

[Dr.Venom]

Originally Posted by Toni Wilen

I probably won't bother with this until DX11+ renderer exists, it seems to allow much more flexibility (both latency and variable refresh rate stuff), instead of letting the driver decide too many things..

That would be absolutely fabulous. The new features in DGXI apparently allow for easier creation of a low latency swapchain, making it perfect for our case (could be Windows 8.1+ only possibly?).

This blog had an interesting comment about it:

In Windows 8.1, DXGI introduced the DXGI_SWAP_CHAIN_FLAG_FRAME_LATENCY_WAITABLE_OBJECT flag for the swap chain, which allows apps to easily reduce this latency without requiring them to implement heuristics to keep the Present queue empty. Swap chains created with this flag are referred to as waitable swap chains. Figure 3 shows the approximate lifecycle and response to an input event when using waitable swap chains:

The following page seems to dive extensively into that topic (you may have crossed it already.. ): Reduce latency with DXGI 1.3 swap chains

To quote from the article, it seems there may be potential to reduce latency by a full frame versus current methods? If that would be the case it would be truly great stuff.

Imagine 1 frame gained in the swap chain, plus extraframewait implemented, i.e. an additional ~0,8 frame gain (on a fast pc at least) for a total of 1,8 frames of latency reduction. This could - potentially - bring us on par with real hardware..

How does waiting on the back buffer reduce latency?

With the flip model swap chain, back buffer "flips" are queued whenever your game calls IDXGISwapChain::Present. When the rendering loop calls Present(), the system blocks the thread until it is done presenting a prior frame, making room to queue up the new frame, before it actually presents. This causes extra latency between the time the game draws a frame and the time the system allows it to display that frame. In many cases, the system will reach a stable equilibrium where the game is always waiting almost a full extra frame between the time it renders and the time it presents each frame. It's better to wait until the system is ready to accept a new frame, then render the frame based on current data and queue the frame immediately.

[Dr.Venom]

I guess you know most of these, but as always just in case..

I came across these two MSDN pages, they look like helpful (or maybe even mandatory) guides on transitioning from d3d9 to 11.

Direct3D 9 to Direct3D 10 Considerations (Direct3D 10)

Migrating to Direct3D 11

This following looks especially helpful for a way to maintain d3d9 compatibility:

Direct3D feature levels

To handle the diversity of video cards in new and existing machines, Microsoft Direct3D 11 introduces the concept of feature levels.

With Direct3D 11, a new paradigm is introduced called feature levels. A feature level is a well defined set of GPU functionality. For instance, the 9_1 feature level implements the functionality that was implemented in Microsoft Direct3D 9, which exposes the capabilities of shader models ps_2_x and vs_2_x, while the 11_0 feature level implements the functionality that was implemented in Direct3D 11.

Now when you create a device, you can attempt to create a device for the feature level that you want to request. If the device creation works, that feature level exists, if not, the hardware does not support that feature level. You can either try to recreate a device at a lower feature level or you can choose to exit the application.