NULL Filter discrepancies

[rsn8887]

Hello Toni,
I think I found a problem that might affect all user-made D3D filters, or it might just be my stupidity. I made my own D3D Null filter and it gives different results with tons of pixel wobble compared to the preset "Null Filter" in the menu and the filter setting "none". All NULL filters should give the same result. The filter and example pics are attached.

I have a hunch it might be related to the quad aligment as in this article:
https://msdn.microsoft.com/en-us/lib...(v=vs.85).aspx

I tried all combinations of re-aligning the quad in the vertex shader and/or applying the 0.5 pixel offset to texture coordinates, but there's always distortion.

The fact that the internal Null Filter gives a nice image without pixel distortion is encouraging. If I can reproduce that behaviour in a user D3D filter, I can make the perfect bilinear_prescale work also.

Do you have the .fx source for the internal NULL filter, so I could compare what is going on?

To test, simply use Integer scaling, only point filtering (bilinear obscures the differences), and look for the differences when you switch filters. The internal filters give much nicer pixels, the NULL filter attached shows tons o wobble.

Here's the shortest NULL Test filter that should give the same results as internal NULL filter, but doesn't:

string combineTechnique : COMBINETECHNIQUE =  "NULL_TEST";
float4x4 WorldViewProjection	: WORLDVIEWPROJECTION;
sampler	sourceSampler = sampler_state {
	Texture   = (source_tex);
	MinFilter = POINT;
	MagFilter = POINT;
	MipFilter = NONE;
};

struct VS_OUTPUT {
	float4 oPosition 	: POSITION;
	float2 oTexCoord 	: TEXCOORD0;
};

VS_OUTPUT vShader (float3 pos : POSITION, float2 texCoord : TEXCOORD0) {
	VS_OUTPUT OUT = (VS_OUTPUT)0;

	OUT.oPosition = mul(float4(pos,1),WorldViewProjection);
	OUT.oTexCoord = texCoord;
	return OUT;
}

float4 pShader ( in VS_OUTPUT inp ) : COLOR {  
	return float4(tex2D(sourceSampler, inp.oTexCoord).rgb, 1.0);
}

technique NULL_TEST {
   pass P0 {
     VertexShader = compile vs_3_0 vShader();
     PixelShader  = compile ps_3_0 pShader();
   }  
}

[Toni Wilen]

.fx is inside direct3d.cpp (includes null filter, overlay and mask handling)

[rsn8887]

I see. I realize there are also some post effect shaders in direct3d.cpp that are output as _winuae.fx. The file _winuae.fx can be used to override at least those without re-compiling direct3d.cpp. Excellent. Those seem to always be active.

And they give the "correct" picture.

So _winuae.fx, and presumably in Null Filter (are they the same?), you use the variable mtx for your WorldViewProjection.

In the user filters, the only semantic available for this is WORLDVIEWPROJECTION.

I have a hunch that they are not doing exactly the same, but it is just a hunch:

If I remove the projection from the vertex shader in _winuae.fx. I get a certain result. It is an Amiga screen that is too small. Note: I simply replaced the
Out.Position = mul(float4(pos, 1.0f), mtx);
with
Out.Position = float4(pos, 1.0f);

to remove any effect of mtx.

If I replace the equivalent projection
OUT.oPosition = mul(float4(pos, 1.0f), WorldViewProjection);
with
OUT.oPosition = float4(pos, 1.0f);
in my user "NULLTEST" shader, I get a different result (an Amiga screen that is about two times smaller again).

Is there any way to access "mtx" from within user shaders?

[rsn8887]

Toni I verified that all user D3D filters are affected by this slight distortion. Although it is seen easiest in my Nulltest filter switching back and forth between it and the internal NULL filter.

It is almost completely invisible when bilinear scaling is on instead of linear but it is clearly there and reduces picture quality.

I think I understand that the post processing filters from _winuae.fx are applied after all other filters, and that the filter labeled "0" is the one that takes the source material and renders it to the screen.

I think filters labeled <0 are the ones applied to the source texture before it is rendered to the screen, and filters labeled >0 are the ones applied after the source is rendered to the screen but before the _winuae.fx filters are applied?

My sharp_bilinear_filter has to be applied as the "0" filter I think, because it needs to take the source texture and modify it as it is being scaled to the screen size or, better, to the target rectangle, which is not necessary the screen size but could be the window size of UAE's window when it is in windowed mode. A quick test with the filters not equal "0" reveals that there's no difference between the internal null and d3d "nulltest" there. it is Just the important "0" filter where it happens. Of course the other filters from the collection also have to be applied as "0" otherwise they give the wrong results.

I don't quite understand why the internal null filter gives nicer results than a user-made direct3d null filter but something must be different. This is perplexing since that _winuaefx filter is applied anyways at the end and when there's no mask or any other user-selected post processing, that basically IS a d3d null filter. I suspect it doesn't cause distortion because it is only acting on the pre-rendered, pre-scaled surface using that mtx projection.

I can minimize the distortion a bit by subtracting 1/2 texelsize from texcoords in either the vertex or the pixelshader inside NULLtest but it there is still clear distortion visible.

Note that all of these are very subtle effects most easily seen observing the dithered scrollbar background in a workbench window, using auto scaling and POINT filtering.

The internal null filter results in an almost perfect checkerboard, while the user d3d filters will have some checkerboard parts thinner than others.

[Toni Wilen]

First, shader stuff is something I am not that interested. Whole code is just a hack made from multiple parts collected from here and there and hacked until it worked like I wanted.. I remember having those same issues and I think some resolution/scaling combinations still make them visible, probably something wrong in some coordinate calculations.

<0 = source and destination size equals internal Amiga display size.
0 = source is Amiga size, destination is host display size.
>0 = both sizes are host display size.

I can do tests if needed but you need to include good, impossible to misunderstand, instructions

[rsn8887]

Thanks! Since I have access to all the matrices within the shader, I think I can debug by concatenating my own worldview projection.

A quick look at direct3d revealed this suspicious line of code that is used for the "middle" shaders:

MatrixOrthoOffCenterLH (&m_matProj_out, 0, w + 0.05f, 0, h + 0.05f, 0.0f, 1.0f);

That absolute offset of 0.05f is probably there for a reason, but since it is absolute it might be the culprit?! I will test some more and get back to you.

[rsn8887]

Toni, I made some more tests. I see now that the two cases of
a) internal Null Filter
b) user-created Null Filter
are quite different.

In case a) the input texture that goes to _winuae.fx for post-processing is small, just the Amiga screen size. This small texture is the output of any of the "BEFORE" shaders and becomes the input to _winuae.fx.

In case b) the input texture that goes to _winuae.fx is much larger, ~the size of the host screen. It is the output of the "MIDDLE" shader.

Therefore it is quite understandable, although disappointing, that the two cases give slightly different results.

In addition, I realized that VPOS cannot be used to accurately calculate the magnification factor that happens inside the "MIDDLE" shader.

I understand each shader renders it's final output to its own s->lpTempTexture. (_winuae.fx post processing is an exception).

I would therefore like to ask is it possible in a future version of WinUAE to export a shader semantic called "TARGETDIMS" that simply contains the size of the s->lpTemptexture for that shader? For the BEFORE and AFTER shaders this would be identical to SOURCEDIMS but for the MIDDLE shader this would be very useful since it will allow me to calculate the magnification scaling factor between source and target etc.

In the source, these targetdims are given as w2,h2 in "createtexture."

static int createtexture (int ow, int oh, int win_w, int win_h)
{
	HRESULT hr;
	bool haveafter = false;

	int zw, zh;

	if (ow > win_w * dmultx && oh > win_h * dmultx) {
		zw = ow;
		zh = oh;
	} else {
		zw = win_w * dmultx;
		zh = win_h * dmultx;
	}
	for (int i = 0; i < MAX_SHADERS; i++) {
		if (shaders[i].type == SHADERTYPE_BEFORE || shaders[i].type == SHADERTYPE_AFTER || shaders[i].type == SHADERTYPE_MIDDLE) {
			int w2, h2, w, h;
			if (shaders[i].type == SHADERTYPE_AFTER) {
				w2 = zw; h2 = zh;
				w = zw; h = zh;
				haveafter = true;
				if (!allocextratextures (&shaders[i], window_w, window_h))
					return 0;
			} else if (shaders[i].type == SHADERTYPE_MIDDLE) {
				// worktex_width = 800
				// extratex = amiga res
				w2 = zw; h2 = zh;
				w = zw; h = zh;
				if (!allocextratextures (&shaders[i], ow, oh))
					return 0;
			} else {
				w2 = ow;
				h2 = oh;
				w = ow;
				h = oh;
			}
			if (FAILED (hr = d3ddev->CreateTexture (w2, h2, 1, D3DUSAGE_RENDERTARGET, D3DFMT_A8R8G8B8, D3DPOOL_DEFAULT, &shaders[i].lpTempTexture, NULL))) {
				write_log (_T("%s: Failed to create working texture1: %s:%d:%d\n"), D3DHEAD, D3D_ErrorString (hr), i, shaders[i].type);
				return 0;
			}
			write_log (_T("%s: %d*%d temp texture:%d:%d\n"), D3DHEAD, w2, h2, i, shaders[i].type);
			shaders[i].worktex_width = w;
			shaders[i].worktex_height = h;
		}
	}
	if (haveafter) {
		if (FAILED (hr = d3ddev->CreateTexture (window_w, window_h, 1, D3DUSAGE_RENDERTARGET, D3DFMT_A8R8G8B8, D3DPOOL_DEFAULT, &lpPostTempTexture, NULL))) {
			write_log (_T("%s: Failed to create temp texture: %s\n"), D3DHEAD, D3D_ErrorString (hr));
			return 0;
		}
		write_log (_T("%s: %d*%d after texture\n"), D3DHEAD, window_w, window_h);
	}
	return 1;
}

[Toni Wilen]

TARGETDIMS added: http://www.winuae.net/files/b/winuae.7z warning: some things may not work.

[rsn8887]

It works great thanks. Of course I tried all the combinations of offsets and scalings using both targetdims and sourcedims and the discrepancy between still remains.

But at least the sharp_bilinear_simple filter can now find the correct integer prescale factors by itself

[rsn8887]

Toni is it possible to add SOURCEDIMS and TARGETDIMS also to the post filter that uses the _winuae.fx? That way I can find out what's going on by playing with _winuae.fx

Its SOURCEDIMS vary depending on whether there is a "0" shader selected or not. I think its targetdims are always window_w and window_h.

From direct3d.cpp SOURCEDIMS and TARGETDIMS for that shader should be set somehow like this around line 3055 in direct3d.cpp:

if (FAILED (hr = postEffect->SetTexture (postSourceTextureHandle, srctex)))
			write_log (_T("%s: SetTexture(srctex) failed: %s\n"), D3DHEAD, D3D_ErrorString (hr));

<<<<<<<<<<<<<<
D3DSURFACE_DESC Desc;
D3DXVECTOR2 fDimsSource, fDimsTarget;
srctex->GetLevelDesc (0, &Desc);
fDimsSource.x = (FLOAT) Desc.Width;
fDimsSource.y = (FLOAT) Desc.Height;
fDimsTarget.x = (FLOAT) window_w;
fDimsTarget.y = (FLOAT) window_h;

hr = posteffect->SetVector(posteffect->m_SourceDimsEffectHandle, &fDimsSource);
		if (FAILED(hr)) {
			write_log(_T("%s: SetTextures:SetVector:Source %s\n"), D3DHEAD, D3D_ErrorString(hr));
			return 0;
		}
hr = posteffect->SetVector(posteffect->m_TargetDimsEffectHandle, &fDimsTarget);
		if (FAILED(hr)) {
			write_log(_T("%s: SetTextures:SetVector:Source %s\n"), D3DHEAD, D3D_ErrorString(hr));
			return 0;
		}
<<<<<<<<<<<<<<

		if (after >= 0) {
			if (FAILED (hr = d3ddev->GetRenderTarget (0, &lpRenderTarget)))
				write_log (_T("%s: GetRenderTarget: %s\n"), D3DHEAD, D3D_ErrorString (hr));
			if (FAILED (hr = lpPostTempTexture->GetSurfaceLevel (0, &lpNewRenderTarget)))
				write_log (_T("%s: GetSurfaceLevel: %s\n"), D3DHEAD, D3D_ErrorString (hr));
			if (FAILED (hr = d3ddev->SetRenderTarget (0, lpNewRenderTarget)))
				write_log (_T("%s: SetRenderTarget: %s\n"), D3DHEAD, D3D_ErrorString (hr));
		}

		uPasses = 0;
		if (psEffect_Begin (psEffect_None, &uPasses, s)) {
			for (uPass = 0; uPass < uPasses; uPass++) {
				if (psEffect_BeginPass (postEffect, uPass)) {
					if (FAILED (hr = d3ddev->DrawPrimitive (D3DPT_TRIANGLESTRIP, 0, 2)))
						write_log (_T("%s: Post DrawPrimitive failed: %s\n"), D3DHEAD, D3D_ErrorString (hr));
					psEffect_EndPass (postEffect);
				}
			}
			psEffect_End (postEffect);
		}

[Toni Wilen]

Done

[Toni Wilen]

Any ideas if http://eab.abime.net/showthread.php?t=86420 is also related?

[rsn8887]

I don't know. Even with the new Shader variables, I got very close but I could not get my own NULL FILTER to exactly reproduce the "None" or "Null Filter" settings. It always introduces some pixel wobble/distortion. E.g. a checkerboard pattern of perfect squares becomes an alternating pattern of squeezed and expanded rectangles.

Since additional transformation rules are hardcoded (but editable) in winuae.fx, I think only a direct hack/modification of winuae.fx with my "sharp-bilinear" filter code could have been able to give me the filter behaviour I wanted. However, in winuae.fx I am missing the SOURCEDIMS and TARGETDIMS variables again

I think the inconsistencies are due to the fact that WinUAE does several transformations to add black borders and center the screen as necessary AFTER the "0" filter is applied. These preset transformations are not simply taking the Amiga Screen texture and magnifying it. The image is also centered. I think it might be those transformations that cause my observed Shader inconsistencies. I did so much testing but could never really pinpoint it down. If I remember right, the transformations are different depending on whether there is a user-defined "0" shader or not... so even if the user-shader is a NULL filter, there is a net difference in what winuae.fx does. If I just bypass winuae.fx, the image is way too small and not centered etc.

[Dr.Venom]

Originally Posted by rsn8887

I don't know. Even with the new Shader variables, I got very close but I could not get my own NULL FILTER to exactly reproduce the "None" or "Null Filter" settings. It always introduces some pixel wobble/distortion. E.g. a checkerboard pattern of perfect squares becomes an alternating pattern of squeezed and expanded rectangles.

Just in case, maybe this is of some use:

Solving DX9 Half-Pixel Offset