Shaders for various emulators

[Dr.Venom]

Quick question:

When I have my monitor set to 6500K, then to reach "9300K" via the shader I use the XYZ values for 9300K. This is simulated 9300K on a D65 monitor.

But, my monitor also allows for a hardware 9300K mode via its display settings. If I set that, to what value do I then need to set the whitepoint in the shader to really be at the monitor's hardware 9300K?

Having both the shader and the monitor at 9300K gives a strange result .

[guest.r]

From my experience both shader and monitor adjustments do their own thing, so the results might be oveblown if we are using both solution at the same time.

If you set the monitor to 9300K it's IMO best for shaders to stay at "neutral" position.

[Dr.Venom]

Originally Posted by guest.r

From my experience both shader and monitor adjustments do their own thing, so the results might be oveblown if we are using both solution at the same time.

If you set the monitor to 9300K it's IMO best for shaders to stay at "neutral" position.

I see. I'm finding that using the monitor hardware 9300K point allows for a more accurate white point approximation of the PVM 9300K mode, so I'll add one extra profile for the PVM 9300K mode that is targeted at using 9300K hardware whitepoint.

A small update on the CRT color profiles: there's some good news and some bad news. The good news is that when using the AdobeRGB colorspace the profiles are virtually indistinguishable from the real CRTs. I've now done both the PVM in 6500K and in 9300K mode as well as the Philips based CRT, and it's great to see the shader run next to the real CRT's with virtually no difference in color appearance. The older Sony trinitron to be finished.

The bad news is that sRGB space is a limiting factor. So on most current LCDs out there the CRT color profiles will not be as accurate compared to AdobeRGB . The amount to which this is noticable very much depends on the colors used in the games. But personally now that I got used to the accurate profiles it's a bit hard to go back to my other monitor

So then I've spent a little bit too much time on seeing whether given the sRGB colorspace limitations it would still be possible to create separate "second best" profiles specifically for sRGB, but I've given up on that. It's a continuous (did I mentious tedious) game of pulling a string on one end and seeing it disappearing on another end. So we'll have to do with some "clipping" of the accurate profiles when viewed on standard sRGB monitors. (Or just use the sRGB primaries...)

The good thing about this all is that we're now also prepared for next gen monitors with HDR and Wide Color Gamut. Apparently the intention is to use Rec. 2020 colorspace for Wide Color Gamut, which is even wider than AdobeRGB, so the future looks bright .

In total I think we'll end up with some 10 profiles; 3 different real CRTs compared/calibrated profiles of which the PVM has both 6500K and 9300K mode, plus a PVM 9300K profile for monitor @9300K, 4 different "CRT specs" and one spectrophotometer measurement I found from a Dell VGA monitor.

I think it would be nice if we could incorporate the profiles in the shader in a way that it's clear for a user that sRGB is a limiting factor. Maybe by calling the AdobeRGB "switch" something like "Accurate CRT color for Wide Color Gamut" and then currently only have "0" (i.e off / sRGB) and "1" (AdobeRGB). In the future we can then add "2" (ITU-R REc.2020 / HDR-WCG).

Sorry to keep this still in the "vaporware" stage , but intend to round this off somewhere next week...

New wersion has some extra parameters, one for scanlines and one for border, but the guide is quite complete. It will be included OFC.

That's great. I'm curious what will the new parameter for scanlines do and also the border setting?

Could you possibly add something to the guide about Gamma IN, OUT and Tweak? Or provide we with some quote? Otherwise I'll just write something like "don't touch this!"

[Dr.Venom]

I'm not fully understanding the subject of gamma correction in these shaders. Sorry to go on about this, but I really want to understand the "gamma chain".

From what I understand:
1. Gamma correction was originally invented to correct the non-linear response of CRTs. I.e. CRTs have a non-linear voltage response when it comes to displaying dark versus lighter colors, so the colors come out wrong by default. So picture input is unequal to picture output. For this the gamma correction was done to the input signal, such that the output signal is linear again.
2. The gamma correction is done by a power law function which uses a lambda value for the correction. Generally it seems a value of 2.4 was taken for this.
3. LCD monitors have the gamma correction build into the hardware to simulate CRT gamma.

So from here I'm trying to understand the Gamma chain.

Without a shader we have:

Input signal -> LCD hardware gamma correction -> correct output?

With the shader the chain is as follows:

Input signal -> Gamma In -> Gamma Tweak -> Gamma Out -> LCD hardware gamma correction -> correct output?

I really wouldn't mind a detailed description of what the shader chain part is actually trying to achieve relative to the built in LCD monitor hardware gamma correction. Sorry to bold this, but I'd really like to understand what each of these steps is doing? Could you explain, or is this some sort of elusive it was there in most of the the shaders and it seems nice / helpful so let's not remove it?

Additional case in question: I'm reading a Gamma of 2.25 on 4 CRT monitors, so it must be 2.25 that is correct right?

Now my LCD monitor is set to a gamma of 2.2, which apparently is the "official" target these days for modern LCD monitors. It looks like we're already correct then aren't we? Why are the Gamma In, Out and Tweak settings in the shader added?

Yes, I'm a noob at this, but I'm trying to learn.

[guest.r]

It's quite simple, not too much to it.

Setting input gamma to 2.4 affects following:

- horizontal interpolation is done in gamma space, where brighter colors spread a bit more over darker ones. Should match interpolation of CRT's.
- scanlines are applied differently
- masks are better distributed over the spectrum

Ofc. we have to switch back to the normal colorspace, so there is the gamma out functionality. In some CRT shaders it's about 10% lower compared with the input gamma. Different input/output gamma values affect saturation and brightness.

Since there is an option to use neutral input/output gamma (1.0), you can observe the difference within the shader.

I think most importantly is that this part of the shader functions as intended and can be tweaked to personal preference.

I think the most catchy part here is that CRT's have this 2.2-2.25 gamma and i defaulted 2.4. It roots in sRGB a bit and i got used to it. You can set output gamma (often referred to as CRT gamma) to 2.25 np.

As i mentioned some crt shaders use different sets of gamma values, i like to use fast gamma of 2.0 in WinUAE, add some saturation and then use same or a bit higher output gamma.

[Dr.Venom]

Cool, I see.

Any idea how monitor hardware gamma interacts with Gamma Out in the shader?

I.e. having shader gamma out at 2.25 and then having the monitor hardware gamma at 2.2 or 2.4, the difference? Should monitor hardware gamma ideally also be at 2.25 when shader is at 2.25 to get "real" 2.25 gamma?

[guest.r]

If shader gamma in equals gamma out, then you should be OK by setting monitor gamma to 2.2.

If shader gamma out is lower, then you get a darker appearance and increased saturation.

As i mentioned, it's mostly a shader internal thingie, gamma doesn't get forced to the display or similar.

I use display gamma 2.2 and am happy with shader 2.4 in/out. If you use display gamma 2.4, then you might set shader gamma out to 2.2 or similar. One of the reasons i like shader gamma 2.4 internally is that masks 1-4 look better with their default values.

A also like to mention this, that it's important that if you use LCD gamma 2.4, shader output gamma is to be about 10% lower so combos like 2.2/2.0 or 2.0/1.8 work ok.

[Dr.Venom]

@guest.r Could you add DCI-P3 gamut to the test shader?


Also I've been running the various profiles and it seems there's only 4 to 5 really distuingishing profiles, so I think we'll end up with 4 or 5 profiles.

The whitepoint on the CRTs is definitely not a "clean" D(65), even when some CRT specs define it as such. E.g. for D65 it seems the target has been more a CCT of around 6500K, instead of an absolute D65 target. There's definitely a slight hue to them, so maybe ANSI 6500K was targeted instead? Given this whitepoint difference from spec, I was wondering whether it would make sense to split the profiles in separate RGB primary profiles and whitepoints profiles?

So instead of 4 to 5 profiles to choose from it would be possible to mix and match the RGB and whitepoint profiles between official specs and the calibrated ones, for a total of 16-25 possibilities. Could that split in two CRT profile parameters be made without a performance penalty and more importantly what would be your preference?

[guest.r]

Originally Posted by Dr.Venom

@guest.r Could you add DCI-P3 gamut to the test shader?

Np. Test shader added.

Also I've been running the various profiles and it seems there's only 4 to 5 really distuingishing profiles, so I think we'll end up with 4 or 5 profiles.

Sometimes less is more. I'm quite pleased with this number, must say.

The whitepoint on the CRTs is definitely not a "clean" D(65), even when some CRT specs define it as such. E.g. for D65 it seems the target has been more a CCT of around 6500K, instead of an absolute D65 target. There's definitely a slight hue to them, so maybe ANSI 6500K was targeted instead? Given this whitepoint difference from spec, I was wondering whether it would make sense to split the profiles in separate RGB primary profiles and whitepoints profiles?

There were some differences on the fourth digit od similar, i fixed this in test case. The default white point temperature should be about 6504K now. I double checked my sources...

So instead of 4 to 5 profiles to choose from it would be possible to mix and match the RGB and whitepoint profiles between official specs and the calibrated ones, for a total of 16-25 possibilities. Could that split in two CRT profile parameters be made without a performance penalty and more importantly what would be your preference?

I would rather default the D65 whites and then add the option to change this by recycling the color temperature pass. OTOH i don't know how the 4-5 profiles look like. I can add crt profiles and colorspaces options in the first pass and whites in the second. This would function quite Ok i guess.

[Dr.Venom]

Originally Posted by guest.r

Np. Test shader added.

Thx, it seems DCI-P3 is even more useful than AdobeRGB. (Btw, shader would not work at first, but I think you forgot to update the c1 = ToAdobeRGB*c1; near the bottom to c1 = ToDCI*c1; .. Works fine with that change though. I doublechecked the matrix was updated..)

So, now finally I've included the profiles (3 x spec, 2 x calibrated) with some comments in the attached "CRT Colors for CRT-Guest-Dr-Venom v1.0.zip". I've also attached some documents that support the specs.

The calibrated profiles are "quite accurate" when running them on a 10bit DCI-P3 wide color gamut panel, side by side with the real CRTs. It is quite important that in the shader they will use the exact whitepoint from the calibration, hopefully this is possible.


Which brings me to a few important points that I just need to highlight for people who haven't been following this thread for longer:

1. The color gamut / RGB primaries for many CRT monitors and TVs are different from sRGB (used by most LCDs). Generally speaking they will be slightly more saturated, mainly for green and red, which means a small part of the CRT color spectrum cannot be displayed correctly with sRGB gamut. IMO this gamut "clipping" gets "worse" when moving away from high-end / newer CRT specs (like Sony BVM), which are closer to sRGB actually, to lower end and older TV's and monitors.
2. At least a gamut of DCI-P3 or wider is needed for these CRT color profiles to be displayed properly.
3. Trying to match CRT colors on a non-wide color gamut / sRGB monitor is a futile mission, it will not work. I've been there. Especially when talking about older Trinitron monitors, they use a very deep saturated green, which gets heavily clipped on the sRGB gamut as are colors involving a mix with those saturated greens.

So the good news is that we've got "CRT colors" largely covered now with the correct specs and two "quite accurate" calibrated profiles. The bad news is that with a default sRGB monitor the profiles will be more or less clipped and look wrong, depending on the content. Then we have some good news, as it seems that DCI-P3 or some other wide color gamut will be part of the HDR-500+ spec (see here. ) So within a few years wide color gamut should become mainstream in monitors.

Lastly, let's not forget we are talking about subtle differences from the default sRGB colors here. Let's just say you have to be slighly OCD on CRT tech to really appreciate the difference .

@guest: With regards to the shader. Maybe it's useful to rename the LUT based "Trinitron color" to something different, like "LUT based CRT color" or something? , it may lead to confusion otherwise with these new profiles..

Once the next update is here I'll update the guide accordingly. In that sense it may be useful if you comment (in the code) which setting is which profile, just to have them line up in the documentation accordingly.

Anyway, I guess we're now getting really close to wrapping things up and calling it a final release, *finally*

[guest.r]

Sorry about the bug lol, was not intended.

I managed to get the profiles in the shader setup, need some confirmation that they work OK though.

Lut colors are also included, aswell the color temperature pass.

Feedback is welcome...

[Dr.Venom]

Profiles 1 to 5 work OK, nice work

Profile "0" (off I guess) is a bit wonkey though..

Could you elaborate on what the scanline beam shape high and low are doing?

Also, is there maybe a chance you could bring the color profiles with color space option to the WinUAE shader?


Edit: I've updated the documentation to "Rev1", incorporating CRT Color docs in the main guide, and adding description for Gamma In, Out, and LUT colors. Only beam shape high and low to add and its ready too

[guest.r]

Could you elaborate on what the scanline beam shape high and low are doing?

Sure. The previous parameter (value 8.0) was split into two parameters, where "low" denotes the shape near middle of the beam and "high" on edges. With some tweaking (like 5.0,15.0) a less rounded - more flat like look can be achieved with higher resolutions, deeper scanlines achieved without darkening the semi-middle parts too much etc..

Profile "0" (off I guess) is a bit wonkey though..

Profile "0" should be a direct throughput without any changes. All green here, so i'm a bit curious.

Edit: I've updated the documentation to "Rev1", incorporating CRT Color docs in the main guide, and adding description for Gamma In, Out, and LUT colors. Only beam shape high and low to add and its ready too.

That's nice indeed, i'm not planing any changes, so a finalized version should be close.

[Dr.Venom]

Originally Posted by guest.r

Sure. The previous parameter (value 8.0) was split into two parameters, where "low" denotes the shape near middle of the beam and "high" on edges. With some tweaking (like 5.0,15.0) a less rounded - more flat like look can be achieved with higher resolutions, deeper scanlines achieved without darkening the semi-middle parts too much etc..

Great, will give us some more to tinker with

Originally Posted by guest.r

Profile "0" should be a direct throughput without any changes. All green here, so i'm a bit curious.

Ah, I see it's meant that way?

I think users may expect "0" to mean OFF / no CRT color profile. Personally I expected the sRGB color profile to be at 0. That green / blue, looks like a shader error now..

That's nice indeed, i'm not planing any changes, so a finalized version should be close.

It's been a bit of a longer road than expected I guess, but IMO very much worth it. The end result is nothing short of excellent.

Guide revision 1 attached. This incorporates the CRT colors stuff, so no need to include the seperate document. If there's anything in the guide which you would like otherwise or subtly change, please feel free.

[guest.r]

Personally I expected the sRGB color profile to be at 0.

Done. Now "Everything OFF" is at -1.0.

If there are no more suggestions, this is the candidate for the final version.

Edit: some tweaking.

[Dr.Venom]

Originally Posted by guest.r

Done. Now "Everything OFF" is at -1.0.

If there are no more suggestions, this is the candidate for the final version.

Edit: some tweaking.

Hi guest.r,

Just a quick note that I downloaded the shader from the libretro buildbot, but there's something very off with the saturarion with that version. All the CRT color profiles look very desaturated and are basically inaccurate / unusable.

I also tested by loading a clean preset with the buildbot version, but the issue with the desaturated look of the CRT color profiles persists.

The last version attached here (post #235) is fine on the other hand (so please don't delete these). Any idea what's wrong with the version from the libretro buildbot?

[Dr.Venom]

Hi m8,

In addition to my previous post, I've tracked down what's causing the wrong colors on the CRT color profiles when using the libretro Online Updater. It's caused by this commit:

Commits on May 3, 2019: optimize color profile arrays in crt-guest


This May 3 commit causes wrong colors for the color space and CRT color profiles, at least on my NVidia setup. Attached zip contains an example comparison, showing how green is different/desaturated before and after the commit. This is just one example, other colors are affected too with the the color spaces and profiles.

I'm not getting grumpy, but maybe it's just not such a good idea for hunterK (or anyone for that matter) to change your code AFTER we've already wrapped up the final with quite extensive beta testing, defeating the whole purpose of RC and final...

Anyways, the error is also there in the slang version, it would be nice if for both GLSL and SLANG the May 3rd commit could be reverted.

[thellier]

Hello

Thanks for your hard work on those shaders

I was considering the idea to port those shaders to OS4 Warp3D Nova as they are GLSL too

So could you explain a few how it works ?
I mean not the math formulas in the GLSL code but how the shaders are called ?
For example there a 9 GLSL shaders in Guest.zip, 4 in /fast and 1 in /lut : are they all called one after one to make several pass effect ? or alternatively to obtain différents effects ?

Also crt-guest-dr-venom-ntsc-composite.glslp reference 2 other shaders
shader0 = ../../../ntsc/shaders/ntsc-pass1-composite-2phase.glsl
shader1 = ../../../ntsc/shaders/ntsc-pass2-2phase-gamma.glsl
Where are they ?

I suppose that if I create a small nova program that draw a quad from a bitmap/texture
, that have your uniforms, in, out variables and all the textures needed it can works, no?

I dont mean creating or porting an emulator but just having a small prog that do the CRT effect to a given bitmap/window

[guest.r]

Hi there,

@Dr. Venom:

I think i found the glitch, a matrix had an error, i fixed it now and checked other matrices too, should be legit now.

Could you test this version? Hope we don't have to revert...


@thellier

Bad news here, the shaders in the preset work as an interlaced composition of effects and calculations which require a proper meta-environment currently only Retroarch and maybe Bsnes can provide. But you could try to port the crt-trinitron for MAME, it's a single pass GLSL shader...

[Dr.Venom]

Originally Posted by guest.r

I think i found the glitch, a matrix had an error, i fixed it now and checked other matrices too, should be legit now.

Tiny lady bug squashed, GLSL all good now

So I checked the slang version from the online updater, and it has a very different issue:




note: this is with the default preset loaded, so nothing changed on any parameters.

Doesn't happen with all cores, like PSX is good, but SNES and MSX show these weird colors, I think mostly on white. Also, crt-guest-dr-venom-fast is NOT affected by this issue, only the full version. Reminds me a bit of this error some time ago: post #40 , only in reverse.