Shaders for various emulators

[guest.r]

Both luts are now in the shader. Old is under "1.0", new under "2.0". Latter was also patched a bit...

I advise to delete the "guest" folder prior extraction to avoid junk files. Presets are also altered to load both luts.

[Retro-Nerd]

Excellent! It's good to have 2 different Trinitron profiles. Works fine.

[Dr.Venom]

Originally Posted by guest.r

Both luts are now in the shader. Old is under "1.0", new under "2.0". Latter was also patched a bit...

I advise to delete the "guest" folder prior extraction to avoid junk files. Presets are also altered to load both luts.

Thanks for putting both in . For me personally the first one is the most accurate, the second one has some more color differences that I don't see in my CRTs.

For some reason the below got a bit of a long write-up. I guess since this is not my expertise I need many words. The old "get a cup of coffee and read on" may apply.....

The short version is: I think a CRT color accurate LUT, that correctly replicates CRT color with the shader on a sRGB accurate LCD is the last missing piece of the puzzle for this excellent shader.

Anyhow, here goes the long version...

With regards to the first LUT, I have both a trinitron and a slotmask sitting next to me and I'm seeing the exact same thing as Retro-Nerd:

Originally Posted by Retro-Nerd

Your current colors are pretty accurate for the most Trinitron colors. Except for all greys+white. They have strong sepia tint, which my Sony Trinitron doesn't have at all. Would be great if you could tweak this (or making it tweakable via an option setting). I know that the strong sepia tint is pretty normal for "movie" profile settings, but older CRT TVs mostly didn't have this back then.

When you think of it, it seems peculiar it's apparently so difficult to have a true to life CRT color profile on a sRGB accurate LCD. Given that you named the LUT torridgristle, I noticed he posted 20 CRT LUTs in the libretro forum. But every one of them has very peculiar things that entirely don't match the stuff I'm seeing on my real CRTs!

OK, so then I stumbled upon the following article: History of the Very Odd sRGB Color Space

This is a very interesting story, worth a read IMO. The bottomline: The era of (early) CRTs was The Wild Wild West of Digital Color Management. The sRGB standard was only created in 1996 (!) by Microsoft and HP to end this mess. It goes without saying that the CRTs we were using back then (from the middle of the 80's to the early nineties, like Commodore 1084 etc..) were using their own (non-)standard color calibration. At least they were not calibrated to an industry standard norm, as that only was created over a decade later.

This would explain why both Retro-Nerd and myself and Nobody are seeing different color profiles on our CRTs. These CRTs are from an era when every company did its own thing, hence the chance of having a slightly different color cast per CRT Manufacturer / Type(?)

So after reading the above article I think our (THE) problem with regards to accurate color representation in emulation is the following:

8 and 16-bit system (Non-)RGB color output (A) -->> (pre 1996) Non-standardized CRT color output (B) -->> sRGB color accurate LCD (C)

(A) is sort of at the emulator author level. For RGB system (Amiga etc) this will be quite correct. For non-RGB systems (C64 and the like) it will be very tricky (I'm quite sure most emulator authors and discussion comes from the layer of (A) AND (B) mingled)
(B) CRT color reproduction is layering a massive problem on top. Pre 1996 CRT color reproduction was the Wild Wild West, there was no factual standard, and so CRTs from different types / manufacturers have their own specific color cast?
(C) To make something accurate, not only the sources must be correctly represented, but the target monitor must be color accurate to a standard, or you'll never know how accurate the representation is on the users end. So one needs this to be close to the current sRGB standard accurate. (Which a lot of post 2000 LCDs are not.)

So let's simplify above problem and only care about

(A) emulated system is of RGB color output era (emulator is color accurate)
(B) (The Wild Wild West of) CRT color cast
(C) Target is "high-end" post 2018 LCD with 99% accurate sRGB.

So simplified we only really need to care about (B): a GOOD CRT color mapping to a sRGB accurate LCD. I.e. "80's/early '90s CRT color accurate transformation into sRGB" LUT!

If that sounds simple, it isn't. As said, I have tried all 20 of torridgristle's CRT LUTS, and literally not one of them represents the color I'm seeing on any of my Philips CRT's or the Sony Trinitron or PVM. And my IPS LED has fairly accurate color reproduction, as far as I may believe the reviews.

The big issue I think is that torridgristle's CRT Luts were converted from drivers of CRT monitors. Which already shows the issue, they were drivers written for CRT technology (using phosphors and all!), and for who knows what standard to match. Not for LCDs, which have a very different color reproduction technology. I.e. they would map colors correctly when the medium was a CRT, but colors will (and do) look wrong when you use those maps without any further conversion on a LCD.

So there we have it. The missing link is some LUTs that actually represents CRT color cast on a LCD correctly.

To me personally this really is the missing piece of the puzzle for this excellent CRT shader.

Since I'm seeing on my real CRTs mostly different color cast between manufacturers, e.g. Philips and Daewoo (both also produced the Mighty Commodore monitors) versus the color cast of Sony (trinitron).

My suggestion for the shader would be to rename the parameter "Trinitron Colors" to "CRT Colors" (which is (B) above) and see if some effort can be put into creating a few accurate CRT color profiles. I.e. color profiles that actually match the color of a real (Manufacturer and Type mentioned) CRT, and one that can stand the direct color comparison between a live CRT setup and the shader on a sRGB accurate LCD next to it. How hard could it be

I guess a long write up, but I really feel this is the missing piece that would actually be able to make this shader make me retire my actual CRTs (especially when HDR lands!)

The real test would be for Retro-Nerd, Nobody and myself, or anyone else with a live real CRT setup (please no "memories of 20 years ago", just plain cold comparison to a live CRT setup), to no longer perceive any difference in color reproduction when running the same system side by side.

Personally I'll try to remove the sepia color cast from the torridgristle1 LUT, while remaining the other benefits. I've installed GIMP and it seems to have some nice tools for color cast correction.

But I'm really wondering whether there isn't a mathematical way of mapping CRT color output from the pre 1996 era (the wild wild west) into LCD sRGB?

In that regard the following link may be interesting: LCDs and CRTs: How different are they?

and the VERY technical Image Display Technology

Or maybe there are limitations in current LCD technology that prevent it from even being technically possible, as below quote may suggest?

The fourth goal is totally beyond your control. The tristimulus values of sRGB were chosen to be a good match to the tristimulus values of the type of phosphors used in most CRTs. LCDs use a completely different technology to make colors, so very likely profiling your LCD monitor will result in tristimulus values that are close to, but not the same as, the sRGB tristimulus values. Compared to CRT monitors, even many professional grade wide-gamut LCD monitors are deficit in the most saturated sRGB blues, though they can display more saturated greens and reds.

[Retro-Nerd]

Mmh, my Sony Trinitron is from 2001, The KV21FX30E. Pretty much the last generation i would assume. Don't know if they already used a sRGB colorspace, can't find the correct infos.

[guest.r]

@Dr. Venom, some interesting reading there. I think the problem is that different CRT models have and had different color calibration/reproduction, so it's hard to make everyone happy. I'll rather concentrate on few nice looking alternate color profiles therefore.

LUT's are the correct way to go since they can store complex transformations.
They may also mean some hard work, even with proper tools.

I'll include corrected luts if you like, np here.

It's quite possible to grasp the general color reproduction of specific crt's, that's what the lut pack is trying to do, but i guess a NEC Multisync 9300K will look "wrong" compared with a 5000K calibrated Sony Trinitron. I read there are also some issues with gamma, but i can correct that in the shader.

Can modern LCD's reproduce the CRT look? To some degree and colors are the least problematic or the most problematic issue, it mainly depends from our perspective of looking.

[Dr.Venom]

Originally Posted by guest.r

@Dr. Venom, some interesting reading there. I think the problem is that different CRT models have and had different color calibration/reproduction, so it's hard to make everyone happy. I'll rather concentrate on few nice looking alternate color profiles therefore.

Indeed, a few alternate color profiles would be enough. The Trinitron profile you've included is already a very nice looking one and my default one also (I'm using the shader parameter Color Temp % at -25% with it ).

I guess my main goal is to replicate both my Trinitron, which the above goes a long way of doing it already.

And secondly the mighty Commodore 1084 monitor (which is he same color profile as almost all Philips 600TVL CRT monitors that were sold with MSX, Amiga etc in the heydays..)

LUT's are the correct way to go since they can store complex transformations.

They may also mean some hard work, even with proper tools.

Yeah I noticed... I'm seeing clear differences with MSX, so I thought I'd focus on that. After getting it very close to correct and be very happy with it I then loaded up PSX, only to notice al lot of colors are off

It's clearly a difficulty that the low "on-screen" color computers/consoles only pick a very small subset, and if that small subset is very slightly off it's much more noticable than when you use a high-color image as reference.

So approaching it from the other end, with a high-color system image as reference the individual colors stand out much less, so calibration may look good, only until you use that LUT on an emulated computer/console with much less colors, where it's too easy to spot the differences. And it's back to square one. Pfff... I understand why you sticked with the torridgristle LUT that came from driver conversion...

As such, I have a feeling manual calibration is not the way to go?

If we would resort to some sort of configurable mathematical transformation, maybe we could test some more stuff in the shader and see if that would bring us closer?

The first idea for testing ground would be if it would be possible to make the chromacity coordinates for R, G, B configurable in the shader? (E.g. a plus and minus from their default X and Y location.) As in below picture. Basicly it would allow the shape of the triangle to be determined and we already have white point adjustment in there, so would it mean we could actually get where ever we would want?




Or another possibility to include the relevant color changing parameters from the "image-adjustment" shader? (see here: https://github.com/libretro/glsl-sha...djustment.glsl )

It seems it may have some relevant stuff in there?

Again this would purely be for testing ground, just to see whether it would allow us to recreate the exact CRT colors we're seeing on our CRT's.

I'm grasping to straws maybe, but the least I can do is try

[Retro-Nerd]

@guest.r:

Could you please remove the sepia tint for your Mame warmer colors shader too? Would be nice to have it for bsnes/higan too.

[guest.r]

NP, here are versions for both emulators.

[Retro-Nerd]

Great. Both works fine. But do you remember, you've first posted a Trinitron shader for bnes/higan which didn't look right? Can you update this one for warmer colors too? It works pretty good with Denise (C64 emulator).


http://eab.abime.net/showpost.php?p=...&postcount=128

[guest.r]

@Retro-Nerd:
Will be finished soon.

@Dr. Venom:

If we would resort to some sort of configurable mathematical transformation, maybe we could test some more stuff in the shader and see if that would bring us closer?

The first idea for testing ground would be if it would be possible to make the chromacity coordinates for R, G, B configurable in the shader? (E.g. a plus and minus from their default X and Y location.) As in below picture. Basicly it would allow the shape of the triangle to be determined and we already have white point adjustment in there, so would it mean we could actually get where ever we would want?

I wrote a test shader to configure the chromatic coordinates. I know such work can be PITA, but it can get you close to what you want. I'm not adding this to the main setup since it's way to hard to utilize it for most users. What i have in mind is to create some presets we can choose from. For that, i'll need all parameters from the shader to create a proper matrix. Currently it's created on the fly and there is no "print" function with shaders. I think if you save the preset all parameters are there.

Or another possibility to include the relevant color changing parameters from the "image-adjustment" shader? (see here: https://github.com/libretro/glsl-sha...djustment.glsl )

It seems it may have some relevant stuff in there?

Since passes can be added by the user this shader is quite popular. I think it's best we don't include it by default since it cannot influence the shromatic values in a fundamental way, only some nice tweaks.

Again this would purely be for testing ground, just to see whether it would allow us to recreate the exact CRT colors we're seeing on our CRT's.

Please give the test shader a try. "New" Desktop Mode isn't as good as the old parameter tweaking (prior RA 1.3 i think), but it can speed testing of shaders considerably.

[guest.r]

Originally Posted by Retro-Nerd

Great. Both works fine. But do you remember, you've first posted a Trinitron shader for bnes/higan which didn't look right? Can you update this one for warmer colors too? It works pretty good with Denise (C64 emulator).

http://eab.abime.net/showpost.php?p=...&postcount=128

It's finished now. Happy playing.

Edit: Updated version.

[Retro-Nerd]

Mmh, not looking right (both).





Just to be sure: check the attached ones. They work as expected in Denise (without the warm colors though).





Yeah, integer scaling is missing yet in Denise. But it will follow.

[guest.r]

Strange, it worked with me. Nevertheless, i updated the shaders.

[Retro-Nerd]

Great. Now it's fine.

about your chromatic coordinates test: Tried that, but matching CRT TV colors of a specific model seems impossible (not even close). Maybe we should stop it at this point. The LUT+color temperature option seems good enough.

[Dr.Venom]

Originally Posted by guest.r

I wrote a test shader to configure the chromatic coordinates. I know such work can be PITA, but it can get you close to what you want.

Thanks for this, I'm getting some very promising results .

Yes it can be (is) a PITA. I found that converting the peak Color Wavelength of the most commonly used CRT phosphors to Spectral Locus coordinates and using those as a starting point lessens the pain a bit .

Only now I need more precision. Could you possibly update the test shader such that it allows for 4 digit decimal precision, in a way that it actually shows the configured values? It may be overkill, but with the current granularity there are very visible tipping points at certain coordinates. (Three digit decimal precision will probably be enough already, but I'd rather check this than assume...)

Lastly in one of my test screens there's a patch of grey that is "same color" (grey) as on the LED, but much darker on the CRT (while all other colors are visually indistinguishable between LED and CRT) . What would be the proper way to calibrate for that? I'm noticing changing Gamma In, Gamma Out or Gamma Tweak have little to no influence on really darkening that specific grey patch. Could this be something related to the Gamma Curve? I have no real understanding of Gamma, but could it be that the gamma decoding curve on a (this) CRT is of a different function than the one in the shader, such that certain shades are emphasized in darkness on the CRT? Just checking...

(I'm also using the test shader as the first pass in the main shader, so that you understand why I'm also testing including gamma.. )

[guest.r]

Originally Posted by Dr.Venom

Thanks for this, I'm getting some very promising results .

Yes it can be (is) a PITA. I found that converting the peak Color Wavelength of the most commonly used CRT phosphors to Spectral Locus coordinates and using those as a starting point lessens the pain a bit .

Only now I need more precision. Could you possibly update the test shader such that it allows for 4 digit decimal precision, in a way that it actually shows the configured values? It may be overkill, but with the current granularity there are very visible tipping points at certain coordinates. (Three digit decimal precision will probably be enough already, but I'd rather check this than assume...)

Thanks for the hard work on this. It's bothering me also, that RA interface allows only 2 decimal digits. So i changed the range from 1.0 to 10000 to allow more precision. The numbers get divided in the shader, so no worries here. Maybe you'll have to re-enter the numbers from a saved preset, but otherwise i think this should work out nicely.

Originally Posted by Dr.Venom

Lastly in one of my test screens there's a patch of grey that is "same color" (grey) as on the LED, but much darker on the CRT (while all other colors are visually indistinguishable between LED and CRT) . What would be the proper way to calibrate for that? I'm noticing changing Gamma In, Gamma Out or Gamma Tweak have little to no influence on really darkening that specific grey patch. Could this be something related to the Gamma Curve? I have no real understanding of Gamma, but could it be that the gamma decoding curve on a (this) CRT is of a different function than the one in the shader, such that certain shades are emphasized in darkness on the CRT? Just checking...

(I'm also using the test shader as the first pass in the main shader, so that you understand why I'm also testing including gamma.. )

Maybe it's some annoying difference in technology, that grey colors have a different luminance/gamma distribution. You could also test it as a vanilla version to make sure the shader isn't doing it's own thing. Otherwise it's only fixable by a nasty "check saturation" hack (grey has 0 saturation) , to have the greys have different gamma. If darker gamma is applied to all colors, they'll get much too saturated.

[Dr.Venom]

Hi

It's been a little while, but I've (slowly) been making headway with a number of profiles for the Philips based CRTs, the Trinitron and PVM, but it's proving to be a bit more of a challenge than I anticipated.

I'm striving for the profiles to be accurate enough that when running the shader side by side with real CRT there are no real perceivable differences. In that regard, it's especially been the older Trinitron consumer monitor (not the PVM) that has been giving some headaches. So to get a better understanding I've been measuring a number of my CRTs with a colormeter (which unfortunately is not as accurate as a ten grand spectroradiometer, but at least it gives ballpark measures that are good enough for illustration.)

One of the main takeaways of that effort is that part of the Trinitron Color Gamut is outside of sRGB space. As an illustration see below image. Trinitron saturated green (to blue) falls slightly outside of sRGB space and as such cannot be displayed correctly on normal LCD monitors. This is also the reason why for example the teal background in Turrican, when viewed on a Trinitron, is not replicable on a normal LCD monitor (hi Retro-Nerd! ).





The issue is much less apparent on the Philips based CRTs, as the green phosphor seems much closer to the default sRGB green primary. Also the Sony Trinitron PVM/BVMs tend to have a less saturated green phosphor than the older Trinitron consumer monitors (and TVs?). Lastly I was a bit surprised to notice that my older Trinitron monitor (not the PVM) has a whitepoint between 8500K-9300K. Who would've thought that!

I hadn't used the Sony PVM much, but I was pleased to find out it has a switch on the back for 6500K and 9300K mode, so that was nice to compare with the other Trinitron and see how far off the colormeter is (it reports 9300K mode as 8500K, the same value it gives for the older Trinitron).

Lastly some issues / questions:

I now have access to a wide color gamut monitor (Adobe RGB), and it indeed allows to replicate Trinitron green, and things like the teal background in Turrican quite accurately. But, I'm noticing that when switching between AdobeRGB and sRGB that a single chromacity pair doesn't give the same result when switching modes. Default sRGB chromacity values for the primaries give a much more saturated result on the Adobe RGB monitor. Since the cie chromacity coordinates are absolute, shouldn't they give the same result? Is this maybe because you are doing some conversion relative to sRGB space in the test shader? Anyhow, hopefully it's possible to make the test shader behave correctly for AdobeRGB space also. That would make things a bit easier..

Next is a finding on Gamma. I'm measuring a gamma of ~2.25 on all of my CRT monitors consistently (which sits nicely between what is generally said for CRTs to be between 2.0-2.5), how would I adjust for that in the shader settings? I assume I need to set "gamma out" to 2.25?

[guest.r]

Hello there!

I'm glad for your response, gearing up and such. I'll wait a bit with my display, prolly full sRGB range and HDR, but need new (super)computer first.

I'll add an Adobe RGB color tool, so you can tweak and alter the values to better authenticity.

The basic values are altered also, so the colors look neutral out of the box, hopefully the magic will work.

About the gamma, there are reasons the output gamma is set to 2.4 - scanlines add some saturation and lowering it to 2.2 would be an overshot. Generally output gamma is about 10% lower then input one since we'arent forcing it directly to the display and this does the trick, so yes, if you want to use 2.25 it's ok, since you can compensate with brightboost for example.

[Dr.Venom]

Originally Posted by guest.r

The basic values are altered also, so the colors look neutral out of the box, hopefully the magic will work.

Works great, thanks . I can use the earlier calibrated values now in AdobeRGB mode and have the same output. This will definitely help in better calibration and better understanding where sRGB space is limiting.

Does it mean that it would be possible to add a sRGB / AdobeRGB switch parameter to the shader, such that the shader does the appropriate conversion depending on which color space is set? It could possibly pave the way for Rec.2020 standard later on?

I think I like the gamma out indeed better at 2.25, it seems to result in closer matches. Anyhow, the calibration process will take some more time. It's a bit of a tedious process, in the sense that you've got to run through various testcases for various systems every time. Not being limited by sRGB in the calibration process may help though. I also found the specification of the phosphors used in Sony Broadcast Monitors and their tolerances, so that may help as a good starting point for the PVM also. Let's see.. In that regard if you or @Retro-Nerd have any special testcases, please let me know. I'd rather know about them now .

Since the shader may be a bit overwhelming in features to people who haven't been following the thread, I wrote a quick and dirty explanation on the shader parameters. If you like you could include it with an upcoming release. It needs a bit of polishing though, as I may have written some silly stuff in there. Please feel free to update / change / revise / mangle it in any way. Not to forget it needs some explanation on the Gamma In, Out and Tweak parameters .

[guest.r]

Originally Posted by Dr.Venom

Does it mean that it would be possible to add a sRGB / AdobeRGB switch parameter to the shader, such that the shader does the appropriate conversion depending on which color space is set? It could possibly pave the way for Rec.2020 standard later on?

NP, i can add these options, dunno about Rec.2020, it's seems we haven't crossed this bridge yet. But i can add it.
It'll need some testing afterwards.

Not being limited by sRGB in the calibration process may help though. I also found the specification of the phosphors used in Sony Broadcast Monitors and their tolerances, so that may help as a good starting point for the PVM also. Let's see.. In that regard if you or @Retro-Nerd have any special testcases, please let me know. I'd rather know about them now .

I guess one good PVM setup will do. I can't test this by myself, but if you got the coordinates it should do.
All i really need are the 3 coordinates and white point, so i can recreate the matrix.

Since the shader may be a bit overwhelming in features to people who haven't been following the thread, I wrote a quick and dirty explanation on the shader parameters. If you like you could include it with an upcoming release. It needs a bit of polishing though, as I may have written some silly stuff in there. Please feel free to update / change / revise / mangle it in any way. Not to forget it needs some explanation on the Gamma In, Out and Tweak parameters .

Hey, that's some nice work. New wersion has some extra parameters, one for scanlines and one for border, but the guide is quite complete. It will be included OFC.

And happy work...