Potential improvements for sky colors

[Biology 1,347]

On 10/9/2023 at 7:43 PM, bobcat999 said:

It would be nice if someone, like @Biology maybe, could briefly explain what the numbers mean, and why some have more 'numbers' in than others for example.

The number for Blue at the bottom has ten digits in it .  Does the 096 at the end really make much difference?

Hi and sorry for being MIA for a while, I've been really busy lately.

The numbers are absorption coefficients, which are physical constants that depend on the compositon of the atmosphere and are expressed in units km^-1, in other words, per kilometer. So for instance the absorption coefficient for red tells how much of the red light is absorbed when it travels through a column of air 1 km long. Honestly the 096 at the end makes little to no difference - the input parameters I used to calculate the coefficients have a lot of significant digits and as a result the calculated coefficients also have a lot of significant digits, but 2 significant digits are more than enough and higher amount of significant digits won't make much, if any, perceptible difference.

Edited October 11, 2023 by Biology

[Biology 1,347]

On 10/9/2023 at 5:21 PM, filou said:

Even with Nvidia Freestyle which I squeezed like a lemon I couldn't do it, the problem is at the root. Nvidia Freestyle cannot improve every situation. Which is why this makes it completely obsolete.

"I'm not here to discuss which simulation is better," 

That doesn't stop you to start one discussion anyway... Comparisons are an integral part of our daily lives.Unless you're an impressionable child, what else are you risking? There are non-essential things I like about MSFS and essential things I hate. It's the same for X Plane as well.

 

The MSFS sky gives the impression that it is not infinite but that it stops not very far from my plane with a whitish luminous glare of the worst effect. A sky without depth.And it happens in the most modern simulator... but MSFS is making a huge step forward thanks to Biology's Work.That even excuses mediocrity.MSFS is average in many areas, sky color is unfortunately one of them. We shouldn't even have discussed it 3 years later because it should have been mastered from the start.

 

Here is a little color,  subtlety and finesse although there is also room for work here .

As much as I understand your frustration things are not that simple. Both MSFS and XP12 use the exact same physically-based scattering equations, just implemented using different methods. The discrepancies you see are caused by 2 separate issues which need to be addressed separately.

1. Both MSFS and XP12 use autoexposure to have as much dynamic range for a given lighting condition as possible. MSFS underexposes during sunset / sunrise, making everything, including the sky, look too dark. If you switch to the drone camera and manually increase the exposure through its key binding you will see that the sky looks much more realistic with a higher exposure. Not that XP12 is perfect in that regard, it has the opposite problem - it overexposes the sky and causes clipping during sunset / sunrise.

2. The method used by MSFS (Bruneton's method) significantly underestimates multiple scattering. This also causes the sky to be too dark. Even the method used by XP12 (Hillaire's method) underestimates multiple scattering, though to a much lesser degree. Even then, to overcome this XP12 uses a non-physically based hack to artificially boost the multiple scattering amount during the sunset / sunrise. Asobo seems to be really reluctant when it comes to using non-physically based hacks to the degree that we still don't even have accurate visibility, so I'm not surprised that it's not implemented by them.

For those who are curious, neither Bruneton's method nor Hillaire's method are superior to each other, they have distinct advantages and disadvantages. For instance Bruneton's method gives you a full LUT for the scattering and transmittance integrals, so you can easily take the the effect of the clouds into the atmosphere into account, which is the case in MSFS. You can see how the aerial perspective / haze on the ground disappears when there is cloud coverage and how the clouds cast light shafts into the sky in MSFS, neither of which are simulated in XP12. It's because it's much harder to implement these effects accurately with Hillaire's method as it doesn't pre-calculate a LUT for the scattering and transmittance integrals but renders a low resolution sky each frame. On the other hand Hillaire's method is much better at calculating multiple scattering, resulting in a much more accurate sky luminance during sunset / sunrise. In the end, it's a compromise. MSFS and DCS use Bruneton's method while XP12 and P3D v6 use Hillaire's method, so both methods are very common and are "state of the art". However based on the last developer Q&A, it appears that Asobo is switching to Hillaire's method for MSFS 2024, which will eliminate both the horizon line issue and the inaccurate multiple scattering.

Also you seem to be using a third-party add-on, namely VisualXP, in those screenshots and while I agree that it looks simply beautiful, in fact even better than default XP12 with ARPC coefficients, in the end it's a based on visual references and not physicaly based whatsoever, which is not desired for physically based rendering. So I wouldn't expect a VisualXP-like look from either of the simulators any time soon. What they achieved is insanely beautiful, but it comes with its own compromise.

Lastly, I think it's more than fair to compare both simulators, but it's also not surprising that you were met with such a negative response, not because you were criticizing things but due to how you phrased these criticisms. To be more clear, I think it's expected that constantly calling things ugly and making fun of the cameras of people who are offering reference screenshots would annoy people.

P.S. Here's XP12 during the time it was newly released, when it wasn't using the non-physically based hack to artificially boost the multiple scattering amount:

You can see that it suffered from the exact same issue as MSFS. The hack for the multiple scattering was implemented only relatively recently, and my work on ARPC was integrated to XP12 with 12.06, so literally just a few weeks ago. I pointed out the issue with the coefficients to LR, they were extremely responsive and they quickly implemented it. Now I did the same with Asobo and they said that it will be implemented starting with SU14, so only one SU after I reported it, I don't think it's fair to Asobo to be this negative.

Edited October 11, 2023 by Biology

[lwt1971 7,418]

24 minutes ago, Biology said:

1. Both MSFS and XP12 use autoexposure to have as much dynamic range for a given lighting condition as possible. MSFS underexposes during sunset / sunrise, making everything, including the sky, look too dark. If you switch to the drone camera and manually increase the exposure through its key binding you will see that the sky looks much more realistic with a higher exposure. Not that XP12 is perfect in that regard, it has the opposite problem - it overexposes the sky and causes clipping during sunset/sunrise.

2. The method used by MSFS (Bruneton's method) underestimates multiple scattering. This also causes the sky to be too dark. Even the method used by XP12 (Hillaire's method) underestimates multiple scattering, though to a lesser degree. But to overcome this XP12 uses a non-physically based hack to artificially boost the multiple scattering amount during the sunset / sunrise. Asobo seems to be really reluctant when it comes to using non-physically based hacks to the degree that we don't even get accurate visibility, so I'm not surprised that it's not implemented by them.

For those who are curious, neither Bruneton's method nor Hillaire's method are superior to each other, they have distinct advantages and disadvantages. For instance Bruneton's method gives you a full LUT for the scattering and transmittance integrals, so you can easily take the the effect of the clouds into the atmosphere into account, which is the case in MSFS. You can see how the aerial perspective / haze on the ground disappears when there is cloud coverage and how the clouds cast light shafts into the sky in MSFS, neither of which are simulated in XP12. It's because it's much harder to implement these effects accurately with Hillaire's method as it doesn't pre-calculate a LUT for the scattering and transmittance integrals but renders a low resolution sky each frame. On the other hand Hillaire's method is much better at calculating multiple scattering, resulting in a more accurate sky luminance during sunset / sunrise. In the end, it's a compromise. DCS and MSFS use Bruneton's method while XP12 and P3D v6 use Hillaire's method, so both methods are used very often. However based on the last developer Q&A, it appears that Asobo is switching to Hillaire's method for MSFS 2024, which will eliminate both the horizon line issue and the inaccurate multiple scattering.

...

P.S. Here's XP12 during the time it was newly released, when it wasn't using the non-physically based hack to artificially boost the multiple scattering amount:
You can see that it suffered from the exact same issue as MSFS. The hack for the multiple scattering was implemented only relatively recently, and my work on ARPC was integrated to XP12 with 12.06, so literally just a few weeks ago. I pointed out the issues to LR and they were implemented, now I pointed out the issues to Asobo and they said that it will be implemented starting with SU14, so I don't think it's fair to them to be this negative.

Good to always hear from actual experts, thanks Biology for these fascinating insights and details on the inner workings of atmospheric light rendering/calculations. Bottom line, it's great that your work has benefited both sims (I didn't even realize it was your input that fixed the other sim as well, going by certain narratives that some seem desperate to push around here 🙄 ).
 

[Biology 1,347]

For those who are interested, I'm aware that my comments regarding multiple scattering are a bit too technical and don't actually give a feel of how it affects the visuals, so here's a demonstration using XP12: (For debug purposes or not, I love that LR exposes these parameters and I wish Asobo did too)

Initial release day XP12: (Before the aforementioned multiple scattering hack and before ARPC coefficients were integrated into the simulator)

Current XP12 multiple scattering, but without ARPC coefficients:

Initial release day XP12 multiple scattering, but with ARPC coefficients:

Current XP12: (After the aforementioned multiple scattering hack and after the ARPC coefficients integrated into the simulator)

Currently what we can achieve with this .exe hack is basically akin to the initial release day XP12 multiple scattering, but with ARPC coefficients. When Asobo tweaks the multiple scattering and exposure during sunset / sunrise it should look like current XP12.

Edited October 11, 2023 by Biology

[Alvega 2,124]

4 hours ago, cianpars said:

As well as Bob for creating the simplified tips and tricks post, can I also thank Biology, Drunq and Alvega for their incredible work on this too.

Be aware that Rob posted the wrong parameters by mistake in the tips and tricks post. Those are the parameters from the video, the ones by MortThe2nd and that I also think are better because they reduce the green tint a bit are:

[COEFFICIENTS]
RED = 0.00508
GREEN = 0.00318
BLUE = 0.0

Rob told me he asked a mod to edit the post in the tips and tricks forum yesterday, but they didn't do it yet. 😉

[Biology 1,347]

29 minutes ago, filou said:

Incredible technical demonstration and what talent! I found it very interesting and enriching .But that doesn't excuse you.

I have no lesson to receive on the content of my answers or my observations concerning certain people and certain contents, I owe you nothing.

VisualXp exceeds everything I've seen, a little visual slap without speech, just effective. This is what I'm waiting for. That's my personal opinion.

After the color of the sky what is your next project?

 

I'm sorry that my comment sounded like a lesson on the content of your answers, that wasn't my intention. It was merely an observation about everyone involved in that argument and why your criticisms despite being valid weren't taken well.

As said above I agree that VisualXP exceeds everything I've seen, including what can be achieved with ARPC in either of the simulators. It looks simply beautiful. My point was that sadly it's very non-physically-based which means neither LR nor Asobo would be willing to integrate it into the simulator for very valid reasons. With physically based rendering everything works like a clockwork, if a component is not physically based, rest of the components in the chain also become non-physically-based and things quickly turn into a tuning nightmare.

My next project is a new custom tonemapper for XP12 called Enhanced Visuals, which replaces the default filmic tonemapper with a more natural hue-preserving one, fixing the cyan tinted sky during day and yellow tinted sky & clouds during sunset / sunrise. I won't get into more detail as it's not a MSFS project and completely off-topic, but it is indeed what I have been spending my time on recently.

Edited October 11, 2023 by Biology

[cianpars 1,166]

Now I'm torn between Biology's settings and Alvega's settings and having tried them both, I'm not sure which I prefer.  Whichever I end up going for, they are both a lot better than the original game settings.

It seems that you now have a choice of ASOBO'S settings or Biology's settings and Alvega's are something in between, though closer to Biology's settings (or we can experiment and create our own).

Both are nicer than the original.

[bobcat999 8,484]

2 hours ago, Alvega said:

Be aware that Rob posted the wrong parameters by mistake in the tips and tricks post. Those are the parameters from the video, the ones by MortThe2nd and that I also think are better because they reduce the green tint a bit are:

[COEFFICIENTS]
RED = 0.00508
GREEN = 0.00318
BLUE = 0.0

Rob told me he asked a mod to edit the post in the tips and tricks forum yesterday, but they didn't do it yet. 😉

Thanks Al!  Still no response from the mods yet.  I will try again shortly.

Even if they just corrected those values themselves I would be happy.

[B777ER 2,486]

Asobo just needs to sign @Biology to a contract to get this into MSFS and be done with it. This guy is playing 4D chess while Asobo is playing checkers. 

[Alvega 2,124]

45 minutes ago, B777ER said:

Asobo just needs to sign @Biology to a contract to get this into MSFS and be done with it. This guy is playing 4D chess while Asobo is playing checkers. 

Then they wouldn't allow him to work on the other sim anymore. 😁

[turbomax 3,286]

6 hours ago, Biology said:

MSFS and DCS use Bruneton's method while XP12 and P3D v6 use Hillaire's method

interesting. how did you come to this conclusion?

[Biology 1,347]

32 minutes ago, turbomax said:

interesting. how did you come to this conclusion?

For the case of MSFS, the horizon line is a giveaway artifact in Bruneton-like implementations, and Asobo mentioning the scattering & transmittance LUTs, which is how of Bruneton-like implementations work, alongside with the leftover text in FlightSimulator.exe confirms it. For the case of DCS, their shader code is open, and indeed they use Bruneton's method (in fact I already found a way to have the ARPC coefficients on DCS as well)

For the case of XP12, the exposed art datarefs were a giveaway and then I got a confirmation from the developers. For the case of P3D v6, the same reason as DCS, their shader code is open. They also had told in their promotional material that they were using Unreal Engine's sky, which uses Hillaire's method. In fact Sebastien Hillaire works as the principal graphics engineer on Unreal Engine.

Edited October 11, 2023 by Biology

[Virtpilot 151]

I did it! Used the Xbox App and changed the exe file location which now lives in Content folder. One of the bravest thing I´ve done for a long time.... 🙂 Managed to put the 3 files in there. However, the toggle switch (CTRL I assume) is still not working... 

[charliearon 3,024]

7 hours ago, Alvega said:

Be aware that Rob posted the wrong parameters by mistake in the tips and tricks post. Those are the parameters from the video, the ones by MortThe2nd and that I also think are better because they reduce the green tint a bit are:

[COEFFICIENTS]
RED = 0.00508
GREEN = 0.00318
BLUE = 0.0

Rob told me he asked a mod to edit the post in the tips and tricks forum yesterday, but they didn't do it yet. 😉

No request made, but I went ahead and made the changes you show here!

[vrdubin6 184]

6 minutes ago, Virtpilot said:

I did it! Used the Xbox App and changed the exe file location which now lives in Content folder. One of the bravest thing I´ve done for a long time.... 🙂 Managed to put the 3 files in there. However, the toggle switch (CTRL I assume) is still not working... 

He made HOLD_CTRL to true by default now so if you enable the toggle it should look like this:

[TOGGLE]
ENABLED = true
; See https://learn.microsoft.com/en-us/windows/win32/inputdev/virtual-key-codes for all the key codes
; Default: INSERT
KEY_CODE = 0x2D
HOLD_ALT = false
HOLD_CTRL = true
HOLD_SHIFT = false

So "ENABLED" needs to be set to true and then you'll need to press CTRL+INSERT in-game to toggle it on/off.