Biology

Hi, Just to prevent a misunderstanding, this will not be just a "fudge" to make a group happy at the expense of the other. Quite frankly I don't think the drastic difference in opinions is detrimental to us being able to come up with a quantitative approach of solving the issue in a way which will make the majority happy (in other words, perceptually accurate) while being physically accurate at the same time. Yes whatever we like something or not is subjective, but at the same time color science is a real science and it allows us to quantify human perception. We have standard observers, color appearance models and much more which helps us to model human perception accurately. And the variance between people when such differences are properly quantified is much lower than one would think given the drastic difference in opinions, but it's not all that surprising after all given that we all share the same DNA and hence the same general brain and eye structure. Because of that I genuinely believe that there is a "no compromise" solution which most people will like, and I believe we will get to that solution by using color science. In short, it's not an either physical accuracy or perceptual accuracy situation - the goal is to take perceptual facets into account without sacrificing physical accuracy. To give a loose photography analogy, physical accuracy is like getting the scene illumination etc. right, while perceptual accuracy is like using a good film & camera and mastering photography techniques such as dodging and burning. People have been able to get perceptually accurate results using basic film cameras for decades, despite the fact that print media has a much lower dynamic range than an average monitor. So it certainly is possible with a monitor, whether it's straightforward or not. From the analogy you can see that both physical and perceptual accuracy must be achieved to get realistic results, and this is exactly what we are aiming to achieve. As I explained in my previous post currently we need improvements on both fronts, but especially on the perceptual accuracy front, and rest assured that they're all being worked on.

I mean if you remember the thread, you probably also remember that I wasn't just giving technical insights but also pretty much arguing with a lot of people who didn't believe this was an issue, so it's not like I haven't dealt with the same thing. Either case we are not responsible from what people in forums or other places say, from LR's side the dark cockpits issue has always been acknowledged & worked on and hopefully will be fixed by 12.3.0 as per the timeline. Regardless I hope it is reassuring that one of the people who vehemently argued about existence of this issue the most is now personally working on the issue, especially given that my opinions on the issue hasn't changed much.

Hi, I would like to add that LR has not found it solely by themselves but by listening to the feedback from their users. In other words, user feedback has always been listened to and taken seriously. LR has been aware of the issue since the beginning and never denied its existence, I don't know where some people heard the "cockpits are still not dark enough" comment but it certainly is not from LR. For context, you probably know me from the original dark cockpits thread and how I vehemently argued with the people who didn't believe there was an issue. This was before I was hired by LR as a graphics programmer. And one of the main things I was "tasked" with right after my hiring is the dark cockpits issue. So this issue is certainly being taken seriously and prioritized. However the issue is multifaceted and hence it's not straightforward at all. In short, there are 3 parts that play a role together: 1. Sky brightness being inaccurate - especially the day sky is too bright and this is causing a harsher contrast between highlights and shadows, this will be solved by calibrating the sky into real world sky models such as CIE Standard Skies. 2. The ambient lighting / cubemap being overly sensitive to the differences in cockpit geometry and cockpit albedo - unfortunately not all cockpits in real life are light colored and it is apparent that the cubemap doesn't do a great job estimating the ambient lighting for cockpits with darker albedos. 3. Autoexposure, tonemapping and other post processing aspects needing refinement - the human visual system is really complex so modelling the physics of lighting is only half of the story, taking perceptual facets into account is equally as important as getting the physics right. You can get the physics right as much as you want but if it's perceptually wrong it's still wrong. The goal is tuning autoexposure, tonemapping etc. to resemble human vision as much as possible. And this is kind of why it is unfortunately taking so long - all of these aspects are interdependent and they need to be addressed all at once. Otherwise it will be a chasing game between developers and LR as the lighting model gets changed again and again. The goal is to measure twice and cut once, and hopefully fix the dark cockpit issue once and for all, as a part of this big lighting recalibration. But long story short, the user feedback is without a doubt taken seriously in LR.

Really nice screenshots! Sorry, it's a bit unrelated but what do you use for the scenery? Is it orthophoto or something like SFD Global? If it's ortho, I'm surprised by how color accurate the terrain looks - if you used some kind of color correction for the tiles I would be really appreciated if you post the settings for it.

You already told the entire story so there's not much left for me to tell. Honestly yeah, given how much research it took I probably should've done what you said. My reasoning was that I really wanted to have it added to both sims as I was really annoyed by the purple skies, and the best chance of achieving that was making it free, open source and easy to implement. In the end as much as I'm a developer I'm also a simmer, and the purple skies were really taking the enjoyment out of the sims.

Are you sure that these are in the right order? These are the exact same values ARPC outputs, just in the wrong order. ARPC outputs 0.00229 for red, 0.00154 for green and 0.0 for blue. In any other order the sky would look completely wrong, so I'm pretty sure it's just the order they're stored in the memory being different, not the order they're used. This means that Asobo used the ARPC values as is (thankfully), and the purpler sunsets / sunrises compared to applying the ARPC values in SU13 is solely caused by the corrections they made to the Rayleigh scattering coefficients and the direct sunlight color.

The issue with the mod is that it only worked because we knew the values to search for - as Asobo used the values from the original paper as is, it was really easy in SU13. Now no one knows the exact values they used, which means it will be much harder to find them.

Sorry for the late response, but indeed I'm pretty happy with the changes. Without seeing the new coefficients it's hard to comment but it appears that they chose something in between the old default values and what ARPC outputs for the ozone absorption coefficients. If I had to guess why, I'd say it's likely because the sunsets / sunrises look too green when using the ARPC coefficients as is without the required changes to the multiple scattering, so they likely had to make a compromise between the accuracy of the coefficients and the greenness of the sunsets / sunrises. Another unexpected but welcome change is the day sky colors. While ozone absorption coefficients affect day sky colors, the effect is minimal, so it seems like they also changed the rayleigh scattering coefficients based on ARPC and not just the ozone absorption coefficients, making the day sky colors have a deeper blue hue with less green, a change that I really appreciate. I would have been even happier if they tried to tweak the multiple scattering so that they could use the ozone absorption coefficients from ARPC as is, but they never said they would do that so I'm not disappointed as I had no expectations in that regard to begin with.

On other good news, the wishlist item for ARPC is now marked as "Fixed (SU_14)", so it sounds like they've finished.

@Drunq sorry for bothering but how does your live memory changing tool works? I run MSFS on Linux so I was trying to make a similar tool that runs natively on Linux, but I couldn't make much progress.

Those are identical to the FSX sky textures and not used, so if I had to guess they were either left there when Asobo was studying FSX code or added for some weird backwards compatibility reasons.

Hi, this is indeed my concern. Hillaire's method assumes that the sky is rather uniform and lacks high-frequency components which is how it can get away with rendering everything at real time at a lower resolution. But it is only true for a clear sky and not a sky with clouds. Thankfully there are potential workarounds for it, like falling back to raymarching for such atmospheric effects, which is why I decided to say "Hillaire's method or something similar" instead of just Hillaire's method as it's hard to know exactly what they're planning without more information - even though it's not straightforward, it's possible that Asobo, who are really skilled in computer graphics, already found a workaround for it or maybe even developed their own Hillaire-like method. But it's also possible that they are going to use Hillaire's method as is and think that compromising some atmospheric effects is worth fixing the horizon line issue and having more accurate multiple scattering, only time will tell. Directly quoting Hillaire's paper: "An important visual effect to reproduce is volumetric shadowing, for example from mountains onto the atmosphere. It is not possible to use epipolar sampling because the atmosphere is not a homogeneous medium. And it is also not possible to use a shadow volume approach from Bruneton et al. because our LUTs do not allow that kind of integral sampling over view ray paths in the atmosphere." As Hillaire's method doesn't have a lookup table like in Bruneton's method to evaluate the atmospheric scattering integrals in any direction, they can't add the effects of other things like clouds or mountains to the sky in a straightforward way, but again this doesn't mean it's completely impossible. In fact Hillaire's paper itself mentions that it's possible to use raymarching and CSMs to achieve light shafts in real time and the results can be seen in Figure 14, but it doesn't go into much detail on how they implemented it.

They will be completely ditching Bruneton's method for Hillaire's method or something similar, so instead of pre-calculating the atmospheric scattering integrals into a lookup table, they will calculate them in real time. This means they no longer will have to deal with the precision issues near the horizon which is what caused the horizon line issue, so it will be gone for good. Another advantage is that Hillaire's method is significantly better at estimating multiple scattering, so the luminance during sunset / sunrise will be much more accurate. Lastly, they talked about an "all new color engine". Given the context of it was that they are going to do more than simply rendering in RGB with more accurate coefficients, it could mean only one thing - ditching RGB rendering altogether and going for full spectral rendering, which means the colors will be even more accurate than what can be achieved with ARPC coefficients.

In case anyone hasn't seen, here are some good news: This week's MSFS development update is out and the wishlist status for ARPC went from "Under Investigation" to "Started (SU_14)" and the wishlist description says "Work is progressing. Looking good for SU_14."

For the case of MSFS, the horizon line is a giveaway artifact in Bruneton-like implementations, and Asobo mentioning the scattering & transmittance LUTs, which is how of Bruneton-like implementations work, alongside with the leftover text in FlightSimulator.exe confirms it. For the case of DCS, their shader code is open, and indeed they use Bruneton's method (in fact I already found a way to have the ARPC coefficients on DCS as well) For the case of XP12, the exposed art datarefs were a giveaway and then I got a confirmation from the developers. For the case of P3D v6, the same reason as DCS, their shader code is open. They also had told in their promotional material that they were using Unreal Engine's sky, which uses Hillaire's method. In fact Sebastien Hillaire works as the principal graphics engineer on Unreal Engine.

I'm sorry that my comment sounded like a lesson on the content of your answers, that wasn't my intention. It was merely an observation about everyone involved in that argument and why your criticisms despite being valid weren't taken well. As said above I agree that VisualXP exceeds everything I've seen, including what can be achieved with ARPC in either of the simulators. It looks simply beautiful. My point was that sadly it's very non-physically-based which means neither LR nor Asobo would be willing to integrate it into the simulator for very valid reasons. With physically based rendering everything works like a clockwork, if a component is not physically based, rest of the components in the chain also become non-physically-based and things quickly turn into a tuning nightmare. My next project is a new custom tonemapper for XP12 called Enhanced Visuals, which replaces the default filmic tonemapper with a more natural hue-preserving one, fixing the cyan tinted sky during day and yellow tinted sky & clouds during sunset / sunrise. I won't get into more detail as it's not a MSFS project and completely off-topic, but it is indeed what I have been spending my time on recently.

For those who are interested, I'm aware that my comments regarding multiple scattering are a bit too technical and don't actually give a feel of how it affects the visuals, so here's a demonstration using XP12: (For debug purposes or not, I love that LR exposes these parameters and I wish Asobo did too) Initial release day XP12: (Before the aforementioned multiple scattering hack and before ARPC coefficients were integrated into the simulator) Current XP12 multiple scattering, but without ARPC coefficients: Initial release day XP12 multiple scattering, but with ARPC coefficients: Current XP12: (After the aforementioned multiple scattering hack and after the ARPC coefficients integrated into the simulator) Currently what we can achieve with this .exe hack is basically akin to the initial release day XP12 multiple scattering, but with ARPC coefficients. When Asobo tweaks the multiple scattering and exposure during sunset / sunrise it should look like current XP12.

As much as I understand your frustration things are not that simple. Both MSFS and XP12 use the exact same physically-based scattering equations, just implemented using different methods. The discrepancies you see are caused by 2 separate issues which need to be addressed separately. 1. Both MSFS and XP12 use autoexposure to have as much dynamic range for a given lighting condition as possible. MSFS underexposes during sunset / sunrise, making everything, including the sky, look too dark. If you switch to the drone camera and manually increase the exposure through its key binding you will see that the sky looks much more realistic with a higher exposure. Not that XP12 is perfect in that regard, it has the opposite problem - it overexposes the sky and causes clipping during sunset / sunrise. 2. The method used by MSFS (Bruneton's method) significantly underestimates multiple scattering. This also causes the sky to be too dark. Even the method used by XP12 (Hillaire's method) underestimates multiple scattering, though to a much lesser degree. Even then, to overcome this XP12 uses a non-physically based hack to artificially boost the multiple scattering amount during the sunset / sunrise. Asobo seems to be really reluctant when it comes to using non-physically based hacks to the degree that we still don't even have accurate visibility, so I'm not surprised that it's not implemented by them. For those who are curious, neither Bruneton's method nor Hillaire's method are superior to each other, they have distinct advantages and disadvantages. For instance Bruneton's method gives you a full LUT for the scattering and transmittance integrals, so you can easily take the the effect of the clouds into the atmosphere into account, which is the case in MSFS. You can see how the aerial perspective / haze on the ground disappears when there is cloud coverage and how the clouds cast light shafts into the sky in MSFS, neither of which are simulated in XP12. It's because it's much harder to implement these effects accurately with Hillaire's method as it doesn't pre-calculate a LUT for the scattering and transmittance integrals but renders a low resolution sky each frame. On the other hand Hillaire's method is much better at calculating multiple scattering, resulting in a much more accurate sky luminance during sunset / sunrise. In the end, it's a compromise. MSFS and DCS use Bruneton's method while XP12 and P3D v6 use Hillaire's method, so both methods are very common and are "state of the art". However based on the last developer Q&A, it appears that Asobo is switching to Hillaire's method for MSFS 2024, which will eliminate both the horizon line issue and the inaccurate multiple scattering. Also you seem to be using a third-party add-on, namely VisualXP, in those screenshots and while I agree that it looks simply beautiful, in fact even better than default XP12 with ARPC coefficients, in the end it's a based on visual references and not physicaly based whatsoever, which is not desired for physically based rendering. So I wouldn't expect a VisualXP-like look from either of the simulators any time soon. What they achieved is insanely beautiful, but it comes with its own compromise. Lastly, I think it's more than fair to compare both simulators, but it's also not surprising that you were met with such a negative response, not because you were criticizing things but due to how you phrased these criticisms. To be more clear, I think it's expected that constantly calling things ugly and making fun of the cameras of people who are offering reference screenshots would annoy people. P.S. Here's XP12 during the time it was newly released, when it wasn't using the non-physically based hack to artificially boost the multiple scattering amount: You can see that it suffered from the exact same issue as MSFS. The hack for the multiple scattering was implemented only relatively recently, and my work on ARPC was integrated to XP12 with 12.06, so literally just a few weeks ago. I pointed out the issue with the coefficients to LR, they were extremely responsive and they quickly implemented it. Now I did the same with Asobo and they said that it will be implemented starting with SU14, so only one SU after I reported it, I don't think it's fair to Asobo to be this negative.

Hi and sorry for being MIA for a while, I've been really busy lately. The numbers are absorption coefficients, which are physical constants that depend on the compositon of the atmosphere and are expressed in units km^-1, in other words, per kilometer. So for instance the absorption coefficient for red tells how much of the red light is absorbed when it travels through a column of air 1 km long. Honestly the 096 at the end makes little to no difference - the input parameters I used to calculate the coefficients have a lot of significant digits and as a result the calculated coefficients also have a lot of significant digits, but 2 significant digits are more than enough and higher amount of significant digits won't make much, if any, perceptible difference.

Sorry for the confusion, I was referring to the projector / OHU there which I assumed to be an LCD one, but either case it appears that most 737s use a HUD with CRT projectors, so a polarizing filter shouldn't be needed, and even if they used LCD projectors, the polarizer would likely be right after the LCD and long before the image arrived at the combiner. Then it seems like the hue shift is solely caused by the coating blocking the green light from the windshield.

I assumed there would have to be a polarizer somewhere after the liquid crystals (assuming the display would be an LCD), really interesting. I'm aware of the coating but it shouldn't affect the light it transmits from the outside, so what makes it wavelength specific like that?

I love this photo because it shows a really cool effect - Rayleigh scattering causing light to be polarized in a wavelength-dependent way. Notice how the sky looks very purple like default MSFS through the HUD, which likely has a polarizing filter, but then it looks blue through the windshield just like predicted by the spectrally rendered ground truth, which ARPC agrees with. Even though in this case it's caused by the HUD, as I mentioned before there are indeed relatively rare cases where twilight sky actually looks purple, which typically occur when clouds block the path of light from the sun or cause light to bounce around the ozone layer more.

Hi, P3D v5, which uses trueSKY, doesn't take light absorption from ozone into account at all, which is why it looks really green out of the box. While the coefficients aren't exposed just like MSFS, by editing the shaders I was able take ozone contribution into account in a freeware mod I released long ago, but it used the same inaccurate coefficients from the Hillaire's paper. P3D v6 on the other hand uses Unreal Engine's (Hillaire's) atmosphere implementation and you should be able to change the coefficients as they are exposed in a configuration file, I don't remember the name of the file but it should be in %AppData%/Roaming/Prepar3D v6. Firstly you should set the Rayleigh and Ozone scale parameters to 1.0 and then you can use the same coefficients in MSFS and XP12. However I'm unable to test it myself as I switched to Linux a while ago and P3D doesn't work in Linux, I only use MSFS and XP now.

That was a typo on my part, it's indeed supposed to be 0.00065 instead.

Hi, to be fair, it's not enough to back up the original file as if the modification is done at the wrong offset, it might cause unpredictable behavior and corrupt the entire installation, which is exactly why I don't recommend it. It's also possible that it might interfere with some antivirus programs as you're changing the .exe file and the original .exe file is signed. Lastly I really don't think the thread will be closed, they don't like the discussion about using a hex editor for the aforementioned reasons, but I think, or at least I hope, that this doesn't mean they will close the entire thread.