On 1/20/2025 at 8:12 AM, mSparks said:

 

having solid vis done properly has always been a strong point of X-Plane - and that hasn't changed.

 

I don't remember much in XP11 (I was avoiding clouds in that sim, because 2D and flashing cloud shadows...), but I got some lovely low level clouds forcing me to perform scud running in XP12 for sure. 

I also got similar conditions around Talkeetna no later than yesterday in MSFS too, and it was looking equally impressive, especially when I reached a place in the center of the valley where I had the opportunity to go above the cloud layer, and discover the cloud tops (and the Denali mountains) lit up by the raising sun 🥰

1 hour ago, Daube said:

I don't remember much in XP11 (I was avoiding clouds in that sim, because 2D and flashing cloud shadows...), but I got some lovely low level clouds forcing me to perform scud running in XP12 for sure.

long running "discussion" going back a few years

 

9 hours ago, mSparks said:

Turn on Stochastic Kinetic Energy Backscatter (SKEB) scheme in GDAS ensemble forecasts

Use a Local Ensemble Kalman Filter (LETKF) with model space localization and linearized observation operator to replace the Ensemble Square Root Filter (EnSRF)

Both under the section of data assimilation. GDAS is merely used to set up the initial condition for GFS:

https://www.ncei.noaa.gov/products/weather-climate-models/global-data-assimilation

"...uses the Global Data Assimilation System (GDAS) to interpolate data from a variety of observing systems and instruments onto a three-dimensional grid. NCEI provides access to this gridded output data, which is used to initialize the NCEP Global Forecasting System (GFS) model."

GDAS is not the final solution. It's the tool that ingests/assimilates the measured atmo data to define the flow field at time=0. The GFS CFD model then iterates on the solution for 3hr in the future (and beyond), writes gribs, and XP downloads the GFS solver output. 

4 hours ago, blingthinger said:

CFD

so close.

There is no "CFD model" afaik.

the NOAA take the data from the satellites and variety of other sources to estimate what the state of weather over the entire earth over the last x hours, then put that into a regression model to estimate the same weather parameters over the next x hours.

in steps of 6 hours.

Laminar slurps the last and next of those estimates and constructs the weather now and the weather in 1 minute from now, over a tile of a few hundred miles around the aircraft, feeds that into their environment variables (datarefs) which also draw it.

4 hours ago, blingthinger said:

GDAS

GDAS and GFS are basically the same thing, but GDAS also uses data from say weather balloons which take several hours to return measurements. 

So while GDAS gives a better estimate of what the weather was doing 24 hours ago, it cannot produce estimates as regularly. pbuckner explained that very nicely in his post I already linked.

looking at the specs, gdas produces a more accurate forecast for the next 9 hours, gfs produces a forecast for the next 15 days.

So those gribs in the XP weather folder may very well be gdas data.

Edited January 21, 20251 yr by mSparks

2 hours ago, mSparks said:

GDAS and GFS are basically the same thing

Not even close. And pbnucker is wrong about a couple things in his post and even admits that that is what he "thinks" is happening. Though he also does conclude that XP is using GFS, so I do agree with at least something there (in addition to his actual specialty: his awesome python plugin).

You are correct that GDAS is a statistical model of the global weather state based on measurements (satellite, etc.).

 

GFS is a CFD solver. Finite volume method, to be precise. 

https://www.emc.ncep.noaa.gov/emc/pages/numerical_forecast_systems/gfs.php

"The forecast component uses the Finite Volume Cubed (FV3) model with a resolution of ~13 km..."

 

https://www.gfdl.noaa.gov/wp-content/uploads/2020/02/FV3-Technical-Description.pdf

"FV3 is a natural evolution of the hydrostatic Finite-Volume dynamical core (FV core) originally developed in the 90s on the latitude-longitude grid...."

and later on:

"FV3 is the solver that integrates the compressible, adiabatic Euler equations in a weather or climate model..."

Where Euler equations are:

https://en.wikipedia.org/wiki/Euler_equations_(fluid_dynamics)

"... the Euler equations are a set of partial differential equations governing adiabatic and inviscid flow. They are named after Leonhard Euler. In particular, they correspond to the Navier–Stokes equations with zero viscosity and zero thermal conductivity."

Finite-volume is a numerical technique used to discretize differential equations. CFD is the discretization of the Navier Stokes "family" of partial differential equations.

 

GDAS does not simulate the flow field of the full global air volume. It uses statistics to fill in the voids on the CFD grid that exist around the measurement sites and satellite data. Repeating the NOAA link I posted a bit ago: it's used to initialize the CFD solution. It is not a prediction in and of itself in this context. That said, it appears that GDAS can be used to predict weather at specific stations, but only for "...evaluation/development purposes." It is not the primary forecast tool used by NOAA but it is part of the tool chain.

https://registry.opendata.aws/noaa-ufs-gdas-pds/

Hanging a hat on claiming that a rudimentary stats model (ai/ml aside) is more effective at predicting 3D global weather than a CFD solver converging the PDEs of conservation of the mass, momentum, and energy, is your own funeral.

Not to mention the fact that GDAS data dumps are released publicly on a weekly basis. It's in a 6hr cadence but backwards in time, not forward. Hardly the forward-looking 3hr cadence used by XP.

https://www.ncei.noaa.gov/data/ncep-global-data-assimilation/access/202501/

Edited January 21, 20251 yr by blingthinger

25 minutes ago, blingthinger said:

Finite Volume Cubed

gribs are finite volumes, so not sure what your point is, that has nothing to do with Navier–Stokes equations

25 minutes ago, blingthinger said:

CFD

you keep calling it CFD, but nowhere is this mentioned?

https://www.ncei.noaa.gov/products/weather-climate-models/global-forecast

The Global Forecast System (GFS) is a National Centers for Environmental Prediction (NCEP) weather forecast model that generates data for dozens of atmospheric and land-soil variables, including temperatures, winds, precipitation, soil moisture, and atmospheric ozone concentration. The system couples four separate models (atmosphere, ocean model, land/soil model, and sea ice) that work together to accurately depict weather conditions.

The model is constantly evolving, and regularly adjusted to improve performance and forecast accuracy. GFS is a global model with a base horizontal resolution of 18 miles (28 kilometers) between grid points. Temporal resolution covers analysis and forecasts out to 16 days. Horizontal resolution drops to 44 miles (70 kilometers) between grid points for forecasts between one week and two weeks. 

https://www.ready.noaa.gov/gdas1.php

The GDAS is run 4 times a day, ie, at 00, 06, 12, and 18 UTC. Model output is for the analysis time and 3, 6, and 9-hour forecasts.

NCEP post-processing of the GDAS converts the data from spectral coefficient form to 1 degree latitude-longitude (360 by 181) grids

____

The main significant difference between gfs and gdas, is gfs includes the land/soil model which are needed to forecast inland cloud formation from lakes and rivers, allowing it to go out beyond just the 9 hours of gdas.

Edited January 21, 20251 yr by mSparks

@jcomm

Thank you for your clarification post in pbnucker's org discussion:

https://forums.x-plane.org/index.php?/forums/topic/282159-what-the-heck-is-a-grib-file/&page=2#findComment-2500617

"...EDR is getting deprecated sooner or later. The ARPÈGE and ECMWF models I work with already have parameters that are based on average u/v/z norms and wind intensity. They characterize turbulence irrespective of the source better than EDR."

It was hurting my head that they use(d) dissipation to estimate CAT intensity instead of turbulent kinetic energy!!

30 minutes ago, mSparks said:

that has nothing to do with Navier–Stokes equations

https://en.wikipedia.org/wiki/Finite_volume_method

 

33 minutes ago, mSparks said:

but nowhere is this mentioned?

You mean other than those NOAA links precisely describing the algorithm? Yikes. You're not helping yourself here.

 

31 minutes ago, mSparks said:

The main significant difference between gfs and gdas

tsk tsk tsk. Still no references other than a comment on update cadence and trying to correlate that to soil model in GFS? Keep digging your grave on this one.

54 minutes ago, mSparks said:

https://www.ready.noaa.gov/gdas1.php

The GDAS is run 4 times a day, ie, at 00, 06, 12, and 18 UTC. Model output is for the analysis time and 3, 6, and 9-hour forecasts.

Did you not investigate the links on that url? What are you even thinking on this one?

This one: https://www.ready.noaa.gov/data/archives/gdas1/ 

appears to be the archive. Last uploaded file was Oct of last year.

They run the model 4 times a day (24/4 = 6 hr cadence) but they don't appear to release the data to the public except on a weekly basis. All internal usage. Do you know where else GDAS data is released to? Cause this ain't it.

Edited January 21, 20251 yr by blingthinger

4 hours ago, blingthinger said:

Last uploaded file was Oct of last year.

its 2025 now not 2026 🤣

And I think I'm lucky when I gain an extra day.

Edited January 22, 20251 yr by mSparks

4 hours ago, mSparks said:

its 2025 now not 2026

Ah! So it is. Now I see them. Hidden in plain sight right at the top of the list with nondescript govn't file names.

Not gribs though. Can't figure how to open that archive file to take a peek.

Found this repo too:

https://nomads.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/

which does provide the individual gribs. Can't read those either though. I'm trying to use XyGrib like @Franz007 was in that org thread. No luck. And can't get metview to install at the moment. Task for later.

 

 

I'm now agreeing with this:

13 hours ago, mSparks said:

GDAS and GFS are basically the same thing

Found a past status review that mentions "FV3GDAS". In my mind FV3 was GFS only but there it is in the same acronym as GDAS.... so I was wrong there.

https://www.emc.ncep.noaa.gov/gmb/wx24fy/doc/CCB_FV3GFS_24sept2018_v4.pdf

Which would mean that GDAS is also CFD, not a simple stats model. This would also explain away my confusion with

On 1/20/2025 at 12:05 PM, mSparks said:

Stochastic Kinetic Energy Backscatter from GDAS

KE Backscatter is a Large Eddy Simulation thing. LES is a turbulence model for CFD. I was racking my brain trying to figure out how/why a stats model was using a CFD turbulence model. If GDAS is CFD too, that statement makes sense.

4 minutes ago, blingthinger said:

I'm now agreeing with this:

👍

4 minutes ago, blingthinger said:

also CFD

I don't believe its CFD, I would expect it to be "just" a regression io model, there is no need to specify how volumes change over time when they have all the data needed to measure how volumes change over time, then refine it based on maximum likehood results.

10 minutes ago, blingthinger said:

Can't figure how to open that archive file to take a peek.

I would expect it to all be open source. US govt is great for stuff like that

https://github.com/NOAA-EMC/GDASApp

6 hours ago, mSparks said:

I don't believe its CFD, I would expect it to be "just" a regression io model, there is no need to specify how volumes change over time when they have all the data needed to measure how volumes change over time, then refine it based on maximum likehood results.

The algorithm descriptions are 100% CFD. Hurricane trajectories are not estimated with statistics. When you see descriptions of Kalman filter, they're describing how they initialize the CFD grid with the various measured data sources. 

9 hours ago, blingthinger said:

statistics

gdas will likely be based around a large nearest neighbour DTMC with a 3 hour cycle length. (hence producing weather states at 0, 3, 6 and 9 hours)

that's the whole baysian stats vs frequentist stats "theory that wouldnt die" we just spoke about in the cfd vs AI thread. 

afaik CFD wouldnt give you the probability of a volume developing cloud based on how wet the soil underneath it is. DTMC does.

But it's open source, so if you are really interested there is no need to guess. link a LOC

Edited January 22, 20251 yr by mSparks

1 hour ago, mSparks said:

need to guess

 

Guess? There's no guessing here. I've already linked many NOAA links showing that it's CFD. Which one are you struggling to understand?

On 1/21/2025 at 11:49 AM, blingthinger said:

GFS is a CFD solver. Finite volume method, to be precise. 

https://www.emc.ncep.noaa.gov/emc/pages/numerical_forecast_systems/gfs.php

"The forecast component uses the Finite Volume Cubed (FV3) model with a resolution of ~13 km..."

 

https://www.gfdl.noaa.gov/wp-content/uploads/2020/02/FV3-Technical-Description.pdf

"FV3 is a natural evolution of the hydrostatic Finite-Volume dynamical core (FV core) originally developed in the 90s on the latitude-longitude grid...."

and later on:

"FV3 is the solver that integrates the compressible, adiabatic Euler equations in a weather or climate model..."

Where Euler equations are:

https://en.wikipedia.org/wiki/Euler_equations_(fluid_dynamics)

"... the Euler equations are a set of partial differential equations governing adiabatic and inviscid flow. They are named after Leonhard Euler. In particular, they correspond to the Navier–Stokes equations with zero viscosity and zero thermal conductivity."

Finite-volume is a numerical technique used to discretize differential equations. CFD is the discretization of the Navier Stokes "family" of partial differential equations.