Next-Gen Scenery first details!

[Inu 60]

https://developer.x-plane.com/2024/02/the-turtle-needs-a-new-shell/

Some key takeaways from this post & the following comments:

1) New file formats to replace DSF are WIP. DSF will still be supported though so current sceneries will still work.

2) New formats will be based on 3D tiles structures, like: https://github.com/CesiumGS/3d-tiles

3) Mesh patching, will finally be possible, including sculpting/modelling terrain.

4) They are moving to raster images; this will probably make mesh reacting to shaders & pathcing possible (think tessellation stuff), also since they mentioned they want 3d grass for the future, I can imagine the vegetation layers may move to raster images as well, meaning that we may be able to use masks to "grow" vegetation out of a texture, so no more flat 2d baked trees textures.

Edited February 20 by Inu

[tonywob 4,769]

3 hours ago, Inu said:

They are moving to raster images; this will probably make mesh reacting to shaders & pathcing possible (think tessellation stuff), also since they mentioned they want 3d grass for the future, I can imagine the vegetation layers may move to raster images as well, meaning that we may be able to use masks to "grow" vegetation out of a texture, so no more flat 2d baked trees textures.

This one is a good logical step. I'd requested the ability to define tree/vegetation coverage using rasters as it's much easier and space efficient to do this (it's also how MFS does it) and Ben had already mentioned he is well ahead on this. Especially in this new world of AI detection, AI could be used to classify vegetation areas on aerial imagery and then that could be fed straight to X-Plane rather than trying to store it as points or polygons.

 

[Murmur 3,944]

9 minutes ago, tonywob said:

define tree/vegetation coverage using rasters as it's much easier and space efficient to do this

Yeah Ben mentioned the new scenery will be raster-centric, but what does it actually mean? Instead of point and vectors, the new scenery will be divided into discrete "scenery pixels"?

[Inu 60]

29 minutes ago, Murmur said:

Yeah Ben mentioned the new scenery will be raster-centric, but what does it actually mean? Instead of point and vectors, the new scenery will be divided into discrete "scenery pixels"?

He did mention there is another blog post coming about that, but generally, I think the main difference is algorithmic generated vs pre-baked into assets.

So now, to get a 3D forest, you have to draw a polygon, then X-Plane has an algorithm driven by the .for asset it self to determine density/patterns using a perlin noise (that can't be made to correlate with the terrain texture).

With a raster approach, you can have a mask to go with the terrain textures that defines where & how dense vegetation is in every given texel.

The other advantage is vegetation density LODs, those in a raster approach can simply be driven by mipmaps in a seamless manner.

Here is an example:

 

 

For mesh, It really depends on what effect you are trying to achieve. Personally I like to think about a procedural 3 step approach, you have the real world base mesh itself, then you can have a height map texture to define some medium-scale detail (think about stuff like large rocks, cliff patterns), then another height-decal for small scale aspects (pebbles LOL) OR just a normal map/ parallax effects to fake 3D little details.

The good thing is X-Plane terrain texturing already kind of conforms to this scheme & will be even better once 12.1.0 comes out with normal decals, it even has a slope shader (to pick textures based on slope, so you can have custom cliffs & other stuff), various tiling/detailing control maps & other effects to expose other textures.

But the mesh pipeline will have to change along with the new formats imo, to support tessellation/mesh shader alternatives.

 

Edited February 20 by Inu

[tonywob 4,769]

34 minutes ago, Murmur said:

Yeah Ben mentioned the new scenery will be raster-centric, but what does it actually mean? Instead of point and vectors, the new scenery will be divided into discrete "scenery pixels"?

In MFS, it's basically a raster image for each bing tile. Each pixel dictates the height and species of the tree, it can be efficiently stored in a grayscale image. Developers could even use photoshop filters to extract their own trees from aerial imagery, or use AI (e.g. https://github.com/martibosch/detectree)

In my own projects, I have tree level accurate scenery (with each tree having a height from lidar data), and I'm storing this per point in the DSF which explodes out the size to over 100-200mb per DSF. With rasters, this would be significantly easier to store, manipulate and load (it would be attached to the base ortho texture instead of stored as point data)

e.g. This shot below, each tree is stored as a single point (lon, lat, height) in the DSF. There are millions of them. 

 

If I could instead just store it as an image with the different values representing heights and species it would still give a similar realistic appearance/distribution, but would use significantly less space

[scotchegg 4,186]

10 hours ago, tonywob said:

Especially in this new world of AI detection, AI could be used to classify vegetation areas on aerial imagery and then that could be fed straight to X-Plane rather than trying to store it as points or polygons.

This was something I was hoping they could do, AI-based translation of freely available satellite imagery into super granular land class / textures.
 

Theoretically would do away with a lot of the problems of simply streaming the imagery (and many of its imperfections) itself.

Looking forward to what’s next.

[Bjoern 1,157]

Wouldn't AI scanning require some kind of color normalization first to positively identify vegetation? I mean you could get an ortho tile that was recoded in late spring while its neighbor was recorded in the deep of autumn. Surely that would confuse the algorithm, no?

 

On 2/20/2024 at 1:47 PM, tonywob said:

e.g. This shot below, each tree is stored as a single point (lon, lat, height) in the DSF. There are millions of them.

Wouldn't it make more sense to find a way to drape a forest polygon over any contiguous number of points? I can imagine that there's some odd Python package out there that can do just that...

[Daube 353]

On 2/20/2024 at 1:47 PM, tonywob said:

e.g. This shot below, each tree is stored as a single point (lon, lat, height) in the DSF. There are millions of them. 

Beside the superb forest coverage and lighting of the tree tops, is that an approach to Concrete ? 🙂 

[tonywob 4,769]

1 hour ago, Daube said:

Beside the superb forest coverage and lighting of the tree tops, is that an approach to Concrete ? 🙂 

It is indeed, one of the best areas to fly in the sim

On 2/21/2024 at 9:14 PM, Bjoern said:

Wouldn't it make more sense to find a way to drape a forest polygon over any contiguous number of points? I can imagine that there's some odd Python package out there that can do just that...

Tried that, there was nothing satisfactory that could do it without needing a lot of processing per tile and if I kept the grouping as 1 metre/species, it still produced a LOT of polygons and X-Plane's point fill tree coverage in polygons was too sparse for the results to be as pleasing. I tried decreasing the grouping size to larger units, but it starts to lose the realistic and varied look. In addition, the points no longer exactly match where there are real trees. Either way, I already have the output as images for MFS as well, so once XP is updated to support rasters, I'll be able to use them as well. 

I did this with the TrueEarth series BTW for Orbx to keep the file sizes down, but had to sacrifice some accuracy

Performance was never an issue and the legacy DSF format could handle it, it was simply the size of the files that becomes an issue.

On 2/21/2024 at 9:14 PM, Bjoern said:

Wouldn't AI scanning require some kind of color normalization first to positively identify vegetation? I mean you could get an ortho tile that was recoded in late spring while its neighbor was recorded in the deep of autumn. Surely that would confuse the algorithm, no?

Yes, generally it works on black/white images and looks for noise and patterns using a sliding window. As long as the imagery is taken in mid summer and not blasted out by sunlight, I found it worked fine. If you also combine it with NIR imagery, it's pretty accurate. It actually failed in the Concrete area in my screenshot because the mountains are all in a dark black shadow, so I had to use other techniques to detect the trees

[Inu 60]

Here is the last follow-up blog post:

https://developer.x-plane.com/2024/03/we-are-all-raster-farians-now/

The last paragraph is interesting to read, it won't be just an improvement for orthos based meshes but also for procedural landclass based texturing.

The ability for more varied & complex data to play with the current already procedural texturing methods in X-Plane sounds very promising especially with 3D vegetation masks, I can already imagine some seasonal 3D crops in farmlands and such, even 3D pebbles 😝 

But more importantly, the situation with current meshes should be solved with the new file formats, hopefully no more chatoic "mesh compatibility patches" for users and developers alike.

Edited March 4 by Inu

[mSparks 2,086]

7 hours ago, Inu said:

The ability for more varied & complex data to play with the current already procedural texturing methods in X-Plane

yep, there is a lot of cool new toys that can be played with once they get the groundwork (pun intended) done.

By faaaaar the most exciting potential for me is that it should put an end once and for all to most if not all of the garbage scenery that is still hanging round from gawdknowswhen. There was a lot of complaining when XP12 first dropped that this wasnt ready, great that we finally not just have some info on it, but to also know the plan is a good one. 

mountain areas already look phenominal, but flat fielded and desert areas seriously end up lacking.

a solid, performent, solution to that is going to be epic.

just landclass "color" will make a massive difference imho. fields and base areas that fully react seamlessly with the seasons and weather.... "perfected" by overlaying some reasonably low res ortho.. omg.

plus they already "proved it" with the snow stuff (which is already raster layers that got added to the existing dsf formt)

I almost cant look because I get so excited

Edited March 5 by mSparks

[Bjoern 1,157]

"Raster" is still fairly negatively connotated in my book because of acute "Falcon terrain" trauma.

(Not my screenshots.)

 

[Inu 60]

4 hours ago, Bjoern said:

"Raster" is still fairly negatively connotated in my book because of acute "Falcon terrain" trauma.

(Not my screenshots.)

Look at those curved edges though 😝

17 hours ago, mSparks said:

yep, there is a lot of cool new toys that can be played with once they get the groundwork (pun intended) done.

By faaaaar the most exciting potential for me is that it should put an end once and for all to most if not all of the garbage scenery that is still hanging round from gawdknowswhen. There was a lot of complaining when XP12 first dropped that this wasnt ready, great that we finally not just have some info on it, but to also know the plan is a good one. 

mountain areas already look phenominal, but flat fielded and desert areas seriously end up lacking.

a solid, performent, solution to that is going to be epic.

just landclass "color" will make a massive difference imho. fields and base areas that fully react seamlessly with the seasons and weather.... "perfected" by overlaying some reasonably low res ortho.. omg.

plus they already "proved it" with the snow stuff (which is already raster layers that got added to the existing dsf formt)

I almost cant look because I get so excited

Absolutely, also being a more than one trick solution, I could imagine you can patch orthos just where they are actually useful, not so much for dense urban areas or forests (for those you'll just need a vegetation mask).

Edited March 5 by Inu

[mSparks 2,086]

5 hours ago, Bjoern said:

"Raster" is still fairly negatively connotated in my book because of acute "Falcon terrain" trauma.

(Not my screenshots.)

 

this is what it looks like in unigine 

 

 

[tonywob 4,769]

14 hours ago, Bjoern said:

"Raster" is still fairly negatively connotated in my book because of acute "Falcon terrain" trauma.

Default MFS uses rasters for some features (vegetation, water, elevation etc)... and that looks really good. I see this as a positive step, as long as we can fall back to vectors if we need to for really tight detail that rasters can't encode.