Sign in to follow this  
Guest

OOM workaround for FlyTampa's Copenhagen

Recommended Posts

After many hours of trial and error testing I was finally able to identify the single file causing this OOM issue.

 

filename: cvxOresund_Terrain.bgl

location: ...\Lockheed Martin\Prepar3D v2\Addon Scenery\FlyTampa-Copenhagen_LC\scenery

 

rename the file to cvxOresund_Terrain_blg.OFF

 

While investigating this issue I also notice that the following files caused stutters and performance issues in P3D V2.5 (12945):

 

filenames: ekch_rwy*.bgl

location: ...\Lockheed Martin\Prepar3D v2\Addon Scenery\FlyTampa-Copenhagen\scenery

 

Hope this helps others running into OOM problems.  Note the file size is Tiny (1KB), but it's memory impact is huge, it'll cause 1GB VAS to be consumed in just a matter of a few seconds.

 

Here is the original issue I was having with FT Copenhagen and P3D v2.5:

 


 

After applying my workaround above:

 


 

I have been in contact with the dev.

 

Cheers, Rob.

Share this post


Link to post
Help AVSIM continue to serve you!
Please donate today!

Great find, Rob! Thanks for sharing. :)

Share this post


Link to post

Awesome, Rob, thank you!

I wonder if the same would do the trick for FSX.

Share this post


Link to post

Many thanks Rob !

 

Do you test by taking out files and them add them back and test or do you have a tool for that !

Share this post


Link to post

I don't have this airport but the implications of your find are interesting and possibly far reaching. Your a star by the way as I would imagine that it took an age to eliminate.

 

I know zero about BGL's but would imagine that they are a script created either manually or with a scenery programme then compiled. So is it the scenery programme, the compiler or how a particular section of code is handled within P3D?

 

Rafal (above) mentioned FSX which implies this is old scenery, which further implies that perhaps when it was released FSX didn't OOM because it was not loaded enough with other scenery therefore the issue never showed up?

 

Perhaps this is a latent bug and what you have found will be amazing. Perhaps I'm making too many assumptions from too little knowledge lol

Share this post


Link to post

Thank you very much Rob.

Share this post


Link to post

Lost 100mb more VAS disabling this file. Default Cessna, 2.GB free VAS. Stayed like that for an hour.

So possible LOD may effect this file. I run a LOD of 4.5. I think it depends what settings you use that effects VAS.

I use Cloud shadows, but not vege or terrain.

Share this post


Link to post

You are relentless, Rob, thank you for all the work!

While investigating this issue I also notice that the following files caused stutters and performance issues in P3D V2.5 (12945):

 

filenames: ekch_rwy*.bgl

location: ...\Lockheed Martin\Prepar3D v2\Addon Scenery\FlyTampa-Copenhagen\scenery

A lot of lights in those files, they probably cost some performance:

C:\>dir "F:\P3D2\Addon Scenery\FlyTampa-Copenhagen\scenery\ekch_rwy*.bgl"
 Volume in drive F is FFS
 Volume Serial Number is BA54-A9B9

 Directory of F:\P3D2\Addon Scenery\FlyTampa-Copenhagen\scenery

18-10-2014  20:32           122.354 ekch_rwy_04L22R_lights.bgl
18-10-2014  20:32           112.056 ekch_rwy_04L22R_pads.bgl
18-10-2014  20:31            53.008 ekch_rwy_04L_appr_lights.bgl
18-10-2014  20:33             1.828 ekch_rwy_04L_papi.bgl
18-10-2014  20:32            94.634 ekch_rwy_04R22L_lights.bgl
18-10-2014  20:32            50.642 ekch_rwy_04R22L_pads.bgl
19-10-2014  02:20            35.904 ekch_rwy_04R_appr_lights.bgl
18-10-2014  20:33             1.828 ekch_rwy_04R_papi.bgl
18-10-2014  20:32            87.074 ekch_rwy_1230_lights.bgl
18-10-2014  20:32            35.102 ekch_rwy_1230_pads.bgl
18-10-2014  20:29             3.492 ekch_rwy_12_appr_lights.bgl
18-10-2014  20:34             1.828 ekch_rwy_12_papi.bgl
18-10-2014  20:27            53.008 ekch_rwy_22L_appr_lights.bgl
18-10-2014  20:34             1.828 ekch_rwy_22L_papi.bgl
18-10-2014  20:30             4.754 ekch_rwy_22R_appr_lights.bgl
18-10-2014  20:33             1.828 ekch_rwy_22R_papi.bgl
18-10-2014  20:29            14.272 ekch_rwy_30_appr_lights.bgl
18-10-2014  20:34             1.828 ekch_rwy_30_papi.bgl
18-10-2014  20:33               546 ekch_rwy_loader.bgl
28-10-2014  05:20         4.651.081 ekch_rwy_units.BGL
              20 File(s)      5.328.895 bytes
               0 Dir(s)  110.544.683.008 bytes free

Share this post


Link to post

Thanks, Rob. I've seen VAS usage with this scenery is dependent on which way I take off from. When pointing directly against the airport (sitting on the runway, 04R, L), VAS usage is 600 MB higher in contrast to if I'm faced the other way.

Share this post


Link to post

Thanks very much, Rob. I sense a great deal of patient trial and error here. I don't suppose there's a shortcut to finding the problem file, because I suspect there may be one (or two) in Aerosoft Dublin, which consumes a ludicrous amount of VAS for what it is, and it would be good to track that one down.

 

With these .bgls being disabled, does anything obvious go missing from the scenery?

Share this post


Link to post

I'm curious Rob, what amount of VAS do you measure when standing on the runway (04 R or L)? I disabled this file but did not experience any drop in VAS used. With LOD radius set to 4.5 and autogen set to normal, after activating ASN and FS2Crew in the NGX I have already eaten up 3600 MB !!!

Share this post


Link to post

 

 


Do you test by taking out files and them add them back and test or do you have a tool for that !

 

I wish I had a tool for that, but I don't.  Sadly the process is VERY tedious and is just a matter or removing files and testing.  But the bigger picture I'm more interested in is why such a small file could cause such a major VAS issue ... get the devs and LM discussing is my priority -- don't really care who's issue it is here just looking for solutions.

 

 

 


A lot of lights in those files, they probably cost some performance

 

Some of the lights are done with ASM process which is "probably" the source of the stutters/performance issues in P3D (only) ... the dev's are aware and they're working on a solution.

 

 

 


I'm curious Rob, what amount of VAS do you measure when standing on the runway (04 R or L)?

 

Sorry, don't have VAS numbers for that runway ... I have the VAS values in my videos above but they hard to read unless you run 4K and even then the video is highly compressed so the VAS text is difficult to make out.  But with my workaround I'm starting at about 1.4GB free after takeoff it drops to about 1.0 GB free, then increases again to 1.2 GB free once over Bunkeflostrand.  (note this is with FTX Global 1.30, PILOT's Mesh, and Orbx OpenLC EU, MT6, and ASN 'saved' weather).

 

 

 


With these .bgls being disabled, does anything obvious go missing from the scenery?

 

Yes, but I haven't spent enough time to determine exactly what as my goal was to resolve OOM issue first, the performance issue I just stumbled into while testing.

 

LM are aware of my testing also and are looking into it.

 

Cheers, Rob.

Share this post


Link to post

Thanks so much for that Rob. I was going crazy about my oom's and your workarround seems to work cause currently i fly where i get oom' before withour your "tweak"

 

Cheers, carsten

Share this post


Link to post

Just did a test flight from EKCH to 29 palms new airport EDBC

with LOD set to 6.5

 

a.jpg
 
b.jpg

 

c.jpg

 

d.jpg
 
e.jpg
 
Perfect

Share this post


Link to post

 

 


I know zero about BGL's but would imagine that they are a script created either manually or with a scenery programme then compiled. So is it the scenery programme, the compiler or how a particular section of code is handled within P3D?

 

I don't know about FSX, as far as I know the ASM process is "safe" with FSX but it's a big problem for P3D.  

 

However, in the  case of the cvxOresund_Terrain.bgl and it causing rapid VAS depletion to eventual OOM, I've been informed it was a BGL produced by the P3D SDK.  So if that's the case then it would warrant some investigation by LM and yes it could have far and wide implications.  Even if it's some conflict with another 3rd party product, it should not exhibit massive VAS depletion like I demonstrated above.

 

BGL's are produced via the SDK and hold all kinds of info ... there are "outside" tool's and process that work with BGLs that aren't a part of the SDK but are used by developers primarily to work around the limitations of the SDK.

 

Identification is key to resolution ... no amount of statements about "code is junk" or "code needs to be re-written" is going to help or solve problems ... I see a lot of these comments and frankly their painful to see ... what makes people think that re-writing entire code base is going to solve anything?  Since programmers are human (and do make mistakes), re-writing an entire code base would just introduce even more bugs and issues to be resolve ... I'm not aware of any programmer from top to bottom that can write bug free and highly efficient code of this complexity from scratch and get it done in one shot (and I know A LOT of very talented programmers, way more talent than I).

 

It's a process of working with 3rd party and working with LM to come to resolution (or at least identification).  Now is the time to use that opportunity as LM do listen, do adjust, and do fix issues.  With my "end user" hat on (not my developer hat), I'm just trying to help pin point specific issues as this will save both 3rd party and LM time ... in the hope of more rapid problem resolution.

 

Cheers, Rob.

Share this post


Link to post

Hi Rob mate,

 

What has the developer said? He's a nice guy and usually responds pretty quick.

 

Alex

Share this post


Link to post

VAS depletion to eventual OOM

 

Hi Rob,

 

From my experience the main problem is arriving at a major airport, as VAS depletes enroute and usually leads to an OOM on final or shortly thereafter.

 

Interestingly enough I've never experienced the OOM you mention leaving EKCH, though I've had plenty on arrival. There are certain areas I now avoid flying over altogether, as trying to engineer a flight plan around OOM hotspots like London is just tedious.

 

It's a shame that we now have hardware to run FSX and Prepar3D at a decent framerate, but are stuck in this OOM-bubble due to the 32-bit VAS limit. I hope LM will eventually come up with a solution for all of us to enjoy complex scenery without having to worry about any program crashes.

 

Cheers,

 

Jerome

Share this post


Link to post

Strange indeed. Never had this issue leaving my home airport but had to disable UTX2 or vector when i was planning arrival. I do have the HD pack installed but using 1024 tml in the NGX and 777 with lod 4.5.

 

Anyone tested if there is a impact with hd textures using TML=1024?

 

Michael Moe

Share this post


Link to post

Interesting.. I'm experiencing the same high VAS issue with FT Dubai in P3DV2. Anyone else having that issue?

 

ROMAN78

Share this post


Link to post

The high VAS at OMDB is not surprising - look at all the autogen for the entire city of Dubai surrounding it, including all those skyscrapers.

 

OMDB is another order of magnitude more complex than EKCH.

Share this post


Link to post

What has the developer said? He's a nice guy and usually responds pretty quick.

 

Yes, the dev is VERY nice guy and has responded and is looking into it ... the dev has been open to discussion which is pretty rare and I certainly appreciate it.  

 

3rd party development can be a coveted world (especially from an "outsider") which can make problem identification/resolution difficult ... and finding end users to spend hours pin pointing a specific file (or files) that causes problems in certain situations is pretty rare.  I just happened to have a free weekend to dig in as I was working on some other testing for LM.  

 

 

 

but are stuck in this OOM-bubble due to the 32-bit VAS limit.

 

In this specific instance 64bit would not have solved the problem (memory was being consumed at a rate of 400MB every couple of seconds) ... 64bit would have extended it a little until all your RAM was consumed and then started paging memory to your OS paging file until that ran out and then OOM.

 

The surprise was that just one tiny 1K file could cause this issue ... I'm waiting to see exactly what this triggered and why.  Like I said 32bit or 64bit will not matter, identification is they key to problem resolution regardless of 32bit or 64bit code.

 

We all want a 64bit product and it will come, but for here and now figuring out existing VAS consumption issues is important moving forward ... if it turns out there is a problem with P3D SDK when it builds a BGL with specific set of parameters then it could have wide spread implications.  BUT, we'll need to wait and see what comes of this ... step 1 is to identify, now we wait for step 2, 3, etc.

 

Cheers, Rob.

Share this post


Link to post

I completely concurre that identification is the main ingredient. As an example, when I was on the technical side with a major retailer and DVD was introduced, as a company some off our players would not play certain disks. Some major manufacturers also had this problem with their players. Was it the disk or was it the player? As you can imagine arguments ensued. I was given the unenviable task of finding the solution.

 

The dvd consortium issued two huge documents one was for the player manufacturers and one for the disk manufacturers. Call it the SDK of player and disk manufacture/authoring. Of course, as you mention, everything was written and made by humans. Like all documents of this type, some things were clear and some things were open to interpretation. A DVDs player is like a computer it has to have its own operating system written and it had to have the correct memory etc. The disk had to be authored so it too was reliant on a computer programme being written to correctly author the disk according to the SDK. So, was it the player? Was it the Disk? Was it the authoring tool? Who was suffering? Everyone! We were getting returns of players, the retailers selling the disks were getting returns and the big studios releasing blockbuster movies were not happy.

 

To cut a long story short, before a certain first for DVD blockbuster title was issued by one of the worlds biggest studios, so sales were expected to be huge I was already working with them to find the problem. Let's just say that by this time the legal eagles were starting to extend their tendrils so their was a lot of pressure. We brought together the manufacturers the studios and the authoring houses and because of one certain disk (a little like you finding the BGL) step one was accomplished.

 

People that say the words "bad code" simply show their lack of experience. Finding problems is about the detail and sometimes bringing all the different parties to the table. It's all about finding the hook and sometimes facilitating to solve the problem. In my example above, knowone had made an error everyone had done everything correctly, by the book, and the two books were huge. The issue was interpretation in this case. Step two followed quickly but their was also a step three which was solved and by getting everyone to talk and finding the hook, the problem had a resolution. I call it the Appolo 13 solution. The Luner module was made by one manufacturer, the command module by another. NASA issues the SDK and when that one in a million problem occurs, all the parties got together and solved it. Software is no different.

Share this post


Link to post

A little bit off topic but this also appliable to this airport :

 

In the last week I done at least 10 flights.

 

It is always the same : 1.4-2.0 Gb free VAS at the departing airport.

Then in flight VAS gets less and stabilizes at 800-1000 mb.

 

When descending VAS slowly gets less till 500-700 mb.

When the airport is 20-30 mls away free VAS gets less quick.

 

After landing ( only with very light settings ) free VAS stabilizes....

 

As free VAS doesn't get higher when leaving the airport it looks like the scenery stays in the memory.

However when flying at 10.000 + VAS stabilizes.

That looks like there is scenery unloading.

 

But when descending free VAS is getting less again till OOM or with ligt settings I can land....

That looks as if scenery is not unloading again...

 

Can someone explain what is happening ?

Share this post


Link to post

 

Just did a test flight from EKCH to 29 palms new airport EDBC

with LOD set to 6.5

 

Hi David,

 

I dare you to reverse your route and now head back into EKCH with a 6.5 LOD  :lol:

 

Cheers,

 

Jerome

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this