Smoothing terrain near airports

[Guest jkvato]

Last fall I began working on a utility to smooth terrain near airports to alleviate the problem of terracing that often happens with add-on mesh. Holger gave me a good idea back then on how to approach this problem, and my results weren't very good. After a hiatus of a few months, and now that I've delved into the world of BlackArt, I think that this approach is viable.Here are a couple of screenshots that illustrate this "airport plateau" problem, or "terracing":http://www.jktaylor.com/flightsim/SVCS1a.jpghttp://www.jktaylor.com/flightsim/SVCS2a.jpgThe approach to fixing this is to create two polygons. The inner polygon is the shape of the airport footprint and filled at airport elevation. The outer polygon is an area that surrounds the inner polygon and is filled with missing data, as shown below:http://www.jktaylor.com/flightsim/fsscr020.gifInitially, I tried using MicroDEM's interpolation routine to fill in the missing data, hoping that it would result in a nice, gradual transition from nearby terrain up to the airport. This didn't work so well, but in my limited testing so far BlackArt is proving to be an effective way of accomplishing this task. Here is what the interpolated terrain looks like in MicroDEM (after interpolating with BlackArt):http://www.jktaylor.com/flightsim/fsscr019.jpgAnd here are some shots of the final result in FS9:http://www.jktaylor.com/flightsim/SVCS1b.jpghttp://www.jktaylor.com/flightsim/SVCS2b.jpgYou can see in the last shot that I was also able to get rid of the little hill at the end of the runway.There are a couple of hangups, though. As far as I can tell, BlackArt can only read SRTM HGT tiles, and therefore this technique can only be used on SRTM mesh. I don't know of a way to use this method on NED data. Maybe someone can shed more light in this area.Another thing is the problem when an airport stradles two SRTM tiles, such as SVMI (Maiquet

[Holger 0]

Hi John,great to see you're back working on this nifty utility! Can you remind me/us how you create those two polygons? Do you "draw" them in with the utility or does this involve hand-editing of the raw data? And how do you get the shape of the airport's apron? Is there an option for an overlay of the AB9*.bgl files?I'm sure that John can extend Blackart to read other formats or, if not, he or us might be able to come up with an external translator (NED>SRTM>NED).The tile boundary issue is a very good point! I believe the reason Blackart is currently restricted to a single tile is that any of the more involved algorithms would overwhelm most computers when applied to larger areas. Would be nice though...Please keep us posted on your progress!Cheers, Holger

[Guest jkvato]

The utility is script-driven, so it's unfortunately rather tedious to define the polygons. In my testing I've just slewed around the airport and jotted down the coordinates. It's been a while since I've used Steve Greenwood's flatten utility, but that could probably be used to collect the data points for the polygons which could then be "cut-n-pasted" into the script.Both polygons are done this way, so getting the footprint of the airport is done by slewing.What are the AB9*.bgl files? I haven't delved into that side of mesh/scenery design yet. Are those the scenery files that contain the flatten polygons for airports? Maybe if this utility catches on, I might see what could be done to automate this process to a higher degree.In the example above, the inner polygon was easy to create, but it took a few trials to get the outer polygon right. In fact, that example could still be improved. If you make the missing data go out too far, you can easily wipe out surrounding terrain; yet if you don't go out far enough you still end up with cliffs.

[Guest sgreenwood]

Hi John, Your work looks very promising.My only experience with BlackArt has been the testing done today, so I may be overlooking critical issues, but that said, I may be able to help with the NED BIL data conversion. I've been converting BIL to Microdem DEM format for some time now, so I took a little time to pull out that routine and add a VB interface. The only difference between hgt and bil data is the byte-order, so it was easy to modify the code to do the conversion. The problem with the hgt "format" is that it is just a square array of elevation values. The filename provides the positioning information.Neither of these is a problem for my program, but loading my output into BlackArt had mixed results. My first test was with a symmetrical array (1800x1800 - pure luck), and the rendering in BlackArt looked pretty much like the source BIL data did in Microdem. The second data set was not symmetrical (1800x3600), and the rendering in BlackArt was not good!So the number of rows/columns doesn't seem to be critical, just the symmetry.As for the problem with merging adjacent SRTM data files, my first reaction would be to merge them in Microdem and Save As 16-bit BSQ files. This data could then be converted to hgt format with just a bit more work than the above conversion process. (This requires rotating the matrix, as well as reversing the byte-order).If you (or any other patient tester) would like to try my Alpha :-) version, you can download it from:www.fs-traveler.com/cgi-bin/fstbil2hgt.zipThere is a brief readme file included. If you need more specific info, or have suggestions/problems/requests, post them here or email me at support@fs-traveler.com.Regards,Steve

[Guest jkvato]

Thank you, Steve, for jumping into this subject and making your converter available.I tried converting a 2deg x 2deg block of SRTM BIL data that encompasses Aconcagua (SW corner: S34W071, NE corner: S32W069) into an HGT file using your converter (resulting in a 2400 x 2400 file, 11.25 MB in size). I renamed the resulting HGT file S34W071.hgt. Blackart loaded it and displayed the entire file in its viewer, but although graphically speaking it seems to be loaded fine, it is still considering the data to be a 1deg x 1deg block (SW corner: S34W071, NE corner: S33W070). I also noticed that all missing data was set to zero elevation, and not considered to be missing data in Blackart. (Is this inherent in BIL data, or did FSTbil2hgt do this?)I decided to save the data to see what happens, and Blackart wrote a 25.327 MB file, which is the same size as a 1 arc-sec SRTM tile. I opened this new file in Blackart and found all my data squeezed up in the upper left corner, and the data went roughly 2/3 the way to the right and 2/3 the way to the bottom, with the rest of the data being zero elevation. This, too, was also considered to be a 1deg x 1deg block of data.These results lead me to believe that my initial (unstated) fear that Blackart is assuming that the HGT file it is opening is either:1. a standard 3 arc-sec data file occupying 1 square degree, or2. a standard 1 arc-sec data file occupying 1 square degree.And I think that Blackart shouldn't really assume anything else, because that's essentially the definition of SRTM HGT files.I could be mistaken, but I'm not seeing any way we can convert SRTM BIL data into HGT files unless the input BIL data fits exactly 1 square degree and lies on whole lat/long boundaries. We probably can convert 30m NED BIL files into 1 arc-sec HGT files only if that data is also exactly 1 sq. deg. lying on whole lat/long boundaries. And I can't see any way to convert 10m NED BIL files to HGT at all because of the smaller spacing.Now for the purpose of straight interpolation in Blackart, it doesn't really matter what the spacing is nor what the coordinates of the data are. I think that these things are immaterial to the mathematical manipulations that the interpolation algorithms do. Where these things do matter is when Blackart is used to merge SRTM30 or DTED0 files into the missing data.So, if we are interested only in using Blackart's interpolation routines without merging SRTM30 or DTED0 data, we can manipulate the input data to suit our needs and "fool" Blackart into thinking that it's standard data. Let me use an example:Aconcagua sits almost exactly on the W70 line, and the missing data there stradles two SRTM tiles. We could conceivably merge the S33W070.hgt and S33W071.hgt tiles, and subset one square degree covering the following area: SW corner: S33 W70.5, NE corner: S32 W69.5. We can then call this subset of data whatever we want, like N00E000. Blackart will think it's a 1deg x 1deg block of data with the SW corner of N0 E0, but that doesn't matter. We could do our interpolation routines then save the data to another file, like N01E01. We could then take this tile, split it in half, replace the right half of S33W071.hgt with the left half of N01E01.hgt, and replace the left half of S33W070 with the right half of N01E01.hgt. That's tedious, no doubt, but it would solve the issue of missing data stradling two SRTM tiles.We could also take a 1/3 deg x 1/3 deg block of 10m NED BIL data, convert it to an HGT file and run Blackart's interpolater on it, then convert it back to a BIL file. Again, tedious, but probably doable. Blackart will think it's working on a 1 arc-sec SRTM file, but that wouldn't matter.If we ever wanted to merge SRTM30 or DTED0 data into the missing data, then these workarounds would not work. And at least in the case of Aconcagua, a merge of SRTM30 data would probably be better than just straight interpolation.As far as airport terrain smoothing goes, merging SRTM30 data is not necessary, so we probably could use the above workarounds.I think it would be more practical for Blackart to be able to read BIL data of various spacing and dimensions. Now if the main concern is that a computer can't handle interpolating large blocks of data, then users will just have to make sure that the BIL files are small enough for their computers to handle. The downside is that we're asking someone else to do the programming work -- someone who's already got plenty to do.Anyway, I've rambled enough for today. *:-* What do you guys think?

[Guest jkvato]

OK, maybe I haven't rambled enough today. :)I'm ready to let you guys play with the beta version if you're so inclined. The link is:http://www.jktaylor.com/flightsim/downloads/smooth110.zipInside you'll find the program, a long readme file, and three sample scripts. The first script is just to help point out all the little nuances of the program. The second script illustrates the POLY method and is what produced the results of SVCS in the original post of this thread.The third script is an example of the RECT method which I'll describe now. This is a simpler approach to smoothing terrain around an airport. What you do is define a north-oriented rectangle that encompasses the airport you want to fix, and the utility will set all data points within that rectangle at the airport elevation, then slope away from the rectangle into the surrounding mesh. Here is what Las Vegas-McCarran looks like in map view after using this approach:http://www.jktaylor.com/flightsim/vegas.gifAnd then here are the before and after shots of KLAS in the sim:http://www.jktaylor.com/flightsim/vegasa.jpghttp://www.jktaylor.com/flightsim/vegasb.jpgThis method is advantageous for the following reasons:1. It requires less work to write the script2. You don't need to run the result through Blackart3. Therefore you can use it on any type of elevation data (as long as it's in Microdem format)It is also disadvantageous because it is less flexible -- it levels out a lot of terrain for odd-shaped airports and it produces somewhat unnatural pyramid-like terrain (though it looks worse in map view than it actually is in the sim). It only works well for airports surrounded by flat terrain.If you have any problems, please let me know. You can email me at the address included in the readme file, but I prefer that you post them in this forum so others with similar issues can participate in the discussion.

[Guest sgreenwood]

Well, that's a bit disappointing, but not unexpected. I checked the NED BIL data and both Microdem and a Hex editor confirm that the missing data is set to 0! So this data is not very useful to us.I've done some more cutting and pasting today and created a new utility that reads Microdem DEM files and converts them to hgt format. It allows you to define a one-degree subset for the output file.So you can implement the approach you described above by merging the two (or more) hgt files in Microdem, extracting the subset you suggest with my tool, processing it in Blackart, then using the Microdem I/O->Edit->Header tool to restore the correct coordinates. The file can then be merged back into the original dem. A little bit simpler than the splitting and replacing process you suggest, if you're comfortable with Microdem.It would be best if BlackArt handled all of this directly, But until then, the new utility is available at:www.fs-traveler.com/cgi-bin/fstmdem2hgt.zipThe source hgt files use -32768 for missing values, but Microdem converts these to 32767. You may be able to get around this by redefining "null" in BlackArt's Images menu. If this is a problem, I already provide a field where you can specify the value used for missing data in the DEM, but this is only used to count them. I can add another field for specifying a replacement value (e.g. back to -32768) if this would be useful.Steve

[Guest jkvato]

The tool worked great, Steve ... only it will be necessary to allow the user to specify a replacement missing data value. Blackart allows the user to change the null data value, but it also considers all numbers below that value to be null data as well: Null data = Elevations < -32000, where -32000 is the number the user can change.After using your utility, however, I ran it through my smoothing utility which I had programmed to convert all 32767 values to -32768 when saving HGT files, and this turned out to be a simple and efficient workaround.So thank you for putting your subsetter/converter together. I know I'll be using it.

[Guest sgreenwood]

I've just uploaded a new version (same link as above).I added the second field for the replacement missing value. (I anticipated the need after actually using BlackArt.) :)I've also cleaned up the interface a bit and improved the byte-swapping algorithm so it runs about 4 times faster.Steve

[Guest jkvato]

Sweet. Thank you!:)