1s model from 3s sources

[Guest vlada stoje]

Hi all,the oversampling bringing denser result from given source was already often discussed here. The standard steps are well known, at least as resize in the graphic tools. The equivalent functions as resize, sharpen and other are available also for elevation data in professional terrain software and some were introduced here in free programs, as Microdem, BlackArt and Slartibartfast. I want offer here for testing my very small utility Vera specialized for only one case.For some areas there the better elevation data as SRTM 3s is already available, but for big piece of the Earth there the SRTM 3s is still the best source. The original 1s datasets were not released except USA. Oversampling is the basic way, how to get artificial 1s (30 m) grid from the 3s (90 m) grid. Maybe we could use also additional information here. In the SRTM JPL-ftp 3s at ftp://e0srp01u.ecs.nasa.gov/srtm/ there one point is the average of corresponding nine points in the original 1s source and in the USGS-seamless SRTM 3s at http://seamless.usgs.gov/ there one point is identical with the middle point from these original nine 1s points (explanation is in the file SRTM_Topo.pdf from the documentation at ftp://e0srp01u.ecs.nasa.gov/srtm/version2/Documentation). I was curious if this difference can be used to improve little bit the model and tried create utility for it. The testing program Vera is a line command working with two parameters, for examplevera N40W076.hgt 26563845.bilwhere the first parameter is the name of the 1deg square from the JPL-ftp 3s dataset, the second parameter is the name of the file *.bil containing the identical square from the seamless server (it also must have 1201 x 1201 points, and in described example, the upper left corner must have the coordinates N41 and W076). The resultant file, in this example N40W076b.hgt, is 1s grid with 3601x3601 points, created from the two input files by standard oversampling steps and also with the use of the difference between ftp and seamless data. When the difference in given point is corresponding with the trend of the preliminary resized model then it is used to improve this place, else both parameters are minimized by averaging. The comparison between source ftp 3s data and the resultant artificial 1s model is visible very well for example in the GlobalMapper http://www.globalmapper.com/ : http://www.volny.cz/stoje/vera/vera1.gifFor USA there are also original SRTM 1s data available, so here is also comparison between real 1s and 3s data, compare it with the first picture:http://www.volny.cz/stoje/vera/vera2.gifAs expected, the main contribution to the combined effect is from the resize by bicubic resample and only small part is from the difference of the two source files. The artificial model is very different from the 1s original in the shapes of the smallest details, but looks quite similar in the overall impression as well as the tops of hills and bottoms of valleys have mostly more accurate elevations. In the program Vera here are several processing steps in balanced combination. If you wish to try it you can download it from http://www.volny.cz/stoje/vera/vera1.zip. Please let me know about your experiences, I tested it only for several 1deg squares. Thanks for your patience to reading this all and eventually for your comments!CheersVlada Stoje PS:1. don't use the JPL ftp data version 2 above N50 and under S50, it is little bit blurred due to resampling to 6x3s and then back to 3x3s, use the ftp version 1 there2. Program Vera doesn't handle the voids in SRTM data, use other tools to fill the voids

[LuisFelizTirado 33]

A very, very clever method, Vlada. And the results look great.Just wondering how this compares to just using a bicubic filter to sharpen - is there a large visible difference?I shall give your program a try and report back here.Best regards.Luis

[Guest vlada stoje]

Hi Luis,in my confusing explanations I forgot one useful help, both hgt and bil files must have the same size 2 884 802 B (1201x1201x2). I would estimate the contributions: 30% sharpen (hi-pass), 60% resize (bicubic), 10% difference ftp-seamless. The dif component is actually very small, but without it the result wouldn

[Guest vlada stoje]

Hi all,very sorry but in the first version in vera1.zip from april 21 there was stupid mistake, after zooming bad artifacts in intersections of rows and columns 1,4,7,10,13,... In the new version Vera11 from april 25 athttp://www.volny.cz/stoje/vera/vera11.zipthis is fixed (but how many bugs are still there)Thanks for your patience,best regardsVlada Stoje

[Guest vlada stoje]

Hi all,sorry there was also mistake for the negative elevations (huuh...), correction is athttp://www.volny.cz/stoje/vera/vera12.zipCheersVlada Stoje

[Guest vlada stoje]

Hi all,sorry no development in this small program, just another mistake found. When the elevations in chosen area are inside the span 256m, it looks that the seamless server produces in the case of the bil format just the 8-bit values. The fixed Vera1.3 at http://www.volny.cz/stoje/vera/vera13.zip handles it somewhat, but I tested it just for Bermudas, for which is the minus threshold set to 220 (for example the value 221 means 221-256 = -35m). The 1x1deg tile in bil format from the seamless server can be 1442401B for 8-bit or 2884802B for 16-bit data.CheersVlada Stoje

[Guest renatus]

Looks really great, Vlada.However, I have a question, and I think you or the subscribers of this forum can help me easily. Since your software doesn't cope with the holes in HGT mesh files and BIL ones, It's necessary to filling the holes prior to apply vera.My question is about the BIL data:How can I fill the holes present in BIL files, and then save them in the original BIL format?Thanks in advance!:-smile12

[Guest vlada stoje]

Hi Renato,thanks for trying this experiment! I was very busy last months, spending also evenings for my profession, so sorry I have no new experiences or ideas for it.About the holes in bil files, it has two aspects. First, the format - the basic matrix file *.bil is similar as hgt, just with swapped bytes (hgt is most significant byte first MSB, while the 16bit bil file is mostly the least significant byte first LSB). I remember for example two free programs, which can open bil file, fill the holes and save the file. One is MicroDem by prof. Peter Guth http://www.usna.edu/Users/oceano/pguth/website/microdem.htm and second is FSDEM by Emerson de Oliveira, here in Avsim library http://library.avsim.net/download.php?DLID=51976There are perhaps also other tools, but I cant check it (not installed in PC now

[Guest renatus]

Vlada,Very good... I have the solution 'at home', since I used FSDEM, by my contryman Emerson de Oliveira. Filled the holes in the raster, and then saved the raster back as LSB... the same with hgt data, and after the Vera process, I've imported the the MSB hgt 'b' data generated using 'Load Raster..' command, instead of 'load single HGT' directly. That's because the coordinates' information gone after the process, I think I forgot something. But anyway, the final result was very good, 2.03MB BGL file generated. SRTM-3 data were 808KB.Thanks a lot for your explanation.Best regards,Renato.

[Guest vlada stoje]

Hi Renato,FSDEM is brilliant program, it seems that it can be used even outside FS for DEM editing. The bgl files are compressed, so their size depends both on the used FS resolution (LOD) and also on the amount of present small details (which is increased by vera). I hope the result could help little bit for example to the panoramic view around an airfield, with sharpened small local hills, which are often too flat after standard resample of SRTM data.Keeping my fingers crossed for your project!Best regardsVlada

[Guest renatus]

No doubt it's a very good tool. My mistake during the creation is not renaming the S??W???b.hgt files to S??W???.hgt, i.e. removing the 'b' letter, in order to maintain the nomenclature standards required for auto-identification of the coordinates. I needed that when I was creating a high-res mesh with to neighbor tiles. It's all ok, now.. I'm going to my next step: use Sbuilder.Best regards,Renato

[Guest vlada stoje]

Hi Renato,thanks for the useful finding! Creating a high-res mesh: note that the described sharpening is extremely sensitive, small mistake in the source creates unacceptable artifacts in the output, for example in the JPL ftp version 2 the blur in the bounds above N50 and under S50 (the picture is from Argentina): http://www.volny.cz/stoje/scen/srtm_vera_S50_arg.gifI am also in the phase to try more SBuilder (or looking forward to do it)best regardsVlada

[Clutch Cargo 384]

Just curious... was there any future development ade on this applet? Did anyone create 38m mesh from a 76m source? Wuld luv to see any results.

[Guest vlada stoje]

Hi all,the utility Vera is now little bit modified to allow just the initial steps (sharpening and resampling from 3s to 1s), without combining the two different versions of SRTM 3s. The use can be very simple with one parameter, for example by the command linevera n50e014.hgtwhere the hgt files are from the address http://dds.cr.usgs.gov/srtm/version2_1/SRTM3The official hgt files were corrected in the new version v2.1 (now without the problems with light blur beyond latitudes N50 and S50), so from this point of view the situation is simplified. New resized 1s file will be named n50e014b.hgt in this example.The result will be very similar as by use of some picture-shop program allowing open 16-bit grayscale raw data and applying steps "sharpen" and "resize 300%". I am not sure about the ways and results in these programs. We need the exact space relation between lattices 3s and 1s (from 1201x1201 to 3601x3601 in the scale 3.0000000). With the Vera we can be sure that this is correct, but sorry the utility works slowly.The resampled 1s files are the same size and resolution as the new ASTER GDEM, so they are ready for direct combining and analyzing of both sources. The utility is at http://www.volny.cz/stoje/vera/vera14.zip .I hope it could help,CheersVlada