Sign in to follow this  
Guest Alex_Law

blackart doesn't work for me...

Recommended Posts

Hi all,1. It doesn't load standard SRTM3 files. It claims that the file is not SRTM. Other programs do display and manipulate it correctly (including SRTM2BGL).2. I can use only the batch processing, but then I cannot "view" or "save" the "invisible" result. I can hack my way around this by renaming the .hgx (temporary?) file it generates - actually a standard SRTM file - and view it with another viewer...What am I doing wrong?Thanks,Alex

Share this post


Link to post
Share on other sites
Help AVSIM continue to serve you!
Please donate today!

Hi Alex,groping in the dark a bit here...1. Are you using the latest version, 3.75?2. Have you looked at my Blackart thread here in the forum with regards to some of the steps regarding using it?3. Can you list some of the tile names you're working with? I can download it/tem and give it a go, though I don't assume that would be the problems as I and others have used it without those issues.Cheers, Holger

Share this post


Link to post
Share on other sites

;-)Thanks for the attention, Holger.1. I am using 3.752. Yep. I've read all your posts (maybe that's the problem - too much info for the unexperienced ?)3. Here are two example tiles: N32E034.hgt and N32E035.hgt.I did a fast SRTM2BGL and saw that there is a lot of "garbage" in the sea and lakes... Tried the flatten tool but they usualy end up creating sharp rifts at the borders - not very realistic if you ask me. Anyway, I am not sure what tool to use to manually remove these bumps without creating the rifts (something like the smooth brush in PSP).In addition I am not sure how to flatten both the sea which is theoretically at 0m and lakes above or below sea level. I don't know how or if blackart can do that...But maybe I'm looking for the ultimate tool that does everything instead of looking for a toolbox. If this is the case, I couldn't find a list saying: tool A does this and tool B does that, etc... maybe that's impossible too... Am I expecting too much? Is the life of mesh "designers" that hard ;-D ?Thanks,Al

Share this post


Link to post
Share on other sites

Hi alex,you read all of my posts? Heck, I don't even do that ;-)Just tried the two tiles you mention in Blackart and they load without any problems!?! Perhaps re-install or tell us the exact steps you're using in trying to load a file (you ARE switching to "SRTM height files" when loading, I presume).What's your mesh "destination": FS or CFS? In FS you don't have to worry about non-flat water because the default LWM water files will flatten them in the sim. Not always in the correct place but that's another story (many threads about that here and in the scenery design forum). Ummm, I don't think the life of mesh designers is that hard (compared to, say, modeling an airplane in GMax) but it does take a bit of know-how and experience to provide high-quality work, just like for any other type of add-on. Matthew's great utility has the unintended consequence that the mesh design process appears to be a 1-2-3 thing but that's definitely not the case, as some recent uploaded examples show. Anyway, great to hear that you're into this and trying to use Blackart instead of the simple linear interpolators of the other utilities!Cheers, Holger

Share this post


Link to post
Share on other sites

LOL,You solved me one (big) problem. I was NOT switching to SRTM when loading... There simply was no hint that that can be done so I was using only the batch processing... Well, I tried again and really looked at the load window and discovered the "elevator" with all the formats. Now I see the light! ;-).The mesh is for FS so thanks for putting my fears to rest with the LWM hint.I guess that now it's all down to manually fill those big lakes (represented by voids) and then run Black Art again.So, is my following procedure OK?1. Load each tile and manually fix the huge water surfaces which happen to be covered by voids (maybe using PSP instead of blackart?). Save the modified files as SRTM.2. Batch process all the corrected files with the Laplacian set to 1000 and LSQR 10 for example.3. SRTM2BGL with NO interpolation.There is one thing that bothers me though, What should I do in order yo ignore the -32000 issue? I DON'T want to clip anything to 0 or to a fixed value. I want voids to be interpolated. What do't I understand?Hehehe - and you beat me with the gMax "mine is bigger than yours". Yes, I agree. Creating a plane with gMax is difficult - but then again - I am sure that there might come an aeroplane designer who will disagree .Have fun,Al

Share this post


Link to post
Share on other sites

Hi Alex,Blackart's interpolation routine will fully interpolate all voids (unlike SRTM2BGL, which often leaves patches in large voids) and thus takes care of the -32000 issue too. Since the water areas will be flattened in FS all you need to do is run the batch process once with your choice of Lapl and LSQR; in other words, forget about your first step and start with 2.Big voids are only an issue when they occur above land. In that case you should attempt to fill them with other elevation data (John describes the process with DTED0 data on his website, and there are other options as well). It didn't look like the tile you were interested in had any of those voids so you should be fine with steps 2 and 3.Good luck! Cheers,Holger

Share this post


Link to post
Share on other sites

Hi Holger,1. Yes, but what can I do when a lake is *under* the sea level. In that case the clipping "thing" simply raises it to 0m. What I've done is "paint" the lake (covered by void) it it's real altitude, and the run Black art. Is there a better - and simpler - solution to that? Please remember that I have boath sea (0m alt) and lake (negative alt) on the same tile ;-).2. What can I do about the occasion "Devil's tower-like" spikes that appear here and there. I tried to pinpoint them but it's next to imossible due to their very similar coloring to the "legal" alt points. I tried low pass filter and selective smooth but not with much success. Overdoing that would defeat the very purpose of SRTM - much better resolution.Any ideas?Best regards,Al

Share this post


Link to post
Share on other sites

Hi Alex,I took a look at one of the tiles myself and finally understand what you're getting at. Sorry, but I never dealt with below-sealevel data myself. It seems like there might be a bug in Blackart.Here are 3DEM images of the raw data (voids in blue), an interpolation with Blackart (10/1000), and a comparison interpolation with MicroDEM (search radius 200, bi-directional). It seems that Blackart attempts to interpolate all missing data to sea level if the surrounding terrain is less than 0 m, which is why even those small voids end up as spikes. Unless we both are doing something wrong here I suggest you use MicroDEM instead. Actually, the better idea is to send an email to John Childs (perhaps with my images or a link to this thread) and ask him about the issue.Cheers, Holgerhttp://forums.avsim.net/user_files/72224.jpghttp://forums.avsim.net/user_files/72225.jpghttp://forums.avsim.net/user_files/72226.jpg

Share this post


Link to post
Share on other sites

;-))Thanks for the research and pics Holger. MicroDem pics look DEFINITELY better - I had a hunch that Blackart cannot cope with both sea-level and below-seal level surfaces at the same time.I tried to use Microdem but I didn't find a way to export to SRTM or even BMP so I use it mainly as a preview tool ;-(.Meanwhile I did some manual paint job and closed the "big below sea level void", and some other small artifacts. I also cleaned the sea while at it - no big deal . Now it works perfectly. And I noticed that the many spikes that I saw (beside the Blackart-generated ones) were due to an over-zealous landclass developer that messed a bit with the mesh. I cancelled the landclass and voila! 90m SRTM mesh in all it's glory!Now I'll try to convince the landclass developer to cancel his mesh "corrections" and follow the more accurate SRTM mesh ;-D. This might turn out to be more difficult than mesh editing .Phew! This wasn't as hard as I was afraid of - you were defeinitely right! gMax beats this anyday!Best regards,Al

Share this post


Link to post
Share on other sites

Hi.I downloaded those files and had a play myself - here's my attempt. Interesting situation, with sub-sealevel lakes and normal sealevel - um - sea on the same tile.If you want the RAW file let me know and I'll email it but it looks like you've already sorted the problem.Cheers,Jim Keirhttp://forums.avsim.net/user_files/72301.jpg

Share this post


Link to post
Share on other sites

;-)Yes, I sorted out the problem though the method is not to my liking. I had to manually fill-in the "voided lake".But the problem is still n the table:How to interpolate sub-sea level voids when you also have sea on the same tile?Black-art currently does not do it, but i am sure that there are other tools which do...Anyway, my SRTM mesh is ready and extremely good, but I will release later, when I manage to convince the landclass developer to fix his mech mods... Hopefully very soon...Best,Al

Share this post


Link to post
Share on other sites

Hello Al,I am not 100% sure, but I think that my set of DEM tools could interpolate SRTM voids at sub sea level. BTW, I am revising these tools to include in my SBuilder programme.My interpolation algorthim has nothing of special - voids values are replaced by the mean of the neigbhour values. When I have a set of continuous voids, I start by replacing the void(s) which has(have) a maximum number of valid neigbhours, which is 7. When it happens the void value becomes valid value. Then I pick another one but I scan the picture from different directions. The aim is that the void region becomes "attacked" from all sides and not, simply, for example, from left to right and top to down.What triggered my post is the fact that I consider a value to be a void if it is equal to -32768 (the maximum absolute negative number that we can represent with 4 bytes signed integers). Therefore if these values are surrounded by negative values they will get interpolated as any others. But I did not check this and I do not remember the details. I will check though ...Regards, Luishttp://www.ptsim.com/downloads/DEM2BSQ.ZIP

Share this post


Link to post
Share on other sites

Thanks for the heads-up Luis,I asked myself why in the name of G.. would someone simply decide that void=0 - or any other value? The method you're using is the only one I can think of - except interpolation based on other sources - GTOPO or other topographic data.BTW, I am new to these issues - does you tool directly support hgt files load and save?Best regards,Al

Share this post


Link to post
Share on other sites

Hi,The tool converts HGT to BSQ. So the answer is yes. The BSQ is the format that MS uses but I add some positioning data on the bottom in order for my tool to handle the files better. This added information is discarded by the MS resampler. As soon as you convert the HGT to a BSQ, you can do many things with the converted to BSQ. You can filter it, interpolate, increase the sampling rate to 1 sec (in order to make possible a merge with DTED2 data), merge 2 BSQs, crop a BSQ, convert the BSQ to a bitmap, convert a bitmap to BSQ. The problem is that the tool is not very well documented.Regards, Luis

Share this post


Link to post
Share on other sites

Hmmmm - this sounds very interesting. I'll give it a try and let you know.Documentation - hehehe - developers NEVER like writing documentation... but if people ask "how I do this and that" you do not have the luxury of saying "RTFM" .But you need some documentation in order to make your tool accessible. More accessible - more users. More users, more feedback. More feedback - more headacke . Forget I said it!Best regards,Al

Share this post


Link to post
Share on other sites

Hi all,I've sent an email to John about the issue and will let you know his response; he's been very responsive to suggestions and questions and I'm sure he'll fix this "bump" too ;-)Please keep in mind that simple linear interpolation of larger voids will yield poor results when applied to mountainous areas (see example below). The ability to compute more realistic (looking) fill areas is what makes Blackart unique and VERY useful for mesh designers. And, no, I don't own any shares of Blackart Corp. ;-)Cheers' Holger http://forums.avsim.net/user_files/72365.jpg

Share this post


Link to post
Share on other sites

;-DThanks Holger,I sent John an email stating my SRTM-loading problem before joining this forum, but he never came back. I am happy that he's been more responsive to you. BTW, there is another very annoyng issue that you might be interested to pass - the BMP generated by blackart is VERY uncomfortable for processing by a graphic application due to the fact that it translates the pixels index (altitude) directly into the pallette. This creates the artificial "seams" that appear when an index moves from - for example RGB - 00FH to 010H. Everything seems to be mapped to Blue, but when "falls" from full intensity to 0 creates a dark pixel and the "count" goes on. It would have been simpler to have a fully coloured altitude mapping which can lend itself to some "fine-tuning". Oh and the Grey scale does not work (for me?). And last - some documentation regarding the matrix/filter manipulations - what they do and why. That's all for now ;-)I hope I made sense and that my suggestions hasn't overlooked some other critical points.BTW, the linear interpolation seems to be excellent for generating Escher-like scenery. I really like the pic you sent. Seems to be right out of the Lord of The Rings .Best Regards,Al

Share this post


Link to post
Share on other sites

Hi all,just received a response from John about the below-sealevel issue and I'm supposed to pass it on:"BLACKART in general cannot interpolate negative elevations if LSQR iterations are specified. The problem is apparently caused by the not completely adequate way that I formulated the matrix multiplication algorithms that the LSQR solver needs to compute the interpolation. I attempted to pinpoint the problem recently but unfortunately was unable to find it in the short time I had to look at it.. It will probably require some major grief to tear into this complicated solver to find the bug."Um, yes, just what I thought... ;-)John goes on to say that he has other commitments in the next few months and that he doesn't anticipate fixing this particular bug before sometime this summer."On the plus side, I am almost finished with the SRTM30 interpolation/patching utility and hope to post version 3.76 next week with this feature completed."Sorry, Al, I had sent me email before your last post. But I'll pass along your questions with the next one.Cheers, Holger

Share this post


Link to post
Share on other sites

I don't know if this idea would work, but could he find the lowest elevation in the file, raise all the data points in the file by that amount, do the interpolation, then subtract all the data points by that amount?So if your file has its lowest elevation at 140 feet, he could add 140 to all the data points, interpolate, then subtract 140 back off all the data points.Alternatively, is there a way that we could add the offset to the data file, run Blackart on it, then subtract the offset back off ourselves?It's just a thought ... that's a nifty trick I've used in several computer engineering applications, and I thought maybe it could work here, too.

Share this post


Link to post
Share on other sites

>So if your file has its lowest elevation at 140 feet, he could>add 140 to all the data points, interpolate, then subtract 140>back off all the data points.What I meant to say, of course, is assume that your file has its lowest elevation at -140 ...

Share this post


Link to post
Share on other sites

Hi John,clever idea! If MicroDEM was able to write out .hgt format we could follow this procedure outside of Blackart (it's possible to do global add/subtract in MicroDEM). In any case, I forwarded your idea to John; let's see what he thinks about it.Cheers, HolgerP.S. Did you ever complete your airport-to-terrain mesh editing utility?

Share this post


Link to post
Share on other sites

> Did you ever complete your airport-to-terrain mesh editing utility?I haven't touched it since I sent you an alpha copy.I'm just starting to inch my way into the world of Blackart, but it looks like it can't read any format that I'm working with. I was hoping that Blackart could interpolate the void polygon around the airport that my utility makes, but it seems that I'd have to export it in a different format. :(So for now, the utility is only useful for north-oriented rectangles.

Share this post


Link to post
Share on other sites

>Sorry, Al, I had sent me email before your last post. But I'll>pass along your questions with the next one.>>Cheers, Holger Hi Holger,Thanks for the answers - not very encouraging but still - there is hope. Blackart is by far the most accurate SRTM manipulation - apart from the below sea level issue. I was able to circumvent it but this is due to the relatively easy topology. I wonder about the mesh accuracy created by people using the "fire and forget" method for areas which are not personally known by the developers. I've came to respect the SRTM data - it is very accurate and yet treacherous for people who do not closely know the targeted topology.Blindly running apps like Terragen, Blackart etc. without close and smart inspection of the results comparing against real topographic maps will create a lot of junk meshes, and that will make people lose their confidence in the developers and the SRTM data.Some food for thought.Best,Al

Share this post


Link to post
Share on other sites

Hi all,Al, I much agree with your concerns about the lack of (or desire for) quality testing! In many cases the mesh maker doesn't even have to know the area to tell - when checking in LWMViewer or TMFViewer - whether the output is usable or not. For example, this is La Palma (Canary Islands) as depicted in a recently uploaded freeware mesh. The open caldera, the defining feature of the island, is completely missing in the raw SRTM data and a simple interpolation algorithm like SRTM2BGL or SRTMFill makes it look like this: http://forums.avsim.net/user_files/73663.jpghttp://forums.avsim.net/user_files/73664.jpgAgain, this is easy to spot and can be avoided by using better tools (even MicroDEM did a much better job in this case).Cheers, Holger

Share this post


Link to post
Share on other sites

Howdy,more feedback from John Childs:Re the altitude mapping:"I understand the problem with the direct mapping. I plan to improve this soon. As far as the gray scale issue - you must remember that BLACKART is a schizophrenic application that started out as an interpolation algorithm, then became a contour line extraction/interpolation application which then became a contour line extraction/interpolation DEM interpolation application. The grayscale feature was added to help in the contour line extraction process and it, like many other menu selections was not intended to be relevant to the DEM patching process and will not work in this case. I plan to rework the menus to make it clearer which utilities apply to which process."Re the matrix/filter manipulations:"I will add some information regarding the Saunders-Paige solver to the Help file when I rework that code. In the meantime you can search those words and discover the source line at the Dr. Saunders Stanford website. He provides a terse explanation of the main components of the computation."John also agrees that John Taylor's idea - raise, interpolate, lower - would probably be a suitable work-around to the issue of below-sealevel data.Last but not least, version 3.76 has just been released - http://www.terrainmap.com/ - and I'd like to remind all users that there are at least three places to look for new/additional information about Blackart: (1) the in-built help text, (2) the Blackart sections on John's website, and (3) the FAQ section - http://www.terrainmap.com/ - where John summarizes specific questions and issues.Cheers, Holger

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this