Draft tutorial for editing default LWM and VTP landscape files

[Holger 0]

Hi all,Luis, thanks for the input! Having a tool like that is a great idea. Correctly implemented, it would save a huge amount of headaches.Adriano, thanks for the impressive "draft standards". I really don't have much to add to them other than a few points:"1. Default LWM/VTP files must be modified, I believe, ONLY IF THERE IS NO OTHER WAY OUT. (This may limit interventions to the selective suppression of default shoreline flattens, river gorges, and ?). Every time an exclude file or whatever else can do the same thing, please let the default files as they are."Strictly interpreted, this includes any LWM/VTP file because the visible features can be excluded without problem. Instead, what we are dealing with here are the residual flattens associated with water bodies, roads, railroads, and streams, which cannot be altered with excludes. To me, all four are "bad" but the degree to which they interfere with a project - and thus the motivation for removing the data from the default files - depends on the local topography more than on the LWM/VTP type. In other words, I would replace "no other way out" with a more general statement that says one should edit default files ONLY if the residual flattens create problems for the intent of the project. (I realize now that using HL9* shorelines in my tutorial is a poor example in that regard; I'll probably switch to roads or streams to avoid confusion). In No. 3, you list coastlines as "the only good reason"; again, I think that those three VTP line types with flattens/depressions should be included.No. 4: I agree that it would be nice to have a rigorous standard for documenting these changes but I doubt that this will happen (writing read-me's is typically the least liked part by designers). My guess is that a utility as suggested by Luis and yourself, which actually makes it easier for the designer too, is the option more likely to be successful. However, until such a tool exists (if ever) I agree that everyone should provide the minimum information about which files and data were altered.---Coming back to Rhumba's suggestion regarding setting entire LOD5 sections to mesh-clinging. To me the main reason we are creating add-ons is to enhance the subjective (i.e., apparent) and/or objective (i.e., measurable) realism of the FS world. This is a process that proceeds in steps but it should always move forward. I just don't find it useful to sacrifice realism in large parts of a 90,000 sq km LOD5 section - by allowing water to climb up and down slopes - just so that my project doesn't interfer with potential (!) other projects in the same LOD5 (Again, this VERY MUCH depends on the kind of landscape we're in; it won't matter much in Kansas). I'm currently in a situation were two other people are using the "surgical" approach in two LOD5 sections previously altered by me. I'm in contact with both of them and I'm sure we'll figure out a way to make our projects mutually compatible. To me, communication among designers is a very important aspect, one that is perhaps the easiest way to avoid chaos.Cheers, Holger

[Guest luissa]

>Luis, thanks for the input! Having a tool like that is a great>idea. Correctly implemented, it would save a huge amount of>headaches.Hi,I hope that some of the readers of this forum who implemented LWM/VTP disassemblers consider the challenge.While such a tool is not available, may be designers could divide the "default LOD5" in 1024 "default LOD8". The possibility for conflict would be much smaller.Regards, Luis

[Guest jimkeir]

Hmm? Wha? Sorry, sleeping :)Well, it shouldn't be too tricky to put together something that could merge or split LWM/VTP bgls as long as they were split along cell boundaries. Arbitrary boundaries would be entirely possible of course but would require a bit more work.It could take all cells from the 'input' file and substitute them in the 'output' file, if they already exist there. In either case, the first patch applied to any file should store a copy of the unmodified original in a central location. This would be used to uninstall any add-ons. Uninstallation would require the add-on BGL to still be available; this would allow subsequent differences to be spotted and left in place.Simply deleting the matching areas from the original BGL would seem simpler to do, but selective uninstalling would be more difficult since you couldn't compare each cell and see if it's been updated by something else.Of course, this means that users have to run a program with correct parameters to install and uninstall your add-on scenery. Given that a number of people have trouble copying files(!), this may be a problem in itself.Cheers,Jim Keir

[arno 1]

Interesting discussion this is becoming :). Just a few comments.I like the idea of some sort of tool to apply the changes to the default files, this would make it better accessible to more scenery designers. But compared to the AFCAD installer for Fs2002 i also see a problem. With the AFCAD you where always talking about one airport with a fixed name. That means that when installing a different AFCAD of the airport you would just overwrite all data. Having a simple backup file of the origional airport also allowed you to put that back easily.With the mesh elements there is not fixed area however. It is very well possible that different designers could even have some overlap in their areas. This would make uninstall options much more complex. Assume you ahve installed the changes made by designer A and then you install some work by designer B that overlaps part of designer A. Should the uninstall then revert to the status of designer A or to that of the default scenery? And what if you want to uninstall designer A, while keeping the work of designer B? It would be a rather complex process of storing all this data so you can have a flexible uninstall.Therefore I think I like the suggestion of Dick more where the only change to the default files is to remove the flattens information. The addon designer can then redefine the altitudes in his own files as he likes. I think we can still use a tool in this case that allows a scenery designer to let the user remove the flatten information from the default files in a very specific area (the area you are editing). This tool would for the uninstall only have to restore the default flattens.

[Guest luissa]

>Interesting discussion this is becoming :). Just a few>comments.>Yes indeed! After reading the posts by Jim and Arno: I think that the replacement can either be "by Area" or "by Cell". I was thinking in a "by Area" approach in which case the author should modify the complete Area (not a part). Jim refers to a "by Cell" which may be mandatory from a point of view of the coding (?) but can bring problems if authors only change some Areas in the Cell. The point by Arno is an important one! What we need is the removal of the flattens. For the rest we can use the priority mechanism of the sim. So the tool would remove the flattens from the worked areas!Regards, Luis

[Holger 0]

Hi all,"What we need is the removal of the flattens. For the rest we can use the priority mechanism of the sim. So the tool would remove the flattens from the worked areas!"(jumps up and down) But this approach works ONLY for LWM polys, not the flattens/depressions associated with vectors!!! Am I really the only one concerned about vector flattens? I could show you a couple of really ugly screenshots of residual vector flattens but I have reached my screenshot quota for this thread. :-)I think that if a tool existed that restricted its influence to full LOD8 Cells only, then the issue of overlap between adjacent projects would be minimized (it would also make it easier to remove VTP2 vectors, which are organized in LOD8, not LOD13). In case a designer's project area includes only part of a LOD8 Cell he/she can develop a replacement LOD8 Cell that consists partially of his/her work and partialy default Areas; that's what I do all the time already.Basically, the designer would prepare his set of replacement LOD8 Cells, submit them to the utility, and the utility would remove the default LOD8 Cells and replace them with the new ones. It shouldn't be too difficult to address Arno's concern by storing a new "last-edited" version (in addition to the default-only version) each time things are changed so that the uninstall can proceed in reverse succession. Admittedly, this wouldn't catch a case were B is installed after A but the user wants to uninstall A but not B. Maybe it's just easier to "lock" an altered LOD8 Cell after it has been altered once. Or, the utility could alert the user that this LOD8 Cell had already been altered once and give the choice to uninstall the previous edit (for that Cell only) and install the new one instead (for polys this could be implemented on a LOD13 basis, if desired). Since we're talking replacing default flattens and default vectors/polys with more accurate versions there shouldn't be a need for multiple changes anyway. Also, doing this would "force" the designers to coordinate their efforts and restrict themselves more or less to LOD8 Cell boundaries. As for the implementation by users, I think a batch executable that initiates the process should do the trick (one for install, one for uninstall). Cheers, Holger

[arno 1]

Hi Holger,>(jumps up and down) But this approach works ONLY for LWM>polys, not the flattens/depressions associated with vectors!!!>Am I really the only one concerned about vector flattens? I>could show you a couple of really ugly screenshots of residual>vector flattens but I have reached my screenshot quota for>this thread. :-)Just to know I understand you correct. If the tool would also allow you to remove default VTP lines the flattens belonging to them are also gone not?I agree with you that these elements should also be removed, but the principles stay the same I would say.

[Holger 0]

Hi Arno,yes, removing the VTP2 lines (and replacing them with more accurately placed ones) will remove the flattens/depressions. One can also do this via changes to the terrain.cfg but those changes would be global and, IMO, undesirable.Cheers, Holger

[edgarone 0]

Hello Holger,I think you mean "... removing the VTP2 lines (and replacing them with more accurately placed ones) will NOT remove the flattens/depressions.", right?A depressing thought indeed!Cheers, Edgar

[Holger 0]

Hi Edgar,this is interesting; it seems that people are indeed unaware of this option:Removing the default VTP2 lines WILL remove the flattens and depressions. The flattens are associated with the vectors via the terrain.cfg (with the "offset= [flat/0]" parameter), NOT stored in any separate file. If you simply exclude the VTP2's their textures will disappear but because the default RD/RR/ST bgls are still loaded the flattens/depressions are drawn by the engine. Removing those bgls (or parts thereof) will remove the flattens.Cheers, Holger

[Guest jimkeir]

Hi all.I had a quick play last night and have a basic program which will patch and unpatch LWM files. VTP is easily do-able too, Holger, I just started with LWMs!One thing I'd not thought about before is the issue of different versions. Say I produce a LWM2 file which is scuppered by default (i.e. LWM3) flattens? This means that patching new data into existing files isn't the best option.What I've gone for is to *remove* data from the default file for which a cell exists in the new file. The process is like this:- If no backup exists, create one.- For each cell in the new file, remove any corresponding cells in the original file.- Store a count of the number of times each cell has been modified alongside the backup.To uninstall, it basically goes backwards; for each cell in the new file, subtract one from the modified count and when it reaches zero put the original data back in, from the backup.The reason I'm working with cells instead of areas is that it's much easier, you can manipulate at the index level instead of parsing the entire data stream, having to support different versions etc. It's certainly possible, at least for LWMs which are divided anyway but I'm very busy with other stuff right now and don't really want to spend too much time on this.So, in summary:"new" file = the one a designer provides with replacement data;"original" file is the one found in the FS hierarchy, patched or not;"backup" is the unpatched default file provided with FS.- A designer needs to provide data for an entire cell. This is easy enough, but means that cell-level collisions are still possible.- You need to specify the exact files to patch. This is deliberate - a single badly-formed new BGL could trash a user's entire installation otherwise.- A backup folder is created under FS, containing backup files and a modification record.- The original BGL has cells removed (actually, set to zero length) which exist in the new file. Of course, you can patch different originals with the same source if need be.- Different patches covering the same cells are no problem to patch and unpatch, but the new files will of course interfere with each other if they use flattens :) However, it will warn when a cell is already patched. I'll get it to write a logfile so it should be possible to tell which new files may collide.- If the modification record gets lost, you can still un-patch but backup data may be copied back while other patches still exist.I'll try and add the VTP patching over the weekend, it's not difficult. BTW Holger, jumping up and down only affects flattening on *real* terrain :)The program also needs lots of error-checking etc. Comments welcome, of course. I'll release the source when it's done so if somebody wants to implement area-level patching for LWM they can go ahead. VTPs can cover an entire cell anyway, of course, so you'd need to get into scanning and splitting each line or poly...This kind of patching has the potential to unleash bigger problems than it solves, of course! Once the general user population starts hacking their base installations anything could happen. At least when designers hack the base files it's under controlled circumstances and the results can be tested, even if it has the drawback of occasional scenery collisions. Cheers,Jim

[edgarone 0]

Sorry Holger,I misread your message! I was referring to "excluding" but not "removing" VTP2 lines.That's quite plausible, if you remove the original data from the default files, it will not leave any depression.All over all, I think the current FS9 scenery engine lost some flexibility concerning the installation of additional sceneries. Which is a large step back.However, I am feeling very uneasy with the idea of changing other peoples files or some global configurations like terrain.cfg. I am afraid we will open a whole can of worms with the compatibility between sceneries of different people and the order of scenery installation/deinstallation etc.And just think of manually displaying/hiding sceneries or changing the scenery level in the manager...Cheers,Edgar

[rhumbaflappy 113]

Hi Jim.I think your solution is very close to where this type of automation must go. I think it should be a requirement to replace the entire cell or cells ( whether VTP2 or LWM )... it's the cleanest and easiest way to handle the default code.How about this idea:Supress the cell data as required, but the new designer data exists as a separate BGL? We could even use a separate, standardized scenery folder for the altered defaults. Keep them in "C:Program FilesMicrosoft GamesFlight Simulator 9Altered_defaults"... and then the new bgls could be stored in any addon folder desired.This makes the restoration of the original much cleaner, and we could then add whatever we like to the cell(s) and the original code isn't "contaminated" by our new code. In other words, the only alteration we attempt on the defaults ( or their children ) would be to exclude cells. New data would require a new bgl with a different name.Even a popular LOD5 default would only get a few addon BGLs, so the number of variations of the default LOD5 would be kept orderly and separate from everything else... and those variations would be the default code, with the cell indexes altered to exclude the unwanted cells, as you indicate. The new data bgls would also be only a few... their names containing some indication of the cell they are adding to.The newly designed bgls should exist in their own project folder, as usual, so their removal would also be made easier.I agree, we should seriously consider the alteration of entire LOD8 cells in our designs, and should also consider replacing the entire LOD5 data, if we have the ability and the time.I'm hoping Slartibartfast will make design of LOD5 LWM bgls easier. Although it's accuracy depends on the mesh used, it is still much, much better than the defaults.... can be used for streams as well? It's a good idea for most of the world that doesn't have freely available, accurate data.Dick

[LuisFelizTirado 33]

Hello Jim,The utility you are putting together sounds very good and certainly provides a solution for us.As you mentioned, it will probably lead to numerous difficulties for users of scenery. It is important to remember that aside from us, nobody else understands the least bit about the arcane details of scenery files. Users simply install the files, often incorrectly at that, and then forget what they have done. Usually, if an installation is complex, the users are incapable of uninstalling correctly.The modification of such essential files as the default hyp bgls will seriously compound this problem. We are heading towards a very risky area where much confusion will probably arise.At the moment, when designing scenery, we have to choose between faithfully reproducing the area in a realistic manner, or accepting the limitations of the default scenery and creating our project using the default as a basis. If we opt for total accuracy, we butt up against the difficulties that are discussed in this thread. This is because our scenery design has gotten ahead of the capabilities of the FS scenery engine.I can only urge all designers to seriously consider the difficulties that lie ahead if we all start to provide modified default bgls, or even more, if we provide a means for each user to modify the default at will.If we are lucky, the Flight Simulator Development Team has taken this problem into consideration and included it in the list of modifications for the next version. If so, then just a little patience on our part (as usual) and Flight Simulator 10 will (hopefully) allow us to either exclude water and VTP2 flattens or override them by the usual method of placing a new bgl in a higher layer in the scenery library.Best regards.Luis

[rhumbaflappy 113]

Hi Luis.I agree the best solution for FS9 is to replace the entire default LOD5 area with better data. Dick