Tip ... Transparency ... Fences ... Drawing Order

[Guest luissa]

Hi,Consider a transparent fence which encircles a number of objects in an airfield. For clarity, consider only one, X, of these encircled objects. Consider that the viewer, Y, is outside the fence and is looking towards the object as represented:X--------o---------YWe want X to be seen through the fence. Both X and the fence are PerspectiveCalled. The symbol o at the middle of the fence is its RefPoint. As X is nearer the viewer than the center of the fence, the fence is processed before X. That is: the display-buffer is filled with the fence and the z-buffer updated with the fence distances. Afterwards X is processed. The z-buffer is tested first and an indication that an nearer object is already in the display-buffer is passed to the FS engine. So X is skipped and we will not see it.One solution is to divide the fence in several small sections, and each of them called with its own PerspectiveCall.Another would be to force the fence to be processed after everything else. This can be achieved with the insertion of a "cheating RefPoint" as in the following code:...PerspectiveCall( :fence )Jump( :next ):fenceRefPoint( ind :return 1 1BE8 )RefPoint( normal refpoint for the fence here )...Variable 1BE8 holds the longitude, latitude and altitude of the viewer and FS thinks that the fence is at zero distance from the viewer.Hope this is useful. Regards, Luis

[arno 1]

Interesting tip Luis, I never tried such a fake RefPoint. Will give it a try (I think it's interesting to see how the scenery engine works with such a object).Arno

---

Member Netherlands 2000 Scenery Team[link:home.wanadoo.nl/arno.gerretsen]Arno's FlightSim World for scenery design hints, tips and other tricks...

[Guest GerrishGray]

I'm intrigued, Luis. Which version of FS did you test this on? I repeated your test in FS2002 using a 200m long semi-transparent fence with its RefPoint (see box labelled R in illustration) about 80m from the viewpoint and a set of 4 objects all within 15m of the viewpoint, two (numbers 1 & 2) on the near side of the fence, two (numbers 3 & 4) on the far side. Each of the five objects is separate, with its own PerspectivaCall and Refpoint, and each object is centered on its Refpoint with no TransformCall etc.The result is illustrated in the attached file and, as you can see, all objects are correctly displayed with no problems about drawing order (the fence is 'drawn' first in the BGL). I used the floating point routines to program this.My understanding of how FS creates its screen images is as follows (but I have NO inside information on this and you may know much more about it?):[ol][li]The z-buffering and painters algorithm operate on a triangle-by-triangle basis.[/li][li]All visible triangles are painted to the screen frame buffer, with the furthest triangles painted first.[/li][li]Transparency is handled by the transparent image being 'added' to the existing 'background' image already in the buffer.[/li][li]The completed frame is then 'blitted' to the screen display buffer.[/li][/ol]--------------------This is a different subject, but what I don't understand is how FS/DirectX handles intersecting triangles and related situations. The straightforward painters algorithm (drawing the nearer triangles last so that they overpaint the further triangles) does not deal with the intersecting surfaces where the final view needs to display part of one triangle and part of the other. Can you throw any light on this ?RegardsGerrish

[arno 1]

Gerrish, just an idea, I haven't tested it. I assume you have used a transparant color for the fence now? If you try to use a bitmap instead that has transparancy information, does it work the same then. I guess the when the engine knows the object is transparant (due to the color in this case) special attention is given to it during drawing.Arno

---

Member Netherlands 2000 Scenery Team[link:home.wanadoo.nl/arno.gerretsen]Arno's FlightSim World for scenery design hints, tips and other tricks...

[Guest luissa]

Gerrish,I think Arno gave the answer to your question. Here are 2 pictures referring to this problem. In the first one the Indirect RefPoint was deleted. The problem is that my fence is made with a bitmap and the transparency is on the 4th byte of palette entry for the color I made transparent.I have no experience on computer graphics and my interpretation is purely based on observation and guessing. Here is how I understand the operation of the FS display.1. Objects are sorted prior to display based on the distance between the RefPoint and the Viewer. Distant objects are processed first.2. Objects are divided in triangles. When time comes for a particular object to be displayed, the triangles are processed by the order they appear in BGL code.3. I helps to think that a complete "display-buffer" for a frame is constructed in memory and, when completely done, it is copied to the display.4. The operation of the "display-buffer" is related to the operation of another buffer: the "z-buffer". Both buffers have the same size. When a pixel is added to "display-buffer" the distance of that pixel to the viewer is added to "z-buffer" at the same location. The "display-buffer" contains pixels or colors and the "z-buffer" contains distances.5. When comes the time for a triangle to be displayed, its "projection" on both buffers is evaluated. Now the important part: each projected point of triangle on the "display-bufffer" is actually copied to the "display-buffer" if, and only if, the projected distance on the "z-buffer" is shorter than the present value on the "z-buffer" for that location.Using my screenshots: when the triangles of the lowest parts of my iluminating post are processed, there is already something near in the "z-buffer". It is the fence. Those parts are skiped. No check is done to see if the near parts are transparent or not.Using your screenshot and object 4: are you sure that the top part having the sky as background is different from the part that is covered by the fence?Regarding your last question, the "projection" operation that I refer to could explain how things happen, but I really do not know.Regards, Luis

[Guest GerrishGray]

Hi again LuisAs regards the method used by FS to construct display frames, I think you will find that FS uses the painters algorithm as I describe. This is intrinsic to DirectX too, I believe - although I am open to being corrected by anyone with more knowledge ... :-)Older versions of FS were not so good at applying the painters algorithm because they did not calculate distance from viewpoint on a triangle-by-triangle basis, and hence FS98 and earlier used to suffer from those terrible bleedthrough problems and had a great deal of trouble handling transparency correctly. We had to use separation planes to help get the drawing sequence correct. The new scenery engine in FS2000 got rid of those problems by adding each triangle to a display list sorted by the distance of each triangle from the viewpoint. The order of objects in the pseudo-procedural code of a BGL file is no longer of much significance.I asked about the intersecting planes case because I wondered if you had additional knowledge about this? In the absence of any other information, I believe that FS/DirectX has to calculate the line of intersection between the two planes to resolve this problem, impacting performance and sometimes leading to errors (some people have reported occasional display problems with the crossed-plane tree model, for example). gMax recommends use of its built-in 'weld' function when dealing with intersecting surfaces, so that it can resolve the planes into separate triangles along the intersection line and save FS/DirectX the trouble, I believe? (I'm not a fan or user of gMax myself - I usually code stuff by hand).As regards your present problem with the transparent fence, which you have overcome with your clever indirect Refpoint technique, the initial cause must lie either (a) with the use of older polygon rendering methods instead of the modern floating point code, or (:( with the texture you are using for the fence.My first guess is the latter because you are using a CFS/FS2000-style 8-bit bitmap from what you say, and although FS2002 does support this format it is known to have some problems when dealing with transparency. Nevertheless, I replaced my semi-transparent material with an 8-bit bitmap, similar to yours but with the normal, simple, transparency map using index 0 rather than 4, and with no partial transparency (as yours also seems to have). The results, as you can see, remain correct without any need for the indirect Refpoint. It might be worth trying to simplify your texture, or converting it to one of the 16-bit formats which FS2002 prefers.CheersGerrish

[Guest BWIA330]

Hey,did you all create these fences? Because they look fantastic. I am currently looking for a decent fence for my Fs2k2 scenery that look exactly like the one in these pictures you all have posted. Is there anyway you all might be uploading these?Regards,BWIA330

[Guest luissa]

>Older versions of FS were not so good at applying the >painters algorithm because they did not calculate distance >from viewpoint on a triangle-by-triangle basis, and hence >FS98 and earlier used to suffer from those terrible >bleedthrough problems and had a great deal of trouble >handling transparency correctly. We had to use separation >planes to help get the drawing sequence correct. The new >scenery engine in FS2000 got rid of those problems by adding >each triangle to a display list sorted by the distance of >each triangle from the viewpoint. The order of objects in >the pseudo-procedural code of a BGL file is no longer of >much significance. Dear Gerrish,Note that the following is a kind of speculation. 1 - I think that each triangular surface is projected in the finite display rectangle, say a rectangle with 1024 x 865 pixels. The z-buffer would be, in that case, an array of 1024 x 865 distances. So, I tend to think more in terms of "pixel sorting" instead of "triangle sorting".The projected pixels of a given triangle and the distances of the projected pixels to the viewer are evaluated. Each projected pixel will be added (or not) to the display buffer, if the z-buffer, for that pixel, does not (or does) indicate that a "shorter" projected pixel already exists in the display buffer. I assume that when a projected pixel is decided to be added to the display buffer (if the z-buffer indicates so) it "replaces" the existing pixel if the new pixel is opaque, or it "will be mixed" with the existing pixel if it is transparent. And, as soon as the new pixel is added, I think that FS will forget about it (about its possible transparency) except for the fact that the corresponding location in the z-buffer is updated with the new shorter distance.If the objects in a scene were all opaque, there will be no need to sort their drawing order with the "PerspectiveCall" mechanism. Sky, mountains, ground polygons and 3D objects could just be used with a simple Call() and would be properly displayed due to the z-buffer technique.But if the objects have transparent surfaces the ordering problem comes again. Fortunately the old "PerspectiveCall" sorting algorithm helps in confining the problem to the object with the transparent surface, one exception being the one I described in the beginning of this thread. Having the problem confined to a single object and if we have a single transparent surface the solution is simply to place the transparent surface as the last piece of the code. But in more general cases with more than one transparent surface, we may even have to use SeparationPlanes or VectorJumps.>I asked about the intersecting planes case because I >wondered if you had additional knowledge about this? In the >absence of any other information, I believe that FS/DirectX >has to calculate the line of intersection between the two >planes to resolve this problem, impacting performance and >sometimes leading to errors (some people have reported >occasional display problems with the crossed-plane tree >model, for example). gMax recommends use of its built-in >'weld' function when dealing with intersecting surfaces, so >that it can resolve the planes into separate triangles along >the intersection line and save FS/DirectX the trouble, I >believe? (I'm not a fan or user of gMax myself - I usually >code stuff by hand). 2 - If the point by point projections, that I referred to above, and their corresponding distances are actually evaluated, then the intersections would be automatically taken care of by the z-buffer technique.>As regards your present problem with the transparent fence, >which you have overcome with your clever indirect Refpoint >technique, the initial cause must lie either (a) with the >use of older polygon rendering methods instead of the modern >floating point code, or (:( with the texture you are using >for the fence. >>My first guess is the latter because you are using a >CFS/FS2000-style 8-bit bitmap from what you say, and >although FS2002 does support this format it is known to have >some problems when dealing with transparency. Nevertheless, >I replaced my semi-transparent material with an 8-bit >bitmap, similar to yours but with the normal, simple, >transparency map using index 0 rather than 4, and with no >partial transparency (as yours also seems to have). The >results, as you can see, remain correct without any need for >the indirect Refpoint. It might be worth trying to simplify >your texture, or converting it to one of the 16-bit formats >which FS2002 prefers. 3 - Regarding my own specific problem, I was using TexPoly()s in the shown screenshots but just switched to DrawTriList()s with the same results. That calls for the texture. This is a normal 256 colors bitmap of size 256 x 512 made with PSP. After saving it, I open it with a programme that I wrote that changes one entry of the pallette. The entry corresponds to the color that I want to make transparent. As the opaque colors of my fence are similar to green, I chose a greenish (spelling?) non used color for the transparent one. The programme just changes the 4th byte of that entry from FF (completely opaque) to 0 (completely transparent). If you think I could obtain a different result with another format I would like to try. I setup an experiment with this problem and it would be very simple for me to test another texture if you could point me one.Regards, Luisps: it is pleassure to discuss these problems here!

[Guest GerrishGray]

Hi LuisIt definitely sounds as though it is your fence texture that is causing the initial problem - and that seems to be borne out by your screenshots as well. In addition, the shots seem to show the chain-link parts of the fence as being semi-transparent, even though you say that you have only set transparency on one palette entry. You will have read my notes about texture formats in another recent thread on AVSIM. The FS2000 8-bit format is trouble-prone in FS2002 when using transparency, but if you want to go on using it I suggest that you get Martin Wright's Bmp2000 tool from http://www.mnwright.btinternet.co.uk/index.htm(Martin is the guru on bitmap formats and you will find his various tools very useful.)There was nothing special about the fence texture I used - it was only one of many that I have accumulated on my hard disk from various scenery downloads (although it did contain an error that I had to put right with Bmp2000 to get correct transparency !)-----------------As regards the method used by FS to create the screen display, you have obviously put a lot of thought into your ideas about how it might work ...My own original knowledge and experience of graphics programming goes back to days 30+ years ago when we did not consider the use of a rastered bitmap to construct the image but instead did everything with 3D vector geometry. The mathematics of clipping, hidden surface removal, and the particular problem presented by intersecting, mutually overlapping, surfaces was complex and a major barrier to fast creation of computer-generated images. The advent of raster displays which made it possible to use the 'painters algorithm' (whereby everything is drawn, but in sorted order with furthest drawn first) offered a major advance that simplified the process enormously and would allow much quicker generation of images for display on a computer screen. One didn't have to waste time on laborious calculations of which surface would be responsible for painting each pixel, because the sorted drawing order would resolve it automatically by the crude expedient of overpainting ...I certainly like your idea of an algorithm that works on a pixel-by-pixel basis and records the distance from viewpoint of each individual pixel, because this would make it unnecessary to sort the objects into distance order before displaying them. On modern computers with their relatively vast memory resources and tremendous computing speed, this certainly seems to be a valid approach.Who knows, maybe it IS how FS works these days - my knowledge of 3D graphics programming techniques is not sufficiently complete or up-to-date to know. But the published SDK, and information from other people with more inside knowledge than myself, seems to suggest otherwise ...Anyway, discussing these ideas is very interesting, as you say! Kind RegardsGerrish

[Guest luissa]

Dear Gerrish,You are absolutely right on the format of the texture causing my initial problem! Still I think that may be it is useful for other scenery designers to comment this a bit further. Please excuse me if I tend to be repetitive.I just used the ImageTool.exe and converted my original fence texture to different formats. I recall that I was using the 8-bit bitmap format. In this format each pixel is represented by an 8-bit pointer to a pallete of 256 colors. The palette has 4 bytes per color and, in FS, the 4th byte of each palette entry is interpreted as a level of transparency (I think I could called this transparency method as a color key method). This is very different of achieving transparency with an alpha channel, being it 1-bit (ON/OFF) as in the "565" 16-bit format, 4-bit (16 levels of transparency) as in the "444" 16-bit format or 8-bit (256 levels of transparency) in the 32-bit format. The conclusion that I found is that the 8-bit format, with the transparency "contained" in the palette, brings the kind of problem that I was having. I think that I can understand why this happens and that is the reason of this post. In one of my posts on this thread I said:... I assume that when a projected pixel is decided to be added to the display buffer (if the z-buffer indicates so) it "replaces" the existing pixel if the new pixel is opaque, or it "will be mixed" with the existing pixel if it is transparent. And, as soon as the new pixel is added, I think that FS will forget about it (about its possible transparency) except for the fact that the corresponding location in the z-buffer is updated with the new shorter distance.I should have said:... I assume that when a projected pixel is decided to be added to the display buffer (if the z-buffer indicates so) it "replaces" the existing pixel if the new pixel is opaque. If new the pixel is transparent, neither the display buffer is replaced nor the z-buffer is updated. However if the new pixel has its transparency definied in a palette, it is treated as an opaque pixel.Having said this, I still keep the 8-bit format for my fence and rely on the indirect RefPoint to solve the transparency trouble. The reason is related to MIP maps as I pointed out in topic 1969. If the 8-bit bitmap has only 1 MIP, and if one color is transparent, FS will not recreate the other low resolution MIPS and will show the texture in full detail. For all the other formats with 1 MIP only, with or with not transparency, FS creates internally the missing MIPs and uses them as the distance increases.Kind Regards, Luis

[Guest GerrishGray]

Hi LuisYes, I have found other cases where the 8-bit format has problems with transparency. The reason is not obvious, but it happens! Transparency is actually handled by a DirectX 'primitive' I believe, that does the 'addition' sum overlaying the semi-transparent information onto what is already in the display buffer (the background, in other words). But because the 8-bit format is palletised and isn't native to DirectX, FS must do some intermediate work in looking up the palette index before sending the data to DirectX, and the problem presumably occurs somewhere at that stage.Even simple index 0 on/off transparency can cause problems in some cases. If you put a lot of my trees on screen using the 8-bit version of the textures, you can get a nasty 'shimmering' effect that moves across the screen in waves in FS2002 (although they were fine in FS2000). When the textures are converted to DXT1 this no longer happens.But the test I did with that fence was using the 8-bit format, not a 16-bit format, and that worked OK.MipsI don't usually find mips a problem, but it does rather depend on how the texture is applied to the model. The worst example, I think, is the Autogen buildings, where the mipped textures can look very badly blurred when they are stretched too much. The floating point commands have a way of contolling this through the argument in the texture list definition, and it is important that the texture size is set correctly to ensure that the correct mip is used depending on the size of the image on screen. (I think that is the problem in Autogen - because the textures are stretched onto different size models, the texture-size setting isn't used correctly).Incidentally, I believe (although I haven't tried it myself) that the 444-4 format also allows you to create a texture without mips, as the 8-bit format does - you might want to try that out.CheersGerrish

[Guest luissa]

Hi Gerrish,I just discovered what caused the problem in my fence texture. I was using a "colorkey" type of transparency. From FS98 days I declare a non-used color as the transparent one. I do not chose the (normal) black. For trees, for example, I use a transparent color similar to the color of the trees. So I was using a "transparent color" very similar to the color of the fence. And, with that 8-bit bitmap I saw transparency except when the fence was near than an object and the object was processed after the fence. So I wass using the RefPoint indirect. But just yesterday I opened that 8-bit bitmap with transparency with ImageTool. As expected I got the alpha image using View > Alpha Cannel. BUT AS SOON AS I converted it to DXT1 the alpha channel was gone! I think the reason is a transparent color very near to the opaques ones. I changed the transparent color to a brillant blue making a big contrast with my green fence. And with this single modification I got no bleedthrough and I have no need to the indirect RefPoint. I conclude that FS handles textures internally as DXT1 or 3 and due to the conversion loss we can get these kind of probems.Kind Regards, Luis