Sign in to follow this  
Guest cbuchner1

New project: Tile server for MS FSX based on Google Map...

Recommended Posts

Hi everyone!I had a cute idea that I decided to try to "make it happen". I am going to implement a tile server for Flight Simulator X in the form of a proxy that mediates between Google Maps satellite imagery and FSX. FSX will use some predefined custom scenery that points to tiles on a mounted network drive. All tiles are assumed to have known filenames from which the coordinates can be derived. I will start out with a very limited, rectangular area first. Later I might try to cover larger areas, maybe even whole continents.The network drive serving out tiles is connected through the SMB protocol and is provided by a Linux or Mac. I currently use a Mac Mini because I have one available. The Mac Mini runs a Samba server, compiled from source code. It provides read-only access to a volume initially containing just an empty folder. Samba will be configured to use a Virtual File System (VFS) driver that intercepts incoming file I/O operations. When a tile filename corresponding to a certain coordinate is first opened, it will open a HTTP connection to the Google Maps tile server and download the resulting 256x256 pixel image. This image would be converted into a format suitable for Microsoft Flight Simulator X and then is served like a regular file. It will also be cached in that folder such that subsequent read operations do not have to load this data again from Google.I see the following issues popping up:* the VFS driver for Samba has to be developed (C/C++) and tested* the initial transfer from google would drastically delay the MS FSX loading of the scenery. I assume MS FSX reads sequentially, not multiple tiles in parallel.* how can I align the Google tiles at various zoom levels with the coordinate system of MS FS X?* what is the best tile format to use? Megascenery used uncompressed BMP images, is that feasible?* a 1 GBit/s LAN would be nice to get good throughput between Tile Server and simulator PC once the tiles are cached.* One season of the year only. Google-season!* No night scenery available also.I will keep everyone posted on the progress of this project. So far I have Samba compiled and working on a Mac Mini (which is quite an achievement on its own). Will start experimenting with VFS drivers soon.Christian Buchner

Share this post


Link to post
Share on other sites
Help AVSIM continue to serve you!
Please donate today!

Interesting, but how will you get around FS's inability to update its scenery while it's running?

Share this post


Link to post
Share on other sites

And how will you get round the legal restrictions on the use of Google images?

Share this post


Link to post
Share on other sites

FSX would first load all tiles to form the Quadtree of various detail levels around the user. see this link for a visualization of the Quadtree:http://www.fsinsider.com/NR/rdonlyres/5D22...alTerrain04.jpgWhen you move around, it re-loads existing tiles at different detail levels (LOD mipmaps). And this would result in more hits to my network drive - probably loading from cached data if it's just a higher detail version of an existing tile.

Share this post


Link to post
Share on other sites

I will build a prototype first (for limited personal use), but not publicly release it without actually clarifying the legality aspects.If you understand FSX as a "web browser", the network drive as a "proxy server", it would not appear illegal. In fact, there should be some gauges out there that provide web browsing ability in the glass cockpit. And a true web browser is built right into the FSX shell - when you access the fsinsider.com from the main menu!As far as I understand, Google Maps explicitly prohibits the use of "robots" to access the data. The flight simulator does not constitute a robot because it provides interactivity. It only loads tiles around your current location - not the entire world.enjoy the following legal (!) Google Maps Flight simulatorshttp://www.isoma.net/games/goggles.htmlhttp://www.markfennell.com/flash/wings/index.phpI understand this thing as a proof of concept - and a lot of technical issues have to be solved before even the first HTTP request to Google's servers is made. Initially I might even use my own HTTP server to make sure I don't put load on Google's servers during development.

Share this post


Link to post
Share on other sites

Allright folks! Progress on this project!So I completely dumped the new RESAMPLE tool from the FSX SDK. Too complex, no way to change the textures on the fly inside the generated BGL file!The .bmp phototerrain texture tiles used in many FS2002 and FS2004 addons however are in DXT1 format and contain 7 mipmap levels. I figured out the file format and I also found a fast (and free) DXT1 encoder source code which is very fast (Intel SSE2 optimized)The file naming scheme of the BMP tiles of these photo terrains is based on a quadtree notation, and I found a way to convert these filenames back into lat/long coordinates. The resolution (for now) is limited to 4.75m/Pixel (LOD 13)I figured out a way to tap into the Google Earth tile server by means of "libgefetch", part of the open source GAIA project, a reverse engineering project by a Russian programmer. This gives me "on demand" access to tiles without any Google imprint or logo. I wrote some code to retrieve the correct Google Earth tiles (based on lat/long coordinates) which then have to be concatenated, then cropped out and scaled to become a photo scenery tile for FS X (or 2004).The major work that remains is:Decoding of downloaded Google JPEG imagery in RAM using libjpeg, concatenate, crop out and stretch tiles, downsample to various mipmap levels, DXT1 encode and write out to file. This is a proof of concept hack only and will probably not see public release anytime soon. That Russian dude already got threats by Google Inc. - because they are not evil! I will post screenshots though of me flying over the Google Earth, promised ;-)Christian

Share this post


Link to post
Share on other sites

Here is a side by side comparison of a MegaScenery LA tile and one which I downloaded, cropped and compressed using my project. Note the difference in quality, allthough the resolution in pixels is identical (~4.75 m /pixel). The top image is MegaScenery LA, the bottom one is Google Earth. Both tiles are already DXT1 compressed, ready for use in FS 2002, 2004 and FSX162078.jpg

Share this post


Link to post
Share on other sites

Mission accomplished! Flying over the Google Earth in a Mooney Bravo. The location is San Diego, you see the old baseball stadium (Qualcomm Stadium) in Mission Valley and a shot of Mission Bay in the images. The freeway below would be Interstate 8.The tiles are loaded *on demand* when you move around. Loading of tiles IS TERRIBLY SLOW. Before the simulation started I had to wait for one hour downloading tiles. The tile resolution is 4.75m/pixel (LOD 13). Tiles are automatically downloaded and converted to DXT1 compressed BMP by a virtual file system layer inside Samba.http://forums.avsim.net/user_files/162129.jpghttp://forums.avsim.net/user_files/162130.jpgNow I understand that the russian dude got served a nastygram by Google because no-one else is allowed to interface with the Google Earth imagery with a client other than the official Google Earth software. It is simply a licensing restriction imposed by the providers of the sattelite imagery. Bummer!So I am going to write different backend modules for my Samba based tileserver - for example USGS data or maybe I can even switch to Google Maps (not Earth). I can't keep using Google Earth, although it was fun while it lasted ;)Christian

Share this post


Link to post
Share on other sites

Hang on.So you've made a program that replaces default texutres with Google Earth satelite immages? Is that correct? Thats fantastic!!! Actually flying over things that actually exist!! I can spot my own house?I'm all for it. How bad is the loading time in flight? What sort of internet bandwidtg would that use??

Share this post


Link to post
Share on other sites

Well, after about an hour of flight I can say "I love it". The quality of the imagery beats the MegaScenery package hands down. I marvel at the beauty ;-)The link between the tile server and MS FSX is a 100 MBit/s ethernet and it keeps pushing tiles at a rate between 4-10 MBytes/s while flying. These tiles are mostly cached on hard drive already. The download from the Google site is much slower ( my DSL caps out at 3 Mbit/s and each HTTP request has to cross the Atlantic - the ping time appears to be ~25 ms - likely by sea cable ). When I fly within a region that has already been cached by my tile server, Google barely gets any HTTP requests for new tiles. I think I can say that the bandwidth I have used on google's servers today is less than if I would be watching a 90 minute movie off video.google.com.The problem I currently face is that I have to stay within the region for which I already own (commercial) photoscenery based on 256x256 pixel BMP tiles. I have no BGL scenery files that tell the simulator that there is photoscenery outside the region. So I will have to create some photo scenery definitions of my own to cover the entire globe - or at least the continents I want to fly on.I wonder if a fully global coverage of photoscenery would slow down the flight simulator to a crawl? I shall test that soon!Christian

Share this post


Link to post
Share on other sites

Hi Christian,a few links that might be useful ...- NASA Worldwind is similar to GoogleEarth, but its open source and uses public data: http://worldwind.arc.nasa.gov/- Google Maps SDK: http://www.google.com/apis/maps/- Microsoft Virtual Earth SDK: http://dev.live.com/virtualearth/sdk/I would try MS Virtual Earth first, cause maybe you can talk MS into giving you a special license for MS VE access if its used for an addon to MS FSX ...Maybe a different approach for loading the tiles would be faster:Instead of hooking into the load file calls of FSX, you could monitor the aircrafts position and heading using SimConnect to retrieve and convert the tiles even before FSX actually loads them.

Share this post


Link to post
Share on other sites

Thanks for your insights!One question though: Is the SimConnect SDK already providing position and heading data when the simulation first loads (the black screen with the "Loading Scenery..." dialog box) ?The one major problem that I was facing is that FSX loads tiles FAR OUT from the viewer position where it would typically only access the lowest mipmap levels (e.g. 4x4 pixels). However my Samba server cannot possibly know that only a low detail level is required and it would load the entire 256x256 full resolution tile of the Internet. In practice, loading a zoomed out version of this tile would provide the required mipmap level AND THAT OF MANY MORE NEIGHBOR TILES in one go, saving a lot of bandwidth in the process.I could possibly estimate the required LOD level based on the distance from the aircraft, but that would not be a very precise estimate, as it depends heavily on the selected viewing distance and on other graphics settings as well.Christian

Share this post


Link to post
Share on other sites

I dont know exactly when position info is set, but since FSX needs the position to know which part of scenery to load, i bet that it is set pretty early.Via Simconnect, you could monitor aircraft position, heading, speed and altitude. Using the SysinternalsTools ProcessMonitor and FileMonitor ( http://www.microsoft.com/technet/sysinternals/default.mspx ) it should be possible to figure out how FSX loads the individual tiles. You could use fake tiles (e.g. red for MIB/LOD level x, blue for level y and so on) to analyze what resolution FSX actually uses depending on the distance from the aircrafts position. Using the heading and possible GPS / flight plan info it might be possible to make educated guesses what tiles and resolutions FSX will load next.Instead of hooking the file access via Linux/Samba you could use the MS IFS Kit ( http://www.microsoft.com/whdc/DevTools/IFSKit/default.mspx ). One of the samples in that kit simply monitors all file access nd redirects access to the real file system (FAT/NTFS). Maybe the SysinternalsTools use the same trick to monitor file access.To avoid the need for a second PC, you could use VmWare or MS Virtual PC - there are free versions available for both - finally the second core would be less bored then ..I wouldnt mind if i have to tell your scenery loader before start, where i want to fly. I think preloading and caching instead of load-on-demand would simplify the whole architecture of your tool (no file access hooks needed). I think you could get rid of all speed problems during flight with that architecture, cause all required scenery (tiles, bgl, etc) could be generated before flight.What are your future plans for your tool ? Will it be a payware or freeware addon ?

Share this post


Link to post
Share on other sites

Technical update: I just blew up the tile size from 256x256 to 1024x1024, going from 4.75m/pixel to 1.1m/pixel. Initially I didn't think this would work, but actually MS FSX accepted this readily. The tile size grew from 44kb to ~650kb PER TILE. Needless to say that my 100 Mbit/s LAN is outgrown by this massive bandwidth requirement.So in fact, you can use RESAMPLE.EXE to generate 4.75m photoscenery and then you can later simply replace the tiles with higher resolution versions - this works in FSX, I haven't tested in FS 2004...I am currently flying over 1m/pixel scenery and looks georgeous, however I have to put the simulator into pause mode to allow it to catch up loading tiles. Duh.I dont think this "addon" will be available publicly anytime soon, especially not with a tile backend that is an open invitation to sue ;-). I am just a hobbyist experimenting and my code is neither clean, nor easy to install nor easy to use - and the concept currently requires two PCs connected through a fast LAN connection.Speaking of FAST. I should try some IE1394 networking instead of the Ethernet LAN. At least my PC offers an IE1394 network connection, maybe my Mac does also.Christian

Share this post


Link to post
Share on other sites

The MS IFS kit is not free, I do not intend to spend 110 dollars for the "privilege" to write file system drivers for windows. But as you noted, the FileMon tool from Sysinternals hooks into API calls for filesystems and could be used as a basis for development of a tile pre loader.My worries are, since this is nitty gritty kernel mode driver development, that I will bluescreen and crash my trusty old PC more than I would like to.VmWare to run the tile server is an excellent idea, as networking between the host and the virtual pc should be quite fast.Let's see how that IEEE1394 (Firewire) link performs.Christian

Share this post


Link to post
Share on other sites

I am considering this thread a technical project blog ;-) Read along if you're interested.I was able to solve some performance issues with high resolution (1m) texture loading by setting the MAX_BANDWIDTH_MULT parameter to 5000 and additionally increasing the fiber time parameter to 0.25. The tile server is connected to the simulator via Firewire and I have added a read cache feature to the modified Samba server. It now keeps a lot of tiles cached in RAM because Flight simulator is one greedy son of a ####. It keeps reloading the same tiles over and over, requesting the entire file even if it just needs a low resolution mipmap. So if you only have 600 MB of tiles on hard disk for a particular region, it may actually request around 5 GB of data over the network before the plane can even start taxiing. During flight, the sim keeps updating tiles at rates between 2 and 6 Mbytes/sec whereas during the initial loading phase, the Firewire sometimes bursts at 14 MBytes/sec (by courtesy of the new read cache).The 1m ground texture resolution is mindblowing. During a test flight today loading of tiles was keeping up with my aircraft most of the time. I have picked one Pacific island location for testing and when looking out of that airplane cockpit everything seemed so real. It's frightening. Almost spiritual. I'll have to invite some friends over just to show them the deal. Posting the images on this forum (at reduced size) wouldn't do them justice. Next stop: Grand Canyon. In the main FSX forum some people were actually questioning whether my latest screen shots were doctored. Nope, it's all for real.Regarding further development, I might try running the Samba server inside a virtual machine on the simulator on the 2nd CPU. Less networking overhead when using a virtual network connection.

Share this post


Link to post
Share on other sites

A bit more of an update. One major problem was that always the entire tile file would be read by FSX, even if only a 4x4 pixel bitmap was required. This created some insane network load when using 1024x1024 textures - 650 kb per tile. And thousands get loaded, yikes!I created some code to dynamically serve lower resolution tiles from the read cache by changing the BMP header and removing higher resolution bitmaps. This causes no CPU overhead as the tile does not need to be recomputed or rescaled and the network load goes down significantly.I created a small program using the SimConnect SDK that reads out the plane's current coordinates periodically. I would like to to report the current position to the tile server through some particular I/O operation, e.g. a write call to a magic file or an open () call specifying the data as part of the filename. Then the tile server could know which tiles are close by to the aircraft and pick an appropriate tile resolution.The part that tells Samba about the plane's position is not done yet as well as choosing the best resolution for each tile. For sure the tile should not exceed the mip level that FSX intends to load. And it should not have a much lower resolution either. Quite a challenge to "guess" what MIP level the simulator intends to load for a specific tile. ;-)The benefit would be a much increased performance with this Samba thing and hopefully no more blurries even with 1024x1024 textures.

Share this post


Link to post
Share on other sites

Project update!Performance problems of the network connection are solved! A combination of caching and delivering lower resolution tiles (depending on the tile's distance from the aircraft) proved to work. I can now fly in a Learjet over my pacific island location and enjoy every spot in crisp texture quality.Some bad news: I found out that MS FSX always loads BMP tiles fully, even when accessing just low resolution mipmap levels. Don't believe it? Run filemon.exe from Sysinternals.com and check what the Read calls say. That would explain some performance issues and long load times on FS2004 style phototerrain textures.Luckily with Samba I can simply serve less data over the network and the file will appear to be "virtually shorter" without causing a read error on the receiving end. Hence I can provide the lower mip levels while FSX thinks it's still getting the full deal. Ha! ;-)The next step will be to dynamically lower the resolution loaded off the Internet depending on distance to the aircraft. It simply doesn't make much sense to get everything at the max detail level - loading time being the main reason.I should now check out and test the latest incarnation of Gaia. It has been cleaned from Google code and now accesses NASA's World Wind servers. Much more legal.Christian

Share this post


Link to post
Share on other sites

This is incredible work, I never thought that something like this would be possible with current technology. Can't wait to see where it ends up!-Scott

Share this post


Link to post
Share on other sites

Some tests in larger areas showed problems with tiles far out from the plane (the lowest resolution 4x4 MIPs) - they appear black. I did not notice this happening in the Oahu scenario at all - obviously because all terrain was close enough to be loaded at higher detail level. Of course the black tile issue must be solved to allow an immersive flying experience in larger photorealistic areas.Someone noted there is no haze in my screenshots at all - making everything appear quite unrealistic. This could be because I disabled Shader model 2.0 and fly at the best weather setting. I shall try some rain or fog soon ;)On the technical "front", I have now devised several ways of serving lower resolution tiles to reduce the network load - all because the tile loader of FSX is so wasteful with resources.1) - This method provides a modified bitmap header (lower resolution, less mips) and skips the bitmap data for higher MIPmap levels. The file then appears as a low resolution tile to FSX.2) - The second way is to serve the original tile file, but in a "sparse" way, zeroing out the parts of the file which I want to skip. This is deep trickery with Samba and is only tested with Windows XP as client so far. XP reads files in 64k blocks and entirely skips a block if you return a read operation with 0 bytes - thus allowing me to not having to transfer the uninteresting parts of a file across the network.3) - I can also serve the BMP header only when FSX accesses a tile for the very first time. This works on all mipmap levels other than 4x4 - the 4x4 level appears black though because FSX does not access the 4x4 tiles for a second time. This is the fastest way of serving out tiles though and the preferred method in the initial loading phase of a flight. FSX then caches how many MIP level each known tile has, allowing me to use method 1) on the subsequent tile reads.A combination of 3) and 1) has worked for Oahu, but I may have to use method 2) for the 4x4 tiles in the far distance.I will also further investigate the LOD quadtree structure that FSX maintains internally such that I can better predict when FSX will load tiles and at what resolution.

Share this post


Link to post
Share on other sites

Hey, thanks for your encouraging comment.I've had a look at your webpage, in particular the Ni'ihau project and I wonder how you masked out water vs. land texture. Did you create kind of an alpha mask for the land shape and applied it with RESAMPLE.EXE? Did you use the FSX SDK or the FS 2004 SDK when doing your project?The reason I am asking is that I will have to come up with ways to prevent my project from displaying water as a land texture. I might have to use an autonomous water recognition algorithm (image processing). Currently I have no idea how to store the alpha information in the DXT1 texture.There is one SDK sample with water masks in the FS 2004 SDK - I shall try that.Christian

Share this post


Link to post
Share on other sites

Am I understanding that you are using the FS2004 resample to create the bgls for your photo tiles? Or is there a way not to embed the tiles in the bgl in FSX resample?As far as the water masking, it seems like one solution would be to read the cvx files to get the hydro polys, and use this to build the masks. Extracting shorelines from the image I guess is possible, but more of a problem for inland water and bridges. Of course if the cvx data don't match the image there is a problem. For the US, the UT USA quality data seem to match very well. scott s..

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this