Sign in to follow this  
hjsmuc

Crowdfunding for orthoimagery - anyone in?

Recommended Posts

I wonder if fellow simmers would be interested in starting a crowd funding project for orthoimagery. The intention would be to gather the necessary funds to buy orthophotos for the most popular regions which could then be turned into scenery with ortho4xp. I didn't think about details yet. It's my personal opinion that orthoimages and XP 11 are the perfect couple and that we need to start building up an official repository of such photos which can be shared by the community. Maybe with some luck and the restriction to not commercialize the pictures one could negotiate an acceptable price with the providers.

Share this post


Link to post
Share on other sites
Help AVSIM continue to serve you!
Please donate today!

Thanks Jay, it's great to see some fair and free providers. It's an interesting market.

 

Problem with free offerings is that they come in a multitude of formats which makes it more difficult to convert into scenery tiles, IMO.

 

The idea behind organizing the sources on a paid basis is to get data which would be easier to convert.

Share this post


Link to post
Share on other sites

I'm definitely in. What is an additional cost I guess is the server bandwith to host/download such huge files. For US I think this would be the only cost, in Europe you would have to add the purchase of textures from Bing or Google, I know company like Aerosoft or 29Palm buy orthos for their mega sceneries.

 

To be honest some providers made available orthos for Europe without any copiright issue from the two mega corps, they eventually closed the more detailed LOD17 due to the massive cost for managing the download traffic.

 

I still believe sharing orthos is the key, for example I managed to do the Cyclades islands at good L17/L18/L19 resolution, spent lot of hours fixing differences, merging tiles, changing colours...but then I don't know how and if I'm allowed to share my work. I guess many others have done the same work on other parts.

Share this post


Link to post
Share on other sites

I think you will be shocked when you find out how much the providers will want to charge. 

 

Also, USGS is good source, but it is a lot of effort to download the files.  For example, a single 1x1 degree area has something like 400 image files to download and each one is 100 megabytes.  That's 40 gigabytes for a single 1x1 degree area.

  • Upvote 1

Share this post


Link to post
Share on other sites

I think you will be shocked when you find out how much the providers will want to charge. 

 

Also, USGS is good source, but it is a lot of effort to download the files.  For example, a single 1x1 degree area has something like 400 image files to download and each one is 100 megabytes.  That's 40 gigabytes for a single 1x1 degree area.

 

That depends on the ZL.

Share this post


Link to post
Share on other sites

 

 


That's 40 gigabytes for a single 1x1 degree area.

 

That's basically what one will find with most of the high quality orthoimagery. The tiles can be very small. Downloading large numbers of tiles is impractical. The NY orthoimagery is UTM zone 18 and jpg.  I doubt that it's any different than the recent USGS imagery. NY paid for the aerial overflights under some USGS sponsored program.

Share this post


Link to post
Share on other sites

That depends on the ZL.

Probably going to want a range of ZL 16-ZL18.

 

Actually I found a source that provides 0.6 meter orthoimagery for US that can be downloaded by entire county at a time.  Downloading one right now to try it out.  That could be an option.

Share this post


Link to post
Share on other sites

0.6 meter is like ZL18? Based on my calculation for the entire US on ZL18 the images will total to sth like 20TB  Just curious how could you manage to download that much data. 

 

It's great though that you can have them for the whole US. I'm jealous. 

 

I'm wiling to put in some money so we can have a more or less permanent and guaranteed source for orhtoimages or even sharing tiles. Bit-torrent should help on the bandwidth problem I guess. 

Share this post


Link to post
Share on other sites

I can't think of a lot of solutions, but I can sure think of a lot of obstacles. Not the least of which is bandwidth costs.

 

Just using Amazon S3 as a very quick guideline, 20 TB (United States at ZL18) with 5 TB data transfer out a month is close to $1000 per month.

 

Peer-to-peer such as bittorrent would help with initial or perhaps ongoing demand of popular areas, but long term over huge data sets it would never work

Share this post


Link to post
Share on other sites

I'm downloading the entire state of CA (2016 data) at 0.6 m/p right now.  It's 10 GB for entire state of CA.  It's compressed Mr. Sid format so size is reasonable, although quality slightly degraded.  Even though source is ZL18, it would probably be good for ZL16 or 17 due to compression degradation. Also, not sure yet how well the colors are balanced from region to region or if 0.6 m/p is available for entire US.

 

Distribution would have to be through bit torrent.  Or maybe bit torrent for popular areas, and donated servers for unpopular areas.

 

I'm stuck at figuring out how to get Orth4XP to use the downloaded imagery though.  I have the capability of exporting to local files in TMS tiles or even Bing tiles.  But not sure Orth4XP can read local files.  Or I can export to GeoTIFF, etc., but again not sure what to do from there.  ANy ideas?

Share this post


Link to post
Share on other sites

I'm downloading the entire state of CA (2016 data) at 0.6 m/p right now.  It's 10 GB for entire state of CA.  It's compressed Mr. Sid format so size is reasonable, although quality slightly degraded.  Even though source is ZL18, it would probably be good for ZL16 or 17 due to compression degradation. Also, not sure yet how well the colors are balanced from region to region or if 0.6 m/p is available for entire US.

 

Distribution would have to be through bit torrent.  Or maybe bit torrent for popular areas, and donated servers for unpopular areas.

 

I'm stuck at figuring out how to get Orth4XP to use the downloaded imagery though.  I have the capability of exporting to local files in TMS tiles or even Bing tiles.  But not sure Orth4XP can read local files.  Or I can export to GeoTIFF, etc., but again not sure what to do from there.  ANy ideas?

 

 

Post in the Ortho4XP thread on the org.

Share this post


Link to post
Share on other sites

Post in the Ortho4XP thread on the org.

 

I posted over there.  No answer yet, but hopefully will get an answer.  I actually found an even better source. 2016 imagery, ZL18, excellent color correction, and no noticeable compression artifacts.  Looks like the source imagery will end up being about 100 GB for the entire state of CA.  Best of all free and public domain. Still not sure how practical it will be to process the imagery.

Share this post


Link to post
Share on other sites

Well, ZL18 is just overkill, it's unrealistic to try and cover an entire country in that zoomlevel. Realistically, you should mix the zoomlevels, ZL18 for airport zone, ZL17 for cities or interesting areas and the rest ZL16. This has worked for me very well, and most of the ZL16 tiles I only see from 2000ft or above. The average 1x1 tile is about 3GB, and will compress down to about 1GB.

 

Apart for USGS, there are other free sources, e.g. LINZ in New Zealand and some others in Europe. However, to actually try and license or purchase from some providers is going to come as a very nasty shock, and the only way to gain this imagery would be to license it in a single product and not provide them in raw form.

 

There have already been many attempts to set up a shared repository of legal imagery in torrent form which has been edited, but it always fails. You can read about 2 or 3 such efforts over on the X-Plane.org thread. So, maybe try going for free sources first and see how it goes (I'll gladly help if someone sets it up)


 

 


But not sure Orth4XP can read local files.  Or I can export to GeoTIFF, etc., but again not sure what to do from there.  ANy ideas?

 

Ortho4XP will download the tiles for you in jpeg, you can then use the mosaic util to join all the jpegs together, colour blend them, and then split them out again (You will need a lot of RAM, I easily use 10GB in photoshop).

 

An alternative without modifying Ortho4XP is to overwrite the jpegs Ortho4XP downloads with your own images (but you'll need some code to do this), but since there are several USGS WMS imagery sources, this seems unnecessary

  • Upvote 1

Share this post


Link to post
Share on other sites

Well, ZL18 is just overkill, it's unrealistic to try and cover an entire country in that zoomlevel. Realistically, you should mix the zoomlevels, ZL18 for airport zone, ZL17 for cities or interesting areas and the rest ZL16. This has worked for me very well, and most of the ZL16 tiles I only see from 2000ft or above. The average 1x1 tile is about 3GB, and will compress down to about 1GB.

 

Came to the same solution: best compromise unless you're a GA exclusive pilot who loves low trip over country side. Still L17 with L18/19 for town airports is manageable with usually 2 to 3 GB of HD space (jpg not dds).

Sharing only the jpg at this setting coupled with the Ortho4Xp tile.cfg is quite doable, I'd go with this solution for a public repository.

For example Italy+France+Suisse+Austria is roughly 200 tiles X 2.5 Gb = 500 Gb which on a public cloud service means more or less 60 $ a year

 

Of course uploading the full dds converted packages means 2Tb and costs and time explodes exponentially that was prolly the reason wht SimHeaven gave up with their L17 tiles.

Share this post


Link to post
Share on other sites

Great feedback, thanks to everyone. I think Tony and Desaix agree most with my initial thought: It's doable with mostly ZL 16/17 and selected ZL 18/19. My ZL 17 orthophotos are usually about 2-3 GB with the final scenery package being around 5-8 GB. 

 

We also seem to agree on the following:

 

1. Were we can access excellent and free source material we need to understand how this material can be read by Otho4XP. 

2. Were we are not aware of any such sources we should find out what cost would be incurred to buy the imagery.

 

In both cases we need Oscar's support to make it all work with his tool. I'll invite him to join us here. 

 

And last but not least, we should look forward to any news Tony comes up with because Orthoimagery without his World2Xplane data is only half of the cake, at least outside of the US. 

Share this post


Link to post
Share on other sites

Uploading jpgs has the plus to let any user easily customize the tile for his own needing: I like to have my town at L19, no problem at all, I load the tileXY.cfg, add some L19 rectangles at my likings, run Ortho4XP and I've probably already the 95/97% ot the tile downloaded.

SimHeaven never had and have any copyrights issues for european textures, I don't think a private repository would incur in any trouble if mantained for personal and free download, but I'm aware that strictly is not allowed. AT the same time I'm quite sure cost of whole tiles should be prohibitive.

Share this post


Link to post
Share on other sites

As far as I know there is no good free data for France. So this would be a good example for trying to find out how to get the necessary data. Bing data for the country is quite good but they have so many different licensing options, it's very confusing. I'll see if I can get some information from them. Maybe it would be helpful to play through a little case study. 

Share this post


Link to post
Share on other sites

Keeping the files as jpegs is a good option as it lets people customise the colours before generating. It would be more a less a case of downloading the zipped tile folder, dropping the folder into the Ortho4XP/Orthophotos folder and generating. The only issue would be that now and then when the coastline changes it may need to grab an extra image, but chances are low that this would happen. If it saves the repeated bombardment of the public WMS servers' bandwidth by 90% of the time then it's a win.

 

Additionally, as well as images, I think sharing generated DEMs and coastlines is a good idea. I used third-party public data for the coastlines and water in the UK, France, Norway and Poland which are superior to OSM. These could be shared as .osm files to be dropped into Ortho4XP/OSM_data folder.

Share this post


Link to post
Share on other sites

I'm downloading the entire state of CA (2016 data) at 0.6 m/p right now. It's 10 GB for entire state of CA. It's compressed Mr. Sid format so size is reasonable, although quality slightly degraded. ...

 

MrSID is a multiple LOD format with zoom levels. It is great for display efficiency and a challenge to convert to more common image formats.

  • Upvote 1

Share this post


Link to post
Share on other sites

 

 

Additionally, as well as images, I think sharing generated DEMs and coastlines is a good idea. I used third-party public data for the coastlines and water in the UK, France, Norway and Poland which are superior to OSM. These could be shared as .osm files to be dropped into Ortho4XP/OSM_data folder.

Bingo.

 

Another advantage: any user could choose his preferences, water ratio, dem ratio, whether selecting mask for inlands or not...indeed the best solution for customization and to save cloud space.

 

We just need an organizer, someone who can check uploaded tile and choose the best or set some guidelines for setting LODs.

My preferences would be for example L17 for base lod and L18 for airports. Maybe offering a base tile both at L17 and L16 would satisfy all users. Places were I just fly with liners I usually set at L16 but airport surroundings, but tiles close to my home base I set to L17.

 

Maybe there is an automatic script to downsize and rename L17 to L16 without the needing to download again the 16 jpgs.

Share this post


Link to post
Share on other sites

Is there a huge difference in quality for USGS vs Bing for a given ZL?  Sounds as if many favor the USGS, I was curious?

Share this post


Link to post
Share on other sites

Easiest place to start would be with the USGS for popular regions such as east/west coasts.  However, I think reality is that as long as Bing and USA_2 servers allow unrestricted downloading people will just use Ortho4XP since that is very simple.  If we ever get to the point where Bing/USA_2 start to limit downloads or ban IPs, then other options (torrent, etc.) will be more possible.

 

Different situation for other parts of the world where good/easy source imagery is not available.


 

 


Is there a huge difference in quality for USGS vs Bing for a given ZL? Sounds as if many favor the USGS, I was curious?

 

I've been using Bing, but trying USA_2 now and it also looks pretty good.  I suspect the answer to your question might depend on the specific area of the US that you're looking at.  For some, Bing might be better, for others USGS might be better.

Share this post


Link to post
Share on other sites

I think instead of trying to prebake all the textures for download, time could be better spent creating utilities to make the process as easy and foolproof as possible and then provide detailed instructions for downloading from a variety of services.

 

I'm thinking something where you download the files (or the software could even download it for you), click "Go", and out pops a read-to-use texture package for X-Plane.  That's what I want to see.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this