OSM2XP is a great piece of software which is used to generate X-Plane scenery from OpenStreetMap data. 

 

I know the simheaven website provides scenery for OSM, which also supports multipolygon forests amongst other things, however for people who like to play round with osm2xp and also test their own changes without waiting for a new download. I've been playing round with a few scripts to try and speed up the processing and add in the forest relations, and I've managed to drop the processing time down for the british-isles scenery from about 4 hours to 30 minutes. I'm adding the instructions here for anyone who is clued up on using the command-line, and hope it will be useful. The osmconvert and osmfilter tools are available freely from the osm wiki site.

 

Firstly convert the downloaded osm.pbf file to o5m using osmconvert

 

osmconvert --drop-author british-isles-latest.osm.pbf -o=british-isles-latest.o5m

 

Then run it through a filter, removing all the stuff osm2xp doesn't use, e.g. roads, names, postcodes, etc.

 

osmfilter british-isles-latest.o5m --drop-tags="addr:city= source= source_ref= created_by= addr:housenumber= addr:postcode= addr:post_code= addr:street= phone= amenity=parking name= building:house_number= building:house_name=" --keep="building=* amenity=place_of_worship man_made=lighthouse man_made=water_tower man_made=crane man_made=windmill landmark=windmill man_made=tower man_made=mast man_made=chimney landmark=chimney generator:source=wind natural=wood natural=scrub landuse=forest landuse=allotments natural=forest leisure=garden leisure=park landuse=cemetery" --drop-relations --out-o5m > british-isles.o5m

 

Then, I process the file again to pull out forest relations

 

osmfilter british-isles-latest.o5m --keep= --keep-relations="type=multipolygon and ( landuse=forest or natural=wood or natural=forest of leisure=park or leisure=garden )" -o=forests.osm

 

I then use the following simplistic ruby script to add forest tags to the forests.osm file. It seems to do the trick and gives more woodland, although probably not correctly as it applies the tag to all items.

 

f = File.open("woods1.osm","w")

File.open("forests.osm", "r").each_line do |line|

  data = line.gsub("</way> "," <tag k=\"landuse\" v=\"forest\"/>\n</way> ")

  f << data

end

f.close

 

Finally, I combine the forests.osm file with the british-isles.o5m file, and convert it back to the original pbf file (ready for osm2xp)

osmconvert woods1.osm -o=woods1.o5m

osmconvert --drop-author woods1.osm british-isles.o5m -o=british-isles.osm.pbf

 

The whole process above takes about 40 seconds and is all in a single script so it's automated. On the entire europe download, it takes about 15 minutes to run, but reduces the filesize down quite significantly, and as well as adding forest relations (i.e. More forests), it also massively decreases the processing time inside osm2xp.

 

Unfortunately it dosn't seem anyone is actually developing osm2xp further, and currently the following features IMO are needed to move it forward:

1. Support for relations/multipolygons, i.e. For large forest areas or buildings with inner court yards.
2. Support for advanced options, such as changing the building facade or object based on a number of criteria (not just a single tag). e.g. You might want a specific phone mast with a height tag to use another object file. Or for houses under a certain height to use a specific facade.
3. Scripting. I would like to be able, for example, to tell osm2xp to only add streetlights to roads if the road is in a residential area. This would require the program to know that the current road is inside a residential zone, etc and isn't a trivial thing.

I've had a look at the osmosis c library for OSM, and have written a very simple program to run through an OSM file and fetch forests and woodlands and produce X-Plane DSF files. It also deals with relations and multi polygons, and also forests crossing boundaries. Osmosis is a pretty powerful library and could easily (for someone with the time), be programmed to create some pretty advanced scenery for X-Plane. I'm really hoping development picks back up on osm2xp, because the OSM data has so much potential in well populated areas, and the libraries available are improving everyday.

 

Couple more ideas I will try on my next europe OSM run,

 

1) Only extract residential streets and motorway junctions, and then OSM2XP will only generate street lights just for those. This should make it look more realistic in some instances, where only roads with houses and motorway junctions are lit. This will slow processing down a little bit, but should give a nice effect at night.

2) Add small shrubs to the landuse=residential (treat it as a forest with very low density). This should add the odd small tree in residential areas aka. like orbx do. However, the chances of a tree being inside a house are high, so it might look odd. But worth a try.

3) Often shrubs and trees and found along drains/ditches. Maybe, I can split these up into their individual nodes using osmfilter and place a tree at each node. 

 

 

 

 

 

 

Added some screenshots with the results of my last run, slowly improving

 

http://forum.avsim.net/topic/429466-buildings-and-town-warning-large-images/

Since development on OSM2XP seems to have stalled (Last change was back in July), I decided to see if there is anything I can do get multipolygons and polygons crossing tile boundaries working. Unfortunately I had problems getting the code to run and compile on my Mac, and it would require some work to get it to support the requested stuff. So I have started to write my own tool, concentrating just on forests at the moment.

 

In the below screengrab from OSM, there are a few forests shapes, two of them have holes in the middle, and the border shading indicates DSF tile boundaries inside X-Plane. When using OSM2XP, only the small square forest on the right would work, the others would not show up (after trying various things).

 

 

The tool I'm working on takes a different approach than OSM2XP, in that it makes several initial passes of the OSM file to get geometry information. X-Plane can handle multipolygons for forests, but for items on tile boundaries I had to break the multipolygons up into smaller shapes, which seems to produce the desired effect:

 

 

So far, the tool does the following:

 

• Supports multipolygons and relations in OSM (Lots of forests in Europe are done like this). 
• After the initial passes of the OSM file, only the tiles that are needed are generated. Empty tiles (without anything drawable in X-Plane) aren't processed. Speeding up processing time for large maps.
• Allows advanced rules, for example you can determine what type of forest to draw based on a number of tag combinations. This can also be used for using different models for different types of churches, masts, etc based on height, and other criteria.

There is still a lot of work required to get it user friendly and configurable, but hopefully it will come in useful for scenery developers in the future who need much more configurability than currently allowed.

 

Next challenge is multipolygon buildings, which, although the specs say aren't supported, it seems to work :-)

 

 

I've now added initial building support in, and have generated central Berlin:

 

 

 

 

Berlin is a good example as there are many buildings with holes, but also lots of the bulidings have detailed information attached to them, e.g. height, material, color. Also, lots of the parks and woodland are built up of multipolygons which have come across quite nicely.

 

However, buildings with more than one hole inside don't render too well inside X-Plane, so at the moment I triangulate them (Which works, but looks ugly). I need to find a better way to do this, so possibly splitting the building into 2 or 3 parts, removing the holes may work (but is complex to do).

 

The OSM data really could do with a cleanup, lots of areas are unclosed, e.g. Forests, or have duplicate coordinates which makes parsing fun.

 

 

Man that is some serious stuff! You software engineers never cease to amaze me.

Could your process be able to generate something like say Monument Valley Utah?

 

 

Could your process be able to generate something like say Monument Valley Utah?

 

It only generates buildings, trees and points of interest, and not actual terrain. Also, the US is a bit behind Europe in terms of information on OpenStreetMaps. So whilst some areas such as Chicago and New York are really well populated, other cities are almost completely empty. It's improving everyday though :-)

Hey man you doing great, seriously too good.

Tony, I figured if it could make a 3D building out of a flat image then it might be able to generate the individual mesa's in Monument Valley since it is not an entire terrain, just individual models.

Monument valley is an area that has never been done very well in any version of FS. The photoreal stuff is ok but the terrain detail needed to accurately model the mesa's is not there since they are small compared to say a mountain. They are more like extremely large buildings.

You could in theory create yourself a facade for a mesa and then map them in OSM using whatever tag is appropriate for it.. You'd set the height, and then the program would draw a solid object to fill the flat polygon you drawed. But, I'd imagine it would look bad, since it would have to be completely vertical, with no slanting edges.

 

The other approach is to model a couple of mesas as 3D objects using the .obj format (Which is pretty easy to do), and then they could be inserted into the scenery. I'm not sure if the mesas are tagged in OSM, but if they are, then the program could be told to insert your object whenever if finds a mesa. But this all depends on how well the information is tagged and the range of models you have.

Ahh now I understand why, the slanting edges and shapes kill it.

 

Thanks Tony!

Some more screenshots. Reducing the number of red roofs, and replacing some of the facades with real objects.

 

 

Ooops wow.

 

Wouldn't it be possible to import google warehouse based on geo location, i know you development does not concern google warehouse but that stuff looks neat  ? Sorry i am asking this as i don't have knowledge about the osm or scenery things don't understand graphics. 

Yes it could, but the problem with Google Warehouse objects is that each object has multiple texture files, whereas X-Plane only supports one object with one texture sheet. There is a converter for these objects, but it requires the textures to be sorted into one file and the UV coordinates remapped. I don't think there exists an automated way of doing this yet.

 

Also, somebody has already tried this (http://members.ferrara.linux.it/cavicchi/GMaps/). What he does is create multiple objects for each face/texture. Although it works (sometimes), the objects are really bad on the framerates, and huge. Generation also takes forever.

 

Never say never though, maybe one day :-)

Tony, first of all I just want to say your works looks really awesome !

 

I wondered what kind of objekts does your script link into the generated scenery, just the X-Plane Standard ones or also objects from librarys like OpenSceneryX? Using objects from OpenSceneryX could make the impression much more realistic since most american buildings don't really fit into Europe or even Asia .

 

Also it would be really cool if you could tell us what your plans with this tool are? Are you thinking of earning money with it (by selling generated scenerys) or do you consider publishing it as open source on a platform like github (which would enable other coders to send in patchs to support other tags in OSM for example)? The later one would be really awesome and I think this tool could become a very powerful scenery generator to make x plane even more awesome .

Yes it could, but the problem with Google Warehouse objects is that each object has multiple texture files, whereas X-Plane only supports one object with one texture sheet. There is a converter for these objects, but it requires the textures to be sorted into one file and the UV coordinates remapped. I don't think there exists an automated way of doing this yet.

 

Also, somebody has already tried this (http://members.ferrara.linux.it/cavicchi/GMaps/). What he does is create multiple objects for each face/texture. Although it works (sometimes), the objects are really bad on the framerates, and huge. Generation also takes forever.

 

Never say never though, maybe one day :-)

 

 

Hey, I used to try ModelconverterX and scenproc for Fsx had only succeeded once that was last year, don't how it had worked. Maybe you give that a try. Of not scenproc but Modelconverterx.