Jump to content

Sign in to follow this  
FlyIce

Removing baked shadows in Bing images by AI?

Recommended Posts

10 minutes ago, Claviateur said:

Maybe I am mistaken, the AI does not only fetch the data (similar to OSM) but has image processing and 3D generating capability in its deep learning process. 

The devs have stated that Azure is used for buildings and trees.  Here is a video where they talk about using Azure to create the buildings.  https://youtu.be/Q4pKRjNhQKs?t=430

 

Edited by GlideBy

My brother has Coronavirus, confirmed by testing. On 3/26 he almost didn't survive the night.  He had extreme trouble breathing and was given a steroid inhaler.  He was very weak, had nausea, and other issues that aren't pleasant.  As of this update he is feeling much better, and seems he will be fine.  Stay safe out there.  (Updated 4/6)

Share this post


Link to post
Share on other sites
1 minute ago, GlideBy said:

The devs have stated that Azure does building and trees.  Here is a video where they talk about using Azure to create the buildings.  https://youtu.be/Q4pKRjNhQKs?t=430

 

Yes I saw this video and I just heard again his explanation. I think he is saying exactly what I explained in my post above, Azure AI analyzes and brings the data. I don't think he said the AI creates the assets.

You know, if the Azure AI retouches imagery and creates 3D elements over a map, MS would have already used it to make Bing imagery seamless and have worldwide 3D infrastructure, forests, powerlines etc etc....And score some points against Google Maps 🙂

 


________________________________
LEBOR SIMULATIONS

Scenery for Flight Simulators since 1998

Share this post


Link to post
Share on other sites
3 minutes ago, Claviateur said:

I don't think he said the AI creates the assets.

Here are links, and articles showing that Azure and/or AI to create building footprints.  These are not specific to MSFS though.

https://azure.microsoft.com/en-us/blog/how-to-extract-building-footprints-from-satellite-images-using-deep-learning/

https://github.com/microsoft/USBuildingFootprints

https://blogs.bing.com/maps/2018-06/microsoft-releases-125-million-building-footprints-in-the-us-as-open-data

In other interviews the MSFS devs have talked about how they can know a buildings height based on the shadow.  There was another interview where they mention about Azure creating the buildings and trees. 

So all that, and more which I am not going digging for, leads me to believe they are using AI to create buildings, trees, and probably more. 

 


My brother has Coronavirus, confirmed by testing. On 3/26 he almost didn't survive the night.  He had extreme trouble breathing and was given a steroid inhaler.  He was very weak, had nausea, and other issues that aren't pleasant.  As of this update he is feeling much better, and seems he will be fine.  Stay safe out there.  (Updated 4/6)

Share this post


Link to post
Share on other sites
20 hours ago, n4gix said:

MSFS has a binary star system modeled! :laugh:

Guys, this is just a preview of the DLC:

Microsoft Flight Simulator: Tatooine

Edited by TechguyMaxC

Share this post


Link to post
Share on other sites
41 minutes ago, Claviateur said:

Yes I saw this video and I just heard again his explanation. I think he is saying exactly what I explained in my post above, Azure AI analyzes and brings the data. I don't think he said the AI creates the assets.

You know, if the Azure AI retouches imagery and creates 3D elements over a map, MS would have already used it to make Bing imagery seamless and have worldwide 3D infrastructure, forests, powerlines etc etc....And score some points against Google Maps 🙂

 

The AI is definitely color correcting, removing shadows, removing clouds, and creating buildings dynamically based on ortho footprints. We’ve seen all that in action. It’s impossible that it’s all been done by hand.

MS has no reason to do that for Bing maps, though perhaps they will one day.

 

 

Share this post


Link to post
Share on other sites

Bing Map will probably prefer to keep the original untouched data as much as possible, as long as MSFS algorithms are experimental.
It's great to have unwanted shadows automatically removed for a flight sim, but the AI may also occasionally erase dark areas it mistakes for shadow - such artefacts are fine within a flight sim, but it's not for Bing Map which has to remain as accurate as possible, even if that means some data is partially occluded, as long as artefacts aren't artifically introduced. For the same reason, Bing Map may prefer to keep the original, non color-contiguous areas, as smoothing those colored borders could potentially lead to artefacts or loss of information.
I would not be surprised if those great algorithms make their way to Bing Map eventually, but not before several years of refinement and testing.

Edited by divide

Share this post


Link to post
Share on other sites

Regarding shadows being removed, it also depends on the dynamic range of the imagery. Especially under harsh lighting conditions, the shadow details could be totally clipped, or lots of noise would be introduced when bringing up the brightness in those spots, which would create unsightly artifacts. Until AI is trained to completely reconstruct those areas (perhaps using other versions of the imagery at different times of the day), there will not be a perfect solution to this.

Share this post


Link to post
Share on other sites
20 minutes ago, ChaoticBeauty said:

Until AI is trained to completely reconstruct those areas

They use AI to create/recreate areas in the source satellite images that are hidden behind clouds.


My brother has Coronavirus, confirmed by testing. On 3/26 he almost didn't survive the night.  He had extreme trouble breathing and was given a steroid inhaler.  He was very weak, had nausea, and other issues that aren't pleasant.  As of this update he is feeling much better, and seems he will be fine.  Stay safe out there.  (Updated 4/6)

Share this post


Link to post
Share on other sites
6 minutes ago, GlideBy said:

They use AI to create/recreate areas in the source satellite images that are hidden behind clouds.

Thanks for the info. Since clouds can cover a much larger part of the imagery, it will be interesting to see how accurately the AI can reconstruct those areas.

Share this post


Link to post
Share on other sites
55 minutes ago, ChaoticBeauty said:

Thanks for the info. Since clouds can cover a much larger part of the imagery, it will be interesting to see how accurately the AI can reconstruct those areas.

For an example of how this might work you can read about how Google did it. Basically, if you have enough images of an area taken at different times, you can work out what is cloud and what isn't.

https://www.smh.com.au/technology/google-maps-removes-clouds-puts-ground-in-focus-in-700trillion-pixel-upgrade-20160628-gptnuh.html

On a smaller scale, you can use image stacking to remove people from photos by taking multiple shots. A number of image editors these days will automate it for you.

https://photoshoptrainingchannel.com/remove-tourists-stack-mode/

  • Like 1

Share this post


Link to post
Share on other sites
3 hours ago, GlideBy said:

Here are links, and articles showing that Azure and/or AI to create building footprints.  These are not specific to MSFS though.

https://azure.microsoft.com/en-us/blog/how-to-extract-building-footprints-from-satellite-images-using-deep-learning/

https://github.com/microsoft/USBuildingFootprints

https://blogs.bing.com/maps/2018-06/microsoft-releases-125-million-building-footprints-in-the-us-as-open-data

In other interviews the MSFS devs have talked about how they can know a buildings height based on the shadow.  There was another interview where they mention about Azure creating the buildings and trees. 

So all that, and more which I am not going digging for, leads me to believe they are using AI to create buildings, trees, and probably more. 

 

I shared these links already in multiple posts 🙂

And I think we are saying the same thing here.

Yes, this is what the AI does, analyzes and fetch the footprint data and its metadata (height, type etc).

But the 3D assets construction out of this data is done procedurally (not manually) most probably with Asobo's in-house tools or with the engine built in procedural tools.

I use similar tools to generate autogen from footprint data for XP11. For an area that cover 4 tiles (i.e: 10500 km2) It takes about 3 min to generate a complex ready to use autogen package with all you can immagine: buildings, houses, forests, roads, railways, power lines etc and all these sit perfectly on the photoreal scenery you generate with another tool... 

A similar tool Arno G. created for FSX as far as I can I remember it was called Sceneproc. It uses OSM data as well to create procedural autogen for FSX.

Now the only difference here is that Azure AI brings all the things you mentioned above with metadatas and Asobo's tool create the 3D world with it...


________________________________
LEBOR SIMULATIONS

Scenery for Flight Simulators since 1998

Share this post


Link to post
Share on other sites
11 minutes ago, goates said:

For an example of how this might work you can read about how Google did it. Basically, if you have enough images of an area taken at different times, you can work out what is cloud and what isn't.

https://www.smh.com.au/technology/google-maps-removes-clouds-puts-ground-in-focus-in-700trillion-pixel-upgrade-20160628-gptnuh.html

On a smaller scale, you can use image stacking to remove people from photos by taking multiple shots. A number of image editors these days will automate it for you.

https://photoshoptrainingchannel.com/remove-tourists-stack-mode/

Thanks for the links. It is impressive that this could be done when the articles were published four years ago.

Share this post


Link to post
Share on other sites
1 hour ago, ChaoticBeauty said:

Thanks for the links. It is impressive that this could be done when the articles were published four years ago.

The concepts and functionality go back further than that. Here's one article from 2007 about removing people from images, for example.

https://photoshopnews.com/2007/03/27/image-stacks-in-photoshop-cs3-extended/

What makes the new sim impressive is that they're combining a number of different technologies, plus modern hardware, and applying it at a much larger scale.

  • Like 1

Share this post


Link to post
Share on other sites

I see double shadows on some building in this image.  Look at the plane tail, you just see a short shadow that doesn't even cover the fuselage.  Then look at the taller buildings in the lower right, not the ones on the water.  I see a short shadow at a 3:30 o'clock (115 degree), and a long shadow at 5 o'clock (150 degree).  The plane shadow seems to correspond to the short shadow at 3:30.

 

Velkome-to-MIAMI-2.jpg

 



My brother has Coronavirus, confirmed by testing. On 3/26 he almost didn't survive the night.  He had extreme trouble breathing and was given a steroid inhaler.  He was very weak, had nausea, and other issues that aren't pleasant.  As of this update he is feeling much better, and seems he will be fine.  Stay safe out there.  (Updated 4/6)

Share this post


Link to post
Share on other sites

Just to add a little note based on the latest (Airport) Video from Asobo.

The dev mentioned the word "Algorythms" during his explanations. This is exactly what I was refering to... In-house tools programmed to process the data and generate content/assets for the engine.

Then he talked about the thousands of airports they had to work on somehow "manually"... The AI did not do that...

But there are chances the AI detected the airports and airfields on the Bing Imagery and built an additional data for Asobo to work with (in addition to the FSX data) just like it did for footprints, vegetation zones etc etc...

The AI is already doing a super job of dectection and data collection on the world scale... Imrpessive.

If we think about it, it's the equivalent of many years of OSM mapping by (human) volunteers and the OSM data is not consistent across the globe since not every conuntry has the same dedicated volunteers to map their region...

 


________________________________
LEBOR SIMULATIONS

Scenery for Flight Simulators since 1998

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Tom Allensworth,
    Founder of AVSIM Online


  • Flight Simulation's Premier Resource!

    AVSIM is a free service to the flight simulation community. AVSIM is staffed completely by volunteers and all funds donated to AVSIM go directly back to supporting the community. Your donation here helps to pay our bandwidth costs, emergency funding, and other general costs that crop up from time to time. Thank you for your support!

    Click here for more information and to see all donations year to date.
  • Donation Goals

    AVSIM's 2020 Fundraising Goal

    Donate to our annual general fundraising goal. This donation keeps our doors open and providing you service 24 x 7 x 365. Your donation here helps to pay our bandwidth costs, emergency funding, and other general costs that crop up from time to time. We reset this goal every new year for the following year's goal.


    48%
    $12,200.00 of $25,000.00 Donate Now
×
×
  • Create New...