New interview with Jörg Neumann v.2

[Nedo68 533]

Hello another interesting interview with Jörg Neumann (in german language only sry.)

you can find it here: Die Liebe für den Flight Simulator war immer da.

Maybe someone can translate it? (i have no time right now)

[ArchStanton 156]

"Love for the Flight Simulator was always there."

The procedurally created earth, the source code of the last MS flight simulator and a commonality with Star Citizen: Golem.de talked to Jörg Neumann about technology and flying. Neumann leads the development of the Flight Simulator at Microsoft.
An interview with Peter Steinlechner

The German developer Jörg Neumann is responsible for the development of the new Flight Simulator at Microsoft. Neumann has previously worked for a studio founded by Chris Roberts (Star Citizen) called Digital Anvil, which was purchased by Microsoft in 2000 - this is how Neumann joined the software group. The Flight Simulator was developed for Windows PC and Xbox One in a cooperation between Microsoft and the French development studio Asobo. There is no release date yet.

Golem.de: Why is a new flight simulator being developed after all these years?

Jörg Neumann: I'm responsible for that. I've been making games for ages, among other things I used to work with Kudo Tsunoda on games for Kinect motion control. Then Kudo was busy with the Hololens and I was one of the first employees in the responsible department.

We had a program called Holo Tour that allowed us to show parts of the world in 3D within Virtual Reality. Rome, San Francisco and the Inca city of Machu Picchu. I've never been to Peru before, but especially with Machu Picchu, I thought it felt incredibly real.

Then we tried to see if we could make a bigger place with the same graphics quality. I asked around inside Microsoft and got data from Seattle - about 25 GByte, from Bing. The data recorded from the plane was photogrammetric - that is, 3D point clouds that can be displayed as a mash.

Golem.de: But why a flight simulation?

Neumann: With such data, it is best to look at them as they were recorded. So we let a virtual Cessna 172 fly over virtual Seattle. I tried that with two programmers and an artist and it worked.

I showed the result to Phil (Phil Spencer, head of games at Microsoft). And he asks me why I'm showing him a video of Seattle. So I took the controller and turned the plane to the right. Only then did he understand that it was all happening in real time.

Golem.de: Was it difficult to push through the project after the meeting with Phil?

Neumann: No, that wasn't a big problem. By the way, there are quite a few pilots at Microsoft, the love for the Flight Simulator was always there. After all, the series is older than Windows and Office - the oldest thing we have at all.

 Golem.de: Then why wasn't there a new flight simulator from Microsoft before?

Neumann: There was simply no team. Sometimes several things also have to come together. In this case it was the new possibilities influenced by Hololens, among other things. In the last Flight Simulator there was also no cloud and much less data at all.

Golem.de: Which map data from Bing do you use? The map view that I can access via my browser?

Neumann: At Bing we have satellite data, which we call Area Photography, which go up to 30 centimeters per pixel. The better images come from airplanes, they have several cameras and take one image per second, the resolution is then 5 centimeters per pixel. From these points you can build the cities.

But then it gets complicated. For example, skyscrapers have a dark side, and you can't use that in a flight simulation. When the sun goes around 360 degrees, you have to solve a lot of problems.

Golem.de: Is the world created completely algorithmically or is it reworked by hand?

Neumann: There are about 300 sights, like the Space Needle in Seattle, which we build manually - but it was really difficult to make the choice. Otherwise, we use algorithms.

We work a lot with machine learning. In our team there are people who have written their doctoral thesis on topographic analysis of roofs. It analyses the roof so that you can see exactly how the gable is assembled. We take the floor plan from the satellite photos. The shadow lengths tell us how high the building is. In principle, we build 1.5 billion houses in the cloud every day. They don't exist on a hard drive - they don't actually exist anywhere.

Golem.de: They're not buffered either?

Neumann: No. It's the same with the trees. There are two to three trillion trees on earth. We have algorithms that search for trees in photos. We also know which tree species exist where. We have 3D trees and we plant them in the right places all over the world.

Golem.de: What other data does the Flight Simulator use?

Neumann: We use real weather data in real time. And every airplane that is currently on the move can also be found in the air with us. You can fly there yourself and watch the other airplane fly over the clouds.

Golem.de: Which engine do you use?

Neumann: I don't know if it has a name, but it comes from Asobo. I've been working with Asobo for twelve years, first on a Kinect game called Pixar Rush. Around 2008 Asobo already made a game called Fuel, in which the whole world was procedurally worked out on the basis of satellite data. There are very few engines that can do this. Unity and Unreal Engine can't do that.

My friend Chris Roberts rebuilt the Cryengine to 64 bit and Double Floating Point for Star Citizen, and it's still difficult for him. The good thing about the Asobo engine is that it has always been good in this area. In the meantime, however, we have also switched to Double Floating Point.

Weather data and computer resources

Golem.de: A good world generation engine is not necessarily a good flight simulation engine.

Neumann: Right. And Asobo hasn't produced any flight simulation before. When we started talking about the project, the boss immediately booked test flights for his employees. 20 or 30 people of them have taken the pilot's license in the meantime. We need that, you can't do a simulation like that without knowing how flying works.

We also got them the old flight simulator from 2007. There they could see how it was all put together. From day one we had Air Traffic Control in our program, and the airports were there too - not all right, but at least.

Golem.de: Were elements from the Flight Simulator taken over from that time?

Neumann: We took over the code at the beginning, but then rewrote everything. We started with that in 2016, so we're working on it now for about four years. But that's necessary because the aerodynamics, for example, were totally outdated. But the computers at that time couldn't do multithreading either.

Now we have over a thousand control points per plane, we have all the wind vectors. This means that each of our aircraft has around a thousand surfaces for which we calculate the aerodynamics.

Anyone who has ever flown a plane knows that flying is a bit like swimming. But in the early flight simulators, airplanes were very rigid. They were basically a block with a point that you could turn left or right. But airplanes don't work that way.

Golem.de: The weather comes in real time from a weather service. Is there a storm, a cloud or a hurricane where every pixel is reproduced, or do you have some kind of wind shader?

Neumann: We divided the world into squares with a side length of 64 meters each, up to a height of 65,000 feet. For each of these cubes there are values for humidity, air pressure, which particles float around and other sensor data. This also means that the air is moving in the right wind direction and strength.

The clouds in the Flight Simulator consist of voxels and are represented by raymarching - a kind of raytracing. In addition, the clouds are generated dynamically. When it rains and gets warmer, it generates lift.

Golem.de: And that still runs fast enough on a computer available in the trade?

Neumann: Flight simulators require comparatively few resources for artificial intelligence. Normally the CPU in games is used to calculate the opponents and the NPCs. We have an airplane - and a few other airplanes - but they're usually 30 or 100 kilometers away.

We use the computing power of the CPU in principle for the instruments as well as for the autopilot. Everything else is done by the GPU. And this is 'many thread' and offers a lot of computing power.

 Golem.de: Is there a casual mode?

Neumann: No, not at all. We don't want to do this as a game. Our target group are people for whom flying is a hobby. They know a lot about it.

Golem.de: If I can fly in the Flight Simulator, then also a real 747?

Neumann: Many pilots tell us that they have learned to fly with the Flight Simulator. You get very close to real flying. However, our goal is that everyone who has the dream to fly should be able to do it in the Flight Simulator. There are a few planes, like the Piper or the Cessna, which are very easy to fly. You can learn that in a few days.

Golem.de: How important are mods? Especially in Germany they were and are very popular with fans of the previous flight simulators.

Neumann: Absolutely! There are about 240 companies for professional mods. I talked directly when I was 60 or so. We know what they want and need, and we're working on an SDK. Then the developers can also give us feedback. Some people do that when they visit us. I see the Flight Simulator as a platform, not as a game or simulation. And as a platform you have to be very careful that those who are involved are fully involved.

Golem.de: How many people work on Flight Simulator?

Neumann: About 200. At Asobo we have about 120 people. In addition, there is another group of 20 to 30 people at Microsoft who help us with machine learning and a few smaller teams, including in Austria.

Golem.de: How far has development progressed?

Neumann: We're in Pre Tech Alpha. The simulation is there, you can do anything. But not all planes are ready yet. For the 47,000 airports, we would like to go through the data and correct them if necessary. We are still working on accessibility. There is always something to do. We simulate the world, you can go into endless detail."

[Bottle 663]

Quote

We use the computing power of the CPU in principle for the instruments as well as for the autopilot. Everything else is done by the GPU

Interesting information. My PC upgrade planning just got a lot simpler. I think I'll wait for the RTX 3000 series cards to drop and may forego a new CPU (had assumed a higher core count would give better results, but it's looking more like it will be GPU-limited).

[LB777 100]

3 hours ago, ArchStanton said:

Neumann: Flight simulators require comparatively few resources for artificial intelligence. Normally the CPU in games is used to calculate the opponents and the NPCs. We have an airplane - and a few other airplanes - but they're usually 30 or 100 kilometers away.

We use the computing power of the CPU in principle for the instruments as well as for the autopilot. Everything else is done by the GPU. And this is 'many thread' and offers a lot of computing power.

But I was told countless times that flight simulators were the absolute pinnacle of software and that in comparison video games were a trivial matter that required no processing power at all as they were just displaying the sky, some trees and a gun.

Or could it be that the people preaching this were completely clueless??

[dal330200 216]

200 people on the team, met with 60 developers already, wow.  This is going to be big.

[simtom 495]

TL;DR:

• Jörg Neumann once worked for Digital Anvil (founded by Chris Roberts (Star Citizen);

• they had to solve the issue of dark sides of buildings;

• around 300 landmarks are handmade, the rest is made by using algorithms;

• they build 1.5 billion houses in the cloud every day;

• every airplane that is currently on the move can also be found in the simulator. You can fly and watch the other airplane fly over the clouds;

• engine is from Asobo and is using Double-precision floating-point;

• they had ATC from day one working in MSFS (taken from FSX);

• they used code from FSX at the start but then rewrote everything (starting in 2016);

• the world is divided into squares with a side length of 64 meters each, up to a height of 65,000 feet. For each of these cubes there are values for humidity, air pressure, which particles float around and other sensor data. This also means that the air is moving in the right wind direction and strength;

• the clouds are made out of voxels and are rendered with raymarching - a kind of raytracing. In addition, the clouds are generated dynamically. When it rains and gets warmer, it generates lift;

• CPU is used for instruments and autopilot. Everything else is done by the GPU, this offers a lot of computing power because it can use multi threading;

• Is there a casual mode? Neumann: No, not at all. We don't want to do this as a game. Our target group are people for whom flying is a hobby. They know a lot about it;

• How important are mods? Neumann: [...] I see the Flight Simulator as a platform, not as a game or simulation;

• Around 200 people are working on this sim, 120 from Asobo and 20 to 30 people at MS (they help with machine learning). There are also a few smaller teams, including in Austria;

• How far has development progressed? Neumann: We're in Pre Tech Alpha. The simulation is there, you can do anything. But not all planes are ready yet. For the 47,000 airports, we would like to go through the data and correct them if necessary. We are still working on accessibility. There is always something to do. We simulate the world, you can go into endless detail.

[Greazer 1,320]

4 hours ago, ArchStanton said:

My friend Chris Roberts rebuilt the Cryengine to 64 bit and Double Floating Point for Star Citizen, and it's still difficult for him. The good thing about the Asobo engine is that it has always been good in this area. In the meantime, however, we have also switched to Double Floating Point.

I hope he has the same vision and that the flight sim world will be persistent, i.e. bush fires leave burnt out land and snow covers land for weeks etc.

[shivers9 1,174]

Well this interview just blows my mind! This is going to be a simulator that is just going to get better everyday! At this point, I think that this will be much much more than just a flight simulator at it's core. It is going to become a very detailed simulation of our world in near real time that with the help of AI can just become more accurate and detailed as each day that passes. It can be packaged to cover Train, Truck, Ship simulators. Things like military, disaster recovery, search and rescue just to mention a few will fit right into this simulated earth on the cloud. We just happen to be lucky enough to be in a hobby that just like Prepar3d and LM did....we are going to be the test bed and provide a huge on going Beta test group for a product that is going to make zillions of dollars for Microsoft. Their long range plans are for much more than a new x-box game.

[Krakin 5,728]

2 hours ago, LB777 said:

But I was told countless times that flight simulators were the absolute pinnacle of software and that in comparison video games were a trivial matter that required no processing power at all as they were just displaying the sky, some trees and a gun.

Or could it be that the people preaching this were completely clueless??

Oh I'm right there with ya on that one, buddy. I remember being told that the whole world was being calculated all the time and I should be amazed the game was running at all.

[Dominique_K 10,133]

2 hours ago, Bottle said:

Interesting information. My PC upgrade planning just got a lot simpler. I think I'll wait for the RTX 3000 series cards to drop and may forego a new CPU (had assumed a higher core count would give better results, but it's looking more like it will be GPU-limited).

Waiting for the completion of the optimization process is prudent before drawing conclusions in terms of hardware purchase.

[Dominique_K 10,133]

A side note, my German is very rusty so I used the Google translating aid to English. I am stunned, once more, to see the progress automatic translation has made these recent years. Things we saw 15 years ago coming in a uncertain distant future are here now ! It doesn't work as well for every language but in that case it is really good.

Edited November 29, 2019 by domkle

[ca_metal 2,015]

4 hours ago, LB777 said:

But I was told countless times that flight simulators were the absolute pinnacle of software and that in comparison video games were a trivial matter that required no processing power at all as they were just displaying the sky, some trees and a gun.

Or could it be that the people preaching this were completely clueless??

Who do you prefer to believe, on the developer or on radom forumers that knows everything? People wanted to believe there was nothing to do to improve the performance of the simulators, this way they could just accept things the way they were. 

It was just odd the flight simulators were so different from the other games on the market (yeah... yeah... it's not a game, it's a simulator...).

[Jean-Jacques 120]

2 hours ago, domkle said:

A side note, my German is very rusty so I used the Google translating aid to English. I am stunned, once more, to see the progress automatic translation has made these recent years. Things we saw 15 years ago coming in a uncertain distant future are here now ! It doesn't work as well for every language but in that case it is really good.

Modesty at its best! Sincere thanks for undertaking this task. I'm certain you make it sound easier than it actually was.

A very interesting read!

Regards, JJ

[A319c@r 12]

I did read it in my native german language. As I read, MSFS2020 is done and ready so far, just few things here and there to tweak and some aircrafts needs to getting done as well, but at least it is done, so I would say, roll out the rech alpha1,2,3,4......😉☺

[Nedo68 533]

a nice christmas present 😁