PMDG - still no proper DME ARC

[virtuali 4,421]

1 hour ago, lwt1971 said:

What's key to note in what Aamir says is this: "It's worth mentioning, of course, that we use different parts of the SDK owing to the different development approaches. The limitations imposed on other developers may not apply - and vice versa"

That's the key sentence.

Integration with external websites is surely possible using, for example, Javascript. And it's surely not possible using a "pure WASM" approach, like PMDG so, in a way, they are both correct.

The main reasons why a developer would decide to use a "pure WASM" approach ( meaning no external .EXEs required ) are:

- Being accepted on the Marketplace. MS doesn't approve anything that requires an .EXE module.

- Eventual Xbox compatibility. While there are still issues with WASM on Xbox, which were discovered by PMDG with the DC-6, and seem to be mainly caused by the inability to test on Xbox in real world conditions, I'm sure this will eventually fixed someday, which means Xbox users will likely be able to buy the 737 too.

- Raw speed. Sure, Javascript has improved a lot in recent years, but C++ still has the edge, and the reason why it takes so much time to start a WASM airplane, is because when the code it's being run for the first time ( or the sim is updated ), all WASM code is compiled to native, so it's basically equivalent to a real x86/x64 .EXE/DLL, and that's still way faster than Javascript will ever be.

- Ease of portability for existing code. If you have tons of existing C++ code ( and "existing" doesn't automatically mean "bad" ), porting it to WASM is reasonably easy (provided you haven't used exceptions too much...), but converting to Javascript can be extremely complex, not so different than a complete rewrite. And you'll end up with a code that will never run as fast as the previous one, and perhaps you discover that only *after* doing all that work. What if it's too slow ? Work for months to port to JS, and then go back to C++ ? That would be a commercial suicide that could cause any company to go bankrupt.

Right now, the pure WASM approach seem to work, considering how good performance are with the 737. Perhaps they might consider doing *something* with Javascript, to achieve some integration with sites like Navigraph or Simbrief, but the issue is, the way the SDK is made, it doesn't exactly makes it "easy" to integrate those different sub-systems, which sometimes just don't talk with each other, or even if they do, they do it in a clunky/inefficient way. For example, lots of interesting things can be done from Javascript + Coherent but...Coherent can't be called from Simconnect, and Simconnect is basically the only way a WASM module can communicate with the sim or the outside world in general, since it's heavily sandboxed. There are workarounds, but they are inefficient/slow/bad to maintain.

In addition to that, the most important NEW part of the whole sim, that is the HTML/JS subsystem, is still completely undocumented. While this might not be so much of a problem for a freeware product, it's really a no-go for any serious commercial developer. How you are supposed to *support* users and reacts to bugs and/or sim updates, if you based your product entirely on undocumented features ? Being undocumented, Asobo/MS are not bound by any backward compatibility, even between updates. They could redesign major parts of the HTML/JS framework, and if that would break dozen of products...that's too bad, but developers who decided to risk using the wholly undocumented sub-systems can't complain much, they should just try to fix their stuff. Again, this might not be an issue for freeware, but until that important part of the documentation shows up, I'd be wary of risking an high profile payware JUST to offer a couple of auxiliary features more, which are not really the core of the product.

So, I must say PMDG explanation about the lack of these features makes sense, and they are telling the truth, it's just a consequence of the design choices of the product, which are sensible, all considering ( here I'm biased, because I'm a C++ addict, and I consider JS to be as a script-kiddies evil, it's so easy to use that it's not even fun anymore.. ). If another developer made a plane using a different approach, possibly using external software ( I believe the Fenix A320 will require an external .EXE ) or Javascript, that makes sense as well, and doesn't make the PMDG explanation any less valid, every design choice has different benefits for both developers and users as well.

Pure WASM is fast, it's a clean approach and it will get you to the Xbox/MS Marketplace, and it's fairly transparent for users, with the only exception of that initial slow startup.

External .EXE give you both the absolute best speed (pure C++ code is a bit faster than WASM C++) and absolute best flexibility (you can do anything a Windows PC can do), and they are kind of safe too (if they crash, they won't crash the sim), but they might be annoying for users and/or being clunkier to use, and are susceptible to be attacked by bugged antivirus, sometimes even Notepad.exe gets blocked, go figure...

HTML/JS is the new wave, it's probably best if you are starting a new airplane from scratch, and you can find developers more easily, but you might discover some things might be slow and would require a WASM module anyway ( like the advanced autopilot in the FBW ). And, JS it's completely undocumented, not integrated with Simconnect, and a sim update can possibly break the whole product at any time.

 

Full disclosure: GSX will use ALL OF THEM. We have a tiny, lightweight, WASM module to do just a couple of calls we need, mostly to allow other products to integrate with us ( publishing L: variables to the outside world ), an external .EXE that runs the whole GSX code, so it can do lots of things ( including Simbrief integration, yes...) without being limited by anything in the SDK, and we are ALSO using HTML/JS for the UI and menus. But I sometimes envy airplane developers that can achieve almost everything they need to do, and they even have multiple choices available.

Edited May 14, 2022 by virtuali

[MySound 403]

@virtuali thanks for the great insight. Guess the community of pro-developers here will accept and appreciate this. 
 

 

not. 

[MySound 403]

Doublepost

Edited May 14, 2022 by MySound

[spesimen 135]

25 minutes ago, virtuali said:

 when the code it's being run for the first time ( or the sim is updated ), all WASM code is compiled to native, so it's basically equivalent to a real x86/x64 .EXE/DLL, and that's still way faster than Javascript will ever be.

can you elaborate on why the WASM needs to be compiled locally on each users machine instead of just distributing the binaries? like why not just have a amd and intel version precompiled and shipped with the product? is it optimizing the compile specifically to how many cores i have or other unique things?

 

[lwt1971 7,413]

Great insights and thanks @virtuali!! I suppose to develop without too many restrictions and still support the marketplace/XBox means going the Javascript/Html route which I guess is supported in the most "first class" way in MSFS currently. And for those add-ons being developed completely from the ground up that's the best way to go definitely.

For Javascript based add-ons in MSFS, the actual execution of code is being done in a dynamically-interpreted fashion like a script running in a JavaScipt engine, or being compiled to WASM first for fast execution? I thought it was the latter right? And it's great that WASM is only around 20% slower than native code execution (i.e. ML code in .exe): https://blog.sessionstack.com/how-javascript-works-a-comparison-with-webassembly-why-in-certain-cases-its-better-to-use-it-d80945172d79

I've coded and developed using various programming languages over the years including ML, Basic, C, C++, Java, JavaScript, Python etc and I guess unlike you tend to favor the Java way these days, especially with all the great JIT compilation technology out there now... I still love C/C++ in a way, but glad to not have to deal with pointers and double pointers any more 😉 Interesting to hear about the hybrid approach you guys are taking for GSX.

Edited May 14, 2022 by lwt1971

[virtuali 4,421]

15 minutes ago, spesimen said:

can you elaborate on why the WASM needs to be compiled locally on each users machine instead of just distributing the binaries? like why not just have a amd and intel version precompiled and shipped with the product? is it optimizing the compile specifically to how many cores i have or other unique things?

That's good question, for which I don't have a precise answer.

Usually, when something is compiled by-user, locally, security has something to do with it. Not from a DRM point of view ( as far as I can see, WASM binaries are not encrypted ), but with security at large.

The first thing that comes to my mind is, how do you spread a virus, if the WASM system is unable to just run pre-compiled binaries, but must always re-compile from the intermediate code ? Clearly, when you are controlling the compiler ( Flight simulator includes the compiler so, in a way, it IS the compiler ), you can fairly easily prevent to compile dangerous code, since the only thing you allow to compile, are the API you provided with that specific WASM implementation.

But please don't quote me on that, it's just an assumption I'm making.

Edited May 14, 2022 by virtuali

[virtuali 4,421]

5 minutes ago, lwt1971 said:

For Javascript based add-ons in MSFS, the actual execution of code is being done in a dynamically-interpreted fashion like a script running in a JavaScipt engine, or being compiled to WASM first for fast execution?

Almost all Javascript engines today use Just-in-Time compilation, meaning the source code is first parsed, then compiled, and then optimized, the JS engine in Chrome has two compilers, one to compile and another to optimize the code created by the first one. Not sure about the MSFS implementation, but I know they used Webkit, which is the engine mainly developed by Apple, and used by Safari, it's know to be very fast as well.

Note that, the binary code from Javascript is still not real x86/x64 ML, but it's a bytecode that is somewhat "similar", and it's executed by the Javascript VM. In this aspect, Javascript is closer to C#, but with the difference that C# compilation is completely Ahead of Time (AOT), while Javascript is JIT. But the end result, is fairly close: a pseudo-asm, which is not real CPU code, but it's optimized to be executed quite fast on a VM.

The main issue with Javascript, or any other JIT language, is that at any time the compiler might decide to re-optimized stuff and compile again, usually because it realized some assumptions were wrong. Another issue is the in-famous garbage collection. Sure, this will completely shield you, the developer, from having to deal with pointers and memory allocation ( more precisely, memory de-allocation, since it's easy to allocate memory, and it's equally easy to forget to free it ), but that also means garbage collection might happen at any time, and you have no control over it. That's common for Java/JS/C# or any other similar language that has a VM that takes care of memory handling for you.

WASM, instead, at least how it has been used by Asobo, it's not only completely AOT ( hence the annoying long compilation step of all WASM airplanes the first time they run, or if there's a sim update ), but it's also fully Intel. If I decompile a WASM .DLL that has been compiled by the sim at startup using my favorite decompiler ( IDA Freeware ), it's recognized as a real x64 .DLL, it shows actual Intel ASM op-codes and it has imported all Simconnect functions like a real C++ module.

Asobo clarified this when some users wanted to know more about technical details, for example if WASM ended up being a "real" machine code, rather than just another bytecode-based VM, like Java/JS/C#, so they explained their implementation is based on Innative:
 

https://github.com/innative-sdk/innative

This creates real .DLL that can be linked with standard C/C++ object modules, and possibly interact with the OS. Now, this last feature is not possible with MSFS WASM: they chose to create a sandboxed plugin instead, so no OS interaction and no linking with 3rd party libraries, but as far as execution speed, it's functionally equivalent to real ML, because it *is* real ML. Now, it's possible MSFS WASM might not be 100% as fast as a standard .EXE, but I'd say the difference is mostly marginal, and related to different optimization capabilities of the different compilers used, nothing you check without running lots of diverse benchmarks. Also, there's no garbage collection in WASM, so it's the same as unmanaged C++ in this case.

[level7 172]

7 hours ago, virtuali said:

That's the key sentence.

...

So, I must say PMDG explanation about the lack of these features makes sense, and they are telling the truth, it's just a consequence of the design choices of the product, which are sensible, all considering ( here I'm biased, because I'm a C++ addict, and I consider JS to be as a script-kiddies evil, it's so easy to use that it's not even fun anymore.. ). If another developer made a plane using a different approach, possibly using external software ( I believe the Fenix A320 will require an external .EXE ) or Javascript, that makes sense as well, and doesn't make the PMDG explanation any less valid, every design choice has different benefits for both developers and users as well.

Thanks Virtuali that's helpful background. I'm not necessarily trying to say PMDG are malicious in this particular aspect but their lack of fulfilling things they had promised to other customers is constantly a source of doubt from me.

8 hours ago, threegreen said:

Other than that, calling out elements in an addon that aren't included yet or in need of an update and at the same time say why should we care about the framework is basically saying I'm going to complain but don't care about the reasons behind what I'm complaining about, which is a bit disingenuous.

 

8 hours ago, lwt1971 said:

Lol a lot of conflating and assumptions going on here... it's not "parroting" but pointing to what the source (in this case PMDG) are saying.. if some choose to not believe them and/or think up conspiracy theories as to why feature X is not released now, well go at it, but it's not fact.

I was also commenting on some notion that if PMDG can do it no one can do it. When the product is presented to the customer, we compare against the features that are provided - I'm very sure most simmers aren't digging into the JS or WASM implementations. Understand PMDG made some choices on the frameworks that worked best for them. But ultimately it will be lined up against other products that do the things that they cannot do.

[threexgreen 3,139]

10 hours ago, level7 said:

But ultimately it will be lined up against other products that do the things that they cannot do.

True, but it's important to understand that there are good reasons behind certain features not being present at launch, some of which aren't even down to PMDG but Asobo to make possible and, more importantly, that these features are coming.

Edited May 15, 2022 by threegreen

[Manny 1,293]

Who says there no DME ARCS in the PMDG 737. Yesterday I flew a DME Arch. Run way 9 to Mumbai (VABB)

And the Chennai (VOMM) the other day.

 

[tfm 169]

21 minutes ago, Manny said:

Who says there no DME ARCS in the PMDG 737. Yesterday I flew a DME Arch. Run way 9 to Mumbai (VABB)

And the Chennai (VOMM) the other day.

 

Well, unless you have magic access to LNAV2.0, you actually flew a series of pseudo waypoints in a polygon as PMDG’s workaround for still not updating their LNAV to provide ARINC424-capable functionality 🙂  

 

A DME arc is one of 23 (?) leg-types supported by the ARINC424 specification. It connects two waypoints (entry and exit) with no waypoints in between, so what you should see on the ND is a nice smooth curve: not a polygon with multiple waypoints.

 

To save you reading the many preceding posts:  the reasons why PMDG does not offer this have been explained: as I understand it, essentially, PMDG has chosen to rely largely on its legacy code, so it needs to wait for Asobo to produce debugging tools in MSFS that will work with its retained code/methods. Whether those are sufficient reasons given the competition out there, is a matter of opinion. Apparently a new LNAV is in prospect, possibly in the next “24 weeks”, but without any exact assurance and despite being acknowledged as “long overdue” as far back as March 2019.

Edited May 15, 2022 by tfm

[Manny 1,293]

1 hour ago, tfm said:

Well, unless you have magic access to LNAV2.0, you actually flew a series of pseudo waypoints in a polygon as PMDG’s workaround for still not updating their LNAV to provide ARINC424-capable functionality 🙂  

 

A DME arc is one of 23 (?) leg-types supported by the ARINC424 specification. It connects two waypoints (entry and exit) with no waypoints in between, so what you should see on the ND is a nice smooth curve: not a polygon with multiple waypoints.

 

To save you reading the many preceding posts:  the reasons why PMDG does not offer this have been explained: as I understand it, essentially, PMDG has chosen to rely largely on its legacy code, so it needs to wait for Asobo to produce debugging tools in MSFS that will work with its retained code/methods. Whether those are sufficient reasons given the competition out there, is a matter of opinion. Apparently a new LNAV is in prospect, possibly in the next “24 weeks”, but without any exact assurance and despite being acknowledged as “long overdue” as far back as March 2019.

OK. I stand corrected.! 🙂

[Biology 1,347]

On 5/14/2022 at 10:12 PM, virtuali said:

That's good question, for which I don't have a precise answer.

Usually, when something is compiled by-user, locally, security has something to do with it. Not from a DRM point of view ( as far as I can see, WASM binaries are not encrypted ), but with security at large.

The first thing that comes to my mind is, how do you spread a virus, if the WASM system is unable to just run pre-compiled binaries, but must always re-compile from the intermediate code ? Clearly, when you are controlling the compiler ( Flight simulator includes the compiler so, in a way, it IS the compiler ), you can fairly easily prevent to compile dangerous code, since the only thing you allow to compile, are the API you provided with that specific WASM implementation.

But please don't quote me on that, it's just an assumption I'm making.

I think it is possible, however WASM modules still "link to" the inNative environment as inNative acts as a bridge between WASM code and the OS by providing "system calls", which means with every simulator update that changes the inNative environment, the compiled binary would need to be updated too, which is indeed the case for MSFS add-ons right now. If add-on developers end up having to provide pre-compiled binaries, they would have to do so with every update which is not developer friendly I guess. Of course Asobo can combine both approaches and allow add-on developers to provide pre-compiled binaries if they want, but I guess they didn't want to deal with possible complications. I'm not sure how much security plays a role in this case as Asobo could still enforce the pre-compilation to be occur in MSFS, maintaining the security provided by inNative. Also regardless a locally compiled binary could be replaced by a rogue software the same way if Asobo doesn't have checks preventing something like that from happening. It is an interesting topic indeed.