Dear MSFS Team - Third party install outside sim root directory

[Skywolf 992]

Hi MSFS Team,

Thank you for sharing your dev update today.  The video episode was jaw dropping unreal; I could not believe my eyes on what I was seeing.  I am like when is MSFS Team going to approve my Tech Alpha 1 invite .  One thing I would like to state based on my insane experience when it comes to installs of FSX, P3D1.4, 2.x, 3.x, and 4.5x  - is that can you please enforce all third party devs to install outside the sim directory.

The legacy mess that everyone is following from FSX days is just plain sad.  FSX had an addon directory which barely anyone used.  When the sim breaks, third party blames the sim team, and these days it is vice versa with LM's P3D - and at times it is super challenging to figure on which addon breaks the sim or is it sim's legacy esp code which is not interacting with the addon right (in this case we can use P3D 4.5x/FSX-SE).

The files go everywhere and it is a total mess.  You all have gone through it.

The third party addon should not work if it is installed in the sim root directory - the sim should only see third party when it is installed outside the sim period - this should be set from day one.  

I am just one voice here but legacy habits from 2006 should not come in MSFS 2020.

Thank you and have happy holidays,

Cheers,

Skywolf

 

 

 

[stephenbgs 169]

100% agree

[virtuali 4,419]

Interesting how times change...

When P3D4 came out, in June 2017, we released an update of our software the very same day, which went all-in adopting the most proper way to use the add-on.xml method, not installing anything inside the sim and not even forcing users to install the addon under Documents, which is of course the best possible solution. Everybody realized that, today.

- total freedom of the installation destination

- not installing anything in the sim own folder

- easy to temporarily disable an addon for troubleshooting, since it's disabled with a single click, even if it composed by many sub-components ( like textures, effects, scripts, gauges and scenery folders )

Back then, we were called the usual weirdos, because that's what usually happens when you do something before everybody else, and I do recall very well having to defend our choice, and LM choice of suggesting that method, against those so accustomed to the big ball of mess which was the shared DLL/EXE.XML and the shared Scenery.cfg, which protested in spades about our usual new strange stuff.

Flash forwards 18 months, and now we see users suggesting a Sandbox. Which I'm 100% in favor of. The simulator own folder should be left alone.

[Skywolf 992]

1 hour ago, virtuali said:

Interesting how times change...

When P3D4 came out, in June 2017, we released an update of our software the very same day, which went all-in adopting the most proper way to use the add-on.xml method, not installing anything inside the sim and not even forcing users to install the addon under Documents, which is of course the best possible solution. Everybody realized that, today.

- total freedom of the installation destination

- not installing anything in the sim own folder

- easy to temporarily disable an addon for troubleshooting, since it's disabled with a single click, even if it composed by many sub-components ( like textures, effects, scripts, gauges and scenery folders )

Back then, we were called the usual weirdos, because that's what usually happens when you do something before everybody else, and I do recall very well having to defend our choice, and LM choice of suggesting that method, against those so accustomed to the big ball of mess which was the shared DLL/EXE.XML and the shared Scenery.cfg, which protested in spades about our usual new strange stuff.

Flash forwards 18 months, and now we see users suggesting a Sandbox. Which I'm 100% in favor of. The simulator own folder should be left alone.

I was super happy and grateful to you guys 18 months ago when you went to addon method (I own lot of your scenery licenses lol).  I knew you understood the importance of installing outside the sim - which reduces headaches/conflicts with other third party and other sim issues.  Only few developers understood this.

I truly hope you use your Developer star power and make sure MSFS Dev team understands the urgency to force all third party add-ons install outside the sim.  No exceptions to anyone...(that is where all problems begin)

I started bugging all the devs from P3D 3.4+ days lol

 

 

Edited November 22, 2019 by Skywolf

[Tuskin38 6,458]

Based on an interview, add-ons might be installed via a package system.

[Pastaiolo 1,686]

I am still hoping for a Xplane like folder structure for planes and sceneries. Want to install a plane? You drop the plane folder in the Airplane main folder, that's it. No "effects go here, gauges go here" etc etc

[nikita 241]

I agree X-plane's approach would be the dream. Everything is self contained, want to install a plane? Drop the plane's folder in the folder with all the other aircraft. Want to uninstall it? Move/delete the plane's folder. Incredibly easy and user friendly.

[Sethos 4,300]

I agree, an approach closer to X-Plane would be much appreciated. The ESP-platform's hellish way of spreading files over multiple folders upon install, then another explosion of files everywhere for every third party addon, it's almost like the system was meticulously built to be as obtuse as humanly possible.

[SnakeP 39]

10 hours ago, virtuali said:

Interesting how times change...

When P3D4 came out, in June 2017, we released an update of our software the very same day, which went all-in adopting the most proper way to use the add-on.xml method, not installing anything inside the sim and not even forcing users to install the addon under Documents, which is of course the best possible solution. Everybody realized that, today.

[...]

Flash forwards 18 months, and now we see users suggesting a Sandbox. Which I'm 100% in favor of. The simulator own folder should be left alone.

 

I'm a bit surprised that you praise yourself to not touch the sim directories in this way, when your Addon Manager, which has the almost exclusive purpose to stop piracy, caused a lot of issues (including a vast number of CTD's) for many people. Customers of FSLabs and PMDG know what I'm talking about...

Back to the OP: yes, I also modified all my payware add-ons to work via the add-on.xml method, I simply don't use installers anymore. For newly purchased scenery, I simply install them on a "dummy system" with an empty prepar3d folder, so that I can grab all the data that would usually mess up the root directory and put everything nicely into "my" xml structure.

The issue is that this method doesn't save me all the troubles after a clean installation of P3D, because the ORBX scenery usually interferes heavily: to correct for elevation issues and scenery from two sources (ORBX and the add-on airport for instance), I sometimes have to manually deactivate the respective ORBX files (APT, AEC etc.). Long story short: after "installing" the 3rd party content via xml file by just copying it to a custom directory where it gets detected, I would still have to look into each and every airport to see whether or not some other scenery files interfere with it and might have to do the manual corrections once again.

So my personal request to ASOBO and MS (besides having addon scenery installed completely independent from the sim) would be to only allow one source of scenery per tile or per well-defined area. Period. 🙂

Btw, I'm still speechless of what they seem to be able to accomplish with MSFS2020, haven't even dared to dream about somthing even close to this ...!

Edited November 22, 2019 by maggussje

[Skywolf 992]

22 minutes ago, maggussje said:

 

I'm a bit surprised that you praise yourself to not touch the sim directories in this way, when your Addon Manager, which has the almost exclusive purpose to stop piracy, caused a lot of issues (including a vast number of CTD's) for many people. Customers of FSLabs and PMDG know what I'm talking about...

Back to the OP: yes, I also modified all my payware add-ons to work via the add-on.xml method, I simply don't use installers anymore. For newly purchased scenery, I simply install them on a "dummy system" with an empty prepar3d folder, so that I can grab all the data that would usually mess up the root directory and put everything nicely into "my" xml structure.

The issue is that this method doesn't save me all the troubles after a clean installation of P3D, because the ORBX scenery usually interferes heavily: to correct for elevation issues and scenery from two sources (ORBX and the add-on airport for instance), I sometimes have to manually deactivate the respective ORBX files (APT, AEC etc.). Long story short: after "installing" the 3rd party content via xml file by just copying it to a custom directory where it gets detected, I would still have to look into each and every airport to see whether or not some other scenery files interfere with it and might have to do the manual corrections once again.

So my personal request to ASOBO and MS (besides having addon scenery installed completely independent from the sim) would be to only allow one source of scenery per tile or per well-defined area. Period. 🙂

Btw, I'm still speechless of what they seem to be able to accomplish with MSFS2020, haven't even dared to dream about somthing even close to this ...!

 

I have done this too but the biggest equation these days is Time.  I have spend more time tweaking, redesigning the sim than simming itself.  Few years ago, I was able to dedicate time more easily to the sim but these days, I want efficiency and less headaches to deal with.  Software should not be designed from a complete engineer's perspective.  Inefficiencies lead to crappy performance; it just is not optimized.

 

[virtuali 4,419]

10 hours ago, maggussje said:

 

I'm a bit surprised that you praise yourself to not touch the sim directories in this way, when your Addon Manager, which has the almost exclusive purpose to stop piracy, caused a lot of issues (including a vast number of CTD's) for many people. Customers of FSLabs and PMDG know what I'm talking about...

The problem with this sentence, is that is based on years old outdated information.

 

- The Addon Manager, which you said has the "almost exclusive" purpose to stop piracy, since several years DOESN'T HAVE ANY DRM FEATURES. Not a single bit, Zero.

This change happened a while ago, when Esellerate switched to a different kind of support .DLL, which didn't play well with the runtime libraries that came with some Windows security update, so we had to REMOVE Esellerate from the Addon Manager entirely, and move it to the Couatl program. Couatl, being an external .exe CANNOT crash the sim, not even if it tried. That's simply not possible.

Yes, it can crash itself, but not the simulator. In fact, most of the time is the opposite: the simulator crashes for other reasons, and bring down Couatl with it.

Yes, the Addon Manager, being a .DLL CAN potentially crash the sim, just like any other .DLL or Gauge out there. That's another reason why we moved all the anti piracy handling to Couatl, so users won't feel they have to install a potentially dangerous module ( again, like any other .DLL ), that doesn't do much more than DRM. You were also convinced of that, even I we try to explain whenever is possible.

- It's a common mistake for users to assume the Addon Manager has something to do with piracy, but that's just because this is the menu they must open to register a Serial and, I can fully understand why, one might assume the program you are using "must" have something to do with piracy, since it's asking for a Serial Number.

But that's just not the case anymore. The Addon Manager now is just a dumb user interface to insert your Serial Number, but it doesn't know what to do with it, it simply passes it to Couatl, which will do the real verification, using Esellerate until last September, and now SORACO, which is the new system we are using since last July, when Esellerate closed.

What the Addon Manager REALLY does today, doesn't have anything to to with piracy anymore, and it's the following:

- It communicates with the sim to obtain data which is not available through Simconnect, like L: variables which are used for all kind of animations in GSX and sceneries. It does it in different way in FSX or P3D. using the Gauges Panel interface in FSX, and the new PDK in P3D4. 

- in P3D4, it handles as a bridge between GSX and the specific PDK features that are possible only with a DLL, which are basically Mouse Handling, and Render To Texture.

- It handles custom Ground traffic that behaves like AI but can be better controlled compared to default ground traffic.

- It provides GSX ( or XPOI ) with Facilities data (nearby airports, navaids, fixes, etc), using an interface which is also not available with Simconnect. 

That's all what the Addon Manager does now. In a way, it's a bit similar to FSUIPC, but more oriented to scenery needs rather than airplanes.

About your "vast" number of CTD claim, there's only a single case in which the Addon Manager was indirectly responsible for a CTD, but NOT because it was "faulty", but because it called into a simulator own API, which apparently has some bugs resulting in crashes when flying over sparsely populated areas. The Addon Manager DID NOT crash, it was the simulator *itself* which crashed into its own FACILITIES.DLL, and not because the Addon Manager sent bad data to it, only because it *requested* data from it.

There are plenty of other .DLL modules out there that do far more dangerous things than the Addon Manager, because they *write* to memory, yet nobody is worried about that, because users don't "see" a "DRM" interface, so they think is all good and fine. What you don't see, can't hurt you, right ?

Do you know what's the easiest way to tell a module is doing something potentially dangerous ? That's easy: if that module works only with a *specific* simulator build, and it needs to be updated then the simulator is updated, it's because it WRITES to specific memory addresses, because it uses undocumented calls and pointers and, when you write in memory to something that has changed, it will cause malfunctions ( if you are lucky ), but it will more likely crash the sim.

The Addon Manager did lots of this in FS9, some of this in FSX, and NONE of this in P3D3/4, which proven by the fact that, for several years, you don't have to wait for a software update from FSDT whenever a new build of the sim is released. That's because, since we don't use undocumented calls anymore, and we surely don't write in memory.

We even considered, at one time, to rewrite the Addon Manager to remove its user interface entirely, and make it "silent", with no visible menu and no user interaction required, and leave the task of inserting your Serial Number to an external program. 

But this would be all for purely marketing reasons, because the program wouldn't be any more reliable than already is and all this work would be entirely useless for technical reasons, it would be only a placebo effect. 

Now that I hope we finally clear the issue of what the Addon Manager is NOT ( it's NOT a DRM ), let's go to the main point I have a problem with in your sentence:

You said "you praise yourself to not touch the sim directories in this way, when your Addon Manager...".

This thread was about installing ( or not ) inside the simulator, or replacing/modifying ( or not ) files that belongs to the simulator. This is an entirely different issue that the very concept of accepting a (potentially) dangerous 3rd party module, regardless where is installed. To make it more clear:

A module is not inherently safer "just" because is installed outside of the sim. Nor it's inherently dangerous if it is. It depends what it does.

If a module sticks with documented SDK practices and it doesn't try to write anything in the simulator own process space, it's likely SAFER than a module that doesn't. Installing inside or outside the simulator won't make any difference in this case.

The Addon Manager, at least in P3D4, it's 100% pure SDK compliant, which is proven by the fact it's not reliant on the simulator precise build. However, the very fact a module IS a .DLL/.GAU, makes a CTD *possible*, that's just how Windows works.

A discussion about the very idea of allowing 3rd party .DLL into the sim might in fact be quite interesting, in the light of the new upcoming MSFS.

Surely, banning 3rd party .DLLs altogether will make the sim more difficult to crash, but since nothing is free in this world, there's always the other side of the coin, which is PERFORMANCES.

Since I know how the Addon Manager *really* works, there are some things that it does which have basically ZERO impact on performances, *because* it's a .DLL.

Render To Texture, for example, it's a wonderful feature in P3D4, which has allowed us to streamline GSX A LOT, and made it more flexible for users, because instead of several thousands of Simobjects we had in FSX ( all combination of liveries, operators and vehicles ), we have about 7-8 times *less* objects now, and that's because or Render to Texture. Also, we can draw on textures with extremely fast C++ DirectX code, that's why the Information Panels at KORD V2 do not cause any fps hit.

Mouse handling is another example. When GSX is in editing mode, the mouse position is checked at each frame, and when a click happens, there's a call to a specific P3D API that converts that click into actual Lat/Lon coordinates in the World. This makes the GSX editor much easier to use for P3D4 users, and the fps impact of it is basically null *because* it's a .DLL so, in addition to be running an extremely fast native C++ on its own, it also communicates with the sim at blazing speed, because in-process communication is basically immediate.

External .EXE modules, on the other hand, while being incapable of crashing the sim, cannot communicate with it so fast, and when they do, they put some strain on the system, that's why the sim start to struggle when you have too many add-ons sending too many commands through Simconnect all a the same time. The system by Simconnect, which is called "Named Pipes", has way more overhead compared to being a .DLL in the same memory space.

So, if you want more safety, you will probably must be prepared to sacrifice some performances too.

But again, this doesn't really have any relationship in WHICH folder the add-no is installed to. But one of the advantages of installing outside the sim and not modifying any of the simulator own files, is that it allows for an easier TROUBLESHOOTING.

If you suspect an add-on might have a problem or be the cause of your problem, if it uses the add-on.xml method, it's really easy to disable it to verify that. And if that add-on didn't change any of the simulator own files, temporarily disabling it would REALLY bring back the sim to the same status as if the add-on was never installed to begin with.

If, instead, it started to spread inside the simulator own folders or, even worse, it replaced lots of default files, it wouldn't be so easy to return to a "before" state by just disabling it: the only option would be uninstalling it altogether, and trust the Uninstaller to do its job properly and restore all replaced files ( which is not a given ).

But since you say you convert all your add-on to the add-on.xml method, you surely know all of this already. I only wanted to clarify the really outdated information about what the Addon Manager is supposed to do.

Edited November 22, 2019 by virtuali

[Skywolf 992]

19 hours ago, virtuali said:

The problem with this sentence, is that is based on years old outdated information.

Wow - wow - Umberto this post needs to be pinned at AVSim - Thank You.  MSFS Team please contact Umberto during tech alpha 1 build phase.  Infact you can even give him my tech alpha slot (if I am getting one) to him.

Thank you for sharing this.

Edited November 23, 2019 by n4gix
REMOVED OUTRAGEOUSLY LONG QUOTE!

[NZ255 309]

Safe bet he’s already talking to Asobo 😉

[mpo910 2,112]

It would be great to install outside the sim. 100% agree. But it would be greater in addition if there was something like a Config-Center from MSFS where you can check/uncheck ALL Addons I installed or  subscribed one by one ore by multiselecting to decide per MSFS Session which Addons to use or not WITHOUT to start the addon seperatly. I would also like to group /categorize them in groups I create by myself (should be able to generate my own groups/categories).

Example: MSFS has build in weather. Maybe there will by a good addon weather engine.

As a user I would like to start MSFS, go to the "Addon Config Center" (just named it like this....could be named anything), uncheck or check the "addon weather engine X", go back to the main menu and setup my flight, start the flight and fly. This would be great. 

The startup of the MSFS App should not take longer than a few secs (max 20 to 30 seconds or shorter of course). The startup of a flight should not take longer than 30 sec to max 1 min.....or shorter if possible. 

Regards Marcus

[SnakeP 39]

19 hours ago, virtuali said:

The problem with this sentence, is that is based on years old outdated information.

 

- The Addon Manager, which you said has the "almost exclusive" purpose to stop piracy, since several years DOESN'T HAVE ANY DRM FEATURES. Not a single bit, Zero.

[...]

But since you say you convert all your add-on to the add-on.xml method, you surely know all of this already. I only wanted to clarify the really outdated information about what the Addon Manager is supposed to do.

<Thank you AVSIM for the censorship of the first version of the reply, freedom of speech seems to be less valuable here than freedom of the skies, I guess...>

Look, you did a very smart thing by developing something almost everybody needs (GSX) so that you can deliver this Addon Manager thing to everybody and to fight piracy with it (I also support fighting piracy, just to say that I'm on the same page).

But your *text* is nothing but throwing words around in a Trumpian way (I should be able to say that here, shouldn't I?!) to hide the fact that you use the Addon Manager to push your anti-piracy agenda, period. And this thing or something else connected to it caused a lot of problems, as stated by other developers in the past (no, not "many" years ago).

Just answer me this question: why am I not able to grab the files for one of you airports and use the add-on.xml method without the Addon Manager? It's because you cripple the installation by hiding or removing key scenery files with it, isn't it?

Since I really have a life and don't have the time to go on and on and on about this, I'll put this to rest now and want to say sorry to the OP for hijacking this thread.

Again, hopefully ASOBO and MS will find a way to prevent 3rd party contributors to mess around with dll's and stuff and to make scenery installation easy and clean without conflicts.

Edited November 23, 2019 by maggussje