Having done a search on the forum for lat/lon calculations and formula, am I correct in assuming that MSFS 2002 and 2004 both use a simplistic perfect sphere for the earth where longitude is calculated as a function of the cosine of the latitude? Did M$ actually make it that simple?Thanks!Senior Captain, Pier Glass Aviation

And I have a follow up - anyone know how many nautical miles per degree that MSFS uses? or Feet? Do they use 6080, 6075,6000 feet/nm?I seem to be off by about 3 feet in my lat compared to what MSFS has.Senior Captain, Pier Glass Aviation

I can't help directly but I seem to remember other posts dealing with this. Have you tried searching the forums?

I belive the International Nautical Mile is exactly 1852 m or 6076 ft.

This is plagurized from an excel worksheet website, but should work for what you want.Great Circle DistancesI began working with Latitudes and Longitudes to create a worksheet that would allow me to enter the latitudes and longitudes of two cities on Earth, and determine the Great Circle Distance (this "as the crow flies" distance) between them.The formula to compute this distance is:=RadiusEarth*ACOS(COS(RADIANS(90-(Lat1*24)))*COS(RADIANS(90-(Lat2*24)))+SIN(RADIANS(90-(Lat1*24)))*SIN(RADIANS(90-(Lat2*24)))*COS(RADIANS(24*(Long1-Long2))))WhereLat1 is the latitude of point 1, entered as DD:MM:SS.Long1 is the longitude of point 1, entered as DD:MM:SS.Lat2 is the latitude of point 2, entered as DD:MM:SS.Long2 is the longitude of point 2, entered as DD:MM:SS.RadiusEarth is the radius of the earth (3,963 miles or 6,377 kilometers). The function above works only for points in the Northern and Western hemisphere, where Latitudes and Longitudes are considered to be positive. If you are mixing hemispheres, enter Northern and Western coordinates as positive, and Southern and Eastern coordinates as negative, and use the following formula:=RadiusEarth*ACOS(COS(RADIANS(90-Lat1))*COS(RADIANS(90-Lat2))+SIN(RADIANS(90-Lat1))*SIN(RADIANS(90-Lat2))*COS(RADIANS(Long1-Long2)))In this formula, Lat1, Long1, Lat2, and Long2 must be entered as decimal degrees (e.g., 45.5 rather than 45:30:00).

The following link suggests that Flight Simulator uses the WGS84 ellipsoid. As this is now the standard for real-world aviation (although not yet universially adopted) it means that Microsoft can take coordinates and heights directly from AIPs.EDIT: I'm not sure about heights because the WGS84 ellipsoid can't represent local heights accurately. However, on second thoughts does it really matter - they are just numbers in a computer as far a Flight Simulator is concerned? If WGS coordinates were used on a circular earth by Flight Simulator then the distance and bearing between two points would differ between it and the real world. But how far apart would the points have to be before the difference became significant? http://forums.avsim.net/dcboard.php?az=sho...ing_type=search

Well, I don't really care about real world vs. FS. what matters is that my remote sim use the exact same mathematics as FS to calculate position. I am talking about distance in the less than foot (.3m)range range.What is happening is that I am landing a helo on a ship... and the inaccuracies make the helo appear to slide on the deck - which is very disconcerting, and tough on the eyes too. I need the helos position relative to the ship to be constant once "on deck". And for that I need accurate mathematics.Thanks for the advice!Senior Captain, Pier Glass Aviation

Better late than never."In Flight Simulator, the earth is defined as an

HOLY COW! They took into account the flattening of the earth? That's impressive. Are there any equations that MS gives?Thank you so much for that info! I've been working on other things, so this is still very useful to me.Senior Captain, Pier Glass Aviation

Hello,Thank you for these formulas which I would like to apply in my C++ gauges. Unfortunately I am a self made man in programming and miss some basis. So I would like to know how tot enter the values in C++ for degrees, minutes and seconds to calculate for example (90-Lat1).For 40deg 20 min 12 sec, how do I enter it in C++ as FLOAT64Is it 40.2012 ?Thanks for the helpJean-Pierre

(((Seconds/60)+Minutes)/60) + DegreesSo, for your example(((12/60)+20)/60)+40((.2+20)/60)+40(20.2/60)+40.33666+4040.33666 degrees is the floating point value