Skip to content
View in the app

A better way to browse. Learn more.

The AVSIM Community

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.

Rounding problem

Featured Replies

  • Commercial Member

I don't know how many different ways I've tried to resolve this but below are just two of them. The problem is that even after rounding I keep hitting the binary division problem where I end up with a long tail of nines or a long tail of zeroes plus a one. This is making one or two autopilot calculations barf pretty badly.... :( Can anyone help with a method that avoids the problem? Even with the frustration I admit to a wry smile that computers can't do some calculations correctly!precision = the number of places to the right of the decimal point.

double round(double value,int precision){    const int adjustment=pow(10,precision);    return floor(value*(adjustment)+0.5)/adjustment;}double round(double value,int precision ){    double remainder=fmod(value,precision);    if (remainder>(0.5*precision))value=value-remainder+precision);    else value=value-remainder;    return value;}

-Dai

There are many numbers that cannot be represented exactly in floating point. For example 0.1 becomes 0.10000000000000006 as a double. I think the general solution is to use either BCD or integer arithmetic. If the problem arises with comparisions then replace:

if ( a  == B)with if (fabs (a  -  B) <  0.000001)

Gerry Howard

Create an account or sign in to comment

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.