How Protobi rounds raw and calculated values

Updated at June 12th, 2023

Rounding raw values

Protobi rounds to the nearest integer

Rounding to the nearest integer is a simple mathematical operation we apply to decimal numbers. For decimal numbers between two integers like 1 and 2 , rounding to the nearest integer is pretty straightforward. Decimals less than 0.5 are rounded down and decimals above 0.5 are rounded up. 

You may know this rule from elementary school-- "5 and above, give it a shove. 4 and below, let it go". 

How Protobi rounds half decimals

For decimal numbers ending with exactly 0.5 there are two integers that are equally near. There are many different methods to handle rounding half numbers.

For these cases, Protobi uses a convention called round half away from zero, where decimals ending with 0.5 are always rounded to the integer with the highest absolute value. This method is also known as round half toward infinity or commercial rounding because it is often used for currency conversions and price rounding.

Rounding half away from zero treats positive and negative values symmetrically, and therefore is free of overall positive/negative bias.

Some clients have remarked that decimals like 23.49999 should first round to one decimal 23.5 then round to 24. This is known as double rounding and we do not use this method when displaying raw numbers from your survey datafile.  

Rounding calculated values

Protobi uses a slightly different method when rounding calculated values versus rounding raw values to account for the limited precision of arithmetic on computers. Protobi rounds to 6 decimals places first, then to your desired precision.

Limited precision of arithmetic on computers 

Rounding errors occur on computers due to the limited precision of binary mathematics. Most decimal fractions cannot be represented exactly as binary fractions. This problem is easy to understand if you consider the fraction ⅓. You can approximate the decimal value to be 0.3, 0.33 or 0.333, etc. But no matter how many digits you write down you will never exactly represent ⅓. 

For instance in binary arithmetic 6/40 + 7/40 = 0.32499999999999996 which is close to but not exactly equal to 0.325. The value the computer returns is just an approximation of the original decimal fraction, and when rounded to the nearest integer the result would be 32%. But Protobi first rounds 0.32499999999999996 to 0.325000, so when rounded the integer result is 33%.

Protobi rounds using the following method which covers a lot of cases where rounding errors occur due to the limited precision of binary arithmetic. 

Protobi rounding algorithm

fmtPercent (6/40 + 7/40, 0, true) ==> 33%
function fmtPercent(val, precision, percentSymbol) {
    var result
    if (val == null || isNaN(val)) {
        result = '–'
    }
    else {
       if (typeof precision == 'string') result = d3.format(precision)                        
       else { 
          result = (+((val * 100).toFixed(6))).toFixed(precision || 0)
          if (percentSymbol !== false) result += '%';    
       }                
    }
    return result
}
 
 

Example of calculated decimals rounded to integers

Decimal number Rounded integer
23.49 23
23.499 23
23.49999 23
23.499999999 24
23.5 24
-23.49 -23
-23.499 -23
-23.49999 -23
-23.499999999 -24
-23.5 -24

See the Wikipedia page for more information on Rounding half away from zero and other rounding methods:

We can replace the rounding algorithm in your projects with your firm's business rules. Contact support@protobi.com for more information. 

 

Was this article helpful?