[javascript] Dealing with float precision in Javascript

I have a large amount of numeric values y in javascript. I want to group them by rounding them down to the nearest multiple of x and convert the result to a string.

How do I get around the annoying floating point precision?

For example:

0.2 + 0.4 = 0.6000000000000001

Two things I have tried:

>>> y = 1.23456789 
>>> x = 0.2 
>>> parseInt(Math.round(Math.floor(y/x))) * x; 
1.2000000000000002

and:

>>> y = 1.23456789 
>>> x = 0.2 
>>> y - (y % x)
1.2000000000000002

The answer is


From this post: How to deal with floating point number precision in JavaScript?

You have a few options:

  • Use a special datatype for decimals, like decimal.js
  • Format your result to some fixed number of significant digits, like this: (Math.floor(y/x) * x).toFixed(2)
  • Convert all your numbers to integers

> var x = 0.1
> var y = 0.2
> var cf = 10
> x * y
0.020000000000000004
> (x * cf) * (y * cf) / (cf * cf)
0.02

Quick solution:

var _cf = (function() {
  function _shift(x) {
    var parts = x.toString().split('.');
    return (parts.length < 2) ? 1 : Math.pow(10, parts[1].length);
  }
  return function() { 
    return Array.prototype.reduce.call(arguments, function (prev, next) { return prev === undefined || next === undefined ? undefined : Math.max(prev, _shift (next)); }, -Infinity);
  };
})();

Math.a = function () {
  var f = _cf.apply(null, arguments); if(f === undefined) return undefined;
  function cb(x, y, i, o) { return x + f * y; }
  return Array.prototype.reduce.call(arguments, cb, 0) / f;
};

Math.s = function (l,r) { var f = _cf(l,r); return (l * f - r * f) / f; };

Math.m = function () {
  var f = _cf.apply(null, arguments);
  function cb(x, y, i, o) { return (x*f) * (y*f) / (f * f); }
  return Array.prototype.reduce.call(arguments, cb, 1);
};

Math.d = function (l,r) { var f = _cf(l,r); return (l * f) / (r * f); };

> Math.m(0.1, 0.2)
0.02

You can check the full explanation here.


Tackling this task, I'd first find the number of decimal places in x, then round y accordingly. I'd use:

y.toFixed(x.toString().split(".")[1].length);

It should convert x to a string, split it over the decimal point, find the length of the right part, and then y.toFixed(length) should round y based on that length.


Check out this link.. It helped me a lot.

http://www.w3schools.com/jsref/jsref_toprecision.asp

The toPrecision(no_of_digits_required) function returns a string so don't forget to use the parseFloat() function to convert to decimal point of required precision.


You could do something like this:

> +(Math.floor(y/x)*x).toFixed(15);
1.2

Examples related to javascript

need to add a class to an element How to make a variable accessible outside a function? Hide Signs that Meteor.js was Used How to create a showdown.js markdown extension Please help me convert this script to a simple image slider Highlight Anchor Links when user manually scrolls? Summing radio input values How to execute an action before close metro app WinJS javascript, for loop defines a dynamic variable name Getting all files in directory with ajax

Examples related to floating-point

Convert list or numpy array of single element to float in python Convert float to string with precision & number of decimal digits specified? Float and double datatype in Java C convert floating point to int Convert String to Float in Swift How do I change data-type of pandas data frame to string with a defined format? How to check if a float value is a whole number Convert floats to ints in Pandas? Converting Float to Dollars and Cents Format / Suppress Scientific Notation from Python Pandas Aggregation Results

Examples related to double

Round up double to 2 decimal places Convert double to float in Java Float and double datatype in Java Rounding a double value to x number of decimal places in swift How to round a Double to the nearest Int in swift? Swift double to string How to get the Power of some Integer in Swift language? Difference between int and double How to cast the size_t to double or int C++ How to get absolute value from double - c-language

Examples related to floating-accuracy

Dealing with float precision in Javascript Truncate (not round off) decimal numbers in javascript Floating point inaccuracy examples What's wrong with using == to compare floats in Java? Is floating point math broken?

Examples related to numerical-methods

Dealing with float precision in Javascript Check if a varchar is a number (TSQL) How can I use numpy.correlate to do autocorrelation?