Here's another (less efficient, of course) algorithm that performs the same FPU rounding but works for all numbers: local function round(num) Ties are rounded to the nearest even number, rather than always up: Most rounding algorithms you find on the net, including Ola M's answer, (Note that the numbers in parentheses are calculated at compilation time they don't affect runtime).įor example, when the FPU is set to round to nearest or even, this unit test prints "All tests passed": local function testnum(num, expected)Įrror(("Failure rounding %.17g, expected %.17g, actual %.17g") If your Lua uses double precision IEC-559 (aka IEEE-754) floats, as most do, and your numbers are relatively small (the method is guaranteed to work for inputs between -2 51 and 2 51), the following efficient code will perform rounding using your FPU's current rounding mode, which is usually round to nearest, ties to even: local function round(num) Return quantum * (quant + (frac > 0.5 and 1 or 0))Īnd tweak the exact condition on frac (and possibly the sign of exact) to get the edge cases you wanted. Local quant,frac = math.modf(exact/quantum)
One approach that is simple to express and has stable behavior is to write function round(exact, quantum) Rounding to a multiple of something other than a power of ten will still require scaling, and still has all the tricky edge cases. Not that this isn't an issue with string.format(), but a lot of work has gone into making it produce "expected" results. Will round the arbitrary value in exact to a multiple of 0.001.Ī similar result can be had with scaling before and after using one of math.floor() or math.ceil(), but getting the details right according to your expectations surrounding the treatment of edge cases can be tricky. For example mils = tonumber(string.format("%.3f", exact)) A trick that is useful for rounding at decimal digits other than whole integers is to pass the value through formatted ASCII text, and use the %f format string to specify the rounding desired.
0 kommentar(er)
