Resolves: tdf#138360 better accuracy in rtl_math_round()

Decimal negative exponents (powers) are imprecise
1e-1   0.10000000000000001
1e-2   0.01
1e-3   0.001
1e-4   0.0001
1e-5   1.0000000000000001e-05
1e-6   9.9999999999999995e-07
1e-7   9.9999999999999995e-08
1e-8   1e-08
1e-9   1.0000000000000001e-09
1e-10  1e-10
1e-11  9.9999999999999994e-12
1e-12  9.9999999999999998e-13
1e-13  1e-13
1e-14  1e-14
1e-15  1.0000000000000001e-15
1e-16  9.9999999999999998e-17
1e-17  1.0000000000000001e-17
1e-18  1.0000000000000001e-18
1e-19  9.9999999999999998e-20
1e-20  9.9999999999999995e-21

so use the positive exponents instead and swap multiplication and
division when scaling and scaling back the value, which multiplies
the rounded value with a precise integer instead of dividing it by
an imprecise fraction.

For a large (absolute) value check if it is roundable to the
desired decimals at all with the binade's distance precision gap
and if not then diminish the decimals parameter. This prevents
possible inaccuracies due to overly scaling the value.

Change-Id: I41a113078031a552cf98d72f8cb2b10bdc88dea4
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/106830
Reviewed-by: Eike Rathke <erack@redhat.com>
Tested-by: Jenkins

0 files changed, 0 insertions, 0 deletions