Incorrect number renderer precision with scientific notation
Incorrect number renderer precision with scientific notation
Seb33300
Posts: 9Questions: 3Answers: 0
Description of problem:
If I define a column to be formatted with a precision of 2, I am expecting the precision to be the number of digits after the decimal (applied to the float number).
Examples:
Expected (precision 2):
123.456789 => 123.46
0.0000123456789 => 0.00
Current result (precision 2):
123.456789 => 123.46 (OK)
0.0000123456789 => 1.23 x 10-5 (WRONG = precision fixed after the 1 instead of decimal)
Expected (precision 7):
123.456789 => 123.4567890
0.0000123456789 => 0.0000123 => 1.23 x 10-5
Current result (precision 7):
123.456789 => 123.4567890 (OK)
0.0000123456789 => 1.2345678 x 10-5 (WRONG)
Replies
Just did some research and precision may be "significant figures" or "decimals" depending on the context.
So I believe here, the renderer is applying "significant figures"
Hi,
I've just created this little test case showing the issue: https://live.datatables.net/socirone/93/edit .
The problem is this little piece of code.
My intention had been for precision (significant figures), but I do see that the wording in the docs isn't quite right and I accept that this might not be the behaviour that is wanted.
There should probably be an optional flag to say which display mode is to be used. I've noted that down to add.
Allan