C# float? rounding issue with "long" numbers

C# float? rounding issue with "long" numbers

guidolsguidols Posts: 38Questions: 14Answers: 1
edited February 2023 in Editor

Hi,

I have an Editor field defined in C# as float?. In JS I defined the field and the columns without any particular setting.

If I save, for instance, the value 10.12345, everything is fine, i.e. the value is correctly saved in the DB and is correctly re-loaded.

But if I save, for instance, the value 1000.12345, there are issues:
* in the DB the value saved is 1000.123
* the re-loaded value is 1000.12347

With 10000.12345:
* in the DB the value saved is 10000.12
* the re-loaded value is 10000.123

and so on...

So first of all it seems that the maximum length for a number saved in the DB is 8 (with the dot), e.g. 10000.12

Moreover it seems that there are also issue when the value is reloaded from the DB.

Can you please clarify?

Thanks!

Answers

  • guidolsguidols Posts: 38Questions: 14Answers: 1

    Seems that the issue arises even without a floating digit.

    If I save 123456789 in an int? field, everything is fine

    If I save 123456789 in an float? field, in the DB I get 1.234568E+08

  • allanallan Posts: 63,678Questions: 1Answers: 10,497 Site admin

    What is your database data value for this field? If it isn't "numeric" then, you might want to set it to be that to avoid rounding errors on IEEE numbers such as this. You can also use decimal as the number type in C#. This MS document has a good reference for the number types in C#.

    Allan

  • guidolsguidols Posts: 38Questions: 14Answers: 1

    Hi Allan,

    in the DB the field is real. So is a C# float, which is "translated" as real.

    Can I verify the executed SQL query when I update a single value in the editor?

Sign In or Register to comment.