View Single Post
Old 27 December 2022, 09:50   #25
Thomas Richter
Registered User
 
Join Date: Jan 2019
Location: Germany
Posts: 3,233
Quote:
Originally Posted by Bruce Abbott View Post
However for most purposes I am happy enough getting back what was typed into the original source code, eg. 1000.0 disassembles as 1.0E3, not 9.9999986E2. That's why I round. It's more likely that the original number was 1000 than 999.999??.
Both requirements cannot be satisfied - that you get in what you typed is not possible since there is limited precision, so precision has to go. Since decimal and binary have different bases, the loss is not expressible in an integer number of digits either.

And that "nice numbers are more likely", well.... Why? In the end it means that the float to ascii conversion cannot distinguish between some inputs and will represent them by the same output. It will print the "nice number" the same way as the "not so nice number" near to it. That does not look right to me. It means that you loose precision because you prefer some numbers to others, which is a bit bizarre.

Actually, this is exactly the problem I wanted to avoid.
Thomas Richter is offline  
 
Page generated in 0.04503 seconds with 11 queries