View Single Post
Old 01-13-2021, 12:36 AM   #2
mschnell
Human being with feelings
 
mschnell's Avatar
 
Join Date: Jun 2013
Location: Krefeld, Germany
Posts: 10,193
Default

What do you mean by "fixed precision" ?
64 bit float format provides a "fixed precision" of about 48 bits. That is some 10e-14. I suppose that there are not that many relevant algorithms that need more. Do check your algorithm on that behalf. Maybe some math can modify it to be less prone to the effects you see.

To get a better precision some very hard software means need to be applied, as the computer hardware can't do this.

-Michael

Last edited by mschnell; 01-13-2021 at 08:37 AM.
mschnell is offline   Reply With Quote