What do you mean by "fixed precision" ?
64 bit float format provides a "fixed precision" of about 48 bits. That is some 10e-14. I suppose that there are not that many relevant algorithms that need more. Do check your algorithm on that behalf. Maybe some math can modify it to be less prone to the effects you see.
To get a better precision some very hard software means need to be applied, as the computer hardware can't do this.
-Michael
Last edited by mschnell; 01-13-2021 at 08:37 AM.
|