JSFX integer quantization at high numbers (NOT BUG)

[mpl]

JSFX:
test=2^64
> 18446744073709552000

Windows calc:
2^64
>18446744073709551616

[Tale]

I don't think this is a bug. JSFX numbers are stored as 64-bit floating-point, and this is simply a limitation of that format, see:

https://en.wikipedia.org/wiki/Double...g-point_format

[bezusheist]

looks ok to me...

[Tale]

Originally Posted by bezusheist

looks ok to me...
https://i.imgur.com/Urlxaox.png

You're right, 2^64 can actually be stored exactly, but e.g. 2^64-1 can't (it will be stored as 2^64).

Also note that printf("%.0f") can't output such big integers accurately, which is why 2^64 appears as 18446744073709552000, eventough the correct value is stored internally. Again, this is not a bug, but simply how things work (IMHO, YMMV).

[mpl]

Got it, thanks