Describe the bug
For a float64 input, mx.sin, mx.cos and mx.exp return results whose relative error is of order
1e-7 (i.e. float32-class) while the returned dtype is float64. mx.tan, mx.log and mx.sqrt on
the same inputs are correctly rounded (~1e-16), so this is specific to these three functions rather than a
general float64 limitation.
mx.cos is the worst at 3.2e-04 max relative error, which is ~1e12x float64 epsilon.
To Reproduce
import mlx.core as mx, numpy as np, mpmath as mp
mp.mp.dps = 60
rng = np.random.default_rng(0)
xs = rng.uniform(-3, 3, 2000)
with mx.stream(mx.cpu):
A = mx.array(xs, dtype=mx.float64)
for name, fn, ref in (("sin", mx.sin, mp.sin), ("cos", mx.cos, mp.cos), ("exp", mx.exp, mp.exp),
("tan", mx.tan, mp.tan), ("log", mx.log, mp.log), ("sqrt", mx.sqrt, mp.sqrt)):
src = np.abs(xs) if name in ("log", "sqrt") else xs
B = np.asarray(fn(mx.abs(A) if name in ("log", "sqrt") else A))
rel = max(abs(float((mp.mpf(float(B[i])) - ref(mp.mpf(float(src[i])))) / ref(mp.mpf(float(src[i])))))
for i in range(len(xs)))
print(f"mx.{name:5s} float64 max relative error = {rel:.3e}")mx.sin float64 max relative error = 7.746e-07 <-- float32-class
mx.cos float64 max relative error = 3.206e-04 <-- float32-class, and 1e12x float64 eps
mx.exp float64 max relative error = 2.932e-07 <-- float32-class
mx.tan float64 max relative error = 3.077e-16 ok
mx.log float64 max relative error = 2.452e-16 ok
mx.sqrt float64 max relative error = 1.094e-16 ok
Single-value form, if that is easier to triage:
import mlx.core as mx, numpy as np, mpmath as mp
with mx.stream(mx.cpu):
a = mx.array([1.0], dtype=mx.float64)
print(float(np.asarray(mx.cos(a))[0])) # actual: 0.5403022766113281 <- float32 value
print(float(mp.cos(mp.mpf(1)))) # correct: 0.5403023058681398Expected behavior
A float64 input should give a float64-accurate result, within a few ULP (~1e-16 relative), as tan,
log and sqrt already do.
Environment
mlx 0.32.0 · python 3.12 · Apple M2 (arm64), macOS · float64 evaluated under with mx.stream(mx.cpu)
(float64 is CPU-only) · reference: mpmath at 60 decimal digits.
Describe the bug
For a
float64input,mx.sin,mx.cosandmx.expreturn results whose relative error is of order1e-7(i.e. float32-class) while the returned dtype isfloat64.mx.tan,mx.logandmx.sqrtonthe same inputs are correctly rounded (~1e-16), so this is specific to these three functions rather than a
general float64 limitation.
mx.cosis the worst at 3.2e-04 max relative error, which is ~1e12x float64 epsilon.To Reproduce
Single-value form, if that is easier to triage:
Expected behavior
A
float64input should give afloat64-accurate result, within a few ULP (~1e-16 relative), astan,logandsqrtalready do.Environment
mlx 0.32.0· python 3.12 · Apple M2 (arm64), macOS · float64 evaluated underwith mx.stream(mx.cpu)(float64 is CPU-only) · reference:
mpmathat 60 decimal digits.