Fix vmap of scatter placing the batch axis wrongly in the updates - #4322
Open
Adityaj0 wants to merge 1 commit into
Open
Fix vmap of scatter placing the batch axis wrongly in the updates#4322Adityaj0 wants to merge 1 commit into
Adityaj0 wants to merge 1 commit into
Conversation
Under vmap the batch dimension becomes an extra scattered source axis at
position src_ax, so the updates need a matching singleton dimension. The
updates are laid out as the index dimensions followed by one dimension
per source axis, but the singleton was inserted at the front of the
source part rather than at src_ax:
updates = expand_dims(updates, {0, inputs[1].ndim()}, stream());
For a source of shape (2, 3, 4) batched on axis 2, the source part of the
updates has to be (1, 3, 1) but came out as (1, 1, 3), so the scatter
wrote with the last two source dimensions transposed and silently
produced wrong values.
This only showed up for a vmap axis >= 2. For axis 0 the intended
position and the front coincide, and for axis 1 the misplaced singleton
sits next to the scattered axis, which is also size 1, so the shape is
unchanged. From axis 2 on, the singleton displaces a real dimension.
Insert the singleton at src_ax within the source part instead. In the
branch where the updates are themselves batched, the insertion shifts
their vmap axis, so that index is adjusted before the moveaxis.
Affects every scatter reduction and plain indexed assignment, e.g.
mx.vmap(lambda a: a.at[idx].add(u), in_axes=2) and
mx.vmap(lambda a: a.__setitem__(idx, u), in_axes=2).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4321.
Proposed changes
Under
vmapthe batch dimension becomes an extra scattered source axis at positionsrc_ax, so the updates need a matching singleton dimension. The updates are laid out as the index dimensions followed by one dimension per source axis, but the singleton was inserted at the front of the source part rather than atsrc_ax:updates = expand_dims(updates, {0, static_cast<int>(inputs[1].ndim())}, stream());For a source of shape
(2, 3, 4)batched on axis 2, the source part of the updates has to be(1, 3, 1)but came out(1, 1, 3), so the scatter wrote with the last two source dimensions transposed and silently produced wrong values.That is also why it stayed hidden. For axis 0 the intended position and the front coincide. For axis 1 the misplaced singleton lands next to the scattered axis, which is itself size 1, so the shape is unchanged by luck. Only from axis 2 on does the singleton displace a real dimension:
(2,3)(2,3,4)(2,3,4,5)The fix inserts the singleton at
src_axwithin the source part. In the branch where the updates are themselves batched, that insertion shifts their vmap axis, so the index is adjusted before themoveaxis.Affects every scatter reduction and plain indexed assignment (
add,subtract,multiply,maximum,minimum,a[idx] = v).take,take_along_axisandput_along_axiswere never affected — they go throughGather/GatherAxis/ScatterAxisrather than the generalScatter.Before / after on the repro from the issue, where
out[0] = 2 * a[0]makes the expected values checkable by hand:Tests
Added
test_vmap_scatter_higher_ranktopython/tests/test_vmap.py. It sweeps shapes(2,3),(2,3,4)and(2,3,4,5)over every vmap axis and several scatter forms (update derived from the vmapped array, constant update, duplicate indices, and a non-sum reduction), comparing against the equivalent per-slice loop.I confirmed the test is a real regression test rather than a tautology: with the
primitives.cppchange stashed and the extension rebuilt, it fails withand passes with the change applied.
The existing
test_vmap_scatteronly covered a 2-D source within_axes0 and 1, i.e. exactly the two cases that already worked, which is how the gap survived.pre-commit run --all-filesto format my code and installed pre-commit prior to committing changesVerified with a CPU-only build (
-DMLX_BUILD_METAL=OFF):test_vmap,test_ops,test_array,test_autograd,test_einsum,test_linalg,test_blas,test_nn,test_reduce,test_fft(481 tests) pass