[pull] main from pytorch:main#1636
Merged
Merged
Conversation
…it runs in fbcode (#189892) The `TestCUDACodeCache` tests were skipped in fbcode because their availability guard `_has_gpu_codecache_compiler` called `nvcc_exist()` with no argument, which defaults to looking up bare `"nvcc"` on `PATH` via `shutil.which`; in fbcode `nvcc` is not on `PATH`, so the guard returned `False` and disabled the tests even though `CUDACodeCache` can compile fine. `CUDACodeCache` locates the compiler through `_cuda_compiler()` (in `torch._inductor.codegen.cuda.compile_utils`), which in fbcode returns the full path `build_paths.sdk_home/bin/nvcc`. This change imports `_cuda_compiler` and switches the guard to `nvcc_exist(_cuda_compiler())` so the test detects nvcc exactly the way the code under test does, letting the tests run in fbcode. Differential Revision: [D111931279](https://our.internmc.facebook.com/intern/diff/D111931279/) Pull Request resolved: #189892 Approved by: https://github.com/aorenste
… (#189884) The test spawned two threads each running `_compose(m, x)` = `_capture(m, x)` + `compile_to_python(...)`. Only `compile_to_python` is serialized by `_COMPILE_LOCK`; `_capture` (a `make_fx` symbolic trace) ran OUTSIDE the lock. `make_fx` is not thread-safe -- it mutates process-global tracing state, notably `CURRENT_PATCHER` in `torch/fx/_symbolic_trace.py` (saved/restored around each trace) and the `torch/fx/traceback.py` node-meta globals. When one thread's unlocked `_capture` overlapped the `make_fx` nested inside the other thread's `compile_to_python`, the two traces raced on `CURRENT_PATCHER`: one trace's `_new_patcher` `finally` block observed the global reset to `None` by the other and raised `AssertionError: CURRENT_PATCHER is None in finally block`. The test swallowed the exception into `errors`, so CI only surfaced the opaque `assertEqual(errors, {})` failure ("Additional keys in the actual mapping: [1]"). Fix: capture both graphs SERIALLY up front so only `compile_to_python` -- the entry point `_COMPILE_LOCK` actually serializes -- runs concurrently. This exercises the lock's real contract without the incidental concurrent-`make_fx` race. The swallowed traceback is now surfaced in the assertion message so a future failure is diagnosable from CI logs. Alternative considered: making fx tracing thread-safe by moving `CURRENT_PATCHER` (and the `fx.traceback` globals) to thread-local storage. Rejected here as out of scope and high-risk for a flaky-test fix: `make_fx` has never been thread-safe and this test never intended to trace concurrently -- it targets `_COMPILE_LOCK`, not `make_fx`. Fixes #189665 Test Plan: Reproduced the race with an in-process stress harness (surfacing the swallowed exception) using the pre-fix body, then re-ran with the fix: # pre-fix: ~4% of iterations raise "CURRENT_PATCHER is None in finally block" # post-fix: 0 / 550 iterations failed Actual test, full file, and a fresh-process stress loop: python test/functorch/test_compile_to_python.py \ TestAOTCompileToPython.test_concurrent_compile_to_python_smoke # OK python test/functorch/test_compile_to_python.py # 50 passed for i in $(seq 1 12); do \ python test/functorch/test_compile_to_python.py \ TestAOTCompileToPython.test_concurrent_compile_to_python_smoke; \ done # 12/12 passed This change was authored with the assistance of an AI coding assistant. Pull Request resolved: #189884 Approved by: https://github.com/aorenste
…SourceBuilder (#189356) Follow-up to #188298. That PR added PySourceBuilder with a hand-rolled indent buffer, which was a third copy of torch._inductor.utils.IndentedBuffer (already subclassed cross-layer by torch._dynamo.guards.IndentedBufferWithPrefix). This PR removes the divergence. The IndentedBuffer primitive cluster (LineContext, ValueWithLineMap, DeferredLineBase, IndentedBuffer) has zero torch-internal deps (stdlib only), so it is relocated verbatim to a new core module torch/utils/_indented_buffer.py that any layer can depend on without reaching across the _functorch -> _inductor edge or importing the heavy torch._inductor.utils module (sympy, DeviceProperties, ...). torch._inductor.utils re-exports the four names so its existing call sites and subclasses (DualIndentedBuffer, AotOnlyBuffer, FakeIndentedBuffer, DelayReplaceLine) are unchanged. torch._dynamo.guards now imports IndentedBuffer from the light module too. PySourceBuilder then subclasses the hoisted IndentedBuffer and keeps only the genuinely-new exec layer (globals binding, fresh-name generation, build() via _compile_and_exec_source) plus emit() (absolute-indent writes for recursive generators) and a getvalue() override that joins without a trailing newline to keep generated source byte-identical. This PR also completes the runtime_wrappers.py conversion #188298 started: the remaining _codegen_* helpers that appended hardcoded-indent strings to a raw buf.lines list (aliased out of the builder, which left buf._indent stuck at 0 -- flagged in review) now take the builder and write via emit(). No path aliases buf.lines/buf.globals anymore, so writeline()/indent() and emit() flow through a single consistent buffer. Generated source is unchanged. Suggested review order: torch/utils/_indented_buffer.py (the relocation), then torch/_inductor/utils.py and torch/_dynamo/guards.py (re-export + import switch), then codegen_utils.py (the subclass), then runtime_wrappers.py and subclass_codegen.py (the helper conversions and getvalue() calls). Authored with the assistance of an AI coding assistant. Test Plan: ``` lintrunner --take PYFMT,RUFF,PYREFLY,CODESPELL,FLAKE8 -a <changed files> python test/functorch/test_subclass_codegen.py python test/functorch/test_codegen_mutation_epilogue.py python test/functorch/test_codegen_backward_epilogue.py python test/dynamo/test_structured_trace.py ``` The functorch codegen tests assert the generated source via assertExpectedInline; they pass unchanged, confirming the generated source is byte-identical. Pull Request resolved: #189356 Approved by: https://github.com/aorenste
…B ROCm wheels (#189903) ## Summary `.ci/manywheel/repair_wheel.py` repackaged manywheels through the `wheel` CLI (`wheel unpack`/`pack`/`tags`). `wheel pack` and `wheel tags` emit an **invalid ZIP64 extended-information field for archives larger than 4GB** ([pypa/wheel#692](pypa/wheel#692)). ROCm wheels bundle the full ROCm runtime and exceed 4GB, so the produced nightlies carry a malformed ZIP64 record. Lenient unzippers tolerate it, but uv's stricter zip validation rejects it, so `uv pip install` of recent ROCm 7.2 nightlies fails to unzip and silently falls back to the last good build (#189748). ## Fix Repackage with **auditwheel** instead of the `wheel` CLI: - `repair_wheel()` now uses `auditwheel.wheeltools.InWheelCtx`, which unpacks via `zip2dir` on enter and, once `out_wheel` is set, regenerates `RECORD` and repacks via `dir2zip` on exit. `dir2zip` writes with the stdlib `zipfile`, which produces a correct ZIP64 record. - The `linux_* -> manylinux_2_28_*` retag (filename **and** `WHEEL` metadata) now goes through `auditwheel.wheeltools.add_platforms`, folded into the single unpack/repack cycle instead of a separate post-pass. - All patchelf / lib-bundling logic is unchanged (including the rocSHMEM bare-soname symlink from #189110). - `auditwheel` added to `requirements.txt` (Linux only); installed into the build interpreter by `build_install_deps.py`. This switches the whole manywheel repack (cpu/cuda/xpu/rocm, x86_64 and aarch64) to one correct path; ROCm is the variant that was actually breaking. Fixes #189748. Works around pypa/wheel#692. ## Test Plan - `python -m py_compile .ci/manywheel/repair_wheel.py` - Verified `packaging.parse_wheel_filename` accepts PyTorch local-version wheel names (e.g. `torch-...+rocm7.2-...-linux_x86_64.whl`), which auditwheel's `add_platforms` relies on. - CI: manywheel ROCm build; confirm the produced wheel installs via `uv pip install` (unzips under uv's strict validation). - Test was done on repackaging wheels for release 2.13 using: https://github.com/pytorch/test-infra/pull/8256/changes *This PR was authored with the assistance of an AI coding agent.* Pull Request resolved: #189903 Approved by: https://github.com/Skylion007, https://github.com/huydhn, https://github.com/albanD
…89533)" This reverts commit 62cdf55. Reverted #189533 on behalf of https://github.com/frgossen due to Regresses the periodic inductor_huggingface_unbacked_parity job on a100, which isn't run on PRs so there was no land-time signal. Pushes DistillGPT2 (+1.5%) and T5Small (+1.2%) over the job's 1% backed-vs-unbacked threshold, red on every periodic run since it landed. Test-revert #189927 drops both under threshold and the job passes, confirming the cause. Happy to help reland. ([comment](#189533 (comment)))
test_polar_matches_svd invokes the cuSOLVER Xpolar (QDWH) kernel directly via
torch._native.ops.polar.nvmath_impl.polar_xpolar, deliberately bypassing the
linalg_polar override's runtime gate and SVD fallback so it exercises Xpolar
specifically. But cusolverDnXpolar was only added in cuSOLVER 12.2 (CUDA 13.1),
and nvmath resolves the symbol lazily, so on an older CUDA runtime the direct
call raises FunctionNotFoundError instead of falling back. This reds the H100
smoke job (CUDA 13.0):
test_linalg.py::TestLinalgCUDA::test_polar_matches_svd_cuda_float32
nvmath.bindings._internal.utils.FunctionNotFoundError:
function cusolverDnXpolar_bufferSize is not found
@skipIfNoNvmath only checks that nvmath is installed, not that the loaded
cuSOLVER exposes Xpolar. Gate the test on the polar override's own
_check_nvmath() (which requires cusolver_version() >= 12.2 and a NVIDIA build)
and skip when the fast path is unavailable -- the same criterion the override
uses to decide whether to run Xpolar at all.
Note: production is unaffected; the override's cond/_run_xpolar already gate and
fall back to the aten SVD kernel. This only fixes the test's direct call.
Test Plan:
```
lintrunner test/test_linalg.py
python -m py_compile test/test_linalg.py
```
Full run needs a CUDA runtime and is left to CI: on the H100 smoke image
(cuSOLVER < 12.2) the test now skips; on runtimes with Xpolar it still runs.
Authored with Claude.
Pull Request resolved: #189875
Approved by: https://github.com/slayton58
…189507) Fixes #180948 ## Problem `test_saved_lowp_checkpoint_truncates_subsequent_uses` was skipped on XPU with `@skipIfXpu(msg="...#180948")`. The test validates that activation checkpointing with `emulate_precision_casts_on_saved_tensors=True` correctly inserts lowp cast operations in compiled Triton code. ## Fix Remove `@skipIfXpu` and relax the skip condition to allow XPU: ```diff - @skipIfXpu(msg="#180948") @parametrize("lowp_dtype", [torch.bfloat16, torch.float16]) - @unittest.skipIf(not TEST_CUDA, "requires CUDA") + @unittest.skipIf(not (TEST_CUDA or TEST_XPU), "requires CUDA or XPU") ``` ## Verification Tested on PVC (Intel Data Center GPU Max 1550): ``` Environment: Hardware: Intel Data Center GPU Max 1550 (PVC) x8 PyTorch: 2.14.0.dev20260707+xpu oneAPI: 2026.0.0 (fresh environment) Results: torch.bfloat16: ✅ assert_close PASS, code contains .to(tl.bfloat16) PASS torch.float16: ✅ assert_close PASS, code contains .to(tl.float16) PASS ``` ### Test Plan ```bash python test/inductor/test_cuda_repro.py CudaReproTests.test_saved_lowp_checkpoint_truncates_subsequent_uses_bfloat16 -v python test/inductor/test_cuda_repro.py CudaReproTests.test_saved_lowp_checkpoint_truncates_subsequent_uses_float16 -v ``` Pull Request resolved: #189507 Approved by: https://github.com/guangyey, https://github.com/etaf, https://github.com/jansel
Reusing the name of the iterator variable in a list comprehension that graph breaks to store the result causes a segfault (the stack underflows in the generated comprehension function). The current approach using bytecode templates is unable to cleanly detect this, so instead switch to analyzing the comprehension bytecode based on the number of STORE instructions. This also includes a fix for another bug where the resume function could reset the result variable to Null. Generated by an LLM Pull Request resolved: #187631 Approved by: https://github.com/guilhermeleobas, https://github.com/williamwen42
…uire it (#189886) `graph_partition` defaults to `True` in OSS but to `False` in fbcode (`torch/_inductor/config.py` gates the default on `is_fbcode()`). Two tests in `CudaGraphTreeTests` in `test_cudagraph_trees.py` assume `graph_partition` is enabled and therefore fail under the fbcode default: `test_skip_cpp_wrapper` and `test_cudagraph_min_partition_size_skip_small`. The fix pins `graph_partition` to `True` for exactly these two tests -- `test_skip_cpp_wrapper` now patches `{"cpp_wrapper": True, "graph_partition": True}` on its `@torch._inductor.config.patch` decorator, and `test_cudagraph_min_partition_size_skip_small` adds `"graph_partition": True` to its existing `torch._inductor.config.patch({"triton.cudagraph_min_partition_size": 10})` context manager. This makes both tests pass in fbcode while leaving OSS behavior unchanged. The same two tests under the `:cudagraph_trees_expandable_segments` target (same file) are fixed as well. Pull Request resolved: #189886 Approved by: https://github.com/aorenste
…ps` (#184355) Fixes parts of #177259 Pull Request resolved: #184355 Approved by: https://github.com/fffrog, https://github.com/albanD, https://github.com/jbschlosser
…ps` (#184356) Fixes parts of #177259 Pull Request resolved: #184356 Approved by: https://github.com/fffrog, https://github.com/jbschlosser ghstack dependencies: #184355
## Summary Follow-up to #189232, which added the `multigpu` pytest marker and, **on CUDA**, moved the single-process (single-GPU) distributed tests out of the multi-GPU `distributed` config onto the `default` config's 1-GPU runner. The routing in `.ci/pytorch/test.sh` was gated to `*cuda*` only — a deliberate, conservative rollout (per the `TODO(temporary) ... while we validate the split` comment) — so **ROCm** still runs the whole distributed suite on its multi-GPU `distributed` box and runs zero distributed tests in `default`. This PR extends the exact same split to ROCm by flipping the four `*cuda*` gates to also match `*rocm*`: - `distributed` config → runs only the multi-GPU (process-spawning) tests (`test_distributed multigpu`) - `default` config's 1-GPU shards → run the single-GPU distributed tests (`test_distributed_single_gpu`) ### Why this is safe/appropriate for ROCm ROCm's CI topology already mirrors CUDA's: - `default` configs run on **1-GPU** runners (e.g. `linux.rocm.gpu.gfx942.1`) and, for mi300/mi350/mi200, set no `tests-to-include`, so they already fall through to the same generic shard branches CUDA uses. - `distributed` configs run on **multi-GPU** runners (`linux.rocm.gpu.gfx942.4`, `linux.rocm.gpu.mi210.2`). - **navi31** is unaffected: it has no `distributed` config and its `default` job uses `tests-to-include` (the ROCm `TESTS_TO_INCLUDE` branch), so it neither reaches these branches nor changes coverage. ### Motivation Beyond matching CUDA's cost/throughput win (single-GPU tests off the scarce multi-GPU runners), this also fixes a downstream inconsistency: tooling that compares ROCm vs CUDA test results per `(test, config)` currently sees single-GPU distributed tests under `default` on CUDA but `distributed` on ROCm, producing spurious mismatches. Aligning the split removes that at the source. ## Test plan - [x] `bash -n .ci/pytorch/test.sh` (syntax) - [x] `ciflow/rocm` + `ciflow/periodic` — verify on ROCm CI that: - the `distributed` config now runs only multigpu tests and stays green on the multi-GPU runners - the `default` config's shards pick up the single-GPU distributed tests and stay green on 1-GPU runners ## Related - Downstream ROCm parity PR that this fixes at the source: ROCm#3432. That PR works around the mismatch in the parity tooling by folding `default`↔`distributed` for `distributed.*` files; once this change lands and propagates to ROCm/pytorch, ROCm and CUDA attribute single-GPU distributed tests to the same config, removing the mismatch upstream. ## Verification on ROCm CI (mi350 / gfx950) Validated end-to-end on this PR's `linux-jammy-rocm-py3.10-mi350` run — all shards green (default 8/8, distributed 3/3, inductor 2/2): https://github.com/pytorch/pytorch/actions/runs/29345176556 **Old vs new — the split is actually exercised:** Old (merge base `329a5b4`, pre-split) distributed shard ran the whole suite — [job 87124950503](https://github.com/pytorch/pytorch/actions/runs/29342891791/job/87124950503): ``` + test_distributed Testing distributed python tests (all) ``` New (this PR) `distributed` shard runs only the multi-GPU tests — [job 87131790315](https://github.com/pytorch/pytorch/actions/runs/29345176556/job/87131790315): ``` + test_distributed multigpu Testing distributed python tests (multigpu) + python test/run_test.py --distributed-tests --multigpu-filter multigpu --shard 1 3 --verbose ``` New (this PR) `default` shard now runs the single-GPU distributed tests — [job 87131790292](https://github.com/pytorch/pytorch/actions/runs/29345176556/job/87131790292): ``` + test_distributed_single_gpu + test_distributed not-multigpu Testing distributed python tests (not-multigpu) + python test/run_test.py --distributed-tests --multigpu-filter not-multigpu --shard 1 8 --verbose ``` **Downstream parity effect (ROCm vs CUDA)** — parity report for this commit: https://github.com/ethanwee1/pytorch/actions/runs/29370122913 (artifact `parity-results-mi350`, `20260714_all_tests_status_mi350.csv`): - 0 new ROCm failures; `distributed`-config disagreements dropped to **69** (from ~1,700 before the split), because single-GPU distributed tests now attribute to the same config on both stacks. - e.g. `distributed/_composable/fsdp/test_fully_shard_autograd::test_post_acc_grad_hook_runs` is now a single `distributed` row `rocm=PASSED / cuda=PASSED`, instead of the previous doubled `default: rocm=MISSED/cuda=PASSED` + `distributed: rocm=PASSED/cuda=MISSED`. ### XML-level evidence: where the single-process distributed tests land Tracing one single-process (not-multigpu) test — `distributed/test_symmetric_memory::SymmMemSingleProcTest::test_memset32` — through the actual JUnit XML artifacts, before vs after, ROCm vs CUDA: | stack | before this PR | after this PR | |---|---|---| | **ROCm** | `test-distributed` artifact | `test-default` artifact | | **CUDA** | `test-default` artifact | `test-default` artifact | **Before — ROCm** ran it in the **distributed** artifact (whole suite on the multi-GPU box) — [run 29342891791 / job 87124950433](https://github.com/pytorch/pytorch/actions/runs/29342891791/job/87124950433): ```xml <!-- rocm_xml/test-distributed-3-3_87124950433/.../distributed.test_symmetric_memory-*.xml --> <testcase classname="SymmMemSingleProcTest" name="test_memset32" time="0.095" file="distributed/test_symmetric_memory.py" /> ``` **Before — CUDA** already ran it in the **default** artifact (job 87122965489): ```xml <!-- cuda_xml/test-default-2-5_87122965489/.../distributed.test_symmetric_memory-*.xml --> <testcase classname="SymmMemSingleProcTest" name="test_memset32" time="0.165" file="distributed/test_symmetric_memory.py" /> ``` → **config mismatch** (ROCm `distributed` vs CUDA `default`) — this is what produced the spurious parity MISSEDs. **After — ROCm** now runs it in the **default** artifact (moved; 0 remaining in the distributed artifact) — [run 29345176556 / job 87131790292](https://github.com/pytorch/pytorch/actions/runs/29345176556/job/87131790292): ```xml <!-- rocm_xml/test-default-1-8_87131790292/.../distributed.test_symmetric_memory-*.xml --> <testcase classname="SymmMemSingleProcTest" name="test_memset32" time="3.214" file="distributed/test_symmetric_memory.py" /> ``` **After — CUDA** unchanged, still **default** (job 87132976027): ```xml <!-- cuda_xml/test-default-1-5_87132976027/.../distributed.test_symmetric_memory-*.xml --> <testcase classname="SymmMemSingleProcTest" name="test_memset32" time="0.150" file="distributed/test_symmetric_memory.py" /> ``` → **now aligned**: both ROCm and CUDA attribute the single-process test to `default`. The multi-process classes in the same file (e.g. `SymmetricMemoryTest`) correctly stay in the `distributed` artifact on both stacks. Pull Request resolved: #189874 Approved by: https://github.com/pablo-garay, https://github.com/jithunnair-amd, https://github.com/jeffdaily
…ith different kernel types (#188301) with_effects records the ordering dependency between consecutive ORDERED-effectful ops in V.graph.additional_star_deps so the scheduler preserves their program order. The dep was keyed by new_op.get_name() (the buffer name), but the scheduler reads additional_star_deps back by operation name. When two effectful ops lowered to the same kernel type the names happened to coincide, masking the bug; when they lowered to different kernel types (e.g. an ExternKernel followed by a _CollectiveKernel) the key mismatched, the dep was silently dropped, and the scheduler was free to reorder op_b ahead of op_a. Key the dep by new_op.get_operation_name() to match how the scheduler looks it up. Adds test_inductor_preserves_effect_order_across_kernel_types, which compiles two ORDERED-effectful ops that lower to different inductor kernel types (op_a -> FallbackKernel, op_b -> _CollectiveKernel) and asserts op_a is emitted before op_b in the generated code. The test fails without the fix because the dropped ordering dep lets the scheduler reorder op_b ahead of op_a. Pull Request resolved: #188301 Approved by: https://github.com/EikanWang, https://github.com/desertfire, https://github.com/jansel
This reverts commit 37c107a. Reverted #189455 on behalf of https://github.com/huydhn due to Sorry for reverting your change but I suspect that this is breaking backward_compat check on trunk ([comment](#189455 (comment)))
Follow-up to #180926. In review there, @albanD asked that we not spread defensive `with_ieee_matmul_precision` decorators across matmul-using tests, and instead fix the root cause: tests that leave a global fp32 precision flag in a non-default state, plus the base `TestCase` not catching such leaks. This makes any test running under `torch.testing._internal.common_utils.TestCase` start from a known precision baseline, and turns a leak into a loud failure in `tearDown` rather than a silent tolerance failure in an unrelated sibling test. The core change is in `common_utils.py`: a single `_fp32_precision_flag_specs()` enumerates the six flags (cuda.matmul, cudnn.conv/rnn, mkldnn.matmul/conv/rnn); `TestCase.setUp` snapshots them and `tearDown` compares, restores, and raises `AssertionError` naming the leaked flag. `recover_orig_fp32_precision` is refactored to reuse the same snapshot/restore. An opt-out env var `PYTORCH_DISABLE_FP32_PRECISION_LEAK_CHECK=1` exists as a short-term escape hatch; the goal is zero callers. With the detector on, the remaining changes fix tests it would now reject. Inductor suites that set process-global precision at import time move the set into `setUpModule`/`tearDownModule` with save/restore, so importing the module no longer poisons precision for everything imported after it; distributed suites that set `allow_tf32 = False` at import time get the same treatment. The `test_linalg.py` tunableop tests replace a hardcoded `finally: allow_tf32 = False` with `tf32_enabled()`, and `TestLinalg`/`TestMatmulCuda` `setUp`/`tearDown` snapshot the real `fp32_precision` instead of restoring a hardcoded value. Three single-test leaks (kernel_benchmark, mix_order_reduction, torchinductor) get `@recover_orig_fp32_precision`. Finally, with leaks no longer possible, the `with_ieee_matmul_precision` decorator added in #180926 is redundant: its five call sites (`test_old_cholesky`, `test_rnn_fused`, three affine_grid/grid_sample tests) pass at the default precision once nothing leaks TF32 into them, so the decorator and its call sites are removed. Suggested review order: `common_utils.py` (detector + helpers) first, then the offender fixes, then the decorator removal. ## Test Plan Verified on ROCm/MI300 (gfx942); CUDA paths validated in CI. ``` # precision setup/teardown changes in test_linalg HIP_VISIBLE_DEVICES=7 python test/test_linalg.py \ TestLinalgCUDA.test_tf32_tunableop_cuda_float32 \ TestLinalgCUDA.test_tf32_offline_tunableop_cuda_float32 # the five tests that previously needed with_ieee_matmul_precision, now without it HIP_VISIBLE_DEVICES=7 python test/test_nn.py \ TestNNDeviceTypeCUDA.test_affine_2d_rotate90_cuda \ TestNNDeviceTypeCUDA.test_affine_2d_rotateRandom_cuda \ TestNNDeviceTypeCUDA.test_affine_3d_rotateRandom_cuda \ TestNNDeviceTypeCUDA.test_rnn_fused_cuda_float32 \ TestNNDeviceTypeCUDA.test_rnn_fused_cuda_float64 HIP_VISIBLE_DEVICES=7 python test/test_linalg.py TestLinalgCUDA.test_old_cholesky_cuda_float32 ``` All passed. The detector was also confirmed to fire on an intentional leak and to honor the opt-out env var. ## BC-breaking? No. `torch.testing._internal` is private API, and the removed `with_ieee_matmul_precision` was only added recently in #180926. The new `tearDown` assertion can surface pre-existing precision-flag leaks in out-of-tree test suites built on our `TestCase`; `PYTORCH_DISABLE_FP32_PRECISION_LEAK_CHECK=1` disables it if needed. Authored by Claude. Pull Request resolved: #185236 Approved by: https://github.com/albanD
… operators (#189930) Summary: Add [[nodiscard]] to the free comparison operators for weak_intrusive_ptr (operator<, operator==, operator!=). These are pure queries whose result must be used. Mirrored across the fbcode and xplat copies of the header. Test Plan: [[nodiscard]] is enforced at compile time via -Werror; relies on CI to surface external discarding call sites. Local `arc lint` is clean. Differential Revision: D111955860 Pull Request resolved: #189930 Approved by: https://github.com/Skylion007
Summary: Annotate the pure-query methods of c10::utils::bitset: NUM_BITS, get, is_entirely_unset, and the operator==/operator!= comparisons. Mutators (set, unset) and the functor-invoking for_each_set_bit are left untouched. Mirrored across the fbcode and xplat copies of the header. Test Plan: [[nodiscard]] is enforced at compile time via -Werror; relies on CI to surface external discarding call sites. Local `arc lint` is clean. Differential Revision: D111955862 Pull Request resolved: #189935 Approved by: https://github.com/Skylion007
Summary: Annotate the pure-query methods of IntrusiveListHook (is_linked), ListIterator (operator==, operator!=, operator*, operator->) and IntrusiveList (begin/end, rbegin/rend, iterator_to, size, empty). insert() keeps no annotation since discarding its returned iterator is a common and valid use of its linking side effect. Mirrored across the fbcode and xplat copies of the header. Test Plan: [[nodiscard]] is enforced at compile time via -Werror; relies on CI to surface external discarding call sites. Local `arc lint` is clean. Differential Revision: D111955863 Pull Request resolved: #189936 Approved by: https://github.com/Skylion007
…rs (#189934) Summary: Annotate the raw pointer helpers in namespace c10::raw::weak_intrusive_ptr: - lock returns a raw owning strong pointer; discarding it leaks. - use_count is a pure query. incref/decref stay unannotated since they are used for effect. Mirrored across the fbcode and xplat copies of the header. Test Plan: [[nodiscard]] is enforced at compile time via -Werror; relies on CI to surface external discarding call sites. Local `arc lint` is clean. Differential Revision: D111955878 Pull Request resolved: #189934 Approved by: https://github.com/Skylion007
Summary: array_of is a constexpr factory that builds and returns a std::array; discarding the result is pointless, so mark it [[nodiscard]]. Mirrored across the fbcode and xplat copies of the header. Test Plan: [[nodiscard]] is enforced at compile time via -Werror; relies on CI to surface external discarding call sites. Local `arc lint` is clean. Differential Revision: D111955866 Pull Request resolved: #189933 Approved by: https://github.com/Skylion007
…es (#189931) Summary: Annotate the ArrayRef members defined locally in this header (front, back, slice, at) and all makeArrayRef convenience constructor overloads. These are pure queries / view factories whose returned value is the only point of the call. Note: ArrayRef's other observers (size, empty, data, begin/end, operator[]) are inherited from HeaderOnlyArrayRef in torch/headeronly/util/HeaderOnlyArrayRef.h and are not touched here. Mirrored across the fbcode and xplat copies of the header. Test Plan: [[nodiscard]] is enforced at compile time via -Werror; relies on CI to surface external discarding call sites. Local `arc lint` is clean. Differential Revision: D111955859 Pull Request resolved: #189931 Approved by: https://github.com/Skylion007
…ators (#189929) Summary: Add [[nodiscard]] to the free comparison operators for intrusive_ptr (operator<, operator== and operator!= including the nullptr_t overloads). These are pure queries; computing a comparison and discarding the result is always a mistake, and [[nodiscard]] on comparison operators is idiomatic. Mirrored across the fbcode and xplat copies of the header. Test Plan: [[nodiscard]] is enforced at compile time via -Werror; relies on CI to surface external discarding call sites. Local `arc lint` is clean. Differential Revision: D111955875 Pull Request resolved: #189929 Approved by: https://github.com/Skylion007
…189932) Summary: UniqueVoidPtr is a unique_ptr-like owning smart pointer specialized to void with a function-pointer deleter. Annotate: - Pure-query observers: operator->, get, get_context, cast_context, operator bool, get_deleter, and the free operator==/operator!= against nullptr_t. - Ownership-transfer: release_context (returns an owning raw pointer; discarding leaks) and move_context (moves the owning unique_ptr out). - unsafe_reset_data_and_ctx returns a success bool that callers must check (it silently no-ops if a real deleter is already installed). compare_exchange_deleter already had [[nodiscard]] and is unchanged. Mirrored across the fbcode and xplat copies of the header. Test Plan: [[nodiscard]] is enforced at compile time via -Werror; relies on CI to surface external discarding call sites. Local `arc lint` is clean. Differential Revision: D111955876 Pull Request resolved: #189932 Approved by: https://github.com/Skylion007
This PR is auto-generated nightly by [this action](https://github.com/pytorch/pytorch/blob/main/.github/workflows/nightly.yml). Update the pinned torchcomms hash. Pull Request resolved: #189957 Approved by: https://github.com/pytorchbot
### Summary The `multilabel_margin_loss` forward decomposition (`torch/_decomp/decompositions.py`) masks the loss along the input axis (via `is_target`), but never drops the "u-slots" coming from padded target positions (indices at or after the first `-1`). When the target contains `-1` padding — the standard ignore marker — those slots gather `input[0]` and contribute spurious margin terms, so the compiled forward loss and backward gradient disagree with eager. ```python import torch import torch.nn.functional as F x = torch.tensor([[1.0, -1.0, 0.5, 0.3, -0.2]], requires_grad=True) target = torch.tensor([[0, 1, -1, -1, -1]]) loss = F.multilabel_margin_loss(x, target) # eager: 1.48 loss_c = torch.compile(lambda z: F.multilabel_margin_loss(z, target))(x) # 1.96 before this PR ``` Both `inductor` and `aot_eager` are affected, confirming the bug is in the decomposition rather than a lowering. ### Fix The gather-safe mask (`target_mask`) that builds `tidx0`/`tidx1` already identifies the real target positions; apply that same mask to the loss on the u-slot axis so padded slots contribute nothing. When no `-1` is present `target_mask` is all-`True`, so this is a no-op for the unpadded case. The decomposition test did not catch this because `sample_inputs_multilabel_margin_loss` only generates targets with `low=0` (no `-1`); this PR adds a padded-target sample so `test_decomp` exercises the path. (`reference_inputs` already covers `-1`, but those inputs are not used by the decomposition test.) Fixes #185464 ### Test plan Compared the decomposition (`aot_eager`) against eager on CPU over 900 random cases spanning all reductions, batched and 1-D inputs, and targets with and without `-1` padding — forward loss and input gradient matched in every case. ``` python test/test_decomp.py -k multilabel_margin_loss ``` Pull Request resolved: #189552 Approved by: https://github.com/eellison
…ories (#189924) Summary: Annotate the observer surface of SmallVectorBase / SmallVectorTemplateCommon / SmallVectorImpl (size, capacity, begin/end, rbegin/rend, size_in_bytes, max_size, capacity_in_bytes, data, at, operator[], front, back, and operator==/!=/<), plus the free capacity_in_bytes and the to_vector factory overloads. Discarding any of these is meaningless; lvalue uses such as v[i] = x remain uses and are unaffected. empty() and pop_back_val() already carried [[nodiscard]] and are unchanged. Mutators are left untouched. Mirrored across the fbcode and xplat copies of the header. Test Plan: [[nodiscard]] is enforced at compile time via -Werror; relies on CI to surface external discarding call sites. Local `arc lint` is clean. Differential Revision: D111955874 Pull Request resolved: #189924 Approved by: https://github.com/Skylion007
Summary: make_intrusive allocates a new object and returns an owning intrusive_ptr. Discarding the result allocates and then immediately frees the object, which is never intended, so mark it [[nodiscard]]. Mirrored across the fbcode and xplat copies of the header. Test Plan: [[nodiscard]] is enforced at compile time via -Werror; relies on CI to surface external discarding call sites. Local `arc lint` is clean. Differential Revision: D111955864 Pull Request resolved: #189923 Approved by: https://github.com/Skylion007
Summary: Annotate the observer surface of OptionalArrayRef: operator-> / operator* (all ref-qualified overloads), operator bool, has_value, value (all overloads), value_or, and the friend operator== against ArrayRef. These are pure queries whose return value is the only effect. Modifiers (swap, reset, emplace, operator=) are left untouched. Mirrored across the fbcode and xplat copies of the header. Test Plan: [[nodiscard]] is enforced at compile time via -Werror; relies on CI to surface external discarding call sites. Local `arc lint` is clean. Differential Revision: D111955861 Pull Request resolved: #189919 Approved by: https://github.com/Skylion007
#189922) Summary: ExclusivelyOwned is a smart-pointer-like wrapper around an exclusively-owned T. Annotate: - Pure-query observers: get(), operator->(), operator* (const and non-const overloads). - Ownership-transfer: take() && and operator T() &&, which move the owned value out; discarding the result destroys the moved-out value. operator bool is =delete and left untouched. Mirrored across the fbcode and xplat copies of the header. Test Plan: [[nodiscard]] is enforced at compile time via -Werror; relies on CI to surface external discarding call sites. Local `arc lint` is clean. Differential Revision: D111955869 Pull Request resolved: #189922 Approved by: https://github.com/Skylion007
Summary: Annotate the pure-query members of c10::basic_string_view (substr and the std::basic_string conversion operator) and the free starts_with / ends_with helpers. These compute and return a value with no side effect. The common c10::string_view alias is just std::string_view and already carries the standard library's [[nodiscard]] annotations. Mirrored across the fbcode and xplat copies of the header. Test Plan: [[nodiscard]] is enforced at compile time via -Werror; relies on CI to surface external discarding call sites. Local `arc lint` is clean. Differential Revision: D111955871 Pull Request resolved: #189921 Approved by: https://github.com/Skylion007
…189918) Summary: Annotate the raw pointer helpers in namespace c10::raw::intrusive_ptr: - make_weak returns a raw owning weak pointer; discarding it leaks. - use_count is a pure query. incref/decref stay unannotated since they are used for effect. Mirrored across the fbcode and xplat copies of the header. Test Plan: [[nodiscard]] is enforced at compile time via -Werror; relies on CI to surface external discarding call sites. Local `arc lint` is clean. Differential Revision: D111955873 Pull Request resolved: #189918 Approved by: https://github.com/Skylion007
Summary: SmallBuffer is a fixed-size SBO array helper. Annotate its observers (operator[], data, size, begin, end, const and non-const overloads); discarding any of them is meaningless, while lvalue uses such as buf[i] = x remain uses. Mirrored across the fbcode and xplat copies of the header. Test Plan: [[nodiscard]] is enforced at compile time via -Werror; relies on CI to surface external discarding call sites. Local `arc lint` is clean. Differential Revision: D111955879 Pull Request resolved: #189916 Approved by: https://github.com/Skylion007
…line() (#186636) Summary: Add `[[nodiscard]]` to two methods in `c10::impl::SizesAndStrides`: - `size()` — the public dimensionality accessor; discarding it is always a logic error - `isInline()` — private helper that gates the entire inline vs. out-of-line storage path; discarding it silently would corrupt subsequent pointer arithmetic Test Plan: `buck2 test fbcode//caffe2/c10/test:core_tests` — 86 tests pass, 0 failures. Differential Revision: D107115819 Pull Request resolved: #186636 Approved by: https://github.com/Skylion007
### Summary Adds a pybind11 trampoline (PyBackend) so Python classes can subclass c10d::Backend and override collectives, lifecycle methods, and properties. Uses PyObjectSlot instead of the wrap() approach from #188747 and #187479, which created a second C++ object and broke pybind11's `get_override`. Dispatch follows PyProcessGroup's conventions - `WORK_OVERRIDE` for collectives, `PYBIND11_OVERRIDE` where names match, manual get_override with the Python binding name where they differ, and a scoped `getPropertyOverride` fallback for `@property` descriptors (supports_*, options) that `get_override` cannot find. Also exposes previously missing Backend bindings needed for the full override surface. ### Test Plan ``` python -m pip install -e . -v --no-build-isolation python -m pytest test/distributed/test_c10d_pybackend.py python -m pytest test/distributed/test_c10d_pypg.py ``` The test suite is carried forward from #187479 Authored with AI assistance Pull Request resolved: #189817 Approved by: https://github.com/d4l3k
…batches linalg tests (#186122) This PR enabled inverse_errors_large linalg tests that were previously failing for ROCm due to HipBlas errors. The fix was verified locally on rocm-nightly with [torch-2.13.0a0+rocm7.13.0a20260513-cp312-cp312-linux_x86_64.whl](https://rocm.nightlies.amd.com/v2-staging/gfx94X-dcgpu/torch-2.13.0a0%2Brocm7.13.0a20260513-cp312-cp312-linux_x86_64.whl) from https://[rocm.nightlies.amd.com/v2-staging/gfx94X-dcgpu/torch/](https://rocm.nightlies.amd.com/v2-staging/gfx94X-dcgpu/torch/) and release wheels for 7.13 https://[repo.amd.com/rocm/whl/gfx94X-dcgpu/torch/](https://repo.amd.com/rocm/whl/gfx94X-dcgpu/torch/) The test test_inverse_errors_large passed on trunk (ROCm 7.2) as it ran on this shard which successfully ran https://github.com/pytorch/pytorch/actions/runs/27307168302/job/80669673812?pr=186122#logs This PR also enabled test_lu_solve_batched_many_batches linalg test. The fix was validated with docker image ghcr.io/pytorch/ci-image:pytorch-linux-noble-rocm-nightly-py3-7ed27d6a861850867bbd15d52665f27fb821857f and release wheels for 7.13 Although the test passes, however, since it is a slow test, it still gets skipped. Pull Request resolved: #186122 Approved by: https://github.com/jeffdaily Co-authored-by: Jeff Daily <jeff.daily@amd.com>
…den MobileBert (#189813)" This reverts commit 47b744d. Reverted #189813 on behalf of https://github.com/williamwen42 due to #189927 blamed for regression and was reverted ([comment](#189813 (comment)))
…#189981) Matching exec_binary_kernel / exec_ternary_kernel pattern Addresses a review comment on #189572. Pull Request resolved: #189981 Approved by: https://github.com/Skylion007 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
#185236)" This reverts commit 85ac0b1. Reverted #185236 on behalf of https://github.com/pytorch-auto-revert due to Reverted automatically by pytorch's autorevert, to avoid this behaviour add the tag autorevert: disable ([comment](#185236 (comment)))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
16 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )