Pinned memory causes error with GGUF model

[nestflow]

Custom Node Testing

• I have tried disabling custom nodes and the issue persists (see how to disable custom nodes if you need help)

Expected Behavior

Works as pinned memory is not enabled.

Actual Behavior

The bug seems to happen when trying to unload the model. I'm using Wan 2.2 model with ComfyUI-GGUF node, and it breaks between two samplers. Maybe it's an issue of GGUF node, but I reported here since it only happens when enabling pinned memory. Also, the bug seems to only happen with a relatively high video resolution.

Steps to Reproduce

This is not the actual workflow I used since it includes too many unrelated custom nodes, but I can reproduce the bug with it: bugged_workflow.json.

Debug Logs

- **Node ID:** 360
- **Node Type:** SamplerCustom
- **Exception Type:** torch.AcceleratorError
- **Exception Message:** CUDA error: invalid argument
Search for 'cudaErrorInvalidValue' in https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html for more information.
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.


  File "D:\Projects\ComfyUI\execution.py", line 510, in execute
    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, hidden_inputs=hidden_inputs)
                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\Projects\ComfyUI\execution.py", line 324, in get_output_data
    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, hidden_inputs=hidden_inputs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\Projects\ComfyUI\execution.py", line 298, in _async_map_node_over_list
    await process_inputs(input_dict, i)

  File "D:\Projects\ComfyUI\execution.py", line 286, in process_inputs
    result = f(**inputs)

  File "D:\Projects\ComfyUI\comfy_extras\nodes_custom_sampler.py", line 658, in sample
    samples = comfy.sample.sample_custom(model, noise, cfg, sampler, sigmas, positive, negative, latent_image, noise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar, seed=noise_seed)

  File "D:\Projects\ComfyUI\comfy\sample.py", line 65, in sample_custom
    samples = comfy.samplers.sample(model, noise, positive, negative, cfg, model.load_device, sampler, sigmas, model_options=model.model_options, latent_image=latent_image, denoise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)

  File "D:\Projects\ComfyUI\comfy\samplers.py", line 1053, in sample
    return cfg_guider.sample(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\Projects\ComfyUI\comfy\samplers.py", line 1035, in sample
    output = executor.execute(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed, latent_shapes=latent_shapes)

  File "D:\Projects\ComfyUI\comfy\patcher_extension.py", line 112, in execute
    return self.original(*args, **kwargs)
           ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^

  File "D:\Projects\ComfyUI\comfy\samplers.py", line 984, in outer_sample
    self.inner_model, self.conds, self.loaded_models = comfy.sampler_helpers.prepare_sampling(self.model_patcher, noise.shape, self.conds, self.model_options)
                                                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\Projects\ComfyUI\comfy\sampler_helpers.py", line 130, in prepare_sampling
    return executor.execute(model, noise_shape, conds, model_options=model_options)
           ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\Projects\ComfyUI\comfy\patcher_extension.py", line 112, in execute
    return self.original(*args, **kwargs)
           ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^

  File "D:\Projects\ComfyUI\comfy\sampler_helpers.py", line 138, in _prepare_sampling
    comfy.model_management.load_models_gpu([model] + models, memory_required=memory_required + inference_memory, minimum_memory_required=minimum_memory_required + inference_memory)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\Projects\ComfyUI\comfy\model_management.py", line 697, in load_models_gpu
    loaded_model.model_load(lowvram_model_memory, force_patch_weights=force_patch_weights)
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\Projects\ComfyUI\comfy\model_management.py", line 506, in model_load
    self.model_use_more_vram(use_more_vram, force_patch_weights=force_patch_weights)
    ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\Projects\ComfyUI\comfy\model_management.py", line 535, in model_use_more_vram
    return self.model.partially_load(self.device, extra_memory, force_patch_weights=force_patch_weights)
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\Projects\ComfyUI\comfy\model_patcher.py", line 919, in partially_load
    self.unpatch_model(self.offload_device, unpatch_weights=unpatch_weights)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\Projects\ComfyUI\custom_nodes\ComfyUI-GGUF\nodes.py", line 77, in unpatch_model
    return super().unpatch_model(device_to=device_to, unpatch_weights=unpatch_weights)
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\Projects\ComfyUI\comfy\model_patcher.py", line 832, in unpatch_model
    self.model.to(device_to)
    ~~~~~~~~~~~~~^^^^^^^^^^^

  File "D:\Projects\ComfyUI\.venv\Lib\site-packages\torch\nn\modules\module.py", line 1371, in to
    return self._apply(convert)
           ~~~~~~~~~~~^^^^^^^^^

  File "D:\Projects\ComfyUI\.venv\Lib\site-packages\torch\nn\modules\module.py", line 930, in _apply
    module._apply(fn)
    ~~~~~~~~~~~~~^^^^

  File "D:\Projects\ComfyUI\.venv\Lib\site-packages\torch\nn\modules\module.py", line 930, in _apply
    module._apply(fn)
    ~~~~~~~~~~~~~^^^^

  File "D:\Projects\ComfyUI\.venv\Lib\site-packages\torch\nn\modules\module.py", line 930, in _apply
    module._apply(fn)
    ~~~~~~~~~~~~~^^^^

  File "D:\Projects\ComfyUI\.venv\Lib\site-packages\torch\nn\modules\module.py", line 957, in _apply
    param_applied = fn(param)

  File "D:\Projects\ComfyUI\.venv\Lib\site-packages\torch\nn\modules\module.py", line 1357, in convert
    return t.to(
           ~~~~^
        device,
        ^^^^^^^
        dtype if t.is_floating_point() or t.is_complex() else None,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        non_blocking,
        ^^^^^^^^^^^^^
    )
    ^

  File "D:\Projects\ComfyUI\custom_nodes\ComfyUI-GGUF\ops.py", line 58, in to
    new = super().to(*args, **kwargs)

  File "D:\Projects\ComfyUI\.venv\Lib\site-packages\torch\_tensor.py", line 1654, in __torch_function__
    ret = func(*args, **kwargs)

Other

No response

[tester4488]

same on rtx 4060.
Same in no gguf workflows

[comfyanonymous]

Have you tried updating the gguf node and comfyui?

[nestflow]

I can confirm they are both the latest.

Have you tried updating the gguf node and comfyui?

[rattus128]

@nestflow how much VRAM do you have? My guess is you have 24GB based on what happening, but these code paths do depend on how much VRAM you are running and the number will help be reproduce.

[rattus128]

I cant reproduce this on Linux at all with the workflow attached and I put latent upscalers between the two samplers to test both up and down changes of the partial load. I trapped the exact code path in the trace and no reproducer.

Theres some suspicious code in the core around weight backups that doesnt look right, but ill take all of this to windows and see if I get the repro first.

[nestflow]

@nestflow how much VRAM do you have? My guess is you have 24GB based on what happening, but these code paths do depend on how much VRAM you are running and the number will help be reproduce.

I have 16GB VRAM and use Windows.

[hanswurst232]

I've tried everything, but the error persists. I've now switched to fp8_scaled and, to be honest, I'm getting even better results with it and am grateful for the error message :-D

[PopHorn]

same here at Windows :( 16gb vram
No GGUF workflow
I am useing block swap and torchcompile

[comfyanonymous]

Stop using block swap, it's completely useless with native models.

[jovan2009]

I also have 16 GB ( rtx 5060 Ti, 64 GB RAM, Windows) and I run without problems unquantized fp16 Wan 2.2 I2V + 4 steps Loras and fp32 VAE (because I am an obsessive compulsive quality freak). One difference might be that I generally install almost every day nightly torch 2.10.0+cu130 (today is no exception) and the corresponding torchvision and torchaudio. I also run my own builds of xformers, flash attention 2.8.3 and sageattention using latest available git versions and CUDA 13.0.2 (I make new builds when there are updates to those projects maybe within a week time). And of course I always use latest Nvidia driver. For what is worth for the subject of this thread.

[PopHorn]

Stop using block swap, it's completely useless with native models.

Really? But I'd definitely go OOM for higher resolutions of video

The bug persists. The error comes when KSampler changes from high noise to low one. I've just loder the WF that worked perfectly yday

[PopHorn]

Stop using block swap, it's completely useless with native models.

I've just used generic wan22 i2v wf from template lib

[slikvik55]

Same issue for me. This is a serious bug it would seem. Wan 2.2 WFs working fine with v0.3.67-27-g135fa49e (2025-11-01) until I ran update.bat yesterday and it started crashing when switching between high and low.

I restored my install from a backup and immediately start working again. Definately something in the last update causing the problem. Using Unet GGUF loader, all updated.

Windows Portable. RTX 5090. Torch 2.9 cu130.

[slikvik55]

I should add, I dont have pinned memory enabled, at least I don't think I do! Didn't even know it existed until now.