NVIDIA's Windows driver can use system RAM when graphics memory runs short, keeping some CUDA applications running at reduced speed. An external card still has its own dedicated VRAM, even when the computer has much more system memory available.
The F9A's external graphics connection supports adding a desktop GPU through OCuLink. Its unified memory serves the built-in graphics, but an attached NVIDIA card reaches system RAM over its PCIe connection. Same computer, different memory access.
In the NVIDIA Control Panel, the relevant option is CUDA - Sysmem Fallback Policy under Manage 3D settings. Prefer No Sysmem Fallback disables this behavior for the selected program, while Driver Default restores the driver's normal policy. Neither choice adds memory to the card.
For a Python-based image generator, the selected program must be the Python executable actually running the workload. Launch it, find that process in Task Manager, and use its file path in Program Settings.
Pointing the setting at another Python installation won't configure the running interpreter. Apply the change and restart the generator before comparing results, or you'll still be testing the old setting.
Turning fallback off can replace a slow render with an out-of-memory error. Keeping it enabled may suit a job you can leave running, but finishing without a crash says nothing about whether the extra runtime is acceptable.
Task Manager's Performance tab separates dedicated GPU memory from shared system memory. Watch the NVIDIA card's counters during the same job, rather than treating the displayed shared-memory allowance as additional onboard VRAM. A capacity figure alone cannot tell you where your model's data resides.
Windows and WSL don't provide the same full support. Under CUDA's limited-support model, managed allocations cannot exceed physical GPU memory, and the GPU can't fetch individual memory pages on demand. A Windows fallback success doesn't establish that a Linux application inside WSL can do the same.
An application advertising unified memory support needs to check the device's concurrentManagedAccess capability. NVIDIA identifies a zero value with the restricted behavior, so the GPU's model name alone won't determine compatibility.
Linux also distinguishes ordinary GPU allocations from managed allocations. A program using cudaMalloc doesn't get access-based memory migration just because the operating system supports it for cudaMallocManaged, so the application's allocation method still matters.
WSL also limits pinned system memory, which an application keeps resident for GPU access. Training workloads can hit this separate limit before exhausting the machine's ordinary RAM. Free ordinary RAM doesn't prove enough pinned memory is available.
Installing a native Linux NVIDIA display driver inside WSL won't remove those restrictions. WSL uses the Windows host driver, and NVIDIA warns against replacing it with a Linux driver. A Linux CUDA toolkit and a Linux display driver aren't interchangeable installations.
The benefit of keeping idle weights in system RAM is space for the next GPU operation. Your application decides when each part moves, instead of waiting for the driver to react to crowded VRAM.
For image generation in Diffusers, sequential CPU offload transfers small model components as they're needed. This saves more VRAM, but repeated transfers can make generation painfully slow.
Model CPU offload moves larger components and keeps a repeatedly used component on the GPU until its work finishes. It needs more graphics memory, but avoids repeatedly transferring that component during the same stage. The trade-off sits in the offload setting, not the OCuLink cable.
Suppose your image pipeline's text encoder fits on the card, but its denoising model exceeds the available VRAM. Whole-model offload cannot solve that mismatch by itself. You need finer-grained offload or another method that reduces the active component's memory requirement.
With sequential offload, initialization order matters too. Diffusers warns against moving the pipeline to CUDA before enabling sequential CPU offload, because doing so leaves only minimal memory savings. A script can contain the right offload call and still retain the memory problem through an earlier GPU transfer.
The F9A's external graphics connection supports adding a desktop GPU through OCuLink. Its unified memory serves the built-in graphics, but an attached NVIDIA card reaches system RAM over its PCIe connection. Same computer, different memory access.
Windows fallback can hide an overloaded card
Fallback can begin before dedicated VRAM fills. NVIDIA designed that early switch to keep the application running, so a sudden slowdown doesn't require the memory counter to hit its ceiling first.In the NVIDIA Control Panel, the relevant option is CUDA - Sysmem Fallback Policy under Manage 3D settings. Prefer No Sysmem Fallback disables this behavior for the selected program, while Driver Default restores the driver's normal policy. Neither choice adds memory to the card.
For a Python-based image generator, the selected program must be the Python executable actually running the workload. Launch it, find that process in Task Manager, and use its file path in Program Settings.
Pointing the setting at another Python installation won't configure the running interpreter. Apply the change and restart the generator before comparing results, or you'll still be testing the old setting.
Turning fallback off can replace a slow render with an out-of-memory error. Keeping it enabled may suit a job you can leave running, but finishing without a crash says nothing about whether the extra runtime is acceptable.
Task Manager's Performance tab separates dedicated GPU memory from shared system memory. Watch the NVIDIA card's counters during the same job, rather than treating the displayed shared-memory allowance as additional onboard VRAM. A capacity figure alone cannot tell you where your model's data resides.
WSL keeps its own memory restrictions
CUDA managed memory is a separate mechanism, with capabilities that depend on the platform. Supported native Linux systems can oversubscribe GPU memory, meaning an application can allocate managed data larger than the card's physical memory.Windows and WSL don't provide the same full support. Under CUDA's limited-support model, managed allocations cannot exceed physical GPU memory, and the GPU can't fetch individual memory pages on demand. A Windows fallback success doesn't establish that a Linux application inside WSL can do the same.
An application advertising unified memory support needs to check the device's concurrentManagedAccess capability. NVIDIA identifies a zero value with the restricted behavior, so the GPU's model name alone won't determine compatibility.
Linux also distinguishes ordinary GPU allocations from managed allocations. A program using cudaMalloc doesn't get access-based memory migration just because the operating system supports it for cudaMallocManaged, so the application's allocation method still matters.
WSL also limits pinned system memory, which an application keeps resident for GPU access. Training workloads can hit this separate limit before exhausting the machine's ordinary RAM. Free ordinary RAM doesn't prove enough pinned memory is available.
Installing a native Linux NVIDIA display driver inside WSL won't remove those restrictions. WSL uses the Windows host driver, and NVIDIA warns against replacing it with a Linux driver. A Linux CUDA toolkit and a Linux display driver aren't interchangeable installations.
CPU offload can still execute on the GPU
Application-level offloading gives you another way to handle a model that exceeds VRAM. With Accelerate's CPU offload, weights live in system memory between uses and move onto the execution device when needed. Selecting the GPU as that device keeps the calculations there.The benefit of keeping idle weights in system RAM is space for the next GPU operation. Your application decides when each part moves, instead of waiting for the driver to react to crowded VRAM.
For image generation in Diffusers, sequential CPU offload transfers small model components as they're needed. This saves more VRAM, but repeated transfers can make generation painfully slow.
Model CPU offload moves larger components and keeps a repeatedly used component on the GPU until its work finishes. It needs more graphics memory, but avoids repeatedly transferring that component during the same stage. The trade-off sits in the offload setting, not the OCuLink cable.
Suppose your image pipeline's text encoder fits on the card, but its denoising model exceeds the available VRAM. Whole-model offload cannot solve that mismatch by itself. You need finer-grained offload or another method that reduces the active component's memory requirement.
With sequential offload, initialization order matters too. Diffusers warns against moving the pipeline to CUDA before enabling sequential CPU offload, because doing so leaves only minimal memory savings. A script can contain the right offload call and still retain the memory problem through an earlier GPU transfer.
