Skip to content

Cosmos 3 Nano Diffusers CPU Offloading - #371

Closed
atharvajoshi10 wants to merge 2 commits into
mainfrom
cookbook/diffusers-cpu-offloading
Closed

atharvajoshi10 wants to merge 2 commits into
mainfrom
cookbook/diffusers-cpu-offloading

Conversation

@atharvajoshi10

@atharvajoshi10 atharvajoshi10 commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

Adds a Cosmos3-Nano CPU-GPU offloading cookbook built on the Diffusers modular pipeline.

  • Keeps nonresident transformer weights in system RAM and streams blocks to the GPU, while computation remains on the GPU.
  • Supports BF16 and native-policy FP8 generation across text-to-video, image-to-video, video-to-video, sound, Action, and Transfer.
  • Automatically manages prefetching, VAE tiling, and memory-constrained execution.
  • Includes usage instructions and BF16/FP8 benchmark scripts with timing, memory, and safety reporting.

Enables researchers to run full generations on lower-VRAM GPUs, including the 12 GB RTX 5070, without modifying Diffusers. Trades additional host RAM and transfer latency for reduced GPU-memory requirements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants