Skip to content

Add NPU 910B (Ascend) support for Wan2.2-S2V-14B - #385

Open
0xPabloLI wants to merge 2 commits into
Wan-Video:mainfrom
0xPabloLI:feature/npu-910b-support
Open

0xPabloLI wants to merge 2 commits into
Wan-Video:mainfrom
0xPabloLI:feature/npu-910b-support

Conversation

@0xPabloLI

Copy link
Copy Markdown

Summary

This PR adds support for running Wan2.2 on Huawei Ascend NPU 910B (32GB HBM + CPU offload), enabling free inference on platforms like AtomGit which provide NPU 910B 64GB instances at no cost.

What's included

  • scripts/patch_npu.py — A non-invasive patch script that applies 10 runtime modifications to enable NPU support:

    1. torch.cuda.*torch.npu.* mapping + import torch_npu
    2. NCCL → HCCL distributed backend
    3. Staged loading: release T5 encoder before loading diffusion model (critical for 32GB HBM)
    4. device_map='auto' for automatic HBM/CPU offload via accelerate
    5. Skip .to(device)/.cpu() for accelerate-managed models (_hf_hook check)
    6. flash_attention → SDPA fallback (NPU has no flash_attn package)
    7. VAE dtype fix (encode input .float() cast)
    8. dtype assert.float() cast in model_s2v.py
    9. max_memory={0: '15GB', 'cpu': '60GB'} tuning
    10. empty_cache() before VAE decode
  • docs/NPU_SUPPORT.md — Complete documentation with setup instructions, patch overview, and test results.

Test Results

Configuration Value
Hardware NPU 910B 64GB (AtomGit free tier)
Model Wan2.2-S2V-14B
Resolution 480×832
Steps 5
Frames 40
Inference time ~550s (9.2 min)
Core-hours ~2.47
Output 480×832 H264, 40 frames, 1.8MB
Cost Free

Design decisions

  • Non-invasive: The patch is a standalone script rather than direct source modifications, making it easy to review and opt-in. Users run python scripts/patch_npu.py before inference.
  • Idempotent: All patches check for existing modifications and skip if already applied.
  • Single-NPU only: Multi-NPU would require HCCL distributed setup (untested).

Environment

  • CANN 8.5, PyTorch 2.9, torch_npu 2.9

Tested by @0xPabloLI. Happy to address any feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant