Add FP8 to FP16 conversion for MPS compatibility#14606
Conversation
Convert FP8 tensors to FP16 during model loading (safetensors and .pt/.ckpt) and add a fallback safety net in cast_to for any FP8 tensors that slip through at inference time, preventing MPS runtime errors. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (5)
📝 WalkthroughWalkthroughThis PR makes three independent sets of changes. First, FP8 tensor coercion for MPS devices is added: ✨ Finishing Touches⚔️ Resolve merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment Warning |
Summary
cast_toto catch any FP8 tensors that slip through at inference timeTest plan
🤖 Generated with Claude Code
API Node PR Checklist
Scope
Pricing & Billing
If Need pricing update:
QA
Comms