Skip to content

Document that Firefox cannot run the WebGPU notebook's q4 model #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
decitre merged 1 commit into from
Nov 4, 2026
Merged

Document that Firefox cannot run the WebGPU notebook's q4 model #11

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion notebooks/ch13_webgpu.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
{
"id": "intro",
"cell_type": "markdown",
"source": "# Proving inference runs on the GPU\n\n[Back to the introduction](ch01_demo.ipynb). Run in order in a fresh kernel, in a browser with WebGPU (recent Chrome or Edge; Safari 26 and Firefox 141 on some platforms). This loads SmolLM2-135M-Instruct (q4, about 120 MB) twice: once with `backend=\"webgpu\"`, once with `backend=\"wasm\"`.\n\nA successful `backend=\"webgpu\"` load alone is weak evidence, so this notebook collects three independent proofs:\n\n1. **A hardware GPU exists**: the browser exposes a WebGPU adapter that is not a software fallback.\n2. **The model's tensors live in GPU memory**: with WebGPU, Transformers.js keeps the attention key/value cache in `gpu-buffer` tensors; with WASM every output is a `cpu` tensor. lmLite maps `backend=\"webgpu\"` to the WebGPU execution provider only, so a load cannot silently fall back to WASM.\n3. **It behaves like a GPU**: the same greedy generation, timed on both backends.",
"source": "# Proving inference runs on the GPU\n\n[Back to the introduction](ch01_demo.ipynb). Run in order in a fresh kernel, in a browser with WebGPU: recent Chrome or Edge, or Safari 26. **Firefox cannot run this notebook's q4 model on WebGPU:** 4-bit kernels need 10 storage buffers per shader and Firefox allows 9, so loading fails with `Too many storage buffers in shader. Current: 10, Max is 9`. fp16 and fp32 models do run on WebGPU in Firefox. This loads SmolLM2-135M-Instruct (q4, about 180 MB) twice: once with `backend=\"webgpu\"`, once with `backend=\"wasm\"`.\n\nA successful `backend=\"webgpu\"` load alone is weak evidence, so this notebook collects three independent proofs:\n\n1. **A hardware GPU exists**: the browser exposes a WebGPU adapter that is not a software fallback.\n2. **The model's tensors live in GPU memory**: with WebGPU, Transformers.js keeps the attention key/value cache in `gpu-buffer` tensors; with WASM every output is a `cpu` tensor. lmLite maps `backend=\"webgpu\"` to the WebGPU execution provider only, so a load cannot silently fall back to WASM.\n3. **It behaves like a GPU**: the same greedy generation, timed on both backends.",
"metadata": {}
},
{
Expand Down
Loading