Skip to content

feat: add Qwen3-ASR speech-to-text - #1554

Draft
msluszniak wants to merge 4 commits into
mainfrom
@ms/qwen3-asr
Draft

msluszniak wants to merge 4 commits into
mainfrom
@ms/qwen3-asr

Conversation

@msluszniak

@msluszniak msluszniak commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Description

  • New createQwen3AsrSpeechToText task for Qwen3-ASR 0.6B, same transcribe/stream API as Whisper; language is auto-detected or forced via QWEN3_ASR_LANGUAGES.
  • useSpeechToText accepts Qwen3-ASR models; registry entry models.speechToText.QWEN3_ASR.SIZE_0_6B (XNNPACK 8da4w; iOS default Core ML audio tower + MLX decoder; HF react-native-executorch-qwen-3-asr @ v0.11.0).
  • A variant key naming two backends (MLX_COREML_INT4) is the default only when both are linked; loading a method whose backend is not linked now names it.
  • Speech demo screens list the model; docs section added.

Introduces a breaking change?

  • Yes
  • No

Type of change

  • Bug fix (change which fixes an issue)
  • New feature (change which adds functionality)
  • Documentation update (improves or adds clarity to existing documentation)
  • Other (chores, tests, code style improvements etc.)

Tested on

  • iOS
  • Android

Testing instructions

  • apps/speech: pick "Qwen3-ASR 0.6B (CPU)" on the audio file or microphone screen and transcribe.

Screenshots

Related issues

#769

Checklist

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have updated the documentation accordingly
  • My changes generate no new warnings

Additional notes

  • WER on 100 LibriSpeech test-clean clips: 2.46% (fp32 2.14%). Decoder layer 0 must stay 8-bit or transcripts derail.
  • iPhone 16: Core ML + MLX 293 ms per clip vs XNNPACK 468, WER 2.51%.
  • No Vulkan: on a Galaxy S26 Ultra it wins prefill but loses encoder and decode (whole clip 451 vs 360 ms).
  • Exporters and findings: export-scripts !41. ExecuTorch fixes found on the way: Vulkan: keep a reshape that follows a dynamically quantized linear pytorch/executorch#24055, #24056.

@msluszniak msluszniak self-assigned this Oct 21, 2026
@msluszniak msluszniak added the feature PRs that implement a new feature label Oct 21, 2026
Audio tower on Core ML, decoder on MLX. A variant key naming two backends
resolves only when both are linked.
Fixes the Qwen3-ASR docs anchor and lets the feature-map test read backends
from file names.
@msluszniak
msluszniak marked this pull request as ready for review October 21, 2026 13:56
@msluszniak msluszniak linked an issue Oct 21, 2026 that may be closed by this pull request
@msluszniak
msluszniak requested a review from barhanc October 21, 2026 14:01
@msluszniak
msluszniak marked this pull request as draft October 21, 2026 16:07
@msluszniak

Copy link
Copy Markdown
Member Author

@barhanc I forgot that I need to rebuild libs for this one to cover all backends, will let you know when this will be ready since I want to batch this with Szymon's changes to MIGAN.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature PRs that implement a new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

support qwen3-asr in speech to text

1 participant