An open source tool for video subtitle generation and translation

SubMaster is a command-line Python application that takes a video as input and transcribes spoken dialogues into a synchronized subtitle file with optional translation into another language.

Features

• Accepts common video formats such as MP4, MKV, MOV, AVI, MPG/MPEG, and RMVB
• Batch-processes video files in a folder, with optional recursive subfolder scanning, using a case-insensitive video extension allowlist and stream validation
• Extracts and normalizes mono WAV audio using ffmpeg before transcription
• Supports tiny, base, small, medium, large, and turbo Whisper models
• Optionally translates subtitles into another language with Tencent HY-MT 1.5 models through llama.cpp
• Optionally polishes --transcribe output with a local Qwen3.8-9B community distillation cleanup pass through llama.cpp
• Uses Silero VAD 6.2.0 by default to exclude non-speech audio and reduce hallucinated subtitles
• Automatically embeds chapter markers from a same-directory <video-stem>.chp text file into a chapter-capable copy of the input video
• Downloads missing Whisper, VAD, HY-MT, and transcript-cleanup model files on demand into the local models/ cache
• Works on Linux, macOS, and Windows when ffmpeg, whisper.cpp, and llama.cpp are available
• Prefers GPU execution in --device auto when the selected native runtime appears GPU-capable
• Disables rolling Whisper text context by default to reduce long-form repetition loops
• Produces normalized .srt output with clean cue numbering and timestamps

Platform Support

Platform	Status	Notes
Linux	Supported	Includes a bundled Linux x86_64 whisper-cli CPU fallback at whisper/whisper-cli
macOS	Supported	Requires installed or locally built whisper.cpp and llama.cpp executables
Windows	Supported	Requires installed or locally built whisper.cpp and llama.cpp executables; common .exe build outputs are auto-detected

The bundled fallback is only for whisper.cpp on Linux x86_64. Translation and transcript cleanup always require a working llama-cli.

Requirements

• Python 3.10+
• ffmpeg and ffprobe
• whisper-cli and llama-cli executables — provided by setup_runtimes.py (see Quick Setup below)

submaster has no third-party Python runtime dependencies. The package metadata in pyproject.toml allows Python >=3.10. Python 3.12 is the default tested setup in this repository.

Quick Setup

Run the included setup script from the project root to clone and build both native runtimes automatically:

python setup_runtimes.py

The runtime setup script:

1. Detects your GPU (CUDA, Vulkan, Metal, or CPU fallback)
2. Checks for required build tools (git, cmake, C/C++ compiler, ninja) and prints per-platform install hints for anything missing
3. Clones whisper.cpp and llama.cpp into the project root and builds them with the correct flags
4. Verifies the resulting executables — python -m submaster picks them up automatically from those locations

Run python -m submaster from this project root so it uses the local models/ cache and the runtimes built by setup_runtimes.py. Pass videos or folders by absolute path, ~ path, or a path relative to the project root.

Install build prerequisites first:

• Ubuntu or Debian: sudo apt install git cmake ninja-build build-essential
• macOS with Homebrew: brew install git cmake ninja
• Windows: install Git, CMake, and Ninja via winget, Chocolatey, or Scoop

For CUDA builds, the NVIDIA CUDA Toolkit is also required. For Vulkan builds, install the LunarG Vulkan SDK (Windows) or libvulkan-dev (Linux).

Common options:

python setup_runtimes.py --backend cpu      # force a CPU-only build
python setup_runtimes.py --backend cuda     # force CUDA regardless of detection
python setup_runtimes.py --no-update        # offline: skip git pull
python setup_runtimes.py --no-rebuild       # skip build, re-verify executables only

On Windows, Ninja must be installed (via winget install Ninja-build.Ninja, Chocolatey, or Scoop) so cmake can run from any terminal. Without Ninja, cmake falls back to NMake Makefiles, which requires a Visual Studio Developer Command Prompt.

Usage

First cd to the SubMaster checkout, then run the command from there:

• Basic transcription: python -m submaster /path/to/input.mp4
• Batch-process all video files in a folder: python -m submaster /path/to/videos
• Batch-process videos in a folder and all child subfolders: python -m submaster /path/to/videos --recursive
• Translate the generated subtitles into Italian: python -m submaster /path/to/input.mp4 --translate it
• Generate a plain-text transcription and clean it locally: python -m submaster /path/to/input.mp4 --transcribe --cleanup
• Summarize the cleaned transcription: python -m submaster /path/to/input.mp4 --transcribe --cleanup --summarize
• Automatically embed chapter markers: place an input.chp file next to /path/to/input.mp4, then run the normal transcription command

Options

If --translate is set, the written .srt file keeps the original-language subtitles and an additional <output-stem>_<language_code>.srt file is written with the translated subtitles.

--transcribe writes a companion plain-text <video-stem>_transcript.txt file next to the subtitle output. Add --cleanup to also write <video-stem>_cleanup.txt with the cleaned-up transcription. Add --summarize alongside both flags to write <video-stem>_summary.txt from the cleaned text using the same Qwen model. It has no effect if either prerequisite is missing. Summary files follow the same output-directory and --overwrite rules.

For each input video, SubMaster automatically looks for a same-directory <video-stem>.chp text file. For example, processing movie.mp4 checks for movie.chp. If the file exists, SubMaster reads its chapter timestamps and produces a chapter-embedded copy next to the original. MP4 and other modern chapter-capable containers retain their original extension. Legacy RMVB, AVI, and MPG/MPEG inputs are written as <video-stem>_chapters.mkv because their original containers do not reliably support chapters. AVI and MPG/MPEG streams are copied without re-encoding when compatible. RMVB video is converted from RV40 to H.264 and its Cook audio to AAC because copying RV40 into Matroska loses decoder initialization data and produces an unplayable file. The H.264 bitrate is calculated from the source container bitrate, reserving 96 kb/s for AAC audio and a small Matroska overhead allowance so the converted file remains approximately the same size as the source. Other legacy inputs automatically fall back to AAC audio or H.264/AAC conversion only when stream-copy is rejected. If no .chp file exists, subtitle creation proceeds normally without chapter embedding. Each non-empty chapter line must follow the format HH:MM:SS Title. In folder mode, every video is matched with its own .chp sidecar independently.

No additional Python packages are required for these formats. The existing ffmpeg and ffprobe requirements handle remuxing and conversion. RMVB conversion requires an FFmpeg build with the libx264 H.264 encoder and the native AAC encoder; the standard Linux Mint/Ubuntu ffmpeg package includes both.

Example chapter file:

00:00:00 Introduction
00:23:20 Start
00:40:30 First Performance
00:40:56 Break
01:04:44 Second Performance
01:24:45 Crowd Shots
01:27:45 Credits

--range START END limits extraction, transcription, and optional translation to the selected source clip while keeping subtitle timestamps aligned to the original video timeline. Accepted formats are SS, MM:SS, or HH:MM:SS with optional .mmm or ,mmm milliseconds.

Silero VAD 6.2.0 is enabled by default so music and silence are excluded before transcription. Use --no-vad for recordings where quiet, very short, or heavily masked speech is being missed. --vad-model can explicitly select any supported VAD model.

When the positional input is a folder, SubMaster first filters direct child files by video extension (case-insensitive), then validates candidates with ffprobe; add --recursive to include child subfolders. Supported extensions are .3g2, .3gp, .asf, .avi, .divx, .f4v, .flv, .m2t, .m2ts, .m2v, .m4v, .mkv, .mov, .mp4, .mpe, .mpeg, .mpg, .mpv, .mts, .mxf, .ogm, .ogv, .qt, .rm, .rmvb, .ts, .vob, .webm, and .wmv. Other files (including images and text) are skipped without probing. Candidates that are unreadable or do not expose a video stream are also skipped. Batch outputs default to <source-stem>.srt next to each source file, or into a shared directory when --output DIR is provided.

--max-context controls how much previously decoded text whisper.cpp feeds back into later decode windows. Submaster defaults this to 0 to reduce repetition loops on long recordings. Pass --max-context -1 to restore the upstream whisper.cpp behavior.

Translation and Transcription

Translation Backend

Subtitle translation uses Tencent HY-MT 1.5 GGUF models through llama.cpp.

Available translation sizes:

• small: HY-MT1.5-1.8B-Q4_K_M.gguf
• large: HY-MT1.5-7B-Q4_K_M.gguf

The project intentionally downloads the Q4_K_M variants instead of full-precision or FP8 checkpoints. For this CLI, the int4 GGUF models are the most suitable choice because they keep downloads, RAM usage, and VRAM usage reasonable while still fitting the local/offline execution goal on consumer hardware.

The models are downloaded automatically into models/ the first time translation is requested.

Transcript Cleanup Backend

Plain-text transcription cleanup uses Empero Qwen3.8-9B Distill Q4_K_M GGUF through llama.cpp.

This is a community distillation of Qwen3.8 into the Qwen3.5-9B architecture, not an official Alibaba 9B release. It retains nine billion parameters and the Q4_K_M quantization used previously (5.78 GB weights), suitable for an RTX 4070. The official Qwen3.8 family starts at 27B; this size choice preserves local resource requirements. Better transcript quality is not guaranteed by its reasoning benchmarks.

Cleanup uses a non-thinking Qwen chat prompt, including on llama-completion. Chat mode is used only when the runtime also supports disabling thinking through chat-template options; otherwise Qwen framing and an empty reasoning turn are supplied explicitly. Tagged reasoning is removed; unfinished reasoning or a leading “Thinking Process:” response retries the affected section in smaller pieces (at most two split levels). If those attempts fail, the run reports an error rather than saving reasoning as successful output. The same recovery applies to summarization; runtime errors are not retried.

• Enabled with --transcribe --cleanup
• Uses a chunked cleanup pipeline with a 16K llama.cpp context window
• Keeps subtitle generation unchanged and writes a cleaned companion .txt file
• Has no effect when --transcribe is not enabled

Summarization uses TRANSCRIPT_SUMMARY_SYSTEM_PROMPT in submaster/config.py, preserves the source language, and condenses key points and practical steps without adding facts. Long transcripts are summarized in chunks; section summaries are combined in a final pass when they fit the chunk limit, otherwise retained in source order.

The cleanup model is downloaded automatically into models/ the first time transcription polishing is requested.

How Native Runtimes Are Located

whisper.cpp

Lookup order:

• WHISPER_CPP_CLI environment variable
• Conda, PATH, and common local build outputs
• bundled repo fallback on Linux x86_64 only

Common local build outputs (produced by setup_runtimes.py):

• Linux and macOS: ./whisper.cpp/build/bin/whisper-cli
• Windows: .\whisper.cpp\build\bin\Release\whisper-cli.exe

llama.cpp

Lookup order:

• LLAMA_CPP_CLI environment variable
• Conda, PATH, and common local build outputs

Common local build outputs (produced by setup_runtimes.py):

• Linux and macOS: ./llama.cpp/build/bin/llama-cli
• Windows: .\llama.cpp\build\bin\Release\llama-cli.exe

Validation

Run the test suite:

python -m unittest

Upstream References

• OpenAI Whisper: https://github.com/openai/whisper
• whisper.cpp: https://github.com/ggml-org/whisper.cpp
• whisper.cpp model files: https://huggingface.co/ggerganov/whisper.cpp
• whisper.cpp VAD model files: https://huggingface.co/ggml-org/whisper-vad/tree/main
• Tencent HY-MT1.5-1.8B-GGUF: https://huggingface.co/tencent/HY-MT1.5-1.8B-GGUF
• Tencent HY-MT1.5-7B-GGUF: https://huggingface.co/tencent/HY-MT1.5-7B-GGUF
• Qwen3.8-9B community GGUF: https://huggingface.co/empero-ai/Qwen3.8-9B-Distill-GGUF
• llama.cpp: https://github.com/ggml-org/llama.cpp