Skip to content

feat(reader): add manga OCR and translation - #1252

Merged
everpcpc merged 4 commits into
mainfrom
feat/manga-translation
Oct 12, 2026
Merged

everpcpc merged 4 commits into
mainfrom
feat/manga-translation

Conversation

@everpcpc

@everpcpc everpcpc commented Oct 11, 2026 •

Copy link
Copy Markdown
Member

What

  • Add on-device manga text detection and OCR with downloadable, hash-verified model packages, component manifests, and pluggable detector/recognizer adapters.
  • Add a catalog and model-management flow with storage reporting and deletion. The catalog lists all manga_ocr archives and compatibility is validated before download; model binaries remain outside the app bundle, and loaded models are released on memory warning on iOS/tvOS and when the reader closes.
  • Add OpenAI-compatible providers with GRDB-backed configuration, a 30-second request timeout, source/target language controls, optional comic metadata, and an on-device system translation fallback.
  • Translate committed pages automatically and render translated bubbles across paged, rotated, cropped, split, RTL, and Webtoon presentations on iOS and macOS, with phase feedback, reader notifications, cancellation, and original/translated toggling.
  • Make Live Text and manga translation mutually exclusive, including the Live Text keyboard shortcut, and remove shake-to-open Live Text in favor of the floating reader action.
  • Use localized language identifiers for language display names app-wide.
  • Add per-instance translation and provider migrations, settings and model management, full localization, and reader lifecycle cleanup.

Why

Bring manga translation to the reader without uploading page images, while keeping OCR models replaceable and giving users control over their translation provider and on-device resources.

Validation

  • make format — passed.
  • make build — passed on iOS, macOS, and tvOS.
  • make localize and ./misc/translate.py list — no missing translations.
  • make run-ios-sim — installed and launched on the iPhone 18 Pro simulator for manual testing.
  • git diff --check — passed.

@everpcpc everpcpc left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review pass: 4 standards reviewers (Core/storage, services, views/settings, reader integration) + 1 spec reconciliation against the PR description.

Verdict: hard rules clean (no unsafe hatches / force casts / object-style environment; GRDB, RawCodableStore, and localization discipline all pass) and the spec checks out end-to-end — page images never leave the device (provider payloads carry text segments only). 4 blockers inline to fix before merge.

Spec-axis gaps (description, not code):

  • This PR removes "shake to open Live Text" entirely (ShakeGestureHandler.swift + toggles in DivinaPreferencesView/ReaderSettingsSheet + AppConfig.shakeToOpenLiveText). The deletion is clean (zero references remain; superseded by the floating button) and the Live Text ↔ manga translation mutual exclusion in AppConfig.setLiveTextEnabled/setMangaTranslationEnabled is deliberate, but neither behavior change is mentioned in the description — please document them.
  • LanguageCodeHelper.displayName(for:) now uses localizedString(forIdentifier:), which changes language names app-wide (e.g. en-US now shows the region full name, zh-Hant renders as "Chinese (Traditional)") — worth a line in the description.
  • Memory-warning cleanup is iOS/tvOS-only (MangaModelMemoryWarningObserver is #if os(iOS) || os(tvOS)); macOS has no public memory-warning API, but the description states the behavior without the platform qualifier.
  • The catalog UI does not pre-filter incompatible models — validateSupport only rejects at download time. Acceptable, but worth noting.

Minor notes (not worth inline threads):

  • MainApp.swift:79: the startup Live Text mutex can never fire (all write paths already enforce exclusivity) — harmless, keep or drop.
  • ModelArchive.swift:80 isHexDigit accepts full-width hex digits; the digest comparison is fail-closed, harmless.
  • AppConfig.modelLibrary read-modify-write races (catalog fetch vs register/select/remove) can overwrite each other; consistent with existing AppConfig patterns, low probability.
  • MangaOverlayView iOS/macOS branches duplicate ~100 lines of layout logic — follows the existing NativePageItem_iOS/_macOS precedent, noted for the record.
  • SettingsSection.swift:86: mangaTranslation reuses .blue, same as the adjacent reading section.

CI: tvOS passed; iOS/macOS still pending at review time.

Comment thread KMReader/Features/MangaTranslation/Services/MangaTranslationService.swift Outdated
Comment thread KMReader/Features/MangaTranslation/Services/MangaTARArchive.swift
Comment thread KMReader/Core/Storage/GRDB/LocalDatabase.swift
Comment thread KMReader/Features/MangaTranslation/Models/MangaProviderConfig.swift
Comment thread KMReader/Features/Reader/ViewModels/ReaderViewModel+MangaTranslation.swift Outdated
Comment thread KMReader/Features/Reader/ViewModels/ReaderViewModel+MangaTranslation.swift Outdated
Comment thread KMReader/Core/Models/ModelPurpose.swift Outdated
Comment thread KMReader/Core/Models/ModelCatalogService.swift
@everpcpc

Copy link
Copy Markdown
Member Author

Verified 7abf75f against all 29 review threads — every blocker and nit is addressed.

Blockers

  • Inference now runs detached via runInference with cancellation propagation, and all three loops (RT-Detr query, per-region OCR, per-token decode) check Task.checkCancellation() — the actor stays responsive and cancellation is prompt mid-inference.
  • TAR validation strips the trailing slash before isSafe, so standard directory entries pass.
  • Cached .recognitionOnly(.noText) results are treated as terminal when !forceProviderRetry — no more full OCR re-runs on revisit.
  • clearInstanceData now deletes manga_translations rows for the instance.

Nits — all resolved, notably: corrupt packages self-heal (delete + re-download) and load() only drops the offending compiled model on MLModelError.io; stale .part is removed before writing; apiKey semantics unified at "required" (config layer); shared HTTP status mapping lives in MangaProviderRequest; deletingStorageKey split from preparingStorageKey; rotation/borderCropMode compared when matching results; showTranslation persisted on toggle; PageSplitMode gained a String rawValue for stable cache keys; named constants for token IDs and the inference cache version; logits shape validated; the redundant @AppStorage re-syncs are gone.

The PR description now documents the shake-to-open Live Text removal and the Live Text ↔ translation mutual exclusion — spec gap closed. CI green on iOS/macOS/tvOS.

One leftover observation, non-blocking: the book_id/page_number columns and idx_manga_translations_book still have no query consumer (clearInstanceData filters by instance_id), and the schema freezes on merge — wire them up or drop them whenever convenient.

LGTM.

@everpcpc
everpcpc merged commit 5439e1d into main Oct 12, 2026
3 checks passed
@everpcpc
everpcpc deleted the feat/manga-translation branch October 12, 2026 01:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant