Skip to content

Add offline article-image caching behind a setting#5352

Open
davidfwatson wants to merge 2 commits into
Ranchero-Software:mainfrom
davidfwatson:feature/offline-article-images
Open

Add offline article-image caching behind a setting#5352
davidfwatson wants to merge 2 commits into
Ranchero-Software:mainfrom
davidfwatson:feature/offline-article-images

Conversation

@davidfwatson

@davidfwatson davidfwatson commented Jul 2, 2026

Copy link
Copy Markdown

Adds an optional, off-by-default setting that caches article images during refresh and
serves them from a local cache at render time, so articles read with images while offline
(planes, trains, tunnels, metered data). Addresses #1873; a step toward #3460.

Design note: what happens when you turn it on

Prefetch only catches images from future refreshes, so the moment you enable this —
typically right before you lose connectivity — your existing unread articles still have
no cached images. To bridge that, enabling the setting offers to cache images for your
current unread articles right away
(a "Cache Images Now? / Not Now" prompt showing the
unread count). There's deliberately no standalone "cache everything" button: if you want
to backfill again later, toggle the setting off and back on.

image image

How it works

Two halves sharing an on-disk cache:

  • Prefetch: ArticleImagePrefetcher watches .AccountDidDownloadArticles, extracts
    <img> URLs off the main thread (a sync can deliver a large batch at once), and fetches
    them via ArticleImageDownloader — throttled, not one unbounded burst. The pure
    URL-extraction is a nonisolated, unit-tested ArticleImageURLExtractor.
  • Serve: with the setting on, main.js rewrites http(s) <img src> to
    nnwimage://cache/?u=…; ArticleImageSchemeHandler (a WKURLSchemeHandler) resolves
    those memory → disk → network. Uncached images load online-only; cached ones load
    offline. On rewrite it also drops the img's srcset and, inside a <picture>, its
    sibling <source> candidates, so WebKit can't select an original https URL that
    bypasses the cache. The enable flag reaches main.js as a hidden DOM marker rather than
    an inline <script>, so it applies even when the reader has article JavaScript off (that
    setting gates content JS; main.js runs as an injected user script).

ArticleImageDownloader mirrors the existing ImageDownloader/FaviconDownloader pattern
(same Downloader, ImageMetadataDatabase backoff, BinaryDiskCache) but keeps full-size
bytes in a separate ArticleImages folder. Same-URL requests coalesce; the in-memory
NSCache is bounded (~50 MB, cleared on background/low-memory); only plausible image
responses persist (an image/* type or recognized magic bytes, under a 20 MB cap). App-side
fetches reuse the web view's content blocker via ArticleImageContentBlocker, so tracker/ad
URLs stay blocked — including on cross-host redirects. The ArticleImages folder is swept by
the existing 3-day CacheCleaner flush, and purged outright when the setting is off.

The setting

cacheImagesForOffline, off by default — iOS: Settings → Articles; Mac: Settings → General.
While a backfill runs, the settings line shows live progress ("Caching Images… N of M");
when idle it shows the current cache size ("166 images · 63.5 MB used"). iOS account-stats
gains an Offline Images section with the same figures.

Key limitations

  • Custom-scheme routing bypasses WebKit's own image cache and doesn't send its cookies/referer,
    so a few hotlink-protected images may fail when the setting is on.
  • Enclosure/media caching (video, audio, PDF) is out of scope ([Feature] Enclosure caching #5284).
  • No pre-buffer byte cap: the 20 MB limit keeps oversized bodies out of the persistent cache,
    but the shared RSWeb.Downloader still buffers whole bodies first. A streaming cap belongs in
    Downloader (it affects every image path) and is left as shared-infra follow-up.

Testing

  • macOS and iOS both build clean.
  • ArticleImageURLExtractorTests covers absolute / relative / protocol-relative resolution,
    scheme lowercasing, data-canonical-src preference, entity decoding, non-http rejection,
    dedup, ordering.
  • Manual: enable → accept the prompt → Airplane Mode → confirm article images still load.

(Full internals live in the three commit messages: core caching, on-enable backfill, and the
Mac prefs-pane sizing fix.)

@davidfwatson
davidfwatson force-pushed the feature/offline-article-images branch 3 times, most recently from 825dad8 to acba163 Compare July 2, 2026 11:44
@davidfwatson
davidfwatson force-pushed the feature/offline-article-images branch 2 times, most recently from c762b1c to e4a6df7 Compare July 4, 2026 11:03
@davidfwatson

Copy link
Copy Markdown
Author

Rebased onto the latest main (now at 808403b00).

Since this PR was opened, main refactored RSWeb.Downloader.download(_:) to return a DownloadResponse struct instead of a (Data?, URLResponse?) tuple. The rebase merged cleanly textually but broke the build at that API boundary, so I updated ArticleImageDownloader.performDownload to the new signature — matching the sibling ImageDownloader / SingleFaviconDownloader call sites. No behavioral change. Both the iOS and macOS targets archive clean again.

@davidfwatson
davidfwatson force-pushed the feature/offline-article-images branch 2 times, most recently from bff01e2 to 4748ec9 Compare July 14, 2026 05:07
davidfwatson and others added 2 commits July 13, 2026 22:46
Optional, off-by-default caching of the images embedded in article HTML:
prefetched during refresh and served from a local cache at render time, so
articles can be read with their images while offline. Addresses Ranchero-Software#1873; a step
toward the unified image handling in Ranchero-Software#3460.

- ArticleImagePrefetcher observes .AccountDidDownloadArticles and, for each new
  article, extracts its <img> URLs (off the main thread, via the unit-tested
  ArticleImageURLExtractor) and fetches them through ArticleImageDownloader into
  a disk cache, throttled rather than in one unbounded burst.
- With the setting on, main.js / main_ios.js rewrite http(s) <img src> to
  nnwimage://cache/…; ArticleImageSchemeHandler resolves those memory → disk →
  network. The img's srcset — and, inside a <picture>, its <source> candidates —
  are dropped on rewrite so WebKit can't select an original URL that bypasses the
  cache offline. The enable flag is passed to main.js as a hidden DOM marker (not
  an inline <script>) so it still applies when the reader has article JavaScript
  turned off.
- ArticleImageDownloader mirrors ImageDownloader/FaviconDownloader (full-size
  bytes in a separate ArticleImages folder, ~50MB NSCache, 20MB disk-write cap
  with image-type sniffing). App-side fetches reuse the web view's content
  blocker via ArticleImageContentBlocker, so tracker/ad URLs stay blocked —
  including on cross-host redirects. The folder is swept by the 3-day
  CacheCleaner and purged outright when the setting is off.
- Prefetch only catches future refreshes, so switching the setting on offers to
  backfill images for the current unread articles right away (a "Cache Images
  Now? / Not Now" prompt). There's no standalone button — toggle off and on to
  backfill again later. A progress line and the current cache size appear under
  the setting, and in the iOS account-stats screen.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cu69KNiXXmKjfmgcnS8RZU
resizeWindow(toFitView:) sized the (non-resizable) Preferences window to each
pane's fixed storyboard frame, so a pane whose content outgrew that frame — like
General after gaining the offline-images rows — was clipped off the bottom.

Measure the pane's real Auto Layout fitting height at the fixed window width
instead — before the view is added to the window hierarchy, so its height is
free — and size the window to that, letting panes grow to fit rather than
squishing or clipping their last control. A <2000pt clamp guards against the
old unbounded-width measurement ballooning the multiline label.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cu69KNiXXmKjfmgcnS8RZU
@davidfwatson
davidfwatson force-pushed the feature/offline-article-images branch from 4748ec9 to fbc8c46 Compare July 14, 2026 05:46
@davidfwatson
davidfwatson marked this pull request as ready for review July 14, 2026 07:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant