Repository navigation
Conversation
Read JSONL boundaries as bytes before decoding each record so an interrupted multibyte character cannot poison the whole transcript or revision scan. Keep incomplete tails eligible for refresh and preserve unrelated source bytes during metadata updates instead of truncating or replacing them. Cover Unicode boundaries, resumed appends, paged and indexed reads, tool lookup and revisions. Extend the isolated full-desktop scenario to reopen damaged history and regenerate across restart without replay on read.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow up on merged #1565 with a focused host-side fix for the UTF-8 failure discussed
in #1560. The outbox change prevents one refused append from blocking the
queue; this change keeps intact transcript records readable when an interrupted
append leaves an incomplete multibyte character. It is independently mergeable
and does not alter the outbox retry or quarantine policy.
Reproduced failure
The transcript readers use
BufRead::read_line(&mut String). An invalid UTF-8sequence fails during that read, before the existing malformed-record handling
can run. A torn Chinese/emoji record can therefore hide otherwise intact
history, including valid records appended after the damaged line.
All five added regression tests fail on unmodified main
22560e083aa7withstream did not contain valid UTF-8, and pass with this change. This establishesthe torn-record defect; it does not establish every reported occurrence's
origin or prove the proposed connection with #1515.
ends_with_newlinealreadyreads a raw byte and is left unchanged.
Changes
Log invalid UTF-8 without logging its content and skip that record on reads.
eligible for refresh when the writer supplies more bytes.
requiring the entire file to decode as UTF-8.
scenario to reopen a torn Unicode transcript before regenerating.
Reading does not truncate the transcript, replace characters, reconstruct lost
records or change the storage format. Invalid records stay on disk but are not
displayed. Existing explicit full-history rewrite behavior is unchanged.
Validation
Windows; existing Node 24 and Rust 1.90 toolchains; isolated app data and a
controlled local HTTP/SSE provider, without production credentials.
with invalid-UTF-8 errors, not fixture/build failures.
regressions: 28 passed, including the six new fix(desktop): quarantine outbox appends the host keeps refusing #1565 cases.
main/preload/renderer build, fresh sidecar bundle and release Rust host build:
passed.
cargo fmt --check,cargo clippy -p host-core --all-targets --release --locked, and diff checks:passed.
node scripts/e2e-regenerate-quit.mjs --torn-unicode: 8 passed in theactual isolated Electron application. This covers a real Read tool response,
reopening the same damaged session without a fork/model replay/file mutation,
regenerate, native quit, restart and both revision branches. The resulting
history screenshot was also inspected.
main 804 passed, the same 4 failed, 1 ignored. Failures are the two existing
config-sync engine cases and the two data-relocation capability-override
cases. The full suite is not reported as green.
06c04e99dc84after fix(desktop): quarantine outbox appends the host keeps refusing #1565 merged and reran the combinedDesktop build, adjacent regressions and actual Electron scenario.
macOS/Linux execution and the full Desktop/runtime test suites were not rerun
for this focused Rust follow-up.
Related to #1560; complements #1565.