Skip to content

logmessaged: drop records too big for the msgq queue instead of dying - #314

Open
PeterPhuTran wants to merge 1 commit into
FrogAi:MAKE-PRS-HEREfrom
PeterPhuTran:fp-logmessaged-queue-guard
Open

logmessaged: drop records too big for the msgq queue instead of dying#314
PeterPhuTran wants to merge 1 commit into
FrogAi:MAKE-PRS-HEREfrom
PeterPhuTran:fp-logmessaged-queue-guard

Conversation

@PeterPhuTran

Copy link
Copy Markdown

Replaces #312, which was auto-closed by an accidental force-push to its head branch (same fork-tooling mishap). Content unchanged.

Description

logmessaged re-publishes every log record onto the logMessage msgq socket. msgq
queues are 256KiB and msgq_msg_send() asserts that a message fits in a third of
the queue — so any record over ~85KB is a SIGABRT. The existing guard only rejected
records over 2MB, leaving an 85KB–2MB band where a single large log record kills the
daemon. And since a crashed logmessaged takes down the whole logging pipeline, the
crash itself can never be logged: the only symptom is a permanent
"Process Not Running: logmessaged" and silent loss of all swaglogs for the drive.

Hit in the field on 2026-07-10 (same device as #309/#311): updated logged a 130KB
git diff output record for a dirty working tree; logmessaged died seconds after
boot and stayed dead all day.

Fix: lower the guard to fit the actual queue capacity (MAX_PUBLISH_BYTES = 80KB).
Oversized records are still written to the on-disk log (the file write happens before
the publish) — they're only dropped from the msgq stream, exactly like the existing

2MB behavior.

Verification

New test_medium_log in system/tests/test_logmessaged.py, validated on a comma three:

  • red on stock: a 200KB record kills logmessaged (msgq assertion)
  • green with fix: logmessaged survives, later records still flow on the socket,
    and both records land in the on-disk log

Existing test_simple_log passes. Note: test_big_log fails on-device with and
without this change (it globs the live /data/log while a running logmessaged
rotates files — a pre-existing test-environment race; it passes in isolated CI).

Related: #311 flags logmessaged restart_if_crash as defense in depth; this PR
removes the crash trigger itself.

🤖 Generated with Claude Code

logmessaged re-publishes every log record onto the logMessage msgq
socket. msgq queues are 256KiB and msgq_msg_send() asserts that a
message fits in a third of the queue, so any record over ~85KB is a
SIGABRT - but the existing guard only rejected records over 2MB.
Records in the 85KB-2MB band therefore killed the daemon, and since a
crashed logmessaged takes down the whole logging pipeline, the crash
itself is never logged and the processNotRunning alert persists for
the rest of the drive.

Hit in the field on 2026-07-10: updated logged a 130KB 'git diff
output' record for a dirty working tree, logmessaged died at boot and
stayed dead all day (see FrogAi#311 discussion).

Lower the guard to fit the actual queue capacity. Oversized records
are still written to the on-disk log (the file write happens before
the publish) - they are only dropped from the msgq stream, same as
the existing >2MB behavior.

Adds test_medium_log: a 200KB record must not kill logmessaged, later
records must still flow, and both must land on disk.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Thanks for contributing! A maintainer will review your pull request soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant