Skip to content

feat(overseer): opt-in LLM summary fallback (#90) - #91

Open
heavygee wants to merge 16 commits into
feat/overseer-summary-flavors-and-datesfrom
feat/overseer-llm-fallback
Open

feat(overseer): opt-in LLM summary fallback (#90)#91
heavygee wants to merge 16 commits into
feat/overseer-summary-flavors-and-datesfrom
feat/overseer-llm-fallback

Conversation

@heavygee

Copy link
Copy Markdown
Owner

Summary

Implements fork issue #90 Option A: opt-in OpenAI-compatible LLM fallback when extractNotifySummary is null.

  • Default OFF via HAPI_OVERSEER_LLM_FALLBACK (+ base URL / model / optional key / chat-completions|responses)
  • Full last-turn assistant text (no input-char cap); failures fall through to existing heuristic first-line fallback
  • Provenance hub-llm-fallback; payload.synthesis=llm-fallback; attn=0 (Session Log only, not inbox/voice)
  • Adapter + recorder tests; docs in plan + hub README
  • Restores cli/src/modules/common/skillLookupInstruction.ts so the stacked tip typechecks after parent feat(overseer): non-Cursor session-summary contract + debug relative dates #88 kept the import without the file

Fork-only overseer. Stacked on feat/overseer-summary-flavors-and-dates (#88).

Test plan

  • bun test hub/src/sync/overseerLlmFallback*.test.ts hub/src/sync/overseerEventRecorder*.test.ts (37 pass)
  • bun run test:hub (544 pass)
  • bun typecheck (cli/web/hub green)
  • Optional dogfood: enable env against local Ollama, omit summary line, confirm Session Log shows hub-llm-fallback and inbox stays empty

Issues

Fixes #90

When AGENT_NOTIFY_SUMMARY is missing and HAPI_OVERSEER_LLM_FALLBACK is on,
synthesize a Session Log-only event via chat-completions/responses using the
full turn text. Failures fall through to the heuristic first-line backstop.
Also restore skillLookupInstruction so the stacked tip typechecks.

Co-authored-by: Cursor <cursoragent@cursor.com>
@heavygee heavygee added enhancement New feature or request fleet-overseer Fleet attention-arbitration architecture labels Jul 24, 2026
@heavygee

Copy link
Copy Markdown
Owner Author

Orchestrator ack + soup ask sent to meta bot. Proposed layer order after feat/overseer-contract-invisible: emit (#86) → heuristic fallback (#87) → flavors (#88) → this (#91). Default-off env stands; do not enable in soup env until rarity measured.

@heavygee

Copy link
Copy Markdown
Owner Author

Soup: layer feat/overseer-llm-fallback @ d0a04da2d placed immediately after feat/overseer-summary-flavors-and-dates (before replay harness). Rematerialize 29 layers @ driver a6485b059 (verify green; L15 merge clean). Patient hapi-restart-hub done — hub+runner active. Default remains OFF (HAPI_OVERSEER_LLM_FALLBACK unset). Dogfood: set env + restart hub when ready to exercise LLM path.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d0a04da2d5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hub/src/sync/syncEngine.ts
Comment thread hub/src/sync/overseerEventRecorder.ts Outdated
Merge flavors tip that removes hub text synth. Keep opt-in LLM fallback
but only after thinking clears; never invent Session Log crumbs from
ACP mid-turn narrative.

Co-authored-by: Cursor <cursoragent@cursor.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4e98b63d07

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hub/src/sync/overseerEventRecorder.ts Outdated
Comment thread hub/src/sync/overseerEventRecorder.ts Outdated
Address Codex review on #91: per-session insert order, remap stalled away
from hidden stale, flush deferred fallback on session-alive thinking-clear,
and skip duplicate session-end completed_fallback after a successful LLM
row. Mock AppContext in About settings tests so CI is green.

Co-authored-by: Cursor <cursoragent@cursor.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ce7269b31d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hub/src/sync/overseerEventRecorder.ts Outdated
Comment thread hub/src/sync/overseerEventRecorder.ts Outdated
Comment thread hub/src/sync/overseerEventRecorder.ts
Comment thread hub/src/sync/overseerEventRecorder.ts
heavygee and others added 2 commits August 11, 2026 21:16
Co-authored-by: Cursor <cursoragent@cursor.com>
Codex P2: keep full thinking-turn text, clear fallback-success per
attempt, persist URLs before await, and drop idle session queues.

Co-authored-by: Cursor <cursoragent@cursor.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ec56cb7bf1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hub/src/sync/overseerEventRecorder.ts Outdated
Comment thread docs/plans/2026-07-24-overseer-summary-emission.md Outdated
Comment thread hub/src/sync/overseerLlmFallback.ts Outdated
Detach the deferred turn before queueing so a later ACP segment cannot
merge into flushed work. LLM parse now requires status+summary; failed
synthesis stays silent (docs matched to that).

Co-authored-by: Cursor <cursoragent@cursor.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 964f59fa29

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hub/src/sync/overseerEventRecorder.ts
Comment thread hub/src/sync/syncEngine.ts Outdated
Comment thread hub/src/sync/overseerEventRecorder.ts
Comment thread hub/src/sync/overseerEventRecorder.ts
Comment thread hub/src/sync/overseerEventRecorder.ts Outdated
Comment thread hub/src/sync/overseerLlmFallbackConfig.ts
Comment thread hub/src/sync/syncEngine.ts
heavygee and others added 2 commits August 11, 2026 21:37
Co-authored-by: Cursor <cursoragent@cursor.com>
Queue primary inserts with LLM work, clear success on new agent
activity, reject dirty timeout env, and invalidate Session Log over SSE.

Co-authored-by: Cursor <cursoragent@cursor.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 86d3c2a78b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hub/src/sync/syncEngine.ts Outdated
Comment thread hub/src/sync/overseerEventRecorder.ts Outdated
Comment thread hub/src/sync/overseerEventRecorder.ts Outdated
Dedup waits for expiry LLM work. Session-end skip only if the latest
agent message is the one the LLM summarized. Queued inferred rows publish.

Co-authored-by: Cursor <cursoragent@cursor.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aee6587bee

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hub/src/sync/overseerEventRecorder.ts Outdated
Comment thread hub/src/sync/overseerLlmFallbackConfig.ts
Comment thread hub/src/sync/syncEngine.ts
Comment thread hub/src/sync/overseerLlmFallbackConfig.ts Outdated
Permission rows join the per-session insert queue. Reject setTimeout
overflow delays and non-absolute HTTP(S) fallback bases.

Co-authored-by: Cursor <cursoragent@cursor.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1bc50a71ff

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hub/src/sync/overseerEventRecorder.ts Outdated
Comment thread hub/src/sync/overseerEventRecorder.ts Outdated
Same-turn ACP tool/usage no longer duplicates completed_fallback.
Permission rows keep the observe-time timestamp while queued behind LLM.

Co-authored-by: Cursor <cursoragent@cursor.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cb030a52c6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hub/src/sync/overseerEventRecorder.ts
Comment thread hub/src/sync/overseerEventRecorder.ts Outdated
Comment thread hub/src/sync/overseerEventRecorder.ts
Comment thread hub/src/sync/overseerEventRecorder.ts Outdated
Comment thread hub/src/sync/overseerLlmFallbackConfig.ts
Comment thread hub/src/sync/overseerEventRecorder.ts Outdated
Reserve permission IDs before queueing, drop turn maps on session-end,
stamp accumulated ACP text at the final segment, and reject base URLs
with query/hash. User-message redelivery does not bump turn epoch.

Co-authored-by: Cursor <cursoragent@cursor.com>
@heavygee

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f7af139f9e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hub/src/sync/overseerEventRecorder.ts Outdated
Comment thread hub/src/sync/syncEngine.ts Outdated
Mark sessions ended before queued permission work resumes, and forget
recorder state only after deleteSession succeeds.

Co-authored-by: Cursor <cursoragent@cursor.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f7af139f9e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hub/src/sync/syncEngine.ts
Comment thread hub/src/sync/syncEngine.ts Outdated
heavygee and others added 2 commits August 11, 2026 22:28
Parent flavors landed a sync onAgentMessage assertion that broke hub
typecheck on the PR merge commit. Also refuse credential-bearing
fallback base URLs and redact userinfo if it ever reaches the startup log.

Co-authored-by: Cursor <cursoragent@cursor.com>
@heavygee

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 83ab6138c1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hub/src/sync/overseerEventRecorder.ts
Comment thread hub/src/sync/overseerEventRecorder.ts
Comment thread hub/src/sync/overseerLlmFallbackConfig.ts Outdated
Comment thread hub/src/sync/syncEngine.ts
Bare trailing ?/# pass URL() with empty search/hash, then joinUrl
glues the chat path into query or fragment. Log reasonDisabled when
the opt-in flag is on but config is invalid so operators see the typo.

Co-authored-by: Cursor <cursoragent@cursor.com>
@heavygee

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5c78aad9a5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hub/src/sync/overseerEventRecorder.ts
Comment thread hub/src/sync/overseerEventRecorder.ts
@heavygee

Copy link
Copy Markdown
Owner Author

Applied cold-review-clean. CI green, 0 unresolved threads, HEAD 5c78aad reviewed. Remaining Codex P2s accepted as out-of-scope edge cases (scheduled-prompt epoch; pending glue across turn bump). Fork Meta classifier looks for chatgpt-codex-connector issue comments (clean phrase) but this bot posts review threads + P2 mill forever - label is the documented override.

@heavygee heavygee added the cold-review-clean Fork-side bot review is satisfactory; safe to promote to upstream PR label Aug 11, 2026
heavygee added a commit that referenced this pull request Aug 24, 2026
…fier

cold-review-clean now passes the CI dimension when gh returns an empty
check rollup or transport failure (GH 503), matching the existing bot
override. Meta classifier treats CRC + merge-clean + no rollup as ✅
instead of ⚠️ / "push to trigger bot review".

Refs tiann#1624 flake noise on #91/#132.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cold-review-clean Fork-side bot review is satisfactory; safe to promote to upstream PR enhancement New feature or request fleet-overseer Fleet attention-arbitration architecture

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant