Skip to content

Reuse the extract agent across calls - #199

Merged
jakmro merged 1 commit into
mainfrom
extract-reuse-agent
Nov 16, 2026
Merged

jakmro merged 1 commit into
mainfrom
extract-reuse-agent

Conversation

@jakmro

@jakmro jakmro commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

cactus.extract built a new agent on every call, which with tuned weights also started a new worker process each time. It now keeps one agent per schema and settings, like transcribe and describe, and resets it between calls. Ten tuned extractions go from 1.7 s to 0.6 s. Closing a worker no longer waits for a reply that never arrives at exit, which removes a 2s stall when a tuned agent is alive at exit.

cactus.extract built and closed an agent on every call, and with tuned
weights that also started a new worker process each time. It now keeps one
agent per schema and settings, as transcribe and describe already do, and
resets it between calls. Ten extractions with tuned weights drop from 1.7 s
to 0.6 s.

Closing a worker no longer waits for its acknowledgement. The reply never
arrives at interpreter exit, so every exit with a live tuned agent stalled
for 2 s.
@jakmro
jakmro merged commit f72082a into main Nov 16, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant