A memory that cannot be embedded or extracted no longer blocks the memories after it

[WaylandYang]

Why

#1187. Every remembered sentence is a chunk of one document per base. While the embedding model answers with an error, nothing said to remember is extracted, although the chat model works and remember goes on answering "Recorded the sentence". One sentence the pipeline cannot process has the same effect on every sentence after it.

Reproduced on dev with a working chat model and an embedding key that is refused: two sentences recorded, memory_ingest failed twice (Embedding request failed (401)), no extraction queued, no pending statement for either until the embedding model came back.

What changes

• memory_ingest embeds the log one sentence at a time, and a failure there is not the end of the job. A sentence that cannot be embedded is logged and left without a vector, to be tried again with the next memory; the index is rebuilt and extraction is queued all the same. Before, all unembedded chunks went in one call and the first error failed the job before extraction was queued.
• run_open finishes the work for what it extracted before it reports an incomplete attempt. The error used to return ahead of time resolution and the pending notice, so a sentence that could not be extracted kept the confirmation card of every later sentence from appearing. The document is still not marked done and the job still fails; only the order changed. This applies to uploaded documents as well: the chunks that were extracted now get their time resolution and alignment on that attempt, not only after a fully successful retry.

remember still takes a text of any length.

How it was checked

• Two tests, each failing on dev without the change: a_memory_that_cannot_be_embedded_does_not_hold_back_the_others and a_memory_that_cannot_be_extracted_does_not_hold_back_the_next.
• cargo fmt --all --check, cargo clippy -p utopia-server --all-targets -- -D warnings, and cargo test -p utopia-server on a fresh database: 608 passed.
• The same script against a dev build and this build, embedding key refused, chat model working:

	dev	this branch
sentence recorded while embedding fails	0 pending	1 pending
a second one, embedding still failing	0 pending	1 pending
memory_ingest	failed, failed	done, done

Not addressed: a sentence that fails extraction every time (a text longer than the chat model can read, for one) is tried again with each new remember and keeps the log marked failed, and there is no way yet to drop one sentence from the log.

🤖 Generated with Claude Code