Skip to content

blog: Tuning Recall — Hindsight's Four Retrieval Arms, Explained - #3836

Open
benfrank241 wants to merge 1 commit into
mainfrom
blog/tuning-recall-retrieval-arms
Open

blog: Tuning Recall — Hindsight's Four Retrieval Arms, Explained#3836
benfrank241 wants to merge 1 commit into
mainfrom
blog/tuning-recall-retrieval-arms

Conversation

@benfrank241

Copy link
Copy Markdown
Member

A pillar post on tuning recall: what each of the four retrieval arms does, the knob and default that controls each, and how to match the pipeline to the shape of a bank.

What's in it

  • The recall pipeline end to end: query analysis → four arms → RRF → cross-encoder rerank
  • Each arm with its real thresholds and defaults (semantic 0.3, graph seed 0.3, temporal 0.1, semantic-link 0.7, BM25 min score 0)
  • The four per-bank toggles, including the new enable_text_search from feat(recall): add a per-bank enable_text_search toggle for pure vector recall #3819
  • Recall budget mapping (fixed 100/300/1000 vs adaptive), strategy boosts, recency decay
  • Reranker failover chain, and the local reranker's bucket-batching / FP16 wins
  • Retain-side pairing (retain_extraction_mode: chunks, enable_observations: false) for plain-retrieval banks
  • A tune-by-bank-shape table and 8 FAQs

Verification

Every default, threshold, and env var name is taken from hindsight-docs/docs/developer/configuration.md and the hindsight-api-slim source, not restated from memory. The enable_text_search behavior is checked against engine/search/retrieval.py:190 and the knowledge-page path in engine/memory_engine.py.

The post deliberately contains no CLI commands: set_config in hindsight-cli/src/commands/bank.rs accepts only the LLM, retain, mission, and disposition params, so none of the four recall toggles are settable via a CLI flag. Configuration is shown through the config API and environment variables only.

Files

  • hindsight-docs/blog/2026-08-27-tuning-recall-retrieval-arms.md
  • hindsight-docs/static/img/blog/tuning-recall-retrieval-arms.png (1600x900)

Noted separately, not fixed here

The "plain-retrieval (RAG) banks" curl example in configuration.md still lists only the three original recall toggles and omits enable_text_search: false, though that section's premise is making a bank behave like a conventional vector store. #3819 updated the table above it but not the example. Left for its own docs PR.

🤖 Generated with Claude Code

A tuning guide for recall: what each of the four retrieval arms does
(semantic, BM25, graph, temporal), the knob and default that controls
each, and how to match the pipeline to the shape of a bank.

Covers the per-bank toggles including the new enable_text_search
(#3819), the recall budget mapping, strategy boosts, recency decay,
and the retain-side pairing for plain-retrieval banks.

Every default and threshold is taken from configuration.md and the
hindsight-api-slim source rather than restated from memory.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@strix-security

strix-security Bot commented Aug 27, 2026

Copy link
Copy Markdown

Strix Security Review

No security issues found.

Updated for 462ee56.


Reviewed by Strix
Re-run review · Configure security review settings

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant