Skip to content

[task] Add directory ingestion helper for supported text files #82

Description

@AlexanderGardiner

Checks

  • I searched open and closed issues and pull requests and did not find a duplicate.

Area

Source

Goal

Add a small helper for ingesting a directory of supported files so users can load a folder of .txt and Markdown documents without manually iterating through paths.

Once Anchor.ingest_file(path) exists, directory ingestion is the next ergonomic layer. It should stay narrow and predictable: discover supported files, call the existing file ingestion path, and return the IDs that were created.

Suggested file locations

  • src/anchor/anchor.py
  • src/anchor/extractors/detect.py only if helper behavior needs shared support
  • tests/unit/test_anchor_directory_ingest.py
  • tests/fixtures/ingestion/

Scope

Add a public directory ingestion helper for supported text-based files.

The implementation should:

  • Add a method such as Anchor.ingest_directory(path).
  • Accept a str | Path directory path.
  • Recursively discover supported .txt, .md, and .markdown files.
  • Skip unsupported file extensions by default.
  • Call Anchor.ingest_file(...) for each supported file.
  • Return the created chunk IDs in a deterministic order.
  • Preserve existing Anchor.ingest_text(...) and Anchor.ingest_file(...) behavior.

Out of scope

  • Do not add PDF, Word, or source-code ingestion.
  • Do not add URL ingestion.
  • Do not add file watching.
  • Do not add ignore-file parsing.
  • Do not add concurrency.
  • Do not introduce new dependencies.
  • Do not change MemoryStore.

Definition of done

  • Anchor.ingest_directory("docs/") ingests supported .txt and Markdown files.
  • Unsupported files in the directory are skipped without blocking supported files.
  • Returned chunk IDs are deterministic relative to sorted file traversal.
  • Nested directories are handled recursively.
  • Tests cover mixed supported/unsupported files.
  • Tests cover nested directories.
  • Existing file and text ingestion behavior continues to pass.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: sourceApplication or library source code.help wantedMaintainers would welcome a community contribution on this work.kind: featureA user-facing capability request.needs: duplicate-reviewAutomation found similar existing work that needs maintainer review.status: ready-for-workTriaged and ready for a contributor to claim.

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions