Skip to content

cleanup: five small storage items (inert detached affected_rows, serial size probes, fork check reads every ref, latent scoped read retries, retry-count doc) #1386

Open

Description

@ragnorc

Problem

Five small items found while auditing the storage layer against Lance 13.0.0 and object_store 0.14.2. None needs a design decision.

  1. L-detached-affected-rows. commit_detached passes with_affected_rows (table_store.rs:1243-1245). Lance passes affected_rows only to commit_transaction; the detached path never reads it (lance-13.0.0/src/dataset/write/commit.rs:459-480). The argument suggests a conflict check that does not happen.
  2. L-source-bytes-serial. source_file_bytes sends one HEAD per deletion file, and one per data file with no recorded size, one at a time (optimize.rs:1740-1770). It runs during planning whenever a byte ceiling is set, which the served path always sets.
  3. L-fork-absent-listing. named_fork_is_absent (table_store.rs:2320-2329) reads every ref's contents (list_branch_contents) to test whether one name exists. The Blob witness (blob.rs:1450-1455) does the same. Legacy and error paths only.
  4. O-bound-scoped-read-retry (latent). With a captured binding and a scope, binding.adapter builds only the single-attempt S3 client (binding.rs:386-409), and with_scope uses it for both the retrying and the single role (scoped.rs:347-349). Reads and fresh immutable writes would then not retry. The environment path builds both clients (lib.rs:1409-1417). No current caller reaches this combination (the server uses for_storage_root; the CLI's binding admission uses GraphOperation).
  5. Doc mismatch. storage-failures.md:69-71 says a direct client retries up to 10 times by default. A captured binding caps retries at 3 (binding.rs:478-498); only the environment path keeps object_store's default of 10.

Proposed change

  1. Drop the argument on the detached path, or comment that it is inert there.
  2. Use an ordered buffered(io_parallelism) over the files, keeping the early stop at the ceiling.
  3. Test presence from a name listing (list_branch_ref_names) instead of reading every ref.
  4. Mirror the environment path: build a retrying client with the scoped read retry configuration and use with_scope_and_retrying.
  5. State both defaults in the doc.
  • Gain: fewer serial requests in optimize planning and fork checks; no misleading argument; read retries kept when the latent path is wired; an accurate doc.
  • Cost and risk: small each. Item 2 changes request order, so keep it ordered for DST replay.
  • Evidence: code inspection only.

Affected areas

table_store.rs, db/omnigraph/optimize.rs, blob.rs, omnigraph-storage (binding.rs, scoped.rs), docs/user/operations/storage-failures.md.

Alternatives considered

Leave them; each is minor on its own.

Finding ID: L-detached-affected-rows, L-source-bytes-serial, L-fork-absent-listing, O-bound-scoped-read-retry, doc mismatch · Suggested size: S each
Audited commit: 9fb5b48e · Pinned: lance 13.0.0, datafusion 54.0.0, arrow 58.4.1, object_store 0.14.2
Report: underused-primitives-20261113.md (focus: storage and versioning)

Activity

  1. ragnorc commented on Nov 13, 2026

    @ragnorc
    ContributorAuthor

    What happens

    All five items are real on main e32110f5. Only item 4 changes behaviour. A captured S3 binding that has an I/O scope sends every read once and never retries it. The new test binding::tests::scoped_captured_s3_binding_retries_reads_issue_1386 fails on main: one request where two are expected.

    Why

    With a scope, CapturedStorageBinding::adapter builds only the single-attempt client and uses it for reads too. The environment and Azure paths build a second, retrying client for reads. No current caller uses this combination yet. The other four items are as the issue describes: an argument Lance ignores, serial size checks, a ref listing that reads every ref, and a doc that gives one retry default where there are two.

    Pull request

    #1392 fixes all five with small changes. It adds the retrying client, runs the size checks in order up to the store's I/O limit, lists ref names only, drops the unused argument and corrects the doc.

    Checks run
    • Storage, core and engine tests: 2092 of 2093 pass. The one failure also fails on main on this machine, because it runs as root.
    • DST storage-cost golden: passes, unchanged.
    • fmt, clippy with -D warnings, docs check and release-note check: pass.
    • Lance: no open or unreleased change to these paths.
  2. aaltshuler commented on Nov 15, 2026

    @aaltshuler
    Contributor

    Closing as low value. Items 1 to 3 are hygiene, and item 4 is not reachable by any current caller, as the issue says. Item 2's premise is off: the served per-table and run byte ceilings default to unset (docs/user/operations/recorded-operations.md). The doc mismatch in item 5 is real and is better as its own one-line docs change.

  3. aaltshuler commented on Nov 16, 2026

    @aaltshuler
    Contributor

    Reopening for item 4. My closing comment called it unreachable. That holds on main, but #1360 makes it reachable.

    • fix(cluster): release the lock after a converged direct apply #1360 adds store.with_io_scope(..) to apply_direct_bundle.
    • The CLI's direct cluster apply reaches that function through apply_deployment_with_binding, with a store from ClusterStore::for_storage_binding.
    • ClusterStore::with_io_scope rebuilds a binding-backed store with storage_handle_for_binding(binding, Some(scope)), which calls CapturedStorageBinding::adapter(Some(scope)): the path that builds only the single-attempt S3 client and uses it for reads too.

    With #1360 as it stands, a direct cluster apply on an S3 root would send every read once, with no retry. Item 4 needs to land before or with #1360. #1392, which fixed it, was closed without merging.

    Item 5 (the retry count in storage-failures.md) is also real. Items 1 to 3 remain hygiene, and item 2's premise is still off: the served byte ceilings default to unset.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureFeature proposalneeds-triageAwaiting maintainer triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions