Skip to content

fee/aesstream: range decryption API (FIL-472)#8

Open
Peeja wants to merge 1 commit into
mainfrom
petra/fil-472-fee-go-range-decryption-api
Open

fee/aesstream: range decryption API (FIL-472)#8
Peeja wants to merge 1 commit into
mainfrom
petra/fil-472-fee-go-range-decryption-api

Conversation

@Peeja

@Peeja Peeja commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

This will support range requests for objects. A higher layer will translate an object range request into plaintext segment (piece/shard) range requests, and those into ciphertext blob range requests which can go to Piri. This PR provides everything that layer needs to convert from a plaintext range to a ciphertext range (CiphertextRange()), and then turn the ciphertext result of those range requests into the plaintext we're after (SpanReader).


Implements FIL-472 — the range-decryption API for the FilOne File Encryption Envelope (FEE).

Pure STREAM mechanics on top of the existing body cipher — no new crypto, just chunk-index arithmetic and a single contiguous fetch of the chunks overlapping a request.

Design: sequential, one request per segment

A range read turns a plaintext range into one contiguous ciphertext fetch, then streams the result through the decryptor — so the read path is one request per segment with O(chunkSize) memory, no per-chunk fetching.

  • CiphertextRange(ciphertextSize, chunkSize, off, length) (start, n, err) — the single contiguous ciphertext byte span covering a plaintext range. Because the chunks overlapping a range are adjacent, the bytes needed are one span; the consumer (FIL-487) fetches it in one range request. Offsets are stream-relative (add any envelope-header length). Chunk-aligned, so the edge over-read is <2·chunkSize (GCM authenticates whole chunks).
  • SpanReader (io.Reader) / OpenSpan — decrypt that span, delivered as a sequential io.Reader (e.g. a range-request body), into exactly the requested plaintext. The total ciphertext length pins each chunk's index, final-chunk flag and length up front, so there's no last-chunk retry; the first/last chunks are trimmed to the range. OpenSpan guards its int64→int buffer with math.MaxInt (steers huge ranges to the streaming reader instead of panicking in make on 32-bit builds).
  • DecryptedSize — inverse of EncryptedSize; derives and validates the plaintext length from a ciphertext length without fetching any ciphertext.

Semantics

HTTP-range-like: a length past the end clamps to the bytes available from the offset; a negative offset/length or an offset past the end is ErrRange. A structurally invalid declared ciphertext length is ErrCiphertextSize; a tampered/reordered/misframed chunk is ErrCorrupted; a span shorter than the geometry requires is ErrTruncated.

What the tests cover (testify require)

  • TestSpanReader/Ranges — a table of hand-computed literal expectations (span start/length and decrypted length): an independent oracle that pins the exact span CiphertextRange reports, that OpenSpan/SpanReader reproduce the right plaintext from exactly that span, and that the span is consumed in full and no further.
  • TestSpanReader/RoundTrip — reads each sealed object back in full through adjacent range windows of varying widths (chunk-aligned and not), via both OpenSpan and SpanReader, and confirms the reassembly equals the original plaintext (length recovered through DecryptedSize).
  • Tamper detection, short-span truncation, wrong-declared-size detection, range/size validation, default 256 KiB chunks, CiphertextRange input validation, the internal chunkLayout geometry + MaxChunks guard, and EncryptedSizeDecryptedSize inversion.

go test (incl. -race), go vet, gofmt clean.

🤖 Generated with Claude Code

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a range-decryption API to fee/aesstream so consumers can read arbitrary plaintext byte ranges from a chunked AES-256-GCM STREAM ciphertext using only the overlapping ciphertext chunks.

Changes:

  • Introduces RangeReader, OpenRange, and DecryptedSize plus shared chunk-geometry math (chunkLayout).
  • Adds comprehensive tests covering correctness, chunk-selective fetch behavior, tamper detection, and size/range validation.
  • Promotes github.com/stretchr/testify to a direct dependency to support the new test suites.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
go.mod Promotes testify to a direct dependency for new tests.
fee/aesstream/rangereader.go Implements range decryption reader + size/geometry helpers.
fee/aesstream/rangereader_test.go Black-box tests for range correctness, selective fetch, and tamper/validation behavior.
fee/aesstream/rangereader_internal_test.go Internal tests pinning chunkLayout boundary math and MaxChunks guard.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread fee/aesstream/rangereader.go
Peeja pushed a commit that referenced this pull request Jun 25, 2026
OpenRange buffers the whole requested range. On 64-bit platforms r.Len()
(bounded by the plaintext length) always fits an int, but on a 32-bit
build a multi-GiB range would overflow the int make wants and panic with
"makeslice: len out of range". Return a clear error in that case and point
callers at the streaming NewRangeReader instead.

Addresses a review note on #8. (The same note's claim that make([]byte,
int64) fails to compile is incorrect — the Go spec allows any integer type
for make's size; this only guards the 32-bit overflow.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7GkuUdrMgiQhmpoY9FjKN
@Peeja Peeja force-pushed the petra/fil-472-fee-go-range-decryption-api branch 4 times, most recently from f5cbf0f to 22e8271 Compare June 26, 2026 20:48
@Peeja Peeja requested a review from Copilot June 26, 2026 20:59

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

Comment thread fee/aesstream/spanreader.go
Comment thread fee/aesstream/spanreader.go Outdated
Comment thread fee/aesstream/spanreader.go
Comment thread fee/aesstream/spanreader.go Outdated
@Peeja Peeja force-pushed the petra/fil-472-fee-go-range-decryption-api branch 2 times, most recently from ae29d47 to 2a55a29 Compare June 26, 2026 21:35
Range decryption on top of the chunked AES-256-GCM STREAM body cipher
(FIL-470). No new crypto — chunk-index arithmetic plus a single contiguous
fetch of the chunks overlapping a request.

The engine is sequential: a caller turns a plaintext range into one
contiguous ciphertext fetch and streams the result through the decryptor,
so the read path is one request per segment with O(chunkSize) memory.

- CiphertextRange(ciphertextSize, chunkSize, off, length) → the single
  contiguous ciphertext byte span covering a plaintext range. Overlapping
  chunks are adjacent, so the bytes needed are one span — the consumer
  (FIL-487) fetches a segment's range in one request. Offsets are
  stream-relative; add any envelope-header length. Chunk-aligned, so the
  edge over-read is <2*chunkSize (GCM authenticates whole chunks).
- SpanReader / OpenSpan: decrypt that span — delivered as a sequential
  io.Reader (e.g. a range-request body) — into exactly the requested
  plaintext. The total ciphertext length pins each chunk's index,
  final-chunk flag and length up front, so there is no last-chunk retry and
  the first/last chunks are trimmed to the range. OpenSpan guards its
  int64->int buffer with math.MaxInt (steering huge ranges to the streaming
  reader instead of panicking in make on 32-bit builds).
- DecryptedSize: inverse of EncryptedSize; derives and validates the
  plaintext length from a ciphertext length without fetching any ciphertext.
- chunkLayout: the shared geometry behind all three.

No io.ReaderAt form: range reads are inherently sequential (each chunk read
once, in order), so the random-access engine wasn't load-bearing. A
random-access source can still feed SpanReader via
io.NewSectionReader(src, start, n) using CiphertextRange's offsets — noted
in the CiphertextRange doc.

Range semantics are HTTP-range-like: a length past the end clamps; a
negative offset/length or offset past the end is ErrRange; a structurally
invalid declared ciphertext length is ErrCiphertextSize; a
tampered/reordered/misframed chunk is ErrCorrupted; a span shorter than the
geometry requires is ErrTruncated.

Tests use testify require. TestSpanReader/Ranges is a table of hand-computed
literal expectations (span start/length and decrypted length) — an
independent oracle rather than a re-derivation of the range math — that also
checks the span is consumed in full and no further. TestSpanReader/RoundTrip
reads each sealed object back in full through adjacent range windows of
varying widths (chunk-aligned and not), via both OpenSpan and SpanReader,
and confirms the reassembly equals the original plaintext (length recovered
through DecryptedSize). Alongside: tamper detection, short-span truncation,
wrong-size detection, range/size validation, default 256 KiB chunks,
CiphertextRange input validation, the internal chunkLayout geometry +
MaxChunks guard, and EncryptedSize/DecryptedSize inversion.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7GkuUdrMgiQhmpoY9FjKN
@Peeja Peeja changed the base branch from fil-470-fee-go-chunked-aes-256-gcm-stream-encryptdecrypt to main June 26, 2026 22:00
@Peeja Peeja force-pushed the petra/fil-472-fee-go-range-decryption-api branch from 2a55a29 to 23704c9 Compare June 26, 2026 22:01
@Peeja Peeja marked this pull request as ready for review June 26, 2026 22:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants