Add progress-event callbacks for evaluation requests - #28
Open
m-messer wants to merge 9 commits into
Open
Conversation
Implement a unified progress reporting framework that supports optional HTTP callbacks: - Define `Event` and `Reporter` abstractions in `internal/progress`. - Add `HTTPFactory` for building per-request HTTP callback reporters. - Introduce `Emit` convenience method to attach/report progress events via context. - Update supervisor and handler logic to emit lifecycle events. - Include comprehensive unit tests for reliability and correctness.
- Integrate callbackUrl field from µEd spec for progress reporting. - Replace progress callback headers with callbackUrl and request ID. - Include evaluation feedback payload in StageCompleted events. - Update tests to reflect callbackUrl usage and validation.
…ck delivery timeout
…k feature - Introduce `--progress-allowed-hosts` flag to restrict allowed callback hostnames. - Add `--progress-allow-private-networks` flag for optional private network access. - Implement automatic request ID generation for traceability and progress correlation. - Expand documentation with guidance on callback URL safety and SSRF prevention. - Update tests and internal logic for new configuration options and request IDs.
- Introduce IP filtering to block private, loopback, and link-local addresses. - Add hostname wildcards for fine-grained allowed host configuration. - Implement custom HTTP transport with DNS-based IP validation. - Add comprehensive unit tests to cover SSRF scenarios and configuration options.
- Introduce IP filtering to block private, loopback, and link-local addresses. - Add hostname wildcards for fine-grained allowed host configuration. - Implement custom HTTP transport with DNS-based IP validation. - Add comprehensive unit tests to cover SSRF scenarios and configuration options.
- Introduce `--progress-sidecar-unbind-grace-period` flag with default value of 250ms. - Add `UnbindAfterGrace` method to allow delayed unbinding with generation-safe logic. - Update supervisor adapter to utilize `UnbindAfterGrace` for improved POST handling.
MinEventInterval's default (200ms) rate-limited a fast evaluation function reporting two checkpoints from compareSets' evaluation function to at most one event per span: even with delivery now serialized on the client side, two closely-spaced report_progress() calls could still both arrive well under any single fixed interval, since arrival timing is governed by local HTTP round-trip cost, not real application-level delay. Add BurstSize (default 5): the first N events in a span bypass MinEventInterval spacing entirely (still bounded by MaxEventsPerSpan), so a handful of legitimate back-to-back checkpoints go through, while MinEventInterval keeps guarding against sustained event spam once the burst is used up. Also lower the MinEventInterval default itself from 200ms to 10ms, since 200ms had no real abuse-prevention basis and was overly aggressive for normal use.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
POST /evaluateendpoint: shimmy POSTs a small JSON event (preparing→evaluating→completed/failed) to a caller-suppliedcallbackUrlat each stage of processing, in addition to the normal synchronous HTTP response.callbackUrlbody field andX-Request-Idheader (now echoed on every response, generated if the caller doesn't supply one, and reused as the progress correlation key).completedevent carries the actual feedback payload indata.feedback, so a caller usingcallbackUrlgets the final result delivered there too, converging with the spec's "deliver feedback results to this URL" wording even though shimmy always takes the synchronous 200 path rather than the spec's 202-Accepted deferred flow.--progress-allowed-hosts) and an explicit opt-out (--progress-allow-private-networks) for trusted private deployments.internal/progresspackage: a generic, reusable event/progress-reporter abstraction threaded through the dispatcher/supervisor/handler layers viacontext.Context, so future custom events (e.g. emitted by the evaluation function itself) are a natural extension rather than a rework./evaluatecallers that don't supplycallbackUrlsee no behavior change.🤖 Generated with Claude Code
https://claude.ai/code/session_0187xesBCDmvKLY1xYbSVfhA