feat: add list_inference_models and estimate_inference_cost actions#44
feat: add list_inference_models and estimate_inference_cost actions#44rajbhensdadiya wants to merge 3 commits into
Conversation
📝 WalkthroughWalkthroughAdds Floe Inference support to X402ActionProvider with two new Zod schemas and two new actions, re-exports the schemas from ChangesFloe Inference feature
Estimated code review effort: 2 (Simple) | ~15 minutes Sequence Diagram(s)sequenceDiagram
participant Agent
participant X402ActionProvider
participant FloeInferenceAPI
Agent->>X402ActionProvider: list_inference_models()
X402ActionProvider->>FloeInferenceAPI: GET /v1/models
FloeInferenceAPI-->>X402ActionProvider: model list JSON
X402ActionProvider-->>Agent: formatted model list
Agent->>X402ActionProvider: estimate_inference_cost(model, usage)
X402ActionProvider->>FloeInferenceAPI: POST /v1/estimate
FloeInferenceAPI-->>X402ActionProvider: cost estimate JSON
X402ActionProvider-->>Agent: formatted USDC estimate
Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
rajbhensdadiya
force-pushed
the
keyless-inference-gateway
branch
from
d509ea0 to
3c0e8f9
Compare
|
@CodeRabbit review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Pull request overview
Adds two new AgentKit actions to the X402ActionProvider for Floe Inference so agents can (1) browse available inference models and (2) estimate USDC cost for a given usage vector before spending.
Changes:
- Introduces
list_inference_modelsandestimate_inference_costactions + Zod schemas inX402ActionProvider. - Re-exports the new schemas from the package entrypoint.
- Updates README action counts and breakdown to reflect the new actions.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/x402ActionProvider.ts |
Adds new inference schemas and two new facilitator-backed actions for listing models and estimating inference cost. |
src/index.ts |
Re-exports the new Zod schemas from the public entrypoint. |
README.md |
Updates documented action counts and breakdown to include the 2 new inference actions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/x402ActionProvider.ts`:
- Around line 1354-1448: Add test coverage for the new Floe Inference actions in
x402ActionProvider, focusing on listInferenceModels and estimateInferenceCost.
Cover the happy-path formatting plus the empty-model-list branch, missing
upstream_cost_usdc handling, and error/AbortError branches by mocking
facilitatorFetch/readJsonOrError. Use the action method names and CreateAction
registrations to locate the code and verify the returned strings match the
spending/formatting behavior.
- Around line 1436-1442: The Floe Inference Estimate formatter in
x402ActionProvider should not assume margin_bps is always present, since it
comes from an unvalidated JSON cast and can produce NaN%. Update the estimate
output built in the return array to handle a missing or invalid d.margin_bps by
falling back to an explicit placeholder such as — instead of formatting it as a
percentage.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: c1361665-f467-4df7-b415-5782d877476b
📒 Files selected for processing (3)
README.mdsrc/index.tssrc/x402ActionProvider.ts
- guard estimate output against missing margin_bps (unvalidated JSON cast rendered NaN%)
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/x402ActionProvider.ts (1)
1428-1435: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winType annotation still claims
margin_bpsis required.Line 1441's new guard implies
margin_bpscan be missing/non-finite at runtime, but the cast on line 1432 still types it as a requirednumber. Marking it optional keeps the type honest and prevents a future refactor from treating the Line 1441 guard as dead code.♻️ Suggested type fix
const d = result.data as { model: string; rail: string; provider: string; - margin_bps: number; + margin_bps?: number; cost_usdc: string; upstream_cost_usdc?: string; };🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/x402ActionProvider.ts` around lines 1428 - 1435, The `result.data` cast in `x402ActionProvider` still treats `margin_bps` as always present, which conflicts with the runtime guard later in the same flow. Update the local type used for `d` so `margin_bps` is optional or nullable, and keep the existing validation logic that checks it before use; use the `result.data` cast and the `margin_bps` guard as the key symbols to locate the change.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@src/x402ActionProvider.ts`:
- Around line 1428-1435: The `result.data` cast in `x402ActionProvider` still
treats `margin_bps` as always present, which conflicts with the runtime guard
later in the same flow. Update the local type used for `d` so `margin_bps` is
optional or nullable, and keep the existing validation logic that checks it
before use; use the `result.data` cast and the `margin_bps` guard as the key
symbols to locate the change.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 88602543-ee0e-41ce-88d1-a1fe8c1d54e8
📒 Files selected for processing (1)
src/x402ActionProvider.ts
- require at least one non-zero usage field in EstimateInferenceCostSchema so empty vectors fail at the schema instead of returning a misleading $0 estimate - keep mixed modality units allowed: realtime models (and Gemini TTS) bill text + audio tokens on one rate card, so mixes must be priced server-side - use explicit null check for context_window so a legitimate 0 still renders
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/x402ActionProvider.ts`:
- Around line 256-270: Update the estimate_inference_cost action description to
state that at least one usage field must be non-zero. Keep the existing
validation refinement unchanged and add the note to the description associated
with estimate_inference_cost.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 7140a759-ab31-4750-b69f-d3333e8fecf9
📒 Files selected for processing (1)
src/x402ActionProvider.ts
| .refine( | ||
| (a) => | ||
| (a.inputTokens ?? 0) + | ||
| (a.outputTokens ?? 0) + | ||
| (a.cachedInputTokens ?? 0) + | ||
| (a.characters ?? 0) + | ||
| (a.audioSeconds ?? 0) + | ||
| (a.audioInputTokens ?? 0) + | ||
| (a.audioOutputTokens ?? 0) > | ||
| 0, | ||
| { | ||
| message: | ||
| "Provide at least one non-zero usage field (inputTokens/outputTokens, characters, audioSeconds, or audio tokens).", | ||
| }, | ||
| ); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win
Refinement logic correct; consider surfacing the constraint in the action description.
The sum > 0 check is a correct equivalent of "at least one non-zero field" given all inputs are .nonnegative() with ?? 0 fallbacks. However, since custom .refine() predicates aren't representable in JSON Schema, this constraint won't be visible to the LLM in the tool's generated schema — it'll only learn about it after a failed call. Consider adding a short note to the estimate_inference_cost action description (Lines 1418-1427) e.g. "Requires at least one non-zero usage field."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/x402ActionProvider.ts` around lines 256 - 270, Update the
estimate_inference_cost action description to state that at least one usage
field must be non-zero. Keep the existing validation refinement unchanged and
add the note to the description associated with estimate_inference_cost.
Summary by CodeRabbit