Skip to content

Document flaky-tests API conventions and pagination#57

Merged
samgutentag merged 1 commit into
mainfrom
sam-gutentag/api-conventions
Jun 1, 2026
Merged

Document flaky-tests API conventions and pagination#57
samgutentag merged 1 commit into
mainfrom
sam-gutentag/api-conventions

Conversation

@samgutentag
Copy link
Copy Markdown
Member

Summary

  • Adds an "API conventions" section to flaky-tests/reference/api-reference.mdx covering behavior that applies across endpoints
  • Documents page_token pagination with a working curl example and the "first call → omit, subsequent → pass next_page_token" rule
  • Covers rate-limit expectations, webhook-to-API immediacy, the support-only Bundle Upload ID, the auto-analysis monthly cap (Beta), and that the UI's GraphQL endpoint is private
  • No nav change — the conventions sit above the existing endpoint card index on the same page

Why

Sourced from customer feedback mining (cluster api-rate-limits-and-pagination, verdict partial, 7 pairs across 6 customers). The API reference is a card index; conventions like page_token pagination and rate limits are documented in OpenAPI specs but never in narrative form, so customers hit recurring issues (500 on misused page_token, "missing" tests that are actually on page 2, no surfaced answer to "what's the rate-limit ceiling?").

Items flagged for review

  • Auto-analysis monthly cap: cluster brief specifies "100/month default." I deliberately did not commit to a specific number in the docs because I couldn't find it in any other docs2 page or in the OpenAPI spec — confirm the exact value with the agents team and either inline the number or keep the current "default monthly cap" phrasing.
  • GraphQL phrasing: confirm "private GraphQL endpoint... no compatibility guarantee, and may change without notice" is the framing the API team wants, vs. softer language.
  • Bundle Upload ID: confirm there's still an open internal ticket to surface it (so leaving the doc as "support-only today" remains accurate), and whether we want to name trunk-analytics-cli upload specifically as the source or keep it generic.
  • 504/timeout history: deliberately omitted. Phil flagged the get-test-details timeout issue in March and said a fix was rolling out — confirm shipped (otherwise this PR should not be merged without addressing) or leave omitted if the fix is in production and there's nothing actionable for readers.
  • Rate-limit phrasing: "no externally published SLO, but the practical ceiling is high enough for reasonable polling and dashboard use" mirrors Phil's Slack response — confirm this framing is OK to publish vs. a more conservative "contact support for rate-limit guidance."
  • Pagination 500 behavior: I wrote "Passing a value that didn't come from a prior response returns a 500." Confirm this is the actual error mode and that 500 is the documented response code (vs. 400).

Customer signal

@samgutentag samgutentag added the needs review PR sourced from customer-feedback-mining; needs human scrutiny for accuracy before merge label May 20, 2026
@mintlify
Copy link
Copy Markdown
Contributor

mintlify Bot commented May 20, 2026

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
trunk 🟢 Ready View Preview May 20, 2026, 11:55 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@samgutentag
Copy link
Copy Markdown
Member Author

samgutentag commented May 26, 2026

Verification status (2026-05-29): live

Verified: customers can use this. Ready to publish.

  • Flag state: no feature flag involved; LaunchDarkly not consulted (nothing to read)
  • Eng PR: none (customer feedback-sourced API conventions)
  • Flag: none
  • Signals: No engineering dependency and no feature flag. This PR documents existing API behaviors (page_token pagination, rate limits, webhook-to-API immediacy, Bundle Upload ID, auto-analysis monthly cap, private GraphQL endpoint) sourced from 7 customer pairs across 6 channels. All described conventions apply to the current production API.

Ready to publish after editorial review of flagged items (monthly cap number, GraphQL phrasing, rate-limit SLO language).


Generated by Claude Code

@samgutentag samgutentag added ready to merge Verify docs PR: customers can use this. Ready to publish. and removed needs review PR sourced from customer-feedback-mining; needs human scrutiny for accuracy before merge labels May 26, 2026
…ation, Bundle Upload ID

Expand the Flaky Tests API reference index page with an "API conventions"
section covering behavior that applies across endpoints. Sourced from
recurring customer questions: page_token misuse returning 500s, "missing"
tests that were actually on later pages, rate-limit expectations for
polling automation, and webhook-to-API immediacy.

Also documents three things that weren't surfaced anywhere in narrative
form: Bundle Upload IDs are support-only, auto-analysis has a default
monthly cap during Beta, and the UI's GraphQL endpoint is private.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Member Author

Verification status (May 31, 2026): live

API conventions docs cover existing, shipped API behavior — no flag or eng dependency.

  • Flag state: not consulted
  • Eng PR: none
  • Flag: none
  • Signals: documents stable, existing API surface already available to all customers

Next action: merge this PR.


Generated by Claude Code

Copy link
Copy Markdown
Member Author

Docs Verify Sweep · June 1, 2026 · ✅ live

Feature is live in production. Ready to merge.

Unchanged from previous sweep (May 31, 2026).


Generated by Claude Code

@samgutentag samgutentag merged commit 374f3c6 into main Jun 1, 2026
3 checks passed
@samgutentag samgutentag deleted the sam-gutentag/api-conventions branch June 1, 2026 21:04
@samgutentag samgutentag restored the sam-gutentag/api-conventions branch June 1, 2026 21:16
samgutentag added a commit that referenced this pull request Jun 1, 2026
…e propagation, Bundle Upload ID (#57)"

This reverts commit 374f3c6.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready to merge Verify docs PR: customers can use this. Ready to publish.

Development

Successfully merging this pull request may close these issues.

1 participant