feat(events): server conformance for the merged Events design sketch, phase 1

[panyam]

What changes

Adds a requirement-traceability yaml for MCP Events and the first two of five server scenarios, scoring an implementation against the design sketch that merged on main of modelcontextprotocol/experimental-ext-triggers-events on 2026-09-08. events-discovery grades the capability, events/list and the error-code contract; events-poll grades poll delivery, the EventOccurrence shape and cursor lifecycle. Run against mcpkit's examples/events/kitchen-sink, they report 26 pass / 13 fail / 6 warn, and the failures are six real divergences rather than harness noise.

Prerequisite knowledge

• Events design sketch — the document every row is extracted from. The Summary and Cursor Lifecycle sections carry most of what the poll scenario tests.
• AGENTS.md § Scenario design and § Check conventions — why this is two scenarios with 45 checks instead of 45 scenarios, and why a missing prerequisite fails rather than skips.
• src/traceability/ — how a yaml row becomes tested or untested in the manifest, which is what makes declaring all 131 rows up front the right move rather than an overreach.
• No page covers how to score against a document that is not a SEP. This PR is the first case, and the yaml header is where that reasoning currently lives.

Reviewer's guide

A restaurant can cook a dish that is not on the menu. The kitchen has the ingredients, the line knows the recipe, and if you somehow know to ask for it you get it. But nobody reading the menu ever orders it, so from the outside the dish may as well not exist. mcpkit's events surface is that dish: events/list answers, the three delivery modes are implemented, and yet nothing appears under capabilities.events, which is the menu a spec-following client reads before deciding what to ask for. This matters for how the suite is shaped, because the obvious design gets it wrong. Conformance harnesses normally treat an undeclared optional capability as "not applicable" and skip, which is right for a restaurant that genuinely does not serve the dish and wrong for one that serves it off-menu: the skip reports a clean run over a surface no client can reach. So the capability gate here asks before it skips. It calls events/list, and only a server that both declares nothing and implements nothing gets the skip. The same instinct runs through the rest of the diff. Wherever a check cannot be exercised, the scenario says which prerequisite was missing and fails, because the one thing worse than a red conformance run is a green one that tested nothing.

Read in this order:

1. src/seps/sep-9999.yaml — start here, header first. It explains the placeholder SEP number, how 144 keyword occurrences became 131 declared checks and 30 excluded rows, and which rows this phase actually emits. The rows themselves are skimmable once the header lands.
2. src/scenarios/server/events/helpers.ts — the shared vocabulary. Note that EVENTS_EXTENSION_ID is a suite-selection key, not a path into the capability object; Events declares top-level, unlike every other extension in this repo.
3. src/scenarios/server/events/discovery.ts — the off-menu gate is at the top of checks(). Everything after it is field-by-field grading of whatever the server serves.
4. src/scenarios/server/events/poll.ts — the interesting part is quietAdvanceChecks: two polls with the first response's cursor fed into the second, which is what makes cursor advancement gradeable against a server where nothing is happening.
5. src/scenarios/server/events/negative.test.ts — acceptance for the above. Every test pairs a conformant fixture with one broken in exactly one way.
6. src/scenarios/index.ts, src/types.ts — registration and the extension id. Mechanical.

How it works

discovery:
  declared, value <- capabilities.events
  if not declared:
      probe <- events/list
      if probe is -32601:  skip everything          # genuinely does not do events
      else:                fail the declaration check, keep grading
  grade events/list, then each descriptor field across all descriptors,
  reporting the first offender per field so check ids stay stable
  probe one unknown event name -> grades NotFound and the server-range rule

poll:
  pick the first descriptor advertising "poll"; none -> untestable, not skip
  p1 <- poll(cursor=null)        # bootstrap: must replay nothing
  p2 <- poll(cursor=p1.cursor)   # quiet period: cursor must still come back
  p3 <- poll(no cursor field)    # absent must be accepted as null
  p4 <- poll(maxEvents=1)        # batch must be capped
  p5 <- poll(cursor=p1.cursor, maxAgeMs=300000)
  p6 <- poll(cursor=null, maxAgeMs=300000)   # must be ignored, replay nothing
  grade EventOccurrence only if some response carried an event

The truncated rows that need a stale cursor are reported untestable rather than skipped. Cursors are opaque by definition, so the harness cannot mint one old enough to fall outside a retention window, and saying so is more useful than a silent pass.

Decision log

• 9999 is a placeholder, and the work did not wait for a real number. Events has no PR in modelcontextprotocol/modelcontextprotocol, so no number exists to use. Both traceability gates are numeric (the filename regex at src/traceability/index.ts:174, the check-id regex at line 41), and a non-matching filename is dropped from the manifest silently rather than rejected, so there is no honest name that works. The alternative was blocking the extraction on a WG round trip. The rename is mechanical, the yaml header names it as a blocker before upstreaming, and the file stays on this fork branch until then so plan.modelcontextprotocol.io never sees 9999.
• All 131 rows are declared now, though only 45 are emitted. An undeclared requirement is invisible; an undeclared-and-unemitted one is indistinguishable from a requirement nobody noticed. Declaring the full set makes the 86 outstanding rows show up as untested in the manifest, which is the mechanism working rather than a gap in this file.
• Pure MAY and OPTIONAL sentences get no row. Five initially got one and were demoted to excluded: rather than deleted, so the keyword sweep stays auditable against the source.
• Client, host, receiver and SDK-guidance obligations are excluded rather than declared. Roughly a fifth of the normative sentences in the document address the client or the callback endpoint. Declaring them would inflate the denominator with rows no server-side scenario could ever emit.
• Two scenarios rather than one, and five eventually. A server may implement any non-empty subset of the three delivery modes, so poll-only and push-only servers are both conformant and grading them together would misreport each.
• Negative tests use in-process mock servers, not a new example server. Same as the SEP-2640 negative tests. An events-capable TypeScript fixture is real work and belongs with the push and webhook scenarios, which genuinely need one.

Risk / blast radius

• Affects: additive. Two new scenarios, one new yaml, one new entry in EXTENSION_IDS. No existing scenario, suite list or check id changes.
• Not selected by default. source is { extensionId }, so matchesSpecVersion returns false for every --spec-version and the scenarios never join a dated run. They are also in the pending list. Reaching them takes --scenario events-* or --suite all.
• Tests covering this: src/scenarios/server/events/negative.test.ts, 29 cases. Full suite 651 pass across 48 files.
• Be paranoid about: the capability gate. It is the one place this suite deliberately departs from how every other extension scenario here behaves, and it is load-bearing for the headline finding. If you think an undeclared-but-serving server should skip, say so, because that choice is what turns a green run into a red one against mcpkit.
• src/seps/traceability.json will drift once these check ids are emitted in a real run. Per AGENTS.md the traceability workflow refreshes it by PR and it is not a gate, so this PR leaves it alone.

Before / after

There was no events suite before, so the comparison that matters is what an implementation looks like when scored. Against examples/events/kitchen-sink at mcpkit main:

events-discovery   Passed: 8/11, 3 failed, 1 warnings
events-poll        Passed: 18/28, 10 failed, 5 warnings

The six divergences, each verified against the running server rather than inferred:

Check	What the server does
sep-9999-capability-events-object	answers events/list while declaring no capabilities.events
sep-9999-descriptor-input-schema	no descriptor carries inputSchema
sep-9999-descriptor-delivery-subset	events.topology sends delivery: null where an array is required
sep-9999-poll-events-array	omits events instead of returning an empty array
sep-9999-poll-mode-unsupported	answers events/poll for a type advertising only push/webhook
sep-9999-poll-next-poll-ms	sends nextPollSeconds, the name spec commit 197c32b4 retired

Only the last was previously tracked. The other five are new.

flowchart TD
    A[server under test] --> B{capabilities.events declared?}
    B -->|yes| G[grade everything]
    B -->|no| C{events/list answers?}
    C -->|"-32601"| D[SKIP all: does not do events]
    C -->|returns a catalog| E[FAIL the declaration check]
    E --> G
    G --> H{check exercisable?}
    H -->|yes| I[SUCCESS or FAILURE]
    H -->|no| J[untestable: FAIL and name the prerequisite]

Loading

Out of scope

• events-push, events-webhook, events-webhook-delivery — phases 2 and 3. The 86 rows they cover are declared and report untested.
• A --header option on the server runner. Needed to score Peter's metronome-mcp.fly.dev, which requires an OAuth bearer and whose AS offers only authorization_code. Its own PR, ahead of phase 3.
• Renaming sep-9999-* once the WG reserves a number.

🤖 Generated with Claude Code

https://claude.ai/code/session_017kyEZdgh1zjdhURgdcZ4nL

[panyam]

Flagging this deliberately, because it is the one place this suite departs from how every other extension scenario in the repo behaves, and the departure is what turns the run against mcpkit red rather than green.

Everywhere else here, an optional capability the server never declared is a clean SKIP. That is correct when the server genuinely does not implement the feature. It is wrong when the server implements it and never says so: mcpkit answers events/list, serves all three delivery modes, and declares nothing under capabilities.events, so a client that reads capabilities to decide what to call never reaches any of it. A SKIP reports that as a clean run over a surface no client can use.

So the gate asks before it skips. One extra events/list separates "does not do events" (-32601, skip everything) from "does events off-menu" (returns a catalog, fail the declaration check and keep grading the rest).

The precedent is declaredSkillsCapability in src/scenarios/server/skills/helpers.ts, split out from skillsCapability for the same reason: a malformed declaration was being folded into "not declared" and skipping the whole suite, which "reads as a clean run against a server that is plainly wrong". This applies that argument to an absent declaration rather than a malformed one.

The cost is one extra request against servers that do not implement events at all. If you would rather pay nothing there and accept the false SKIP, say so, but the finding seems worth the request.

[panyam]

Superseded by modelcontextprotocol/conformance#504, opened as a draft against upstream so the WG can review it there.

Same branch (feat/events-conformance-suite on this fork), same commit, so nothing moved — only the PR target. Review comments belong on 504 now.