Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions plugins/magpie-contributor-growth/skills/candidate-screen/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ argument-hint: "[target:committer|pmc|both] [window:6m] [end:YYYY-MM-DD]"
capability: capability:stats
surface_hash: sha256:a1767d466ee963d1
license: Apache-2.0
measured_tokens: 3723
measured_tokens: 3994
---

<!-- SPDX-License-Identifier: Apache-2.0
Expand Down Expand Up @@ -130,7 +130,7 @@ Local modifications go in the override file; framework changes go via PR to `apa
| `window:Nm` | the configured window, else `6m` | Activity window |
| `end:YYYY-MM-DD` | today | Last day of the window |

From `<project-config>/contributor-nomination-config.md`: `report_repo` (required), `report_path` (default `reports/`), `screen_prefilter_ratio` (default `0.5`).
From `<project-config>/contributor-nomination-config.md`: `report_repo` (required), `report_path` (default `reports/`), `screen_prefilter_ratio` (default `0.5`), `screen_prefilter_require` (`any` or `all`, default `any`), `screen_prefilter_targets` (`committer` or `both`, default `committer`), `screen_recent_activity_months` (default `4`; `0` turns the check off).
A `shortlist_max_missing` left over from an earlier version is ignored; say so once at the start of the run.
Floors come from `<project-config>/committer-readiness.md`, else `contributor-nomination-config.md`, resolved as `contributor-to-committer` Step 1 does.
Both files are personal configuration, read from the personal layer first and from `.apache-magpie-overrides/` only as a fallback; [they belong in the personal layer](../../../../docs/contributor-growth/README.md#why-the-configuration-is-personal).
Expand Down Expand Up @@ -172,13 +172,20 @@ If even a one-day slice exceeds the cap, stop and say so rather than build a par

## Step 2 — Pre-filter

**`<governance-body>` target:** no pre-filter — the pool is the current committers who are not members, small enough to measure in full.
The pre-filter only drops the long tail; it is the one place the floors are used.

**Committer target:** for each person in the pool, run two count-only queries — changes landed (`list_authored(person, state: landed, count_only)`) and changes reviewed (`list_reviews_given(person, count_only)`), in the window — reading `total` only.
Keep the person when either count is at least `screen_prefilter_ratio` × its floor.
A floor of `0` (an evidence-only metric) is ignored here; it never keeps anyone by itself.
Only these two counts are cheap enough to pre-filter a large pool; list, triage and community activity are measured in Step 3 for everyone who stays.
Log everyone dropped, with both counts, in `dropped`; nobody leaves the pool silently.
**Recent activity (both targets).**
When `screen_recent_activity_months` is above `0`, run one count-only query per person — changes landed in the last that many months of the window (`list_authored(person, state: landed, since: <end> − N months, count_only)`) — and drop anyone whose `total` is `0`.
Someone who stopped contributing months ago has nothing current for the `<governance-body>` to look at, however busy the start of the window was.

**Counts.**
For each person still in the pool, run two count-only queries — changes landed (`list_authored(person, state: landed, count_only)`) and changes reviewed (`list_reviews_given(person, count_only)`), in the window — reading `total` only, and compare each with `screen_prefilter_ratio` × its floor for the person's target.
With `screen_prefilter_require: any`, keep the person when either count reaches its share; with `all`, only when both do.
A floor of `0` (an evidence-only metric) is ignored here: it never keeps anyone by itself, and with `all` it is not required.
The count check applies to the committer target, and to the `<governance-body>` target too when `screen_prefilter_targets` is `both`; otherwise the `<governance-body>` pool — the current committers who are not members — is small enough to measure in full and only the recent-activity check applies to it.

Only these counts are cheap enough to pre-filter a large pool; list, triage and community activity are measured in Step 3 for everyone who stays.
Log everyone dropped, with both counts, the landed count for the recent months when it was checked, and which check dropped them, in `dropped`; nobody leaves the pool silently.

---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,9 @@ and there are no meaningful framework defaults.
| `report_repo` | TODO: `owner/name` of a **private** repository | Where `candidate-screen` commits its report. The skill refuses unless the GitHub API reports the repository as private; restrict it to `<governance-body>` members. |
| `report_path` | `reports/` | Directory inside `report_repo` for the reports. |
| `screen_prefilter_ratio` | `0.5` | `candidate-screen` keeps a contributor for full measurement when merged PRs or reviews reach this share of the floor. |
| `screen_prefilter_require` | `any` | `any`: one of merged PRs and reviews reaching its share of the floor keeps a contributor; `all`: both must. |
| `screen_prefilter_targets` | `committer` | `committer`: the count pre-filter applies to the committer pool only; `both`: to the <governance-body> pool as well. |
| `screen_recent_activity_months` | `4` | Drop anyone with no change landed in the last N months of the window, for both targets; `0` turns the check off. |

---

Expand Down
2 changes: 1 addition & 1 deletion tools/skill-evals/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ Suites are currently implemented for:
- **setup-privacy-llm** — 6 cases across 2 suites (step-1-resolve, step-4-gate)
- **preflight-reconciliation** — 7 cases across 1 suite (step-reconciliation)
- **contributor-calibrate** — 11 cases across 3 suites (step-1-find-nominations, step-4-propose-floors, step-6-write-configuration)
- **contributor-candidate-screen** — 14 cases across 6 suites (step-0-gates, step-2-pre-filter, step-3-measure, step-4-write-report, step-5-deliver)
- **contributor-candidate-screen** — 15 cases across 6 suites (step-0-gates, step-2-pre-filter, step-3-measure, step-4-write-report, step-5-deliver)
- **pr-management-stack-review** — 11 cases across 2 suites (step-3-structural-findings, step-4-reading-plan); the gate, mechanical findings, verdict and posting are pytest-tested in `tools/pr-management`

## Prerequisites
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ skill. Tests six decision points:
|---|---|---|
| `step-0-gates` | A public report repository is refused with no alternative offered; a private one proceeds after the audience is confirmed | 2 |
| `step-1-pool` | Roster ids mapped to GitHub handles (unmapped ids listed, current committers excluded); a merged-PR search over 1000 results sliced by month, never truncated | 2 |
| `step-2-pre-filter` | Pre-filter ratio applied; a zero (evidence-only) floor ignored; the PMC pool not pre-filtered; every dropped contributor logged with counts | 3 |
| `step-2-pre-filter` | Pre-filter ratio applied; a zero (evidence-only) floor ignored; the PMC pool not count-filtered by default; no recent landed change drops a contributor; `require: all` and `targets: both`; every dropped contributor logged with counts | 4 |
| `step-3-measure` | Everyone who passed the pre-filter goes into the report, alphabetically, with no comparison against the floors | 1 |
| `step-4-write-report` | Surfacing note at the top; verified real name; no `@`-mentions anywhere in the report; capped counts shown as minimums; people listed alphabetically, never by activity, with no floors-met count or threshold column | 3 |
| `step-5-deliver` | No write without confirmation; privacy re-checked before the write; nothing posted elsewhere | 3 |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
https://www.apache.org/licenses/LICENSE-2.0 -->

screen_prefilter_ratio: 0.5
screen_recent_activity_months: 0
Floors (committer): prs_merged 40, reviews_total 20.

Count-only results for the window:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

Target: committer.
screen_prefilter_ratio: 0.5
screen_recent_activity_months: 0
Floors (committer): prs_merged 40, reviews_total 0 (evidence only).

Count-only results for the window:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

Target: pmc (the pool is the 14 current committers who are not PMC members).
screen_prefilter_ratio: 0.5
screen_recent_activity_months: 0
Floors (pmc): prs_merged 40, reviews_total 100.

Count-only results for the window:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"kept": [
"india",
"lima"
],
"dropped": [
{
"handle": "juliet",
"merged_prs": 60,
"reviewed_prs": 40
},
{
"handle": "kilo",
"merged_prs": 45,
"reviewed_prs": 10
},
{
"handle": "mike",
"merged_prs": 33,
"reviewed_prs": 175
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->

Target: both. Window 2026-04-10 → 2026-10-10.
screen_prefilter_ratio: 1.0
screen_prefilter_require: all
screen_prefilter_targets: both
screen_recent_activity_months: 4
Floors (committer): prs_merged 40, reviews_total 20. Floors (pmc): prs_merged 40, reviews_total 100.

Count-only results for the window (committer pool):
- india: merged PRs 50, reviewed PRs 30, landed since 2026-06-10: 12
- juliet: merged PRs 60, reviewed PRs 40, landed since 2026-06-10: 0
- kilo: merged PRs 45, reviewed PRs 10, landed since 2026-06-10: 9

Count-only results for the window (pmc pool — committers not on the PMC):
- lima: merged PRs 80, reviewed PRs 220, landed since 2026-06-10: 20
- mike: merged PRs 33, reviewed PRs 175, landed since 2026-06-10: 5
5 changes: 4 additions & 1 deletion tools/spec-loop/specs/contributor-growth.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,10 @@ Adopter config scaffolds live in `plugins/magpie-setup/templates/`
`tools/skill-evals/evals/contributor-calibrate/`.
- Skill: `contributor-candidate-screen` — cuts the long tail of recent
contributors with a deterministic pre-filter against the floors (every
drop logged; the only place the floors are used), measures everyone
drop logged; the only place the floors are used; configurable as `any`
or `all` counts, for the committer pool or both pools, plus a
recent-activity check that drops anyone with nothing landed in the
last `screen_recent_activity_months`), measures everyone
who remains — deliberately more than the PMC would consider,
alphabetically by handle, never scored, ranked, shortlisted or judged —
and writes a report (the list linked to details, plus a
Expand Down