Skip to content

Add pre-review PR checklist and description check - #25496

Merged
dkirov-dd merged 23 commits into
masterfrom
dk/ai-7189-pr-template
Oct 7, 2026
Merged

dkirov-dd merged 23 commits into
masterfrom
dk/ai-7189-pr-template

Conversation

@dkirov-dd

@dkirov-dd dkirov-dd commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Checklist before requesting review

  • I self-reviewed the full diff, and this description matches the final change
  • Tests cover the change, or this description explains why none are needed
  • Automated review comments, including Codex review, were addressed or answered
  • I, the human author, understand this change, can explain it to reviewers, and believe it meets the repository standards

What does this PR do?

Replaces the PR template with a short checklist authors complete before requesting review, modeled on DataDog/cloud-inventory. It only covers what CI can't check:

  • the diff was self-reviewed and matches the description
  • tests cover the change
  • automated (Codex) review comments were addressed
  • the human author understands the change and vouches for it (agents must never tick this)

Each item has a hidden Validation comment so coding agents can verify it. CI, the changelog, and the QA label are left to their existing checks.

Adds a pr-description ddev validation (ddev validate pr-description, run by Run Validations) that fails only when a template checklist item is missing or unticked, or the visible description (HTML comments excluded) exceeds 3,000 characters. It is not a required check, so it never blocks merge. It skips bot-authored and release PRs, and PRs opened before the template change (ENFORCED_SINCE). Repos without the checklist in their template are skipped. validate.yml now also runs on edited, so editing the description re-runs validations. The same rules apply to everyone; it does not try to detect AI-assisted PRs.

Updates the AGENTS.md Pull Requests section to match:

  • agents always open drafts and never mark a PR ready or request reviewers; the human author does
  • explain in Motivation how the change fits in, linking only the references a reviewer needs
  • code owned exclusively by agent-integrations is always qa/skip-qa; otherwise agents ask the user

Motivation

AI-7189: make AI-assisted (and all) PRs easier to review with short, standardized descriptions and pre-review checks done by the author, with a human accountable for each PR.

Before merge: set ENFORCED_SINCE to the merge date.

Replace the PR template with an author checklist to complete before
requesting review, each item with validation steps for agents. A new
Check PR description workflow blocks merge until every checklist item is
ticked and the visible description is at most 3,000 characters.

AI-7189
@dkirov-dd dkirov-dd added the qa/skip-qa Automatically skip this PR for the next QA label Oct 2, 2026
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

evalya-impact-summary

evalya impact analysis
Impact analysis: RUN-ALL — every test task will run
Trigger:         empty diff (default branch, scheduled run, or shallow-clone fallback)
Test tasks:      0 (all selected)
Publish tasks:   2 (always emitted)
Diff:            empty (no diff information)

Learn more about CI impact filtering

@dkirov-dd
dkirov-dd marked this pull request as ready for review October 5, 2026 11:43
@dkirov-dd
dkirov-dd requested a review from a team as a code owner October 5, 2026 11:43

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 905b5541e2

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread .github/workflows/scripts/check_pr_description.py Outdated

@AAraKKe AAraKKe left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @dkirov-dd! Added some comments, the one I would strongly request is removing the workflow and script from this pr and not regress on the effort on simplifying CI and rogue scripts in the repo. We have enough and want to move out of them, adding more is a blocker for me since it is against what we have discussed multiple times already. Agents love to do this because there is precedent but we need to stop them from doing this.

Comment thread AGENTS.md Outdated
Comment thread .github/workflows/scripts/check_pr_description.py Outdated
Comment thread AGENTS.md Outdated
Comment thread .github/PULL_REQUEST_TEMPLATE.md Outdated
Comment thread .github/PULL_REQUEST_TEMPLATE.md Outdated
Comment thread .github/PULL_REQUEST_TEMPLATE.md Outdated
Comment thread .github/workflows/pr-description-check.yml Outdated
@dd-octo-sts

dd-octo-sts Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

✅ Dispatcher tests: passed

Dispatcher beta: informational only
Existing CI remains the merge signal.

  2/2 jobs

✅ 2 passed · nothing failed

Batches · ✅ batch-01 2/2

Dispatcher finished on f520160 — GitHub Run · Dispatcher Logs.

@datadog-prod-us1-5

datadog-prod-us1-5 Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Tests  Code Coverage

✅ All CI checks and tests passed. Datadog automation helped this PR pass.

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🔄 Datadog retried 127 tests - 2 passed on retry View in Datadog

🎯 Code Coverage (details)
• Patch Coverage: 96.83%
• Overall Coverage: 89.52%

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 45d2852 | Docs | View more details | Give us feedback!

@dkirov-dd
dkirov-dd requested a review from AAraKKe October 5, 2026 16:01

@AAraKKe AAraKKe left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @dkirov-dd! Two small requests

Comment thread ddev/src/ddev/utils/github.py Outdated
Comment thread ddev/src/ddev/cli/validate/all/orchestrator.py Outdated
AAraKKe
AAraKKe previously approved these changes Oct 7, 2026

@AAraKKe AAraKKe left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Comment thread ddev/src/ddev/cli/validate/pr_description.py Outdated
@temporal-github-worker-1
temporal-github-worker-1 Bot dismissed AAraKKe’s stale review October 7, 2026 14:30

Review from AAraKKe is dismissed. Related teams and files:

  • agent-integrations
    • ddev/src/ddev/cli/validate/pr_description.py
AAraKKe
AAraKKe previously approved these changes Oct 7, 2026
@dkirov-dd
dkirov-dd enabled auto-merge October 7, 2026 14:34
@temporal-github-worker-1
temporal-github-worker-1 Bot dismissed AAraKKe’s stale review October 7, 2026 14:38

Review from AAraKKe is dismissed. Related teams and files:

  • agent-integrations
    • ddev/tests/cli/validate/test_pr_description.py
@dd-octo-sts

dd-octo-sts Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Validation Report

All 22 validations passed.

Show details
Validation Description Status
agent-reqs Verify check versions match the Agent requirements file ✅
ci Validate CI configuration and code coverage settings ✅
codeowners Validate every integration has a CODEOWNERS entry ✅
config Validate default configuration files against spec.yaml ✅
dep Verify dependency pins are consistent and Agent-compatible ✅
http Validate integrations use the HTTP wrapper correctly ✅
imports Validate check imports do not use deprecated modules ✅
integration-style Validate check code style conventions ✅
jmx-metrics Validate JMX metrics definition files and config ✅
labeler Validate PR labeler config matches integration directories ✅
legacy-signature Validate no integration uses the legacy Agent check signature ✅
license-headers Validate Python files have proper license headers ✅
licenses Validate third-party license attribution list ✅
metadata Validate metadata.csv metric definitions ✅
models Validate configuration data models match spec.yaml ✅
openmetrics Validate OpenMetrics integrations disable the metric limit ✅
package Validate Python package metadata and naming ✅
pr-description Validate the pull request checklist and visible description length ✅
qa-label Validate the pull request declares whether it needs QA for the next Agent release ✅
readmes Validate README files have required sections ✅
saved-views Validate saved view JSON file structure and fields ✅
version Validate version consistency between package and changelog ✅

View full run

@dkirov-dd
dkirov-dd added this pull request to the merge queue Oct 7, 2026
Merged via the queue into master with commit 419abeb Oct 7, 2026
665 of 666 checks passed
@dkirov-dd
dkirov-dd deleted the dk/ai-7189-pr-template branch October 7, 2026 16:30
@dd-octo-sts dd-octo-sts Bot added this to the 7.86.0 milestone Oct 7, 2026
github-actions Bot pushed a commit that referenced this pull request Oct 7, 2026
* Add pre-review PR checklist and description check

Replace the PR template with an author checklist to complete before
requesting review, each item with validation steps for agents. A new
Check PR description workflow blocks merge until every checklist item is
ticked and the visible description is at most 3,000 characters.

AI-7189

* Trim checklist to judgment items and add human-author attestation

* Limit QA skip default to code owned exclusively by agent-integrations

* Never use qa/required for code owned exclusively by agent-integrations

* Test the PR description check against the real template

* Drop PR description check tests and log why the check is skipped

* Exempt PRs opened before the checklist template from the description check

* Require the template's checklist items instead of any ticked task

* Link the human-author item to CONTRIBUTING.md and update its PR standards

* Drop the CONTRIBUTING.md standards lead-in

* Link the human-author item to AGENTS.md and restore CONTRIBUTING.md

* Describe the PR description check as failing, not blocking merge

* Move PR description check into ddev

* Rewrap the PR template checklist comment

* Enable the pr-description validation for integrations-core

* Apply review feedback to the PR template and AGENTS.md

* Detect bot authors by GitHub user type

* Pass the GitHub event to pr-description through options instead of env in tests

* Read the current PR description from the API so re-runs see edits

* Fetch the PR description with the async GitHub client and re-run validations on edits

* Move enforced time up to be in line with time of merge

* Derive pr-description test dates from ENFORCED_SINCE

* Set ENFORCED_SINCE to 2026-10-08 00:00 UTC 419abeb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants