Add pre-review PR checklist and description check

.ddev/config.toml

@@ -27,7 +27,8 @@ validations = [
     "readmes",
     "saved-views",
     "version",
-    "qa-label"
+    "qa-label",
+    "pr-description"
 ]
 
 [overrides.display-name]

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,11 +1,45 @@
-### What does this PR do?
-<!-- A brief description of the change being made with this pull request. -->
+## Checklist before requesting review
 
-### Motivation
-<!-- What inspired you to submit this pull request? -->
+<!--
+  The `pr-description` validation (`Run Validations`) fails until every item below is ticked and
+  the visible description (excluding HTML comments) is at most 3,000 characters. CI, the changelog,
+  and the QA label have their own checks. Request review only once all checks pass:
+  `gh pr checks <PR> --repo DataDog/integrations-core --watch`.
+-->
 
-### Review checklist (to be filled by reviewers)
+- [ ] I self-reviewed the full diff, and this description matches the final change
+  <!--
+    How to verify: run `gh pr diff <PR> --repo DataDog/integrations-core`, review every changed line,
+    and compare the final diff with the description below.
+  -->
+- [ ] Tests cover the change, or this description explains why none are needed
+  <!--
+    How to verify: check the diff for relevant tests and run them as described in `AGENTS.md`.
+  -->
+- [ ] Automated review comments, including Codex review, were addressed or answered
+  <!--
+    How to verify: run `gh api repos/DataDog/integrations-core/pulls/<PR>/comments` and
+    `gh pr view <PR> --repo DataDog/integrations-core --json reviews,comments`, then fix each
+    finding or reply with the reason no change is needed.
+  -->
+- [ ] I, the human author, understand this change, can explain it to reviewers, and believe it meets the [repository standards](https://github.com/DataDog/integrations-core/blob/master/AGENTS.md)
+  <!-- Coding agents: never tick this item. Only the human author may tick it. -->
 
-- [ ] Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
-- [ ] Add `qa/required` if this PR needs QA validation, or `qa/skip-qa` if it does not. Exactly one of the two is required.
-- [ ] If you need to backport this PR to another branch, you can add the `backport/<branch-name>` label to the PR and it will automatically open a backport PR once this one is merged
+<!--
+  QA label: PRs that only change code owned exclusively by @DataDog/agent-integrations (see
+  `.github/CODEOWNERS`) always use `qa/skip-qa`, never `qa/required`.
+-->
+
+## What does this PR do?
+
+<!-- Give a minimal but complete description of the change and how it works. -->
+
+
+
+## Motivation
+
+<!--
+  Briefly explain why this change is needed and how it fits in: is it part of a bigger project, and
+  how does it fit into it? Are there related or follow-up PRs? Link only the references a reviewer
+  needs (e.g. Jira ticket, design doc, upstream docs or changelog, related PRs).
+-->

.github/workflows/validate.yml

@@ -5,7 +5,7 @@ on:
     branches:
     - master
   pull_request:
-    types: [opened, synchronize, reopened, labeled, unlabeled]
+    types: [opened, synchronize, reopened, edited, labeled, unlabeled]
   merge_group:
     types: [checks_requested]
 

AGENTS.md

@@ -266,10 +266,11 @@ These guidelines apply to automated code review (the Codex review bot). They do
 
 ## Pull Requests
 
-- Open PRs in draft mode unless explicitly asked otherwise; mark them ready for review once the work is complete and CI passes.
-- Always populate the PR body using the repository's template at `.github/PULL_REQUEST_TEMPLATE.md`. Read the template first and fill in every section (`What does this PR do?`, `Motivation`, `Review checklist`). Do not omit, rename, or reorder the template sections, and do not add unrelated sections on top.
+- Always open PRs in draft mode unless the user explicitly asks otherwise. Never mark a PR ready for review or request reviewers yourself: that notifies code owners, so it is the human author's decision. Tell the user once all CI checks pass (`gh pr checks <PR> --repo DataDog/integrations-core --watch`; relaunch it if it times out).
+- Always populate the PR body using `.github/PULL_REQUEST_TEMPLATE.md`, keep its visible text at or below 3,000 characters, and make the required sections minimal but complete. In `Motivation`, explain how the change fits into any larger project or related PRs, and link the references a reviewer needs to evaluate the change (e.g. the Jira ticket, upstream docs or changelog for a behavior or version change, related PRs), not every source you consulted. Do not omit, rename, or reorder the template sections, or add unrelated sections.
+- Before handing the PR to the user, check each item in the template's `Checklist before requesting review` by following the `How to verify` instructions hidden under it, and tick only the items you verified. Tell the user about anything you could not verify. Never tick the human-author item; only the human author may tick it. The `pr-description` validation fails until every item is ticked and the description is within the length limit.
 - Keep PR titles short and descriptive in plain words. Do not use conventional-commit prefixes (`feat:`, `fix:`, `chore:`).
-- Every PR must declare a QA decision via a label. Add `qa/required` if the PR ships changes that need QA validation, or `qa/skip-qa` if it does not (e.g., docs, tests, developer tooling, or no agent-impacting changes). The `validate-all` CI check fails until exactly one of those labels is set. Tick the matching checkbox in the PR template.
+- Every PR must declare a QA decision via a label. Set exactly one of `qa/required` or `qa/skip-qa`; the `qa-label` validation fails until one is set. Whether a change needs QA is a team decision, not something to infer from the diff. PRs that only change code owned exclusively by `@DataDog/agent-integrations` in `.github/CODEOWNERS` always use `qa/skip-qa`, never `qa/required`. Otherwise, unless another rule in your context settles it, ask the user which label to use.
 - Push the branch and open the PR before adding the changelog entry so the entry filename can reference the real PR number.
 
 ## Documentation

ddev/changelog.d/25496.added

@@ -0,0 +1 @@
+Add a `pr-description` validation for pull request checklist and length requirements.

ddev/src/ddev/cli/validate/__init__.py

@@ -27,6 +27,7 @@
 from ddev.cli.validate.licenses import licenses
 from ddev.cli.validate.metadata import metadata
 from ddev.cli.validate.openmetrics import openmetrics
+from ddev.cli.validate.pr_description import pr_description
 from ddev.cli.validate.qa_label import qa_label
 from ddev.cli.validate.version import version
 
@@ -58,6 +59,7 @@ def validate():
 validate.add_command(models)
 validate.add_command(openmetrics)
 validate.add_command(package)
+validate.add_command(pr_description)
 validate.add_command(qa_label)
 validate.add_command(readmes)
 validate.add_command(saved_views)

ddev/src/ddev/cli/validate/all/orchestrator.py

@@ -106,6 +106,14 @@ class ValidationConfig:
     "package": ValidationConfig(
         description="Validate Python package metadata and naming",
     ),
+    "pr-description": ValidationConfig(
+        description="Validate the pull request checklist and visible description length",
+        repo_wide=True,
+        failure_guidance=(
+            "**To fix:** Tick every item in `.github/PULL_REQUEST_TEMPLATE.md` and keep the description under "
+            "3,000 visible characters."
+        ),
+    ),
     "qa-label": ValidationConfig(
         description="Validate the pull request declares whether it needs QA for the next Agent release",
         repo_wide=True,

ddev/src/ddev/cli/validate/pr_description.py

@@ -0,0 +1,175 @@
+# (C) Datadog, Inc. 2026-present
+# All rights reserved
+# Licensed under a 3-clause BSD style license (see LICENSE)
+from __future__ import annotations
+
+import json
+import re
+from datetime import datetime
+from typing import TYPE_CHECKING
+
+import click
+from pydantic import ValidationError
+
+from ddev.utils.github_actions import PullRequestEvent
+
+if TYPE_CHECKING:
+    from ddev.cli.application import Application
+
+
+MAX_DESCRIPTION_LENGTH = 3000
+# PRs opened before the checklist template existed are exempt. Set to the merge date of the template change.
+ENFORCED_SINCE = datetime.fromisoformat('2026-10-08T00:00:00+00:00')
+CHECKLIST_HEADING = 'Checklist before requesting review'
+HEADING_PATTERN = re.compile(r'^##[ \t]+(.+?)[ \t]*$', re.MULTILINE)
+COMMENT_PATTERN = re.compile(r'<!--.*?-->', re.DOTALL)
+CHECKBOX_PATTERN = re.compile(r'^[ \t]*-[ \t]*\[([ \txX])\][ \t]*(.+?)[ \t]*$', re.MULTILINE)
+RELEASE_TITLE_PATTERN = re.compile(
+    r'^(?:\[backport\]\s*)?(?:\[release\]\s*|finalize agent release\b|release new integrations\b)',
+    re.IGNORECASE,
+)
+
+
+def _sections(body: str) -> dict[str, str]:
+    headings = list(HEADING_PATTERN.finditer(body))
+    sections = {}
+    for index, heading in enumerate(headings):
+        end = headings[index + 1].start() if index + 1 < len(headings) else len(body)
+        sections[heading.group(1).strip()] = body[heading.end() : end]
+    return sections
+
+
+def _skip_reason(author: str, author_type: str, title: str, created_at: str | None) -> str | None:
+    if created_at and datetime.fromisoformat(created_at.replace('Z', '+00:00')) < ENFORCED_SINCE:
+        return f'PR opened before {ENFORCED_SINCE:%Y-%m-%d}, when the checklist template was introduced'
+    if author_type == 'Bot':
+        return f'bot author {author}'
+    if RELEASE_TITLE_PATTERN.match(title):
+        return 'release PR'
+    return None
+
+
+def _checklist(body: str) -> dict[str, bool]:
+    """Map each checklist item's text to whether it is ticked, ignoring HTML comments."""
+    section = _sections(COMMENT_PATTERN.sub('', body)).get(CHECKLIST_HEADING, '')
+    return {' '.join(text.split()): bool(mark.strip()) for mark, text in CHECKBOX_PATTERN.findall(section)}
+
+
+def _fetch_current_body(app: Application, repository: str, pr_number: int) -> str | None:
+    """Return the PR's current description ('' if empty), or None if it could not be fetched."""
+    import asyncio
+
+    import httpx
+
+    from ddev.utils.github_async import async_github_client
+    from ddev.utils.github_errors import GitHubAuthenticationError
+
+    owner, _, repo = repository.partition('/')
+    token = app.config.github.token
+    if not (owner and repo and token):
+        return None
+
+    async def fetch() -> str:
+        async with async_github_client(token=token) as client:
+            response = await client.get_pull_request(owner, repo, pr_number)
+            return response.data.body or ''
+
+    try:
+        return asyncio.run(fetch())
+    except GitHubAuthenticationError as error:
+        app.abort(str(error))
+    except httpx.HTTPError:
+        return None
+
+
+def _check_pr_description(body: str, required_items: list[str]) -> list[str]:
+    visible_body = COMMENT_PATTERN.sub('', body)
+    items = _checklist(body)
+    errors = []
+    for item in required_items:
+        if item not in items:
+            errors.append(f'Missing checklist item (restore it from the template): {item}')
+        elif not items[item]:
+            errors.append(f'Unchecked checklist item: {item}')
+
+    description_length = len(visible_body)
+    if description_length > MAX_DESCRIPTION_LENGTH:
+        errors.append(
+            f'PR description is {description_length} characters; maximum is {MAX_DESCRIPTION_LENGTH}. '
+            'Trim the description before requesting review.'
+        )
+
+    return errors
+
+
+@click.command(short_help='Validate the current pull request description')
+@click.option(
+    '--event-name',
+    envvar='GITHUB_EVENT_NAME',
+    default='',
+    help='GitHub event name; the validation only runs for `pull_request`. Defaults to $GITHUB_EVENT_NAME.',
+)
+@click.option(
+    '--event-path',
+    envvar='GITHUB_EVENT_PATH',
+    default='',
+    help='Path to the GitHub event payload JSON. Defaults to $GITHUB_EVENT_PATH.',
+)
+@click.pass_obj
+def pr_description(app: Application, event_name: str, event_path: str):
+    """Fail when the PR checklist is incomplete or the description is too long."""
+    if event_name != 'pull_request':
+        app.display_info('Not running in a pull_request context; skipping pr-description validation.')
+        return
+
+    if not event_path:
+        app.display_info('No GitHub event payload path given; skipping pr-description validation.')
+        return
+
+    try:
+        event = PullRequestEvent.load(event_path)
+    except (OSError, json.JSONDecodeError, ValueError, ValidationError) as exc:
+        app.abort(f'Could not read GitHub event payload: {exc}')
+
+    pull_request = event.pull_request
+    if pull_request is None:
+        app.display_info('Event payload has no pull request; skipping pr-description validation.')
+        return
+
+    user = pull_request.user
+    author = (user.login if user else None) or ''
+    author_type = (user.type if user else None) or ''
+    reason = _skip_reason(author, author_type, pull_request.title or '', pull_request.created_at)
+    if reason:
+        app.display_info(f'Skipping PR description check: {reason}.', markup=False)
+        return
+
+    template_path = app.repo.path / '.github' / 'PULL_REQUEST_TEMPLATE.md'
+    try:
+        template = template_path.read_text(encoding='utf-8')
+    except OSError:
+        app.display_info(f'No checklist items found because {template_path} could not be read; skipping validation.')
+        return
+
+    required_items = list(_checklist(template))
+    if not required_items:
+        app.display_info(
+            f'No checklist items found under ## {CHECKLIST_HEADING} in {template_path}; skipping validation.'
+        )
+        return
+
+    # Re-runs reuse the original event payload, so read the current description from the API.
+    repository = event.base_repo
+    body = _fetch_current_body(app, repository, pull_request.number) if repository and pull_request.number else None
+    if body is None:
+        app.display_warning('Could not fetch the current PR description; using the event payload, which may be stale.')
+        body = pull_request.body or ''
+
+    errors = _check_pr_description(body, required_items)
+    if errors:
+        for error in errors:
+            app.display_error(error, markup=False)
+        app.display_info('Update the pull request body using .github/PULL_REQUEST_TEMPLATE.md.')
+        app.abort()
+
+    app.display_success('PR description check passed.')

ddev/src/ddev/utils/github_actions.py

@@ -24,12 +24,23 @@ class PullRequestRef(BaseModel):
     repo: Repo | None = None
 
 
+class EventUser(BaseModel):
+    model_config = ConfigDict(extra='ignore')
+
+    login: str | None = None
+    type: str | None = None
+
+
 class EventPullRequest(BaseModel):
     model_config = ConfigDict(extra='ignore', strict=True)
 
     number: int | None = None
     head: PullRequestRef | None = None
     base: PullRequestRef | None = None
+    body: str | None = None
+    title: str | None = None
+    user: EventUser | None = None
+    created_at: str | None = None
 
 
 class PullRequestEvent(BaseModel):