Dpdk 26.07 - #1692
Open
DawidWesierski4 wants to merge 17 commits into
Open
Conversation
Three lint systems disagreed. format-coding.sh invoked clang-format, shfmt
and the Python tools directly and required each on PATH, so the version a
developer happened to have installed decided the result; linter.yml ran
super-linter against a second set of rule files; neither pinned the same
versions as the other.
.pre-commit-config.yaml is now the only place a tool, a version, an
argument or a file filter is declared. checkpatch.sh chooses which files
to feed it and how to report, format-coding.sh is its write-mode wrapper,
and the git hooks and linter.yml run that same list on Linux, macOS and
Windows. super-linter keeps only what the list cannot reproduce, including
GITLEAKS: its hook scans the staged diff and so cannot scan a whole tree
or a pull request.
.clang-format becomes a real file at the repository root because
clang-format searches upward from each source file, and a Windows checkout
without symlink support materializes a symlink as a text file, at which
point it silently falls back to LLVM style.
Every pin is at its latest release, and each was measured on its own with
the rest of the config held fixed:
clang-format 14.0.6 -> 22.1.8 isort 5.13.2 -> 8.0.1
black 24.4.0 -> 26.5.1 flake8 7.0.0 -> 7.3.0
ruff 0.4.1 -> 0.16.3 shfmt 3.7.0 -> 3.13.1
shellcheck 0.10.0 -> 0.11.0 markdownlint 0.43.0 -> 0.49.1
yamllint 1.35.1 -> 1.38.0 actionlint 1.7.7 -> 1.7.12
gitleaks 8.16.3 -> 8.30.0 textlint 14.0.4 -> 15.8.0
htmlhint 1.1.4 -> 1.9.2 pre-commit-hooks 6.0.0 (new)
Nine of those are byte-identical over the whole tree. Four are not, and
the interesting content of this commit is what was done about them.
**A version bump may not smuggle in a rule change.** Three tried:
ruff 0.16 reported 592 findings on a tree ruff 0.4 passed -- blind
excepts, datetime timezones, pyupgrade rewrites. None of it was new code.
ruff's *implicit* default rule set grew from about 40 rules to 413, and
.ruff.toml named no rules, so it pinned the version rather than the check.
It now selects E, W and F explicitly.
markdownlint 0.49 ships MD059 and MD060, which did not exist when this
config was vendored; MD060 alone reports 309 findings and rewrites 20
files. Both are off, with the reason recorded at the key.
textlint-rule-terminology 5.x rewrote prose in 15 files, including 18
lines of published CHANGELOG.md history, and replaced "blank line" with
"empty line" -- in a document describing git commit format, where "blank
line" is git's own wording. The *engine* is bumped to textlint 15.8.0; the
*word list* stays at terminology 4.0.1, because a word list is rule
content and rule content is versioned separately for exactly this reason.
clang-format 22 changed two things. It no longer reads `(type)-1` as a
cast, so `((mtl_iova_t)-1)` becomes `((mtl_iova_t) - 1)`; that is
whitespace, the tokens are identical, and for `((align)-1)` -- a macro
parameter, not a type -- the new spacing is simply correct. Accepted. But
from 18 on it also breaks a braced initializer whose elements carry
trailing comments to one element per line, which added 480 lines to
st_avx512_vbmi.c and destroyed the layout of six permute tables that are
written one pixel group per row so the pattern can be read against the
422le10 packing they implement. Those six now sit in a
`/* clang-format off */` region: a pin on the layout, not an exemption
from review.
flake8 is kept rather than folded into ruff, for one measured reason:
ruff 0.16 does not implement F824 ("dead `global` declaration") at all --
`ruff rule F824` answers "unknown rule". flake8 7.3 found two, both
genuinely dead (the names are only mutated, never rebound), both removed
here. .ruff.toml mirrors flake8's rule set rather than extending it so
that retiring flake8 later is a delete and not a re-measurement.
black's `--line-length 88` is black's own default, written down because
the documented Python line length was 120 while the formatter had been
wrapping at 88 the whole time. Not a conflict -- black wraps at 88, ruff
only rejects past 120 -- but only one of the two was stated, and a default
is not a pin.
Five pre-commit-hooks guards are added. None is a style check; each
mechanically enforces a claim this repository already makes and nothing
checked: destroyed-symlinks (the .clang-format-as-text failure above),
check-illegal-windows-names and mixed-line-ending (the platform support
claim), check-merge-conflict, and detect-private-key -- the only secret
scan that runs in a bare whole-tree checkpatch.sh, since gitleaks sees
only the staged diff.
Four more from that repo were probed and left off, two because they fail
on pre-existing defects that are not this commit's to fix, both now
recorded in doc/coding_standard.md §3.1:
check-json 20+ tests/tools/RxTxApp/script/**/*.json use
trailing commas; json-c accepts them, strict JSON
does not.
check-case-conflict tests/acceptance/mtl_engine/RxTxApp.py and
rxtxapp.py are both tracked, so this tree cannot be
checked out on a case-insensitive filesystem --
which contradicts the macOS and Windows support
claimed above. Renaming a module mtl_engine imports
is not a lint change.
The remaining source churn is clang-format 22 and black 26 improving what
they touch: `struct st40_meta m {}` becomes `m{}`, single-expression
lambdas collapse, and black hugs a sole `textwrap.dedent` argument.
check-illegal-windows-names earned its place immediately. patches/ was
excluded globally -- a vendored patch series must not be reformatted -- and
a global exclude turned out to be wrong in both directions. It protected
nothing, because every formatter here is selected by language type and a
*.patch file is none of those types; deleting it changed no hook's result
over the whole tree. And a hook-level exclude cannot un-exclude a global
one, so it silently disabled the one hook that reads paths instead of
content. Blinded, that hook reported "no files to check" while
patches/dpdk/26.03/0012-net-ice-e830:-use-direct-MMIO-for-PHC-update.patch
sat in the tree. A colon is not a legal filename character on Windows, so
`git checkout` there refuses the whole clone with "error: invalid path"
and exit 128 -- meaning the Windows support claimed in §6 had been broken
for as long as that file existed, and the new Windows CI job could never
have gone green no matter what the linters said. The file is renamed
(nothing references it by name; script/build_dpdk.sh globs *.patch and the
0012- ordering prefix is preserved), the exclusion now sits only on
mixed-line-ending, the one hook that does read every file regardless of
type, and the guard is what keeps the name legal from here on.
Renaming the CI job broke build.yml's linter gate, so that is fixed here too.
wait-for-linter polled for super-linter's check run, "Lint Code Base",
which stopped existing the moment checkpatch replaced it. A missing check
is not a failure in that action, it is a wait, so every pull request spent
ten minutes timing out and reported infrastructure flake rather than a
configuration error. The gate now names all four check runs that linter.yml
actually produces -- the three checkpatch OSes and the residual job -- and
build still declares needs: [wait-for-linter, checksums], so a lint failure
skips the DPDK build instead of paying for it. wait-for-workflow takes a
newline-separated list and requires every entry to reach success; a single
name is unchanged, which is what the other two callers pass.
The coupling is by check-run name and nothing validates the two lists
against each other, so all three files now say so at the point where the
mistake would be made: both linter.yml job names, the gate itself, and
doc/coding_standard.md §4.1. The action also reads its inputs from env
instead of interpolating them into the script body, which a multi-line
value would have broken outright.
18 hooks, ./checkpatch.sh clean and idempotent, ColumnLimit stays 90.
Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com>
The bare `"terminology": true` form loads the default term list, whose `png` -> `PNG` rule has no word-boundary guard on the left. It rewrote `rte_pcapng_copy`, `*.png` paths, and every other identifier ending in `png`. The explicit rule keeps the default list and excludes a `png` preceded by a dot or by `pca`. Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com>
MTL sends nothing to dev@dpdk.org any more. The plan is a version bump: move to DPDK 26.07, drop the 5 patches 26.07 already carries, keep and renumber the 11 it does not, and add a test at the cheapest tier for each change that can alter behaviour. upstreaming.md keeps the review history only as the evidence for the drop list, and now links every claim to the file it rests on. Three findings correct the earlier record: - The 2 KB scheduler burst also comes from the kernel ICE patch, which is what programs a VF rate limiter. Dropping the DPDK-side patch may change nothing in the normal deployment, so the rl_burst_size devarg needs a measurement before any code. - The accepted pcapng change is not in 26.07, so mt_pcap.c does not break on this bump. - /home/labrat/dev1/dpdk no longer exists. Six upstream commit hashes rest on it and are a record, not a measurement; T-01 re-proves them against the v26.07 source. Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com>
The orchestrator owns the work list in tasks.md and fires Gates 5 and 6, which mtl-developer can only name. It has no Copilot counterpart in .github/agents/, so the developer and planner prompts said "the user" where the invoker may now be an agent; both are corrected to name the invoker instead. mtl-ste-writing is a symlink into .github/skills/, like every other skill, so Copilot and Claude Code share one copy. CLAUDE.md now points at tasks.md and upstreaming.md, so a fresh session finds the active plan without being told where it is. Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com>
requirements.txt read `mcp[cli]>=1.0.0`, which resolved to mcp 2.0.0. That
release removed `mcp.server.fastmcp`, which both mtl_mcp_server.py and
mtl_acceptance_mcp_server.py import, so each server died with
ModuleNotFoundError before any handshake and every mtl-system-setup and
mtl-acceptance-setup tool went missing from the agent inventory. The venv now
rebuilds at 1.29.0, the last 1.x release, and a hand-fed stdio handshake lists
32 tools on one server and 7 on the other.
The pin alone does not heal a venv that already holds 2.x. The old guard
`if ! python3 -c "import mcp"` succeeds there, so pip never ran and the new
ceiling never applied. Both wrappers now probe
`importlib.util.find_spec("mcp.server.fastmcp")` — the module the servers
really import — so a 2.x venv reinstalls itself and no host needs to delete
.github/mcp/.venv by hand.
A floating major version in agent tooling fails silently and looks like a
broken agent, which is the part worth remembering.
Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com>
versions.env is the single pin, and 4 places named a version beside it.
doc/build.md, doc/build_WIN.md and doc/experimental/header_split.md hardcoded
25.11 or 23.03 in a git checkout, a patch glob or both, so a bump left the
instructions pointing at a directory the tree no longer has. Each file now
sources versions.env and uses ${DPDK_VER}.
validation-tests.yml set DPDK_VERSION: '25.11' in env and used it in the DPDK
checkout. It now reads versions.env into GITHUB_ENV instead. The step is
unreachable today, because DPDK_REBUILD is hardcoded 'false' and is not a
dispatch input, so all 4 consumers never run; this fixes a latent defect, not
a live one. T-17 owns the reachability question.
Two version literals stay on purpose. doc/design.md keeps 25.11 as the
Ubuntu 22.04 AF_XDP workaround, which the file already justifies, and
.github/workflows/msys2_build.yml keeps [25.03, 23.11], because bumping that
matrix would answer a product question in silence. T-13 owns it.
header_split.md no longer says DPDK v23.03 "is verified" as though the reader
should stay there. It says the feature is experimental, states how to
reproduce the verified configuration, and says to restore the pin afterwards.
Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com>
The 26.07 set is 11 files: 9 flat patches, plus hdr_split/0001 and windows/0001. It is a copy of patches/dpdk/26.03/, not a move — D5 keeps the 26.03 directory for the maint branches and for a rollback. Five patches are dropped, because the v26.07 source proves the change is already there: the iavf ring-descriptor cap, the inverted iavf_tm guard, the virtchnl queue-vector size, the E830 PHY model test, and the scheduler burst size. The last one is not covered in the ordinary sense. Upstream superseded MTL's approach with an rl_burst_size devarg and did not take MTL's change, so 26.07 still reads ICE_SCHED_DFLT_BURST_SIZE (15 * 1024) and the old patch still applies. It is dropped because the devarg replaces it; the replacement lands in lib/ as an mtl_port_init_params field. Renumbering is 26.03 0004 to 0013 into 26.07 0001 to 0009. script/build_dpdk.sh applies a flat *.patch glob, so name order is apply order, and the two subdirectories are applied by hand. The directory is inert until versions.env moves. That file still pins DPDK_VER=26.03, and the glob is patches/dpdk/"$DPDK_VER"/*.patch, so nothing reaches these files and no running test can change behaviour because of them. The patch metadata was repaired in the same pass. Every header-bearing file reads `From nobody Mon Sep 17 00:00:00 2001` on line 1, in place of hashes that were a keyboard walk, a hand-typed counter and 40 zeros. Fabricated [PATCH nn/mm] series counters are now a bare [PATCH]. Comma-form identities, which DPDK's own devtools/check-git-log.sh rejects against .mailmap, are plain. Cc: stable@dpdk.org is gone, because this tree posts nothing. Eight of the 9 flat patches commit under 5 real author names. 0009 commits as `MTL Contributor <noreply@example.com>`, and that is deliberate: 5 independent routes failed to recover its author, and inventing a better guess is not a fix. Its Signed-off-by: keeps the same placeholder, because a real name there would forge a DCO certification and deleting the trailer would edit the body. T-27 and T-31 own what is left. The index <pre>..<post> lines are not maintained — 13 stale lines in 7 of the 11 files. Repairing them means regenerating bodies, which a metadata pass must not do. Plain patch -p1 and plain git am ignore them, so the cost lands only on a future git am -3. T-21 owns it. Verified against a pristine v26.07 tree: the 9 flat patches git am clean in order with 0 fuzz and 0 rejects, each optional patch applies after those 9, and VERSION ends 26.07.0_mtl_. Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com>
DPDK 26.07 replaces the MTL patch that flipped ICE_SCHED_DFLT_BURST_SIZE from 15 KB to 2 KB with an ice devarg, rl_burst_size. Dropping that patch is not a no-op for MTL: st_tx_video_session.c:580 already compensates VRX for a 2 KB burst, and MTL selects rate-limit pacing on an ice PF with no PF or VF test, so a PF port would run against a 15 KB burst the pacing does not expect. The new field lets a caller ask for the old value. dev_build_pci_devarg() appends ",rl_burst_size=%u" to the BDF for the PCI path only. Zero means unset and builds a bare BDF, which is the safe default. The field is opt-in because it must be. iavf_parse_devargs() passes a valid-key list to rte_kvargs_parse, so one unknown key returns NULL and the VF never probes; ice uses a valid-key list too, so a misspelling breaks the PF probe as well. No silent no-op is possible either way. MTL cannot tell a PF from a VF before rte_eal_init(), so the library cannot make the choice for the caller — mt_user_params_check() only warns when the field is set on a PMD that has no devarg path at all. MTL does not validate the range. lib/meson.build accepts libdpdk >= 25.03, so one binary can link against ice versions with different bounds and a copied constant would drift toward MTL rejecting a value the driver accepts. The ice PMD stays the single source of truth and its failure is loud, -EINVAL from the probe. No ABI break. The field takes 4 bytes of tail padding that uint64_t flags plus int socket_id already forced, so sizeof stays 16 and no member of port_params[MTL_PORT_MAX] moves. The Rust example still needs 1 line, because a #[repr(C)] struct literal is exhaustive whatever the layout does. The residual hazard belongs to the header, not here: a caller compiled against the old header leaves those bytes uninitialized, so a caller that does not zero the whole struct can fail the probe on garbage. A doc comment cannot fix that, since the caller at risk never reads the new header; a size or version field can, and that is a separate change. MT_EAL_PORT_ARG_MAX_LEN replaces 4 open-coded 2 * MTL_PORT_MAX_LEN widths with one named constant, and every write now takes sizeof(port_params[i]) from the row declaration. Worst cases against 128 bytes are 109 for eth_af_packet, 102 for net_af_xdp and 89 for the PCI devarg. Five tests in tests/unit/dev/mt_dev_devargs_test.cpp pin the string the EAL receives, including the unset case and an out-of-range value passed through unvalidated. Four of the 5 failed first, against the bare BDF. Gate 6 is still open: whether the 7-level PF scheduler tree commits on 26.07, which checks depth against hw->num_tx_sched_layers, needs hardware. The field lands either way, because the decision is about the interface. Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com>
MTL_DPDK_HAS_PCAPNG_TS comes from an MTL patch that adds rte_pcapng_copy_ts(). Upstream accepted a different shape for the same feature — a uint64_t timestamp parameter on rte_pcapng_copy(), plus rte_pcapng_tsc_to_ns(). Release v26.07 carries neither shape, so this bump is safe and the next one is not. The comment sits at the guard, not at a call site, because the guard is where the capture is lost: when the define is absent, mt_pcap.h falls back to stubs and packet capture stops with no build error. It names the symbol, the accepted upstream signature, and the patch by Subject: text rather than by number, since the number changes at every bump. The failure analysis stays in upstreaming.md section 7 only, so the 2 copies cannot drift again. Object code is unchanged: build/lib/libmtl.so.p/src_mt_pcap.c.o hashes the same before and after, and ninja -n confirms the object was dirty first, so the rebuild was real. Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com>
tasks.md carries the round's result. Six tasks are DONE, the code half of T-04 closed on Gate 5, T-05 captured 2 of its 3 hardware baseline runs, and the rest is blocked on the host chain T-03, T-35, T-06, T-07 and on a session restart for the MCP servers. Every task numbered T-11 and above was found by a verification pass, not planned. Twenty-seven are open, most of them defects in the carried patch set or in its own record that only a re-measurement could find. The ones that reach outside this move: the unit suite aborts after 46 of 508 tests because a test reaches rte_eal_init(), and no workflow runs that suite at all; 24 files under patches/dpdk/*/windows/ are symlinks a core.symlinks=false checkout turned into text, so the msys2 workflow cannot pass and the version pin is not the reason; the Rust no_std example does not compile and nothing builds it. upstreaming.md keeps the review history only as the evidence for the drop list. Section 3 now records the 5 greps and the 16 dry runs against a real v26.07 tree — 10 apply and 6 fail, which is not the planned 5 and 11 — and section 8 records what the 26.07 patch metadata says now, including the 1 defect that ships on purpose. Two measurements in there change how the host chain must run, and neither was predicted. The installed ice PMD has no rl_burst_size key at all, so a run today returns an unknown-key probe failure and proves nothing about T-04. And /etc/ld.so.conf.d/mtl_local.conf puts a sibling checkout ahead of /usr/local for the same soname, so installing 26.07 does not by itself change what a test loads; every recorded run needs --log_level notice to prove the version from inside the process. report-dpdk-26.07.md is new. It adds no facts. It tells a reader who did not sit through the round what was done, how, and where it stopped, so tasks.md can stay a work list instead of a narrative. Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com>
The file carried 6 closed tasks under `## Done` and a `### Where the work stands` narrative, together 317 lines, and both are now recorded in report-dpdk-26.07.md and in git log. A work list that also holds its own history buries the 30 things still to do, which was the state before this change. What went: the `## Done` section, the progress narrative, and the `## Order of work` list, whose 6 numbered steps sequenced T-01, T-02, T-08, T-09 and T-10 and so described a plan that is already spent. What stays untouched: every open task body, the Decisions table, the concurrency and snapshot rules, the irreversible-step constraint, and the Cancelled table, which exists so a missing task does not read as an oversight. The new `## What needs to be done` section is the index the file lacked. It names all 30 tasks once, in 5 groups by what actually blocks them: the 7-step serial chain of the move itself, 6 that need a person rather than a command, 10 that can run today with no host and no decision, 5 patch-metadata repairs, and the 2 long-tail tasks that are the only way to shrink the patch set below 11. Every line names the 1 thing that has to happen, so the reader does not have to open a 60-line task body to learn whether it is actionable. The header now says the file holds open work only, and that a task which closes leaves it. The `## Done` section was itself a task, T-33, and it grew back the moment it existed. Verified: all 30 `## T-` headings appear in the new index, and no index entry names a task that has no body. The one reference to a closed task inside an open body, T-24's `See T-08`, now points at git log instead of a section that is gone. Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com>
DawidWesierski4
requested review from
Sakoram,
awilczyns,
moleksy and
soopel
as code owners
August 26, 2026 18:38
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.