Add: everything - #1695
Open
DawidWesierski4 wants to merge 6 commits into
Open
Conversation
DawidWesierski4
requested review from
Sakoram,
awilczyns,
moleksy and
soopel
as code owners
August 27, 2026 12:24
DawidWesierski4
force-pushed
the
sqush_cicd
branch
6 times, most recently
from
August 27, 2026 22:02
84a4182 to
1b5e7b3
Compare
The ICE bundle carried a 19-field metadata.env and a cache key built from a hash of every kernel header plus a producer-compiler contract. None of it decided whether a module loads: a bundle that reaches a host is already keyed on the patch hash, the kernel release and the architecture, and what is left is answered by the files themselves. So the bundle is now ice.ko and iavf.ko and nothing else, like every other stash. Validation reads vermagic and the ice_vc_cfg_q_bw symbol -- the kernel it loads into, and that it is the Kahawai build. Activation compares srcversion, and still replaces a module only when the running one differs. Dropping the header ABI and compiler hashes from the key also stops a bundle built for the fleet kernel from missing on a fleet host whose kernel build tree differs in a file that has no bearing on the module. Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com> build-dependencies.sh had two gates and two producers. A standard_miss loop repeated the any_miss gate that build.yml already applies to the step, and it left ICE out, so an ICE-only miss took a second code path into build-ice.sh. That script did what every other component does through MTL_INSTALL_PREFIX: build, then put the output under .local_install. So build_drivers.sh now honors the prefix like build_dpdk.sh does. With one set, the two .ko go to <prefix parent>/ice/<kernel release>/ <architecture> and nothing is installed or loaded, because a kernel module cannot go in a prefix and be loaded. ICE then maps to SETUP_BUILD_AND_INSTALL_DRIVERS_ICE/_IAVF like every other component maps to its own flag, the second gate and the second producer go, and build-ice.sh goes with them. The kernel header check moves with the work, so the build still stops with the package to install instead of a missing make target deep in the driver tree. mh_ice_driver_rebuild clears the prefix: it installs into the running kernel, and a prefix would silently package instead. JPEG XS now follows CI_BUILD_JPEGXS instead of always rebuilding, which is what standard_miss was hiding. FFmpeg asks for a JPEG XS bundle rather than a JPEG XS build, because caching can restore one without the other. Activation loses the process kill -- the cleanup action every job runs first already did it -- and the irdma reload, which nothing in the suite uses. Verified locally through the Taskfile: task ci:build-dependencies with CI_BUILD_ICE=1 alone produces a bundle with the same srcversion the deleted script produced, with no sudo, no system install and no module reload, and task ci:validate-ice and task ci:validate-cache COMPONENT=ice both pass on it. Fix: Derive test port addresses from the host octet test_random_ip built the host octet of every port after the primary one from p_ip[0], which is the network octet: p->sip_addr[i][3] = p_ip[0] + i; That only worked because the random path sets the network octet to 197, so the redundant port took .198. gtest.sh now passes --p_sip, which makes p_ip[0] equal to 192, so the redundant port always takes .193 and the run stops whenever the primary address ends in 193: Error: mt_user_params_check, same ip 192.168.193.193 for port 1 and 0 Error: mtl_init, mt_user_params_check fail -22 gtest.sh made that a 1 in 256 chance, because it built the subnet octet and the host octet from one seed expression. Run 33116774088 drew seed 18881, and 18881 % 256 is 193. So the host octet now counts up from the primary host octet, and stays in 1..254, so no port takes the network or the broadcast address. The seed picks the subnet alone, the primary host octet is 1, and TEST_R_SIP comes from TEST_P_SIP, because the value the seed gave it never reached the test binary. The "Skip if user already set this port's IP" guard goes with it. The loop starts at MTL_PORT_R, so it could never see MTL_PORT_P. Verified with the arguments of the failing job. The old binary reports the same ip error, and the new one gets past mt_user_params_check. Ci: Let provision-runner repair a dpdk build host Nightly run 33121513803 stopped in the build job on mtl-runner-3: meson reported "gstreamer-1.0 found: NO" and ecosystem/gstreamer_plugin stopped. The same commit built on mtl-runner-9, which holds the packages. The build job installs nothing, by the rule this workflow states, so no commit can repair the host. Add the dpdk label to the host choice, and add a step that installs the apt lists setup_environment.sh already holds. The step sources that script, which runs no build step because the script guards its body, and each flag selects one apt list. So there is still one package list. The verify step asks pkg-config the question meson asks. Gate the virtualenv verify step on the same input as its install step, because a run that makes no virtualenv cannot verify one. Run 33152219490 gave no result. A person stopped the three bare-metal gtest jobs before they reached a runner, so no code of this branch ran. This commit changes no file. It only starts the checks again. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com> Run 33155049621 killed KahawaiTest with a general protection fault in librte_eal at St20_rx.detect_uframe_mix_s2 on every NIC label. The DPDK 26.07 patch put rte_free(vf->qtc_map) in iavf_hierarchy_commit(), but upstream 26.07 gets that map from libc calloc(). rte_free() then read a malloc element header that the DPDK heap never wrote and faulted. The same line also freed a block that the next line frees again. This change takes both added frees out and keeps the iavf_dev_info_get() capability gate, which is the purpose of the patch. Upstream already frees the old map in iavf_hierarchy_commit() and clears the map in iavf_tm_conf_uninit(), so no new release is needed. The patch message and the diffstat now agree with the code. Fix: Never advertise runtime queue setup on an iavf VF Run 33161105103 failed the run-gtest (e810) job. St20_rx tests reported fb_rec 0, and the port counter rx_hw_dropped_packets climbed past two million. A session received only when it landed on a queue index that no earlier test had used. The 26.07 iavf patch advertised RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP while vf->qtc_map was NULL. MTL reads dev_capa once in mt_dev_create, before it commits the rate limit hierarchy, so the capability was on. MTL then started and stopped one Rx queue per session, and an E810 VF queue that stops once never receives again. The patch now takes the runtime Rx and Tx queue setup capability away always, as the 26.03 patch does. The two documentation hunks go away with the condition they described. .github/scripts/ci/check-yaml-policy.sh rejected a block scalar after run: or script: in .github/workflows and .github/actions. The Taskfile task ci:check-yaml was its only caller, and no workflow and no script calls that task. This change deletes the script and the task. No other file names either one. Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com>
DawidWesierski4
force-pushed
the
sqush_cicd
branch
from
August 28, 2026 11:02
cdecef5 to
dc6fce9
Compare
The gtest St20_rx.pcap_dump failed on each NIC in run 33166408995. The step "run gtest bare metal" printed "mt_pcap_open(1,535), no pcap support for this build", and st20_rx_pcapng_dump() returned -5. This branch moved versions.env from DPDK 26.03 to DPDK 26.07 and added the patch series patches/dpdk/26.07/, but that series left out "pcapng: add user timestamp support". That patch is the only source of the macro MTL_DPDK_HAS_PCAPNG_TS and of rte_pcapng_copy_ts(), and upstream DPDK 26.07 holds neither. So lib/src/mt_pcap.h took the #else branch and built the stub mt_pcap_open(), which returns NULL. The new file is a copy of patches/dpdk/26.03/0006-pcapng-add-user- timestamp-support.patch. It applies to DPDK 26.07 with no offset and no fuzz, so script/check_dpdk_patches.sh stays as it was. No other patch of the 26.07 series touches lib/pcapng. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com>
The build job of run 33168999034 stopped on mtl-runner-12 with "buildtools/meson.build:14:31: ERROR: python3 is missing modules: elftools". The stop came from `meson setup` of DPDK, after the job had already downloaded the DPDK archive, unpacked it and applied the whole patch series. The message names no package and no host, so the cause of the failure is not readable from the log of the job. The step "system: Check host prerequisites" runs before that work, and its build scope holds what consuming DPDK asks of the host. DPDK's buildtools/meson.build asks python3 for the elftools module, so the check belongs in that scope. The job now stops in seconds with the apt line that repairs the host. mtl-runner-12 was the one host of the fleet without python3-pyelftools, because the watchdog package list of the devtools repository did not name it. That list now holds it. Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com>
The two pr-gate / wait-for-build jobs of run set 33169960xxx failed 8 seconds after they started, with `"build" finished with: cancelled`. The Build workflow of the same commit was in_progress at that moment, so the gate reported a failure on a build that had not finished, and both bare-metal test legs then skipped. Commit 284d27d reached the branch through two events, a push and the pull_request synchronize it caused. Each event started its own run set on the same commit, and the concurrency group cancelled the older set. That left a completed/cancelled "build" check on the commit while the live Build run had not yet created its own check. `checks.listForRef` returns both, so the action found a completed run whose conclusion was not success, found nothing pending, and stopped. A cancelled or a skipped check is not a result. The action now drops those runs before it decides. With only a cancelled check on the commit the state falls back to "not found", and the absent budget waits for the live check. A real failure still fails at once, and a success still passes. Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com>
activate-ice.sh compared the srcversion of the loaded ice and iavf modules with the srcversion of the cached ones, and did nothing when the two matched. So a test could run against the module the host already held. A srcversion match is not enough. It says only that the loaded module was built from the same source. It does not say the driver is in a state a test can use: a job that ran before can leave the driver holding its own state, a set of VFs, or irdma bound to it. A module that the distribution supplies can also carry the same source with other build options, and the hardware rate limiter of the pacing needs the build that the patches of patches/ice_drv/ make. The load is now unconditional. The script takes the VFs of the ice PFs down, removes irdma, ice and a loaded iavf, and puts the cached modules in their place. The check that follows the load stays as it was. `modprobe -r ice` is now asked first whether ice is loaded at all, because that command fails on a module that is not loaded and the path runs on every job. Each of the six workflows that use the validate-host action builds the VF state it needs after the load: the two gtest workflows run `ci:bind-test-ports`, and the four pytest workflows get their VFs from Nicctl.create_vfs() in the acceptance conftest. So a reload that takes the VFs down costs a rebuild and holds no test back. Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com>
configure-host.sh set GST_PLUGIN_PATH to the cached plugin directory and stopped there. That variable adds a directory to the search; it does not take one away. GStreamer also reads its system plugin directories, so a libgstmtl_*.so that an earlier install left in one of them is a second plugin of the same name. Which of the two answers a pipeline follows from the order the registry was built in, so the gstreamer suite could run against a plugin that no job put on the host and that no cache key covers. The seven MTL plugins -- libgstmtl_common, and the st20p, st30p and st40p pairs -- are now taken out of the system plugin directories before the environment is written, so the cache is the one source left. This is what activate-ice.sh does for the driver: remove what the host holds, then use what the cache holds. Only the MTL plugins go. The core, base and good plugins stay, because the pipelines of the suite need videotestsrc, filesink and their like and the packages of the host supply them. GST_REGISTRY now names a file of this job. The registry under ~/.cache names the files it read the last time, and these runners are long-lived, so it can hold an entry for a plugin that was just taken away. GStreamer then reports the element and fails to load it. Signed-off-by: Wesierski, Dawid <dawid.wesierski@intel.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.