confluent-kafka: add build-confluent-kafka.yml for riscv64 wheels

[luhenry]

confluent-kafka publishes per-interpreter compiled wheels (a cimpl extension linked against librdkafka) for x86_64/aarch64 but not riscv64.

Upstream's released wheels take librdkafka from a prebuilt librdkafka.redist NuGet package (tools/wheels/install-librdkafka.sh), which has no riscv64 build. So this workflow uses upstream's other supported path — tools/prepare-cibuildwheel-linux.sh, the Linux CIBW_BEFORE_BUILD that their tools/cibuildwheel-build.sh sets — which runs bootstrap-librdkafka.sh --require-ssl, so mklove builds zlib, OpenSSL 3.5.6, zstd 1.5.7 and curl 8.20.0 from source and links them statically into librdkafka.

Two riscv-only deviations:

• perl. The Rocky 10 manylinux image ships only perl-interpreter, so OpenSSL's Configure dies with Can't locate lib.pm in @INC. Verified in quay.io/pypa/manylinux_2_39_riscv64 under QEMU: before dnf -y install perl that error reproduces exactly; after it, use lib; use FindBin; use IPC::Cmd; use Time::Piece; use Pod::Html all succeed (perl-lib, perl-FindBin, perl-IPC-Cmd, perl-Time-Piece, perl-Pod-Html). Installing the whole distribution rather than picking modules one CI cycle at a time — a first attempt that named four of them still missed lib.pm. wget is not needed: mklove's mkl_download_file falls back to curl, which the image has.
• auditwheel repair --strip, because a source-built librdkafka keeps its debug symbols (58MB against the 11MB prebuilt one upstream bundles); stripped it lands at 9.8MB, i.e. upstream's wheel size.

The librdkafka tag is read out of the checkout's .semaphore/semaphore.yml with upstream's own sed from tools/pre-release-validation.sh, so it tracks whatever the release pins rather than being hardcoded a second time.

Licensing

The wheel statically links OpenSSL, zlib, zstd and curl into librdkafka.so — strings on upstream's own 2.15.0 aarch64 wheel shows STATIC_LINKING ... ZLIB SSL ZSTD CURL and OpenSSL 3.5.6. Upstream's LICENSE has a dedicated "The binary wheel distribution of confluent-kafka-python contains additional software with the following licenses" section, but it covers only OpenSSL, zlib and librdkafka's own bundled licences — zstd and curl are absent (grep -ci curl LICENSE → 0). Both require their copyright notice to travel with binary redistributions, so patches/confluent-kafka/2.15.0/0001-LICENSE-add-zstd-and-curl-notices.patch adds them in upstream's existing format, tagged Upstream-Status: To upstream.

LICENSE is already shipped via MANIFEST.in + setuptools' default LICEN[CS]E* glob, so this is a content-only change with no packaging edit. The first test-command asserts the notices are present in the installed dist-info/licenses/LICENSE, so a patch that silently stops applying fails the job rather than leaving it green — checked against an unpatched 2.15.0 wheel, where the assertion fails.

Tests

Upstream's wheel job only runs pytest {project}/tests/test_error.py. The rest of tests/ needs no cluster, so this runs all of it bar tests/integration (needs a live Kafka cluster) and tests/schema_registry (needs the rules extra, whose tink and google-re2 have no riscv64 build).

Test deps are upstream's requirements/requirements-tests.txt plus the oauthbearer-aws extra and avro/requests — exactly what the collected modules import. The avro extra is deliberately not used: it pulls authlib → cryptography, and PyPI's latest cryptography (50.0.1) has no riscv64 wheel while our registry tops out at 49.0.0, so pip would resolve to the newer one and try a Rust build from sdist in a container with no cargo. Every remaining dep resolves to a riscv64 or py3-none-any wheel (ast-serialize, librt and coverage, the only compiled ones, all publish riscv64 wheels).

Dry-run of the exact test command against upstream's released 2.15.0 wheel on macOS/arm64 (no compile, no QEMU): 670 passed, 16 skipped.

Matrix is cp312/cp313/cp314 — upstream's build-wheels.sh sets CIBW_SKIP="... *t-*" and PyPI carries no free-threaded wheel, so no cp314t.

A latent race in test_callback_exception_no_system_error

patches/confluent-kafka/2.15.0/0002-tests-make-the-raising-callbacks-in-test_callback_ex.patch.

The test arms an error_cb and a stats_cb that raise unconditionally, asserts the exception surfaces out of consume(), and then calls close() with the callbacks still armed. librdkafka keeps delivering both events for the consumer's lifetime — stats every statistics.interval.ms (100ms here), the broker-resolve failure on its retry backoff — and close() dispatches whatever is queued, so once more than one interval elapses between the assertions and close() the callback fires again and its RuntimeError escapes close().

That failed the cp312 job of run 32915860381 while cp313 passed on the same tree — a race, not an architecture problem; the riscv64 runner is simply slow enough to reach it. Reproduced on macOS/arm64 against upstream's released 2.15.0 wheel by inserting time.sleep(1.2) before each close() in the real test: it fails there unpatched and passes with this change.

The patch makes each callback raise on its first invocation only, so every assertion is unchanged and only the redundant later raises during close() go away. Re-running the full reduced suite against the released wheel with both patches applied still gives 670 passed, 16 skipped.

A refcount bug in AdminClient.delete_records()

patches/confluent-kafka/2.15.0/0003-Admin-stop-delete_records-dropping-a-borrowed-refere.patch.

The cp314 job of the same run died with Fatal Python error: Segmentation fault inside re/_compiler.py while pytest compiled a literal regex — the first statement of tests/test_Consumer.py::test_basic_api, immediately after tests/test_Admin.py finished. Nothing about it is riscv64-specific: the identical crash, same file and same line, reproduces on macOS/arm64 under CPython 3.14.7 against upstream's released 2.15.0 wheel, in about a second:

pytest -q tests/test_Admin.py tests/test_Consumer.py     # rc=139

Bisecting isolates it to tests/test_Admin.py::test_uninitialized_admin_client_methods, and within that test to admin.delete_records([TopicPartition("topic", 0, 10)]). Admin_delete_records() parses topic_partition_offsets out of PyArg_ParseTupleAndKeywords("OO|ff", ...) — a borrowed reference it never Py_INCREFs — and then Py_XDECREFs it on both the success and the err: path. Every call drops a reference it does not own, measurable against the released wheel without a broker:

lst = [TopicPartition("topic", 0, 10)]; keep = lst
sys.getrefcount(lst)            # 3
admin.delete_records(lst, ...)  # or the RuntimeError path
sys.getrefcount(lst)            # 2

Both paths, on 3.12 as well as 3.14. In the test the list is a temporary owned only by the argument tuple, so it is freed while still referenced and the interpreter faults at whatever runs next — interpreter shutdown when the test runs alone, an unrelated test when it runs in a suite. That is why cp313 passed on the same tree.

The patch drops the two decrefs. It is the only unbalanced one in the extension: the other Admin_* functions that decref a parsed-in object do so for future, which they Py_INCREF first because options_to_c() hands it to the background event callback as the opaque.

Note on triggers

A brand-new build-<pkg>.yml cannot be workflow_dispatch-ed: GitHub resolves the workflow file on the default branch, so gh workflow run (and therefore pr-trigger.yml's Trigger: line) answers HTTP 404: workflow build-confluent-kafka.yml not found on the default branch — that is what happened on run 32891418686. The pull_request: paths trigger is what produces the first run and registers the workflow, so it is declared here alongside workflow_dispatch, as every workflow on main does.

---

CI

Run 32962091431 is green on all three interpreters — 670 passed, 16 skipped on each of cp312/cp313/cp314 — and the publish job dry-ran cleanly:

Dry run (not on main branch — no upload will happen)
Would upload 3 file(s) to https://gitlab.com:
  dist/confluent_kafka-2.15.0-cp312-cp312-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl
  dist/confluent_kafka-2.15.0-cp313-cp313-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl
  dist/confluent_kafka-2.15.0-cp314-cp314-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl

Each wheel is ~4.1 MB and carries confluent_kafka/cimpl.cpython-3XX-riscv64-linux-gnu.so plus a stripped 8 MB confluent_kafka.libs/librdkafka-*.so.1.

[luhenry]

CI could not be started for this PR: pr-trigger.yml picked up the Trigger: confluent-kafka:2.15.0 line and ran gh workflow run build-confluent-kafka.yml --ref confluent-kafka, which GitHub rejects with

HTTP 404: workflow build-confluent-kafka.yml not found on the default branch

workflow_dispatch can only target a workflow that already exists on the default branch, so a brand-new build-<pkg>.yml cannot be dispatched from its own PR branch. (Run 32891418686.) Nothing in this branch can fix that — either the workflow lands on main first and is dispatched from there, or a temporary pull_request: paths: trigger is added for the review cycle, which is what the recent new-package PRs did (e.g. #360 at 91c9c2c, stripped before merge).

What was verified locally instead, on aarch64 in quay.io/pypa/manylinux_2_28_aarch64 with this exact recipe:

• tools/prepare-cibuildwheel-linux.sh v2.15.0 builds zlib 1.3.2, OpenSSL 3.5.6, zstd 1.5.7 and curl 8.20.0 from source and then librdkafka, --require-ssl satisfied;
• pip wheel + auditwheel repair --strip produce confluent_kafka-2.15.0-cp312-cp312-manylinux_2_28_aarch64.whl containing confluent_kafka/cimpl.cpython-312-*.so and a 9.8MB confluent_kafka.libs/librdkafka-*.so.1 (58MB unstripped; upstream ships 11MB);
• the test command above: 667 passed, 16 skipped (confluent_kafka.libversion() → 2.15.0).

riscv64-specific checks run under QEMU against quay.io/pypa/manylinux_2_39_riscv64:

• dnf -y install perl then OpenSSL 3.5.6 ./config … → "OpenSSL has been successfully configured". Without it Configure dies on Can't locate FindBin.pm — Rocky 10 ships only perl-interpreter;
• yum install -y zlib-devel gcc-c++ (upstream's line, unmodified) resolves: zlib-devel is provided by the preinstalled zlib-ng-compat-devel;
• python3, make, patch, file, nm, ar, autoconf, automake, libtool, pkg-config are all present.

[luhenry]

On cb21ffd, is it an issue known upstream?

EDIT: https://github.com/confluentinc/confluent-kafka-python/pull/2328/changes, make sure to backport that patch rather than doing your own

[luhenry]

Good catch — backported in 5db5778 rather than carrying our own.

patches/confluent-kafka/2.15.0/0003-Fix-reference-counting-in-delete_records-Admin-API.patch is now upstream's commit 253e0c6cfd7a0f07e9c43ab686824dcc8bca1b4b (confluentinc/confluent-kafka-python#2328, merged 2026-08-24), with Upstream-Status: Backport pointing at that PR, so it drops out cleanly once we build a release that contains it. It keeps upstream's authorship, and it also brings the regression test in tests/test_Admin.py that our hand-written version did not have — the Admin.c change is byte-for-byte the same two Py_XDECREF(topic_partition_offsets) removals we had arrived at independently.

The only thing dropped from upstream's commit is its CHANGELOG.md hunk: it does not apply to the 2.15.0 tag and carries no code. Verified git apply --check against a clean v2.15.0 checkout, and ci_scripts/check_patch.py passes on all four patches.