build: cross-compile flow-connector-init for linux/amd64, bump CONNECTOR_INIT_IMAGE - #3290
Open
mdibaiee wants to merge 1 commit into
Open
build: cross-compile flow-connector-init for linux/amd64, bump CONNECTOR_INIT_IMAGE#3290mdibaiee wants to merge 1 commit into
mdibaiee wants to merge 1 commit into
Conversation
flow-connector-init is bind-mounted into every connector container as its entrypoint, and connector containers always run linux/amd64. The build task targeted $(uname -m), so on an aarch64 host it produced an aarch64 binary that those containers cannot exec: every connector dies before starting, reported only as "failed to inspect a started docker container (did it crash?)" with the container stuck in Created. Build for x86_64-unknown-linux-musl unconditionally, cross-compiling when the host is something else. ring and aws-lc-sys compile C, and their objects must see musl headers -- a glibc cross-gcc leaves glibc-only symbols (__isoc23_sscanf, __fprintf_chk) that the musl link cannot resolve -- so use zig cc, which ships headers and libc for every target, and let rustc's bundled lld link against the target's musl. locate_bin() searches PATH, so the local stack and IDE launch configs also need the x86_64 musl directory on it; without it the runtime silently falls back to extracting flow-connector-init from CONNECTOR_INIT_IMAGE. Also bump that pinned image, which was 67 commits older than the commit adding created_at to task specs. Because it decodes and re-encodes every request between runtime and connector, a copy older than a protocol field drops it silently: connectors observed specs with no created_at at Validate, Apply and Open even though the build database and live_specs both carried it.
This was referenced Aug 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description:
Two fixes so that a locally built stack can actually run connectors, and so that connectors see the spec fields the runtime sends them.
build:connector-initnow builds for linux/amd64.flow-connector-initis bind-mounted into every connector container as its entrypoint, and connector containers always runlinux/amd64(hard-coded incrates/runtime/src/container.rs). The task built for$(uname -m), so on an aarch64 host it produced an aarch64 binary those containers cannot exec — every connector container dies before starting. The only symptom is:with
docker ps -a --filter name=fc_showing the container stuck inCreated. On Apple Silicon this makes a locally built stack unable to run any connector at all.The target is now
x86_64-unknown-linux-muslunconditionally, cross-compiling when the host differs.ringandaws-lc-syscompile C, and those objects must be compiled against musl headers: a glibc cross-gcc (x86_64-linux-gnu-gcc) leaves glibc-only symbols (__isoc23_sscanf,__fprintf_chk) that the musl link cannot resolve. Sozig ccis used as the cross C compiler — it ships headers and libc for every target — while rustc's bundled lld links against the target's musl. Two zig quirks are handled by generated wrappers:cc-rspasses an LLVM triple via--target=that zig rejects, andzig ccenables UBSan by default without linking its runtime.locate_bin()searchesPATH, somise/tasks/local/lib.shandbootstrap/ide-settingsnow include the x86_64 musl directory. Without that the runtime silently falls back toCONNECTOR_INIT_IMAGE.CONNECTOR_INIT_IMAGEbumped fromreactor:v0.6.10-62-g8b6aeec1cd3toreactor:v0.6.12-37-g4cb160dc000(the newest published tag). The old pin was 67 commits older than 4d66c8d, which addedcreated_atto task specs. Because that binary decodes and re-encodes every request between runtime and connector, a copy older than a protocol field drops it silently — connectors observed specs with nocreated_atat Validate, Apply and Open even though the build database andlive_specsboth carried it. The comment now says the pin must be bumped whenever a spec/protocol field is added.Workflow steps:
mise run build:connector-initnow produces an x86_64 binary on any host (~40s cross-compiling on an 8-CPU aarch64 VM). No workflow change beyond that;mise installpicks up zig.Documentation links affected:
None.
Notes for reviewers:
ELF 64-bit x86-64, the runtime resolves it from PATH, and the container mounts it (md5 of the mounted/flow-connector-initmatches the built binary). Connectors then receive the full spec — a materialization reportedcreatedAt="2026-08-01"at Validate, Apply and Open, where before all three saw an absent field.zigis added tomise.tomlpurely as a cross C compiler; on x86_64 hosts (CI) the build path is unchanged and zig is not invoked.gcc-x86-64-linux-gnu/g++-x86-64-linux-gnuare added toapt-packages-ci-extra. They are not used by the zig path and can be dropped if you prefer — they were the first approach and are harmless to keep for other cross needs.reactor:stabletag; CI tags images only withgit describe, and:stableexists solely forderive-python/derive-typescript. Note the newest master commits often have no image at all (docs-only commits skip the platform build), so the newest usable tag lagged master by two commits when I picked it.