Skip to content

Fix build system on Windows and test it deliberately - #1389

Merged
salkinium merged 12 commits into
modm-io:developfrom
salkinium:feature/windows_compat
Oct 13, 2026
Merged

salkinium merged 12 commits into
modm-io:developfrom
salkinium:feature/windows_compat

Conversation

@salkinium

@salkinium salkinium commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Our Windows CI compiled whatever examples happened to be in its list, so Windows-only bugs in the build system showed up at random whenever the list changed, most recently in #1118. This fixes the ones I found and makes the Windows job compile examples picked for what can actually break there. Turns out it's always one of the same three things, meh.

Requires lbuild v1.23.0 (modm-io/lbuild#88), which returns all relative paths with forward slashes on every host.

  • SCons does not preprocess .S files on Windows, since the file system is case-insensitive: the LVGL Helium assembly (Cortex-M55 only) is not copied anymore. RTT was already fixed in Add board connections, hosted configuration and fixes for deduplicated examples #1387.
  • Command lines longer than the 8191 characters of cmd.exe: the archive and link commands now use a response file. LVGL alone is about 60000 characters of object files, lol. SCons only writes the file if the command is too long.
  • Backslash paths in generated files turning into escape sequences, like generated\nanopb, generated\xpcc and generated\tests: fixed at the source in lbuild instead of in modm, so the minimum lbuild version is now v1.23.0. This removes the modm.windowsify and modm.posixify filters and the escape_level completely, since having to remember them for every single path is what caused these bugs in the first place.
  • modm/processing.hpp counted the native path separator to find its headers, which would have left it empty on Windows with the new lbuild, whoops.
  • scons compilation_db passed all sources through cmd.exe and failed silently with exit code 0, whoops. The builder is now called without a shell.
  • Hosted CMake targets do not create .bin and .hex files anymore, objcopy -Oihex fails on a 64-bit Windows executable.
  • The XPCC generator finds its DTD for file:/C:/... paths.
  • The generated .gitignore used backslashes, which ignores nothing.
  • Git info timestamps fall back to 0 instead of "Unknown" without Git, which did not compile as uint32_t on any OS.
  • examples_compile.py also compiles CMake examples on Windows using the "MinGW Makefiles" generator. Make stays excluded, it's documented as unsupported.
  • The Windows job compiles an explicit list of examples per failure class: assembly sources, generated paths, long command lines, CMake and CompilationDB, hosted. FreeRTOS and TinyUSB are in there too, but do not reach the limit.
  • Tested locally with the CI toolchains and the Windows Python in Wine: all hosted, STM32, AVR and NRF examples of the new list passed with the conversions still in modm. After moving them into lbuild, the nanopb, XPCC, CMake, CompilationDB, image, fiber and hosted examples pass with v1.23.0 and the generated SCons, CMake and OpenOCD files contain no backslashes, while v1.22.0 is rejected.
  • The docs test job moved from the container of 2022 to the current one, whose patched DoxyPress finishes again. The old one only has Python 3.8, which cannot install lbuild v1.23.
  • CI passed, including the new example list and CMake on the real Windows runner.

Missing: The modm.posixify filter is gone, in case any external repository used it. The Wine container has no Git, so the linux/git example and the hosted unit tests did not run locally and rely on the CI here. The link command limit has no test, since no example has enough source files: I only forced it by lowering MAXLINELENGTH. OpenOCD with forward slashes in its config is reasoned from its startup.tcl, not run.

Conflicts with #1118, which moves all examples and has its own copy of the first two commits. I'll rebase whichever gets merged second.

@salkinium
salkinium force-pushed the feature/windows_compat branch 3 times, most recently from 1bfc853 to 96c4a6d Compare October 12, 2026 21:47
@salkinium
salkinium marked this pull request as ready for review October 13, 2026 07:56
@salkinium
salkinium force-pushed the feature/windows_compat branch 2 times, most recently from 8720316 to 21c983e Compare October 13, 2026 09:19
lbuild v1.23 returns relative paths with forward slashes on all hosts,
so counting the native separator found no header on Windows and
modm/processing.hpp stayed empty.
It returns all relative paths with forward slashes, also on Windows.
Archiving many object files exceeds the command line length on Windows.
SCons only writes the file when the command is too long. It uses forward
slashes, since ar reads a backslash in a response file as an escape.
It is only for Cortex-M55 and SCons does not preprocess .S files on Windows,
so its include guard has no effect there.
An application with about a hundred source files exceeds the 8191
characters that cmd.exe accepts when linking with SCons on Windows.

The linker reads a backslash in a response file as an escape character.
The target and sources are written with forward slashes, but SCons has
no such form for the library paths, libraries and link flags, which
contain absolute Windows paths. Their backslashes are escaped instead,
only for the link command of the program.
Paths were written with backslashes on Windows and every use in a
template had to opt into escaping them with the modm.windowsify filter.
Wherever that was forgotten, a path like generated\nanopb or
generated\xpcc turned into an escape sequence or a syntax error.

lbuild now returns all relative paths with forward slashes, which all
tools accept on Windows as well. This makes the modm.windowsify and
modm.posixify filters and their escape levels unnecessary.
All application sources are passed as arguments, which exceeds the 8191
characters that cmd.exe accepts for about a hundred files, and SCons did
not even report that as an error. The interpreter is also only called
python3 on Windows if it was installed by the GitHub setup-python action.
A 64-bit Windows executable is loaded at an address that does not fit
into an Intel Hex file, so objcopy failed every hosted CMake build.
lxml passes the DTD of an XML file with an absolute Windows path as a
file URL, which was not recognized as an existing file, so the DTD was
searched for in the wrong folder. Together with the forward slashes in
the SConstruct this lets the XPCC examples compile on Windows again.
The CMake generator was never tested on Windows. The commands were
written for a POSIX shell, they now only use CMake itself. MinGW
Makefiles are generated on Windows, since they run their commands with
cmd.exe and need nothing but the make that comes with MinGW.
The STM32 examples were whole board folders, so Windows bugs in the
build system were only found if an affected example happened to be in
one of them. The list now names examples with preprocessed assembly,
generated paths, many object files, CMake and CompilationDB.
The Nanopb generator needs protobuf and grpcio-tools.
Outside of a Git repository or without Git installed the two timestamps
fell back to the string "Unknown", which was generated as a char pointer
and conflicted with their uint32_t declaration, so info_git.c did not
compile.
@salkinium
salkinium force-pushed the feature/windows_compat branch from 21c983e to 7d99e46 Compare October 13, 2026 09:39
@salkinium salkinium added this to the 2026q4 milestone Oct 13, 2026
@salkinium
salkinium merged commit 7d99e46 into modm-io:develop Oct 13, 2026
78 checks passed
@salkinium
salkinium deleted the feature/windows_compat branch October 13, 2026 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

1 participant