Skip to content

feat!: replace the Python CSS implementation with a Rust NIF - #38

Merged
zachdaniel merged 11 commits into
ash-project:mainfrom
shahryarjb:rust-rewrite-biome
Aug 4, 2026
Merged

feat!: replace the Python CSS implementation with a Rust NIF#38
zachdaniel merged 11 commits into
ash-project:mainfrom
shahryarjb:rust-rewrite-biome

Conversation

@shahryarjb

@shahryarjb shahryarjb commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Replaces the Python/tinycss2 implementation with a precompiled Rust NIF built on Biome's lossless CSS CST, and deletes priv/python, plibs/, rebuild_wheel.sh and the pythonx dependency — no Python, no Node, no external process. The load-bearing decision is that the tree is never reprinted: operations parse losslessly, locate byte ranges, and splice text into the original source, so comments, indentation and property order outside an edit are preserved by construction rather than by effort. That yields four properties, each asserted for every operation against every fixture rather than spot-checked — comments are never lost, diffs contain only the lines the codemod meant to change, every operation is idempotent and reports changed: false on a re-run, and a file that cannot be patched safely is refused rather than half-edited. Selector matching is deliberately strict (top-level only, normalised comparison, never substring or fuzzy) and an ambiguous selector is an error rather than a guess, which is how the previous implementation produced surprising diffs.

Related: ash-project/ash#2834

Examples

Inserting an at-rule: it lands after the at-rule prologue rather than at the top of the file, and the inline comment on .btn survives the value change.

css = """
@import "tailwindcss";
@source "../js";

/* ===== Layout ===== */

.btn {
  color: red; /* brand */
  margin: 0;
}
"""

{:ok, a} = IgniterCss.ensure_at_rule(css, ~s|@plugin "daisyui";|)
{:ok, b} = IgniterCss.set_declaration(a.source, ".btn", "color", "var(--brand)")
@import "tailwindcss";
@source "../js";
@plugin "daisyui";

/* ===== Layout ===== */

.btn {
  color: var(--brand); /* brand */
  margin: 0;
}

Re-running is a no-op, which is what makes it safe inside an installer:

{:ok, again} = IgniterCss.ensure_at_rule(b.source, ~s|@plugin "daisyui";|)
again.changed
#=> false

Removing a declaration takes the comments it owns and leaves the section header:

{:ok, c} = IgniterCss.remove_declaration(b.source, ".btn", "color")
@plugin "daisyui";

/* ===== Layout ===== */

.btn {
  margin: 0;
}

It refuses rather than guessing:

IgniterCss.set_declaration(".a{}\n.a{}", ".a", "color", "red")
#=> {:error, "selector \".a\" matches 2 top-level rules; refusing to guess which one to patch"}

IgniterCss.ensure_rule(".broken {\n  color: red;\n", ".x")
#=> {:error, "cannot safely patch this file: braces are unbalanced; refusing to patch"}

Inside an installer:

igniter
|> IgniterCss.Codemods.ensure_at_rule(path, ~s|@plugin "daisyui";|)
|> IgniterCss.Codemods.ensure_rule(path, ".hide-scrollbar")
|> IgniterCss.Codemods.set_declaration(path, ".hide-scrollbar", "scrollbar-width", "none")

Verified

rust tests:        339   (unit, corpus-invariant, property/fuzz)
elixir tests:      268 + 39 doctests
clippy warnings:   0
cargo fmt:         ok
mix format:        ok
credo --strict:    no issues
dialyzer:          Total errors: 0, Skipped: 0

Phase 0 gate, which everything else rests on — parse.syntax().to_string() == source byte-for-byte across the whole fixture corpus, and it stays in CI:

round_trip_is_byte_identical_with_default_options ... ok
round_trip_is_byte_identical_in_strict_mode ... ok
round_trip_is_byte_identical_with_css_modules_enabled ... ok
round_trip_holds_even_when_the_parse_has_errors ... ok
tailwind_v4_at_rules_parse_without_diagnostics ... ok

The corpus is real-world shaped on purpose: a Phoenix app.css, Tailwind v4 (@theme, @plugin with and without a block, @source, @custom-variant, @variant, @utility, @apply, @reference — all parse with zero diagnostics), comments in awkward places, CRLF, a BOM, no trailing newline, minified vendor CSS, non-ASCII content, and files that are simply broken.

Patching a fresh Phoenix app.css end to end — two at-rules plus a new rule — touches 6 lines and no others, with every original line still present verbatim.

cargo test, cargo fmt and cargo clippy pass on the shared ash-ci workflow on both matrix legs:

success  ash-ci (1.17.0-otp-26, 26.0, false) / cargo test, fmt and clippy
success  ash-ci (default, default, true)     / cargo test, fmt and clippy

Notes

Two behaviour changes worth flagging. IgniterCss.CSS.CssProcessor is removed — its pipeline mixed codemods with whole-file rewriting, which this design keeps separate: IgniterCss patches, IgniterCss.Transform (minify/beautify/merge) rewrites and must not be pointed at a file a user maintains. And IgniterCss.Parsers.Parser keeps its function names and {:ok, :function_name, result} shape, but its mutating functions are now diff-minimal instead of reprinting the stylesheet.

Releasing depends on ash-project/ash#PENDING — ash-ci's build-release job currently hardcodes igniter_js as the crate to build, so tagged releases here would build a crate this repository does not contain. Branch and PR checks are unaffected.

shahryarjb and others added 8 commits August 3, 2026 16:24
Rewrites igniter_css on top of Biome's lossless CSS CST, and deletes the
Python/tinycss2 implementation entirely: `priv/python`, `plibs/`,
`rebuild_wheel.sh` and the `pythonx` dependency are all gone. No Python, no
Node, no external process -- a single precompiled native library, matching the
igniter_js layout.

The load-bearing decision is that the tree is never reprinted. Operations parse
losslessly, locate byte ranges, and splice text into the original source, so
comments, indentation and property order outside an edit are preserved by
construction rather than by effort. That gives four properties, each asserted
for every operation against every fixture rather than spot-checked:

  1. comments are never lost
  2. diffs contain only the lines the codemod meant to change
  3. every operation is idempotent and reports changed: false on a re-run
  4. a file that cannot be patched safely is refused, never half-edited

Phase 0 gate: parse.syntax().to_string() == source holds byte-for-byte across
the whole fixture corpus, including Tailwind v4 (zero diagnostics), CRLF, BOM,
minified vendor CSS, non-ASCII content and deliberately broken files. It stays
in CI.

Notable findings handled along the way:

  - Biome lexes a leading U+FEFF into the first identifier, which silently
    breaks selector matching on BOM'd files. ParseCtx now strips the BOM before
    parsing and re-attaches it on output.
  - An unbalanced-brace file makes "top level" meaningless -- an insertion lands
    inside somebody's unterminated block -- so mutating ops refuse it outright.
    Analysis still works on those files.

API:

  - IgniterCss           -- the codemods, {:ok, %Outcome{}} | {:error, reason}
  - IgniterCss.Codemods  -- Igniter-facing wrappers (diff preview, confirmation)
  - IgniterCss.Transform -- whole-file minify/beautify/merge, held deliberately
                            apart from the codemods
  - IgniterCss.Parsers.Parser -- the previous surface, same function names and
                            {:ok, :fun, result} shape, now diff-minimal

Selector matching is strict by design: top-level only, normalised comparison,
never substring or fuzzy, and more than one match is an error rather than a
guess. Comment ownership on delete follows the documented convention -- trailing
and adjacent own-line comments go with the node; blank-line-separated comments
and section headers stay.

Tests: 333 Rust (unit, corpus-invariant, property/fuzz over generated and
malformed input) and 238 Elixir (unit, doctests, corpus invariants, Igniter
integration). clippy, cargo fmt, mix format, credo --strict and dialyzer are all
clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…mment

Found by mutation-testing the comment guarantees: the corpus sweep's
`set_declaration_existing` op targets `.page`/`color`, but no fixture rule has
that property, so the op always took the *append* branch. The *update* branch --
the one rule E governs, where only the value bytes may be replaced -- was never
exercised against a declaration carrying a trailing comment anywhere in the
sweep.

Adds an op that sets `.page`/`display`, which in comments_everywhere.css is
`display: flex; /* trailing on a declaration */`, so the sweep now asserts
comment survival across that path on every fixture.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…vely

Import de-duplication was deciding equivalence by scanning the prelude *text*
for a quoted string or `url(...)`. That worked, but it was the one correctness-
critical path in the codebase still reading characters instead of tokens, and it
could in principle be fooled by a quote appearing somewhere unexpected.

`AtRuleRef` now carries a `target` read from the CST: the first
`CSS_STRING_LITERAL` or `CSS_URL_VALUE_RAW_LITERAL` token appearing before any
block. Equivalence and removal both compare that. Text scanning survives in
exactly one place -- `normalize_target_needle`, which unquotes the caller's
`matching` argument -- because that argument is a bare path from Elixir, not
CSS. It is documented as such.

Consequences, all now asserted:

  - `@import "a.css"`, `'a.css'`, `url("a.css")`, `url(a.css)`, and the same
    with `screen` / `layer(base)` / `supports(...)` trailing, are one import
  - a quote inside a comment is not a target
  - a string inside a block is not a target, but one before the block is, so
    `@plugin "p" { ... }` dedupes against `@plugin "p";`
  - `@layer base, components;` has no target and falls back to prelude
    comparison

Adds test/imports_test.exs (38 tests): the full quoting/wrapping matrix in both
directions, near-miss targets that must stay distinct, ordering against
@charset / prologue / style rules, media queries and modifiers, url quoting
rules, CRLF/BOM/no-trailing-newline shape preservation, removal and Rule D, the
other at-rule families, and an installer-shaped end-to-end run asserting one
copy of each target after two passes.

Rust: 335 tests. Elixir: 39 doctests + 237 tests. clippy, fmt, credo clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Answers "can it change any line, in any class, id or tag?" with a table rather
than an assurance. 22 selector shapes -- class, id, tag, universal, attribute
(quoted and bare), pseudo-class, pseudo-element, functional pseudo, :not(),
:where(), :root, descendant, child, adjacent and general sibling, selector list,
tag.class, compound chain, escaped `\/`, non-ASCII, double class -- each run
through the full lifecycle: update a value, append a declaration, query, remove
a declaration, remove the rule.

The same table exists in native/igniter_css/tests/selectors.rs and
test/selectors_test.exs, so the behaviour is proven directly and through the
NIF boundary rather than assumed to survive marshalling.

Each shape additionally asserts:

  - an equivalent spelling matches and can drive a real edit
    (`.a>.b` == `.a > .b`, `nav    ul   li` == `nav ul li`)
  - a near miss never matches (`.btn-primary` for `.btn`, `.a .b` for `.a > .b`,
    `:nth-child(2n)` for `:nth-child(2n+1)`) and never removes anything
  - comments in every awkward position survive, and an update touches 2 lines
  - all three mutating ops are idempotent

Property shapes get the same treatment: standard, hyphenated, custom property,
vendor prefixed, shorthand, function value, nested functions, a `url()`
containing a semicolon, a non-ASCII string value, grid-template, and the
`!important` lifecycle (set, preserved on update, cleared on request).

Scoping is pinned down too: a rule inside `@media` is unreachable at the top
level, the same selector at two scopes only edits the top-level one, and two
top-level rules with one selector is an error rather than a guess.

Rust: 340 tests. Elixir: 39 doctests + 268 tests. clippy, fmt, credo clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ash-ci's `build-release` job cannot ship this package. It hardcodes igniter_js:

    project-name: igniter_js
    project-dir: "native/igniter_js"
    workspaces:  native/igniter_js

and the `rust-crate-dir` input that would fix it is only consumed by
`rust-check`. Worse, `build-release` is gated on the tag rather than on
`inputs.release`, so pushing `v0.2.0` would have made ash-ci try to build a
crate this repository does not contain — a failed release, not a skipped job.

So elixir.yml no longer triggers on tags; it covers branches and PRs, where
ash-ci is exactly right. It now also passes `rust-crate-dir: native/igniter_css`
so the `rust-check` job (cargo fmt --check, clippy -D warnings, cargo test)
targets our crate instead of discovering every Cargo.toml in the tree.

release.yml handles `v*` tags: it re-runs the full check suite against the tag,
then runs the same ten-target precompiled matrix ash-ci uses — same actions,
same pinned SHAs, same NIF version — pointed at native/igniter_css. It then
generates checksum-Elixir.IgniterCss.Native.exs from the attached artifacts,
publishes to Hex, and cuts the GitHub release from the CHANGELOG.

The release matrix and `targets:` in lib/igniter_css/native.ex are verified in
sync: a target built but not declared is never downloaded, and one declared but
not built is a hard failure for those users. Both lists are the same ten.

The proper fix is upstream: give ash-ci's build-release `rust-project-name` and
`rust-project-dir` inputs, at which point this workflow can go away.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reverts the local release.yml now that ash-ci takes the crate name and directory
as inputs. igniter_css is back to the igniter_js shape: one workflow, tags
included in the triggers, and ash-ci owns build-release, hex publish and the
GitHub release.

`release: false` is gone, so the input falls back to its default of true and a
v* tag builds the ten-target matrix, attaches the artifacts, generates the
checksum file and publishes -- the same path igniter_js takes.

Requires the matching ash change: passing rust-project-name / rust-project-dir
to an ash-ci that does not declare them fails every run, not just tagged ones.
Merge that first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ash-ci resolves the precompiled crate from the calling repository's name, so
igniter_css gets `igniter_css` and `native/igniter_css` without configuring
anything. Passing them explicitly only duplicated the convention.

This also removes the ordering hazard from the previous commit: the workflow no
longer passes inputs that current ash-ci@main does not declare, so it is safe on
either version. Releases still need the ash change merged to build the right
crate, but CI will not fail in the meantime.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two findings from the first real CI run.

REUSE compliance failed on native/igniter_css/tests/probe.rs -- a scratch file
used to inspect the CST shape of `@import` variants. It was meant to be deleted;
the `rm` sat behind a `cd` that had already succeeded, so `&&` short-circuited
and it was committed instead. It carried no SPDX header, which is what caught
it. Removing it drops the Rust suite from 340 to 339 tests: the probe was
counted as one.

The run before that never started at all. ash-ci's jobs request
security-events: write for the credo SARIF upload, pages and id-token for docs,
and contents: write for releases; a reusable workflow cannot request more than
its caller grants, so on any repository whose default workflow token is
read-only the run fails at startup. igniter_js already declares these. Adding
the same block.

Verified against the real workflow: `cargo test, fmt and clippy` passes on both
matrix legs. The remaining job failures in that run are a runner-side TLS error
reaching builds.hex.pm, unrelated to this repository.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
shahryarjb and others added 3 commits August 3, 2026 20:23
The `Could not mix rebar from any hex.pm mirror` failures are not flaky. OTP
27.1.3 cannot complete the TLS handshake to builds.hex.pm -- it rejects the
certificate chain with `key_usage_mismatch` -- and `.tool-versions` pinned
exactly that, so `default` resolved to a toolchain that cannot install rebar.
Compared on the same runner the same day, igniter_js on OTP 28.0.2 compiles and
tests cleanly while igniter_css on OTP 27.1.3 fails every job that needs hex.

Moves `.tool-versions` to erlang 28.0.2 / elixir 1.18.4-otp-28, matching
igniter_js, and drops the 1.17.0-otp-26 matrix leg, which fails the same way and
harder. igniter_css now calls ash-ci as a single job, as igniter_js does.

Tradeoff worth stating: mix.exs still declares `elixir: "~> 1.17"`, and that
floor is no longer exercised in CI. Testing it would need an Elixir 1.17 build
on OTP 28, which does not exist -- 1.17 tops out at OTP 27. igniter_js makes the
same tradeoff, declaring `~> 1.14` while testing only the default toolchain.

`publish-docs` is dropped rather than passed, since its default is already true.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review of the Rust for hand-rolled parsing that Biome already models. Four
places were reading characters where the tree had the answer.

Selector normalisation was a character scanner tracking quote state and bracket
depth. The CST models combinators as tokens -- `>` `+` `~`, and a
CSS_SPACE_LITERAL for descendant -- and selector lists as element children, so
canonical spacing now falls out of a structural walk. Quoted attribute values
and `:not(...)` arguments are copied token by token and need no tracking at all.
A caller's selector string is parsed and rendered through the same function, so
a hand-written selector and one read from a stylesheet cannot disagree. This
fixed a real bug: `:nth-child(2n + 1)` and `:nth-child(2n+1)` are one selector,
and the text scanner kept them distinct, so one could not match the other.

Colour detection scanned for 148 names in raw text and needed a `strip_opaque_runs`
hack so `url(/red.png)` was not read as `red`. Biome distinguishes all three
cases already: a hex colour is CSS_COLOR, `rgb(...)` is CSS_FUNCTION with an
identifier name, a bare `red` is CSS_IDENTIFIER, and a url payload is
CSS_URL_VALUE_RAW -- structurally not an identifier. Matching node kinds deletes
the hack outright. The name list stays because it is spec data, but it is now
compared against identifier tokens rather than substrings. Hex literals are
still length-checked, since Biome parses `#notahex` as a colour node too.

At-rule preludes were compared by collapsing whitespace in raw text. AtRuleRef
now carries `prelude_norm`, built by joining the tokens between the name and the
`;`/`{`; trivia is excluded by construction, so a comment inside a prelude
cannot change the comparison. `collapse_ws` is gone.

`split_declarations` hand-parsed caller text with bracket counting. It now wraps
the text in a throwaway rule and reads the declarations back off the CST, so a
`;` inside `url(...)`, a string or a comment does not split -- because the
parser knows what those are, not because of counting.

Also removes the last two `expect()` calls reachable from a NIF. Both were
provably unreachable, but the rule is that nothing reachable from a NIF may
unwind, and proving it per call site is worse than not needing to.

README rewritten: what it does, how to use it, how to contribute and test.
No migration history.

339 Rust tests, 268 Elixir + 39 doctests, clippy -D warnings clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two problems, both mine.

A `cat > README.md` ran while the shell was inside native/igniter_css and
overwrote the NIF readme with the root one. That dragged the root's personal
copyright line onto a file that never carried it. The NIF readme is restored,
covering the architecture, build, test and dependency-pinning rules for the
crate.

Attribution is now uniform. Every tracked file carries exactly one line:

    SPDX-FileCopyrightText: 2025 igniter_css contributors
      <https://github.com/ash-project/igniter_css/graphs/contributors>

That normalises three inconsistencies at once: the personal line, the
`graphs.contributors` spelling that predated this branch in the .license
sidecars, and the duplicate lines in the two readmes. 24 files updated; the
comment prefix of each file type is preserved.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@shahryarjb
shahryarjb requested a review from zachdaniel August 3, 2026 19:12
@shahryarjb shahryarjb self-assigned this Aug 3, 2026
@shahryarjb
shahryarjb marked this pull request as ready for review August 3, 2026 19:12
@zachdaniel
zachdaniel merged commit 35b4f0a into ash-project:main Aug 4, 2026
26 checks passed
@zachdaniel

Copy link
Copy Markdown
Contributor

🚀 Thank you for your contribution! 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants