Skip to content

PLAN.md

Latest commit

 

History

History
238 lines (201 loc) · 10.7 KB

PLAN.md

File metadata and controls

238 lines (201 loc) · 10.7 KB
Raw

Plan

This is what's planned for proof, along with what's done so far. It changes as we go, so it's a good idea to check it before starting on something big.

Done So Far

  • 2026-09-30: the first version, with the core checks (describe, unknown ops, sessions, eval, stdin and the wire format), server profiles and a daily compatibility matrix for nREPL, Babashka, ClojureCLR, Basilisp, jank and dialtone. The profile for repartee came a day later.
  • 2026-10-07: proof proxy for checking what clients send (#1).
  • 2026-10-08: proof serve for testing clients and -like for acting like a whole server (#2, #3), client profiles for CIDER, Calva, Conjure and vim-fireplace (#4, #6) and -record (#5).
  • 2026-10-09: nREPL 1.8.0 as the reference (#7).
  • 2026-10-10: client profiles for REPLy and neorepl (#8).

0.1.0

The first release, which came out on 2026-10-26. Server authors can run a fixed version of proof in their CI now (jank is already looking into this, see jank#1104), and it covers the rest of what every editor sends: interrupt, load-file, lookups and completions, plus requests that go wrong.

Rough Edges

  • Done: wire.error-terminal no longer fails Babashka for a message with nothing but changed-namespaces that arrives after eval-error. The clients it names (REPLy and rebel-readline) lose only output, values and need-input there, so that's what it looks for now.
  • Done: proof run babashka and proof proxy babashka work, just like proof serve -like babashka, as the profiles are embedded in the binary. People who install proof with go install don't have the profiles folder at all.
  • Done: proof run profiles/babashka.toml -only 'eval.*' works, as flags can come after the profile (and after the scenarios of proof serve).
  • Done: proof run -h says how to use proof run, rather than Go's bare "Usage of run:", and the same goes for proxy and serve.

Releases

  • Done: proof version takes the version from the build info, so a binary installed with go install ...@v0.1.0 knows its version.
  • Done: a CHANGELOG.
  • Done: binaries for Linux, macOS and Windows on every release, so the CI of a server doesn't need Go. The Release workflow builds them when a version gets tagged.
  • Done: a GitHub Action, so a server's workflow needs nothing but uses: nrepl/proof@v0.1.0 and the path to its profile.
  • Done: the CI section of Usage uses the release instead of @latest.

interrupt

  • Done: checks for interrupting a running eval, with and without interrupt-id (Conjure never sends one), the interrupted status, an idle session, an interrupt-id that doesn't match and the session working afterwards.
  • Done: the evals waiting behind an interrupted one, which nREPL 1.8.0 and Babashka drop (nrepl#503), and where an interrupted eval's done goes when the interrupt comes from another connection, which nREPL's main branch gets right now (nrepl#498). The exception of an interrupted eval, which nREPL sends after its done (nrepl#506), is wire.after-done's warning.
  • Done: proof serve interrupts like dialtone and repartee do, and has nREPL's bugs as scenarios (late-exception, drop-queued and interrupt-elsewhere), which -like clojure and -like babashka turn on.
  • Not done: an interrupt sent right after its eval, which can find nREPL's session idle (nrepl#504). That's a race, so a check can't count on hitting it.
  • Not needed: a step option for client profiles that sends a request without waiting for its reply. The interrupt requests of CIDER, Calva and vim-fireplace are plain enough for interrupt.running, and Conjure's for interrupt.no-id.

load-file

  • Done: checks for the value of load-file and for a load that fails, where CIDER needs eval-error, Calva ex and Conjure err, plus what CIDER, Calva and Conjure send to load a file. The spec and the servers don't agree on which field has the contents of the file (spec#3), but every server in the matrix takes file and passes them all.

Lookups and Completions

  • Done: checks for nREPL's completions and lookup and cider-nrepl's complete, info and eldoc, for servers that list them, sending CIDER's prefix and sym and the symbol of Calva, Conjure and vim-fireplace. The checks of completions and lookup skip on servers that have complete and info, as clients use those then. Babashka, Basilisp and ClojureCLR don't say no-info for a symbol that doesn't exist, Basilisp counts columns from 0 and ClojureCLR's file isn't a URI, which Calva needs to jump to a definition.
  • Done: proof serve has nREPL's completions and lookup.
  • Not done: scenarios for cider-nrepl's ops in proof serve, so -like could act like Babashka, Basilisp and ClojureCLR there too.
  • spec#9 (arglists-str and no-info) should be settled by then.

Requests That Go Wrong

  • Done: frames that aren't requests (the server has to keep taking connections), fields of the wrong type (nrepl#477), a client that goes away in the middle of an eval (babashka#2227) and an eval in one session holding up the others (jank#1022). A bencode string takes jank down, and after a client goes away in the middle of an eval, nREPL (1.8.0 and its main branch) never sends the done of the next eval in that session.
  • Not done: an eval whose reply bencode can't carry, e.g. a boolean (babashka#2201), as there's no telling which code makes a server send one.

Before the Release

  • CI is green and the binaries build for every platform.
  • nREPL passes every check with the fail severity, and the matrix makes sense for every server in it.
  • Every new check has a quirk in the fake server, proof serve passes the new checks and it has scenarios for the servers that do things differently.
  • Spec Changes has everything the new checks found.
  • The docs cover the new checks and the release.

0.2.0

The second release, which came out on 2026-11-03. The compatibility matrix has a page of its own now, with nbb and let-go in it, and there are client profiles for clojure-cli.repl and bb repl --connect.

The Matrix

  • Done: the Compatibility workflow publishes the matrix on GitHub Pages, with the version of each server, which comes from the new version snippet. proof matrix -html builds the same page out of any reports.
  • Done: nbb and let-go, and scenarios for what they do differently in proof serve. Neither says eval-error when the code throws or sends what it prints to stderr, nbb stops taking connections after a frame that isn't a dict, and let-go shares the current namespace between sessions and doesn't list interrupt in describe (even though it has it).

More Clients

  • Done: clojure-cli.repl, the new client of the Clojure team. It's meant for the server that comes with it, but it works with others too. As of 0.2.3 it still doesn't answer need-input, so code reading input hangs, and it never closes its session.
  • Done: bb repl --connect, which every server in the matrix gets right.

After 0.2.0

This is what's next for proof.

The Matrix

  • shadow-cljs.

More Clients

  • The client in nrepl.cmdline, which expects an exact list of statuses when interrupting an idle session.
  • vim-iced and rebel-readline.

proof serve in the Tests of Clients

  • CIDER's integration tests jack in to nREPL, Babashka, Basilisp, nbb and shadow-cljs. A test that connects to proof serve instead, with a few -like servers and byte-writes, would catch problems with other servers before users run into them.
  • neat, mezcaml and neorepl should do the same.

Upstream

  • The pull requests for the spec based on proof's findings, spec.nrepl.org#6-#9. The fixes for nrepl-beam, neat and mezcaml got merged on 2026-10-02, except for nrepl-beam#2, which was closed when the branch it was based on went away and has to be opened again.
  • Issues for the spec about the rest of Spec Changes, once the spec pull requests are settled.
  • nREPL: a release with nrepl#498, then nrepl#477 and nrepl#183, which has the same cause as #498. The next eval after a client goes away in the middle of one loses its done for the same reason as nrepl#509. Babashka has the same problem, which is why it fails robust.disconnect in a full run, but not on its own.
  • CIDER and Calva accepting ops as a list, after which describe.ops-dict can allow both formats.
  • Reported on 2026-10-29: basilisp#1356 (no no-info for unknown symbols), basilisp#1357 (columns from 0), jank#1169 (a request that throws stops its connection from answering), conjure#802 (no interrupt-id in interrupt) and cider#4224 (async eldoc ignores no-eldoc).
  • ClojureCLR doesn't say no-info for unknown symbols either, but clr.tools.nrepl takes bug reports on Ask Clojure rather than GitHub.
  • Calva waits for an eval of java.version in the main session before it interrupts, so it looks like it can't interrupt an eval there. That needs a run in VS Code before it gets reported.

Further Down the Road

  • Incorporating Spec Changes into the spec, so that the spec and proof agree and proof can eventually be versioned together with the spec.