Address the review in #9: improper log scales, a new-scheme tutorial, docs

CHANGELOG.md

@@ -6,6 +6,25 @@ All notable changes to MessagePassingRulesBase.jl are documented here. The forma
 
 ## [Unreleased]
 
+### Added
+
+- `logscale = improper`, a declaration for a message rule whose message has no normalising
+  constant, as an exact message may: its log scale is an `UndefinedLogScale` whose cause is
+  `:improper`, and `require_logscale` says that none exists, where an omitted `logscale` says that
+  the log scale is not known (#9).
+- A tutorial, *A new message passing scheme*: natural-gradient message passing for a Poisson count
+  with a log rate, as an algorithm, a dependency declaration and a rule (#9).
+- A Makefile: `make test` (as CI runs it, with `test_args` to select items), `test-fast`, `docs`,
+  `docs-serve`, `format`, `check-format`, `clean` (#9).
+
+### Changed
+
+- Documentation, from the review in #9: what a pure rule may and may not do; what the registry is
+  for, with an example; *resolution* defined in the glossary and linked; improper messages on the
+  *Log scales* page; a shorter first example on the overview; the expectation propagation entry
+  pointing to the tutorial that builds such a rule; and an example in *Algorithms and
+  dependencies* told through its own node.
+
 ## [1.0.0]
 
 The first release: the rule system of the ReactiveMP ecosystem, developed in the

Makefile

@@ -0,0 +1,40 @@
+SHELL := /bin/bash
+JULIA ?= julia
+.DEFAULT_GOAL := help
+
+# The variables CI sets: the items tagged `:slow` run, and FastCholesky throws on a matrix that is
+# not symmetric instead of warning, so a local run fails where CI would.
+CI_ENV := TEST_ALL=true JULIA_FASTCHOLESKY_THROW_ERROR_NON_SYMMETRIC=1
+
+# Optional selection, passed to the suite: `make test test_args="tag:alloc name:routing"`.
+test_args ?=
+TEST_ARGS := $(if $(test_args),test_args = split("$(test_args)") .|> string,)
+
+.PHONY: help test test-fast format check-format docs docs-serve clean
+
+help: ## Show this help
+	@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) \
+	  | awk 'BEGIN {FS = ":.*?## "}; {printf "  \033[36m%-14s\033[0m %s\n", $$1, $$2}'
+
+test: ## Run the suite as CI does; select with test_args="tag:<t> name:<n> <path>"
+	$(CI_ENV) $(JULIA) --project=. -e 'import Pkg; Pkg.test($(TEST_ARGS))'
+
+test-fast: ## Run the suite without the items tagged :slow
+	$(JULIA) --project=. -e 'import Pkg; Pkg.test($(TEST_ARGS))'
+
+format: ## Format the tree with Runic
+	$(JULIA) -e 'import Pkg; Pkg.activate(temp = true); Pkg.add("Runic"); using Runic; exit(Runic.main(["--inplace", "."]))'
+
+check-format: ## Check the formatting with Runic, without changing files
+	$(JULIA) -e 'import Pkg; Pkg.activate(temp = true); Pkg.add("Runic"); using Runic; exit(Runic.main(["--check", "--diff", "."]))'
+
+docs: ## Build the documentation into docs/build, running every example and doctest
+	$(JULIA) --project=docs -e 'import Pkg; Pkg.instantiate()'
+	$(JULIA) --project=docs docs/make.jl
+
+docs-serve: ## Build and serve the documentation with live reload
+	$(JULIA) --project=docs -e 'import Pkg; Pkg.instantiate(); using LiveServer; servedocs()'
+
+clean: ## Remove the documentation build and coverage files
+	rm -rf docs/build lcov.info
+	find . \( -name '*.jl.cov' -o -name '*.jl.*.cov' -o -name '*.jl.mem' \) -delete

README.md

@@ -35,12 +35,11 @@ result = @call_message_update_rule(node = Shift, target = :out, m = (in = 1.0,))
 getresult(result), getlogscale(result)   # (2.0, 0)
 ```
 
-- Documentation: <https://reactivebayes.github.io/MessagePassingRulesBase.jl/stable/>. Build it
-  locally with `julia --project=docs -e 'import Pkg; Pkg.instantiate()'` and then
-  `julia --project=docs docs/make.jl`, into `docs/build`.
-- Tests: `julia --project -e 'import Pkg; Pkg.test()'`; with `TEST_ALL=true` in the environment
-  they include the items tagged `:slow`, and `Pkg.test(test_args = ["tag:alloc"])` selects by tag,
-  `name:<text>` by name and a path by file.
+- Documentation: <https://reactivebayes.github.io/MessagePassingRulesBase.jl/stable/>. `make docs`
+  builds it locally, into `docs/build`.
+- Tests: `make test` runs the suite as CI does; `make test test_args="tag:alloc name:routing"`
+  selects items by tag, by name or by file. `make help` lists the other targets: `docs`,
+  `docs-serve`, `format`, `check-format`, `test-fast`.
 - Depends on BayesBase, MacroTools, Compat, FastCholesky, IrrationalConstants and LinearAlgebra:
   no distribution package and no engine.
   Julia 1.10 or later. MIT licence.

docs/Project.toml

@@ -4,6 +4,7 @@ Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
 Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
 DocumenterInterLinks = "d12716ef-a0f6-4df4-a9f1-a5a34e75c656"
 ExponentialFamily = "62312e5e-252a-4322-ace9-a5f4bf9b357b"
+LiveServer = "16fef848-5104-11e9-1b77-fb7a48bbb589"
 MessagePassingRulesBase = "4a5b64c7-30c9-4471-82f2-bbe8464b157d"
 
 [sources]

docs/make.jl

@@ -19,6 +19,7 @@ makedocs(
             "Your first node" => "tutorials/first-node.md",
             "A deterministic node with a group" => "tutorials/groups.md",
             "A node with its own algorithm" => "tutorials/algorithm.md",
+            "A new message passing scheme" => "tutorials/new-scheme.md",
         ],
         "Defining nodes" => "nodes.md",
         "Defining rules" => "rules.md",

docs/src/algorithms.md

@@ -56,7 +56,7 @@ MessagePassingRulesBase.default_algorithm
 ## Extending the default
 
 A subtype of [`DefaultAlgorithmExtension`](@ref) overrides some rules, or some dependencies, of
-the default, and inherits the rest. Resolution looks for the extension's own rule first. When
+the default, and inherits the rest. [Resolution](@ref glossary-resolution) looks for the extension's own rule first. When
 there is none, it falls back to the default's rule. That rule then runs with
 `DefaultAlgorithm()` in its `algo` slot, the algorithm it was written for.
 
@@ -122,8 +122,25 @@ A node may name a parametric algorithm as its own, `algorithm = T`. A rule that
 
 ## Purity
 
-A rule is pure unless it is declared otherwise. A pure rule mutates neither its inputs nor any
-state shared beyond one call. It draws randomness only from `ctx.rng`, which the caller owns.
+A rule is pure unless it is declared otherwise. Pure means that a call has no effect anyone
+outside it can observe, apart from its result: running the rule leaves nothing changed that its
+caller did not hand it to change. It does not mean that the rule writes nothing.
+
+A pure rule may
+
+- allocate whatever intermediate arrays it needs;
+- write its output into the buffer an [in-place rule](@ref glossary-in-place-rule) is given,
+  since that buffer is handed over to hold the result;
+- reuse its own [scratch](@ref glossary-scratch), working memory that nothing else reads;
+- draw random numbers from `ctx.rng`, a generator its caller owns and passes in;
+- warn or log, for a degenerate input say, which changes no state another computation reads.
+
+A rule is impure if it
+
+- mutates an input, a message or a marginal it reads;
+- mutates its algorithm, a cache kept in one of its fields for instance;
+- writes a global variable, a file or any other state that later code reads;
+- carries its own random number generator in its algorithm.
 
 An algorithm that carries state of its own is impure, and it says so with a method of
 [`ispure`](@ref). A rule overrides its algorithm's purity with `pure = false` or `pure = true`.
@@ -199,9 +216,10 @@ MessagePassingRulesBase.free_energy_partition
 
 A rule may need one input that the factorisation does not give it, while its other inputs
 follow the factorisation as usual. `default` among a target's inputs stands for the inputs of the
-default scheme, and the inputs beside it are added to them. ContinuousTransition's rule towards
-`a`, for example, reads `q(a)`, the expansion point of its transformation, under both of its
-factorisations. The node below declares the same:
+default scheme, and the inputs beside it are added to them. Take a transition `y ~ f(x; a)` whose
+function is linearised around the mean of `a`. Its rule towards `a` needs `q(a)`, the expansion
+point, under every factorisation, including one where the default scheme gives it the message
+on `a` instead. The node below declares that:
 
 ```@example algorithms-extend
 using MessagePassingRulesBase

docs/src/calling.md

@@ -134,7 +134,7 @@ MessagePassingRulesBase.RuleInputError
 
 Code that builds its own [`RuleArgs`](@ref) resolves a rule with the `find_*` functions. They
 never throw: they return a [`RuleNotFound`](@ref) when nothing fits. The code then runs the rule
-with the `message_passing_*` functions, which do throw. Julia's dispatch does the resolution,
+with the `message_passing_*` functions, which do throw. Julia's dispatch does the [resolution](@ref glossary-resolution),
 over every loaded package.
 
 ```@docs

docs/src/fallbacks.md

@@ -9,7 +9,7 @@ reporting the error. A rule fallback is a callable, `fallback(node, target, args
 the node, the target and the rule's arguments. It returns a [message](@ref glossary-message), or
 `nothing` when it has none either.
 
-An engine consults the fallback only when resolution returns a [`RuleNotFound`](@ref). A fallback
+An engine consults the fallback only when [resolution](@ref glossary-resolution) returns a [`RuleNotFound`](@ref). A fallback
 therefore never replaces a rule that exists, and an error inside a rule is never turned into a
 fallback. ReactiveMP takes a fallback as its activation option `rulefallback`. A message that a
 fallback computes has an undefined [log scale](@ref glossary-log-scale).

docs/src/glossary.md

@@ -64,7 +64,9 @@ are always its output and the joint over its inputs. See [`Deterministic`](@ref)
 ### [Expectation propagation](@id glossary-expectation-propagation)
 
 A message passing scheme that approximates each message by projecting the corresponding
-marginal onto a simpler family, usually by matching moments. The Probit node uses it.
+marginal onto a simpler family, usually by matching moments.
+[A node with its own algorithm](@ref tutorial-algorithm) builds a node whose rule towards its
+input is an expectation propagation rule.
 
 ### [Factor graph](@id glossary-factor-graph)
 
@@ -110,7 +112,8 @@ convention. An interface may have aliases, other names a model can use for it.
 
 The logarithm of a message's normalising constant: a message is
 ``\exp(\text{log scale}) \cdot p(x)`` with ``p`` a normalised distribution. Summed over a graph,
-log scales give the model's evidence. See [Log scales](@ref).
+log scales give the model's evidence. Only a message with a finite integral has one: an improper
+message, which an exact message can be, has none. See [Log scales](@ref).
 
 ### [Marginal](@id glossary-marginal)
 
@@ -137,6 +140,13 @@ enters a rule. `PointMass(2.0)` comes from BayesBase.
 The distribution of `f(x)` when `x` has a known distribution. A deterministic node's message
 towards its output is the pushforward of the messages on its inputs.
 
+### [Resolution](@id glossary-resolution)
+
+Finding the rule that runs for a call: the node, the target, the
+[algorithm](@ref glossary-algorithm) and the types of the inputs select one rule. It is Julia's
+method dispatch over the methods the definition macros generate, so it needs no list of rules.
+When no rule matches, the result is a [`RuleNotFound`](@ref). See [Calling rules](@ref).
+
 ### [Rule](@id glossary-rule)
 
 A function that computes a message, a joint marginal or an average energy of a node from its

docs/src/index.md

@@ -20,7 +20,7 @@ You use the package for three things:
 
 A rule is an ordinary Julia function of its inputs. Julia dispatches it on the node, on the
 target and on the types of the incoming [messages](@ref glossary-message) and
-[marginals](@ref glossary-marginal). Because resolution is Julia's own dispatch, a rule defined
+[marginals](@ref glossary-marginal). Because [resolution](@ref glossary-resolution) is Julia's own dispatch, a rule defined
 in any loaded package is found.
 
 The package depends on BayesBase and on small numerical packages (FastCholesky,
@@ -38,55 +38,38 @@ MessagePassingRulesBase
 
 ## A first node and rule
 
-The node below is a [deterministic node](@ref glossary-deterministic-node), `out = in + c`, for
-a known shift `c`. It has one rule, for the message towards `out`. The messages are a small
-normal type that the example defines, a mean and a variance. A real rule package uses
-ExponentialFamily's distributions instead.
+A node, `out = in + 1`, with one rule, for the message towards `out`, called as an engine would
+call it:
 
 ```jldoctest overview
 julia> using MessagePassingRulesBase
 
-julia> struct Gauss   # a normal, by its mean and variance
-           m::Float64
-           v::Float64
-       end
+julia> struct Shift end   # out = in + 1
 
-julia> struct Shift end
-
-julia> @define_factor_node(node = Shift, type = Deterministic, interfaces = [:out, :in, :c])
+julia> @define_factor_node(node = Shift, type = Deterministic, interfaces = [:out, :in])
 
 julia> @define_message_update_rule(
-           node = Shift, target = :out,
-           args = (m[:in]::Gauss, m[:c]::Real),
-           logscale = 0,
-           body = (args) -> Gauss(args.m[:in].m + args.m[:c], args.m[:in].v),
+           node = Shift, target = :out, args = (m[:in]::Real,), logscale = 0,
+           body = (args) -> args.m[:in] + 1,
        )
 
-julia> result = @call_message_update_rule(node = Shift, target = :out, m = (in = Gauss(1.0, 2.0), c = 3.0));
-
-julia> getresult(result)
-Gauss(4.0, 2.0)
+julia> result = @call_message_update_rule(node = Shift, target = :out, m = (in = 1.0,));
 
-julia> getlogscale(result)
-0
+julia> getresult(result), getlogscale(result)
+(2.0, 0)
 ```
 
-The node's declaration names its [interfaces](@ref glossary-interface). The rule names its
-node, its target, the inputs it takes with their types, and its body. The call runs the rule as
-an engine would. It returns a [`RuleResult`](@ref): the message, its
-[log scale](@ref glossary-log-scale) and everything that produced them. An engine such as
-ReactiveMP builds the node in a graph from the same declaration. It runs the same rule whenever
-the rule's inputs change.
-
-[Your first node](@ref tutorial-first-node) builds a complete node step by step, with
-ExponentialFamily's distributions.
+[Your first node](@ref tutorial-first-node) builds a real node step by step, and explains each
+part.
 
 ## The site
 
 - **Tutorials**: [Your first node](@ref tutorial-first-node) declares a normal node and writes
   its belief propagation and variational rules. [A deterministic node with a group](@ref tutorial-groups)
   writes rules for a sum of any number of inputs. [A node with its own algorithm](@ref tutorial-algorithm)
   gives a node a parametrised algorithm and declares what its rules take.
+  [A new message passing scheme](@ref tutorial-new-scheme) writes natural-gradient message
+  passing as an algorithm and its rules.
 - [Defining nodes](@ref): [`@define_factor_node`](@ref), what a declaration records, and the
   queries that read it.
 - [Defining rules](@ref): the three rule macros, targets, the inputs a rule receives, the slots

docs/src/inspecting.md

@@ -101,7 +101,7 @@ A rule package's tests check its rules in two ways:
 
 - against their nodes' declarations, with [`check_rules`](@ref);
 - against each other, with [`check_rule_ambiguities`](@ref). Two rules that some call matches
-  equally well make resolution throw a `MethodError`.
+  equally well make [resolution](@ref glossary-resolution) throw a `MethodError`.
 
 Each check takes the modules to check, every loaded module by default, and returns the problems
 it finds. Both lists are empty for this page's module:
@@ -118,11 +118,53 @@ MessagePassingRulesBase.check_rule_ambiguities
 MessagePassingRulesBase.duplicate_rules
 ```
 
-## The registries
+## [What the registry is for](@id inspecting-registry)
 
-Each module that defines nodes or rules keeps a registry of what it defined. The registry is
-filled when the module loads. It serves introspection only: listings, coverage, checks, and the
-near misses a [`RuleNotFoundError`](@ref) reports. Resolution never reads it.
+Each module that defines nodes or rules keeps a [`Registry`](@ref) of what it defined: its
+[`RuleSpec`](@ref)s, [`NodeSpec`](@ref)s and dependency declarations. The definition macros create
+it in the module, as the constant `__message_passing_registry__`, and fill it as the module loads.
+Each module keeps its own so that a package's precompiled image carries its own entries;
+[`registries`](@ref) gathers them from every loaded module.
+
+[Resolution](@ref glossary-resolution) does not use the registry. Julia's dispatch finds the rule
+for a call, through the methods the macros define. Dispatch cannot say which rules exist,
+though, and that is what the registry is for: everything that lists, counts or checks rules reads
+it. Take a module that defines a node and two rules:
+
+```@example registry
+using MessagePassingRulesBase
+
+module Coins
+using MessagePassingRulesBase
+struct Coin end   # out ~ Bernoulli(p)
+@define_factor_node(node = Coin, type = Stochastic, interfaces = [:out, :p])
+@define_message_update_rule(node = Coin, target = :out, args = (m[:p]::Real,), body = (args) -> args.m[:p])
+@define_message_update_rule(node = Coin, target = :p, args = (m[:out]::Real,), body = (args) -> args.m[:out])
+end
+
+length(MessagePassingRulesBase.registered_rules(Coins))
+```
+
+The registry lists the module's two rules. [`list_rules`](@ref) and [`rule_coverage`](@ref) read
+it, and so do [`check_rules`](@ref) and a test suite's rule-coverage gate:
+
+```@example registry
+MessagePassingRulesBase.rule_coverage(Coins.Coin)
+```
+
+A call that no rule takes fails with a [`RuleNotFoundError`](@ref), and its near misses, the
+rules that come closest, come from the registry too:
+
+```@example registry
+try
+    @call_message_update_rule(node = Coins.Coin, target = :out, m = (p = "half",))
+catch err
+    showerror(stdout, err)
+end
+```
+
+A call that matches never consults it. A rule a registry missed would still run; it just would
+not be listed, counted, checked or suggested.
 
 ```@docs
 MessagePassingRulesBase.Registry

docs/src/internals.md

@@ -38,4 +38,5 @@ fragments have no docstrings of their own.
 MessagePassingRulesBase.default_inputs_match
 MessagePassingRulesBase.WithLogScale
 MessagePassingRulesBase.FromBody
+MessagePassingRulesBase.Improper
 ```

docs/src/keywords.md

@@ -534,11 +534,12 @@ struct Gain end   # out = a ⋅ in
 `logscale` declares the message's [log scale](@ref glossary-log-scale): the scalar with
 `message = exp(logscale) · result`, for the normalised `result` the body returns. The message
 towards `in` is ``\mathcal{N}(a x \mid m, v) = |a|^{-1}\, \mathcal{N}(x \mid m/a, v/a^2)``, so its
-log scale is ``-\log|a|``, a function of the inputs. The keyword takes one of four forms:
+log scale is ``-\log|a|``, a function of the inputs. The keyword takes one of five forms:
 
 - a number, `logscale = 0`, when the constant does not depend on the inputs;
 - a function of the inputs over the slots `(algo, ctx, args)`, named in that order, as above;
 - `from_body`, when the body returns [`with_logscale`](@ref)`(result, logscale)`;
+- [`improper`](@ref), when the message has no normalising constant, so no log scale exists;
 - nothing: a rule that omits the keyword gives an [`UndefinedLogScale`](@ref) naming it.
 
 ```@example messages