Skip to content

About

Model the system. Prove its properties. Assure its operation.

Resources

Contributing

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

System Assurance Model

Model the system. Prove its properties. Assure its operation.

System: Not limited to infrastructure. It can cover distributed services, hardware, networks and operational processes.

Assurance: Correctness, resilience, safety and reliability.

Model: An explicit, executable representation using Prolog and TLA+.

Overview

System Assurance Model is an executable, continuously reconciled model of infrastructure and distributed systems. It combines formal methods, logic programming, infrastructure automation, hardware inventory and runtime observations to reason about system correctness, resilience, operational safety and change impact.

The project aims to connect what a system is supposed to be, what infrastructure changes will produce, and what is actually running. It makes assumptions, dependencies, constraints and verification results explicit and traceable.

Objectives

  • Model system structure: Represent physical and virtual hardware, networks, hosts, clusters, services, storage and dependencies.
  • Reason about system properties: Evaluate topology, capacity, redundancy, failure domains, placement constraints and operational dependencies.
  • Verify system behaviour: Formally specify safety and liveness properties, including failure scenarios, failover, replication and recovery.
  • Observe actual state: Collect hardware inventory, health indicators and runtime information from Linux, FreeBSD and infrastructure services.
  • Detect inconsistencies: Compare declared, planned and observed states to identify drift, missing information and violated constraints.
  • Assess changes: Analyse proposed infrastructure changes before deployment and identify their potential impact.
  • Support safe operations: Use verified constraints and current observations to inform maintenance, placement and recovery decisions.

Core Technologies

Technology Responsibility
TLA+ / TLC Formal specification and model checking of system behaviour, safety and liveness properties. TLC comes from the vendored ext/tla2tools.jar and needs a Java runtime
GNU Prolog Implementation basis: knowledge representation, dependency reasoning, policy evaluation, constraint analysis, and the CLI, collectors and adapters, compiled to native binaries with gplc
OpenTofu / Terraform Infrastructure as Code and planned-state extraction
Kubernetes Workload and cluster state integration
OpenSSH Restricted remote observation of Linux and FreeBSD hosts
Prometheus / Grafana Optional telemetry integration and operational visualization

These technologies have distinct responsibilities. The project does not attempt to replace existing infrastructure management, monitoring or orchestration systems.

Architecture

The model distinguishes three representations of the system:

  1. Desired state: What configuration and policies declare should exist.
  2. Planned state: What infrastructure tools intend to create or modify.
  3. Observed state: What collectors and runtime interfaces report actually exists.

These representations are reconciled into a canonical, versioned data model.

Each observation should retain its provenance, timestamp, source and quality. Facts are classified as declared, observed, derived, assumed or unknown. Missing information must not silently be interpreted as a healthy state or as proof that something does not exist.

Engine and System Repositories

This repository is the generic assurance engine. It contains no production facts, hostnames or credentials. Each modeled system lives in its own repository (named sam-<system>), which holds that system's infrastructure declarations, Prolog facts and policies, TLA+ compositions, validation profiles and selected sanitized observations.

  • A versioned system manifest (system.json) identifies the system, the engine version it needs and its entry points. The engine validates it (sam validate system, make validate-system) before any reasoning or verification starts.
  • A system repository pins the engine as a git submodule at engine/. The engine never references system repositories.
  • The same engine version can validate any number of systems without modification, and each system has independent versioning and access control.
  • Every validation result records the engine and system revisions it used.

See docs/architecture/repositories.md for ownership boundaries and the system repository layout. SBC/sam-test1 and SBC/sam-test2 are the test system repositories; SBC/sam-acceptance_tests validates both with one engine.

Main Components

  • Canonical model: Versioned schemas for infrastructure entities, relationships, hardware, telemetry and operational state.
  • Collectors: Platform-specific inventory and telemetry collection for Linux, FreeBSD and supported infrastructure services.
  • Observer: A restricted, read-only remote observation protocol using a finite set of audited operations.
  • Prolog knowledge base: Rules and constraints for topology, dependencies, capacity, health, redundancy and change impact.
  • TLA+ specifications: Behavioural models for distributed systems, failure handling, maintenance and operational protocols.
  • Adapters: Integrations with Infrastructure as Code, Kubernetes, databases, monitoring systems and hardware tools.
  • Validation pipeline: Automated checks that produce reproducible reports and preserve model-checker counterexamples.

Formal Assurance

TLA+ is used to model behaviour rather than simply describe configuration. Initial areas of interest include:

  • etcd quorum and membership changes
  • PostgreSQL replication, promotion and split-brain prevention
  • Redis availability and failover
  • Kafka replication and durability
  • Kubernetes scheduling, node failures and upgrades
  • Hardware maintenance and safe evacuation
  • Restricted observation protocols and their permitted state transitions

Prolog complements this by reasoning about the current system model, dependencies, constraints and operational decisions.

Formal verification results are conditional on the model and its assumptions. They do not automatically prove the correctness of external software, hardware firmware or unmodelled implementation details.

Hardware Intelligence

Hardware inventory and telemetry are first-class parts of the model.

The collection layer is intended to support:

  • CPU, memory, NUMA and PCI device inventory
  • Storage devices, controllers, capacity and firmware
  • SMART health and temperature observations
  • ECC memory error information where available
  • Network interfaces and error counters
  • ZFS pool, vdev and device health
  • Operating system and hardware identification

The initial implementation targets Linux and FreeBSD. Hardware observations should remain independent of Prometheus; monitoring integrations are optional consumers rather than mandatory dependencies.

Remote collection is designed around restricted SSH access, fixed command allowlists and least privilege. TLA+ can verify the specified observer protocol and its permitted transitions, while implementation testing and security auditing establish confidence in the actual collector.

Development Approach

Development will proceed incrementally:

  1. Define the canonical data model and stable entity identifiers.
  2. Implement inventory collection for Linux and FreeBSD.
  3. Introduce the Prolog knowledge base and topology reasoning.
  4. Add desired, planned and observed state reconciliation.
  5. Specify and verify selected distributed-system behaviours in TLA+.
  6. Implement restricted remote observation and hardware health assessment.
  7. Integrate change-impact analysis into an automated validation pipeline.
  8. Add optional monitoring and visualization integrations.

The initial focus is on producing a reliable model and trustworthy validation results, not on automatically controlling production infrastructure.

Design Principles

  • Explicit models and machine-checkable specifications
  • Least privilege and read-only observation by default
  • Separation of facts, assumptions, derived conclusions and unknowns
  • Traceable provenance for every relevant observation
  • Reproducible validation and model-checking results
  • Small, composable formal specifications
  • Platform-specific collection with normalized common representations
  • No unnecessary dependency on a particular monitoring or orchestration platform
  • Human approval for consequential infrastructure changes

Project Status

This project is under active design and initial implementation. Components, interfaces and supported integrations will evolve as the canonical model and validation architecture mature.

The repository structure, build and test workflow are documented in docs/development.md.

About

Model the system. Prove its properties. Assure its operation.

Resources

Contributing

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages