Native systemd + supercronic deploy for VPS monitoring

[spalen0]

Closes #255. First PR in the VPS migration.

What this does

Adds automation/ — a single source of truth (jobs.yaml) plus a thin runner that ports every existing GH Actions cron schedule — and a native systemd deploy that runs it under supercronic on the VPS. No Docker, no compose, no Caddy: liquidity-monitoring already runs on the same box, so a container boundary buys nothing. The existing GH Actions workflows stay live for a parallel-run verification window; cutover is documented in deploy/cutover.md.

Runs alongside liquidity-monitoring's liqmon on the same host — distinct unit (yearn-monitor) and paths (/etc/yearn-monitoring, /srv/yearn-monitoring), so the two coexist.

Layout

• automation/ — flat package alongside aave/, morpho/, utils/, …
  • jobs.yaml — SSOT: 5 profiles, 32 tasks, one cron each
  • config.py / runner.py / __main__.py — schema+parser; per-task subprocess.run with continue-on-error + one Telegram digest on failure; CLI list / render-crontab / run <profile> [--dry-run]
• deploy/
  • systemd/yearn-monitor.service — one hardened unit. ExecStartPre requires /etc/yearn-monitoring/.env and renders the crontab from jobs.yaml; ExecStart runs supercronic. Restart=on-failure, ProtectSystem=strict, ReadWritePaths=/srv/cache. No /healthz watchdog — no daemon to wedge.
  • install.sh — idempotent provisioning: uv + Python 3.12 + supercronic (SHA-pinned), clone → /srv/yearn-monitoring, uv sync --frozen, create /srv/cache, install the unit.
  • runbook.md — ops (status, logs, manual runs, updates, failure table).
  • cutover.md — GH Actions → VPS migration playbook (shadow week + flip).
• tests/test_automation_{config,render,runner}.py — 21 tests.
• pyproject.toml — adds pyyaml, registers automation. uv.lock synced.

Secrets — plain root-owned .env

This box signs nothing (unlike liquidity's liqmon), so secrets are a plain /etc/yearn-monitoring/.env (0640 root:<deploy-user>) loaded via EnvironmentFile. The unit refuses to start without it. No sops/age ceremony.

Caching — one CACHE_DIR

All on-disk dedupe/cache state resolves against a single CACHE_DIR (set to /srv/cache by the unit; unset locally → repo CWD). utils/cache.py gains cache_path(); the selector cache, stuck-triggers JSON, and the maple/3jane caches (which previously hardcoded repo-relative paths and would have failed writing under the read-only hardened unit) all route through it. jobs.yaml no longer hardcodes /srv/cache paths.

Deployment steps (fresh VPS)

# 1. Provision (installs uv/Python/supercronic, clones repo, venv, systemd unit)
sudo bash /srv/yearn-monitoring/deploy/install.sh
#    …or curl it — see the header of deploy/install.sh.

# 2. Drop the env (copy from .env.example, fill in RPC/Telegram/API keys)
sudo install -m 640 -o root -g <deploy-user> /dev/stdin /etc/yearn-monitoring/.env   # paste, Ctrl-D

# 3. Start it
sudo systemctl enable --now yearn-monitor
systemctl status yearn-monitor

# 4. Verify
cd /srv/yearn-monitoring && uv run python -m automation render-crontab   # expect 5 lines
journalctl -u yearn-monitor -f
uv run python -m automation run hourly --dry-run                         # dry-run, no sends

Updates are git pull --ff-only && sudo systemctl restart yearn-monitor (add uv sync --frozen for dependency changes).

Cutover steps (GH Actions → VPS)

Full playbook in deploy/cutover.md. In short:

1. Shadow week — deploy with all TELEGRAM_CHAT_ID_* funneled to one shadow chat and all TELEGRAM_TOPIC_ID_* removed (works around telegram.py's per-protocol chat_id having no DEFAULT fallback). GH Actions keeps posting to the real channels; compare for ~7 days. This also warms /srv/cache, so no cold-start alert burst.
2. Flip — restore the real channels in the env + systemctl restart, confirm one good real tick, then disable the GH crons:

   gh workflow disable hourly.yml daily.yml weekly.yml multisig-checker.yml --repo yearn/monitoring

   (Go live before disabling GitHub: duplicate alerts are deduped, a gap is not.)
3. Rollback — gh workflow enable … + stop/quiet the unit. Both run in parallel again.
4. Cleanup — keep workflows disabled (not deleted) ≥30 days; later strip the now-dead actions/cache steps from _run-monitoring.yml.

Verification (local)

Check	Result
uv run python -m automation render-crontab	5 flock-wrapped lines
uv run --extra dev pytest	442 passed, 4 skipped
uv run --extra dev ruff check . / ruff format --check .	clean
uv lock --check	in sync
bash -n deploy/install.sh	clean

End-to-end systemd behavior is exercised during provisioning per deploy/runbook.md, not in CI.

Test plan

• CI green
• Reviewer: uv run python -m automation render-crontab matches the 5 expected lines
• Reviewer: skim deploy/systemd/yearn-monitor.service, deploy/runbook.md, deploy/cutover.md
• After merge: provision the VPS, run the shadow week, then flip per deploy/cutover.md

🤖 Generated with Claude Code