feat: esphome noise api encryption#8
Open
deviantintegral wants to merge 15 commits into
Open
Conversation
Ignore the local virtualenv and Python/test caches.
Phase 1 of plan 01 (ESPHome-compatible Noise API encryption). Adds bt_proxy/noise.py implementing the server side of ESPHome's Noise_NNpsk0_25519_ChaChaPoly_SHA256 transport: 0x01-indicator framing with 16-bit big-endian lengths, the hello exchange, the NNpsk0 handshake with the "NoiseAPIInit" prologue, transport encryption, and the inner 4-byte big-endian message header. Wire constants were verified against ESPHome's api_frame_helper_noise.cpp and aioesphomeapi's noise.py rather than recalled, since byte-exact interop with Home Assistant is the acceptance bar. Adds --encryption-key (with BT_PROXY_ENCRYPTION_KEY fallback) to __main__, validated as base64-encoded 32 bytes and fatal on error -- there is deliberately no fallback to plaintext on a bad key, since that would leave an operator who believes they are protected fully exposed. Advertises api_encryption over mDNS when keyed, and warns loudly at startup when not. The API server itself is not yet wired up; that is Phase 2. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgFr3P6634eCsTShr5Vbf3
…t by preamble Phase 2 of plan 01 -- the actual fix for the unauthenticated-access finding. The root cause was not merely the lack of encryption but that the message dispatch table was consulted for the very first frame on a socket, so an attacker could send a BluetoothDeviceRequest as their opening message and drive the BLE radio with no Hello or Connect. Verified: against the previous code that request reached BLEManager.connect_device; it no longer does. Adds an explicit connection State (HANDSHAKE -> CONNECTED -> AUTHENTICATED) and gates dispatch in exactly one place, driven by a per-message minimum-state requirement in the handler table. The table is default-safe: a handler with no declared state requires AUTHENTICATED, so a future handler is gated by default rather than exposed. Every BLE/GATT handler requires AUTHENTICATED. The first byte now selects the transport: 0x01 runs the Noise handshake when a key is configured (and is refused, not hung, when none is); 0x00 serves plaintext when no key is set (with a per-connection deprecation warning) and is refused when a key is set. The plaintext framing is a faithful extraction of the previous _read_message/_read_varint logic, so existing v1.0.2 clients are unaffected -- confirmed by an end-to-end Hello/DeviceInfo exchange. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgFr3P6634eCsTShr5Vbf3
…cument encryption Phase 3 of plan 01 -- verification and documentation. Adds tests/test_wire_and_gate.py (the gate's state-boundary arithmetic, the one case earlier suites did not pin) and tests/test_interop.py, which drives the genuine aioesphomeapi client -- the same library Home Assistant runs -- against the real server: a matching key completes the Noise handshake and returns device info; a wrong key is rejected with InvalidEncryptionKeyAPIError well inside a timeout rather than hanging; plaintext is refused when keyed; and the un-keyed plaintext path still round-trips Hello/DeviceInfo. Using the foreign client, not a hand-rolled one, is what makes the interop claim meaningful. Documents encryption in README (key generation, --encryption-key / BT_PROXY_ENCRYPTION_KEY with the env var preferred in production, the matching Home Assistant api: encryption: key: config, and the deprecation of unauthenticated mode), replacing the stale "not supported" line, and shows supplying the key via Environment=/EnvironmentFile= in the systemd unit. Full suite: 50 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgFr3P6634eCsTShr5Vbf3
Adds a Docker-free deployment for a Pi Zero / Zero W: an install.sh that builds a Python venv and installs bt-proxy using piwheels (prebuilt ARMv6 wheels, so nothing compiles), a systemd unit that runs the venv directly rather than via uv (uv has no ARMv6 build), and a concise deploy/README with the Raspberry Pi Imager -> SSH -> install -> Home Assistant adoption steps. The service reads an optional BT_PROXY_ENCRYPTION_KEY from /etc/bt-proxy/bt-proxy.env so the Noise key is kept out of the unit file and `ps` output. Not yet exercised on real hardware. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgFr3P6634eCsTShr5Vbf3
…ts for the key Home Assistant discovers that a device is encrypted by probing with plaintext first and reading back a reply whose indicator byte is 0x01; aioesphomeapi surfaces that as RequiresEncryptionAPIError, which is what advances HA to the "enter encryption key" step. Our keyed server was instead closing the plaintext connection silently, so the client saw a bare EOF (SocketClosedAPIError) and HA showed a generic connection error and never prompted for the key. Match ESPHome: when a key is configured and a client probes with plaintext (0x00), reply with a 0x01-indicator frame before closing. Verified against the real aioesphomeapi client: the plaintext probe now raises RequiresEncryptionAPIError while a correct-key client still completes the handshake and returns device info. Updates the two tests that asserted the old silent-close behavior and adds a real-client RequiresEncryptionAPIError interop test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgFr3P6634eCsTShr5Vbf3
…e flusher In Noise mode a dense advertisement batch can exceed the 65535-byte frame ceiling, so encode_inner raises ValueError inside write_message. The _adv_flush_loop only caught ConnectionResetError/OSError, so that ValueError killed the flush task and the client silently stopped receiving ALL advertisements until it reconnected -- the proxy's core function, dead, with no error. Plaintext has no cap and was unaffected. _flush_advertisements now splits a batch that a transport rejects and sends it as several frames, dropping no advertisements; the plaintext path still sends a single frame. Adds a unit test that a batch too large for one frame is split across frames with every advertisement delivered. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgFr3P6634eCsTShr5Vbf3
… session read_message() fed a zero-length frame (a peer sending 01 00 00) straight to decrypt(), which raised InvalidTag and tore down an otherwise healthy authenticated connection. A real Noise transport message always carries at least the 16-byte AEAD tag, so an empty frame is never valid post-handshake; skip it and read the next frame. A genuinely corrupt non-empty frame still fails to decrypt and closes the connection, unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgFr3P6634eCsTShr5Vbf3
…ng silently The state gate refused any message that arrived before the connection reached AUTHENTICATED by logging a warning and returning without a reply. A client waiting on that reply (e.g. a login-less or non-standard client that subscribes without sending ConnectRequest) then blocked until its own timeout -- the proxy "appeared dead" -- and the silent per-message refusal was also an unbounded log-flood lever for an unauthenticated peer. Now a pre-auth message closes the connection: the client gets a definitive EOF instead of a hang, and the log is capped at one line per connection. The standard Home Assistant flow sends ConnectRequest first and never trips this; the exploit regression test (BLE request as the first message) still refuses to drive the radio. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgFr3P6634eCsTShr5Vbf3
…lit arg string systemd (and the shell) split an expanded variable on whitespace regardless of quotes, so the documented BT_PROXY_ARGS example `--friendly-name "Living Room Proxy"` reached argparse as three stray tokens, failed to parse, and put the unit into a Restart=on-failure crash loop. Name and friendly name (the values that can contain spaces) now fall back to BT_PROXY_NAME / BT_PROXY_FRIENDLY_NAME environment variables, which carry a multi-word value intact through the service EnvironmentFile. The parser is extracted into build_parser() for testing; explicit CLI flags still win over the environment. The deploy unit drops the unquoted $BT_PROXY_ARGS expansion, and install.sh's env-file template uses the env vars. Adds tests that a spaced friendly name survives and that a CLI flag overrides the env. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgFr3P6634eCsTShr5Vbf3
The root bt-proxy.service had diverged from deploy/bt-proxy.service in ways that could silently start the proxy UNENCRYPTED: its EnvironmentFile comment pointed at /etc/bt-proxy.env while the installer and deploy unit use /etc/bt-proxy/bt-proxy.env, so a key placed per the root unit's instructions was never loaded. It also left the EnvironmentFile commented out entirely. Both units now read /etc/bt-proxy/bt-proxy.env (active, optional via the leading "-"), and the root unit documents the same BT_PROXY_NAME/BT_PROXY_FRIENDLY_NAME env-var config as the deploy unit. Keeps `uv run` for the generic path but states plainly that ARMv6/Pi Zero must use deploy/install.sh (uv has no ARMv6 build); README's service section now points ARMv6 users there. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgFr3P6634eCsTShr5Vbf3
deviantintegral
commented
Jul 12, 2026
| @@ -0,0 +1,317 @@ | |||
| """ESPHome Noise-encrypted Native API transport (server side). | |||
Author
There was a problem hiding this comment.
I really don't have the expertise to validate the low-level implementation here. I'm treating most of this as "if I can connect from Home Assistant, it's probably fine".
| # | ||
| # Designed for a headless Raspberry Pi Zero / Zero W (ARMv6): it uses a plain | ||
| # Python venv and relies on piwheels (the default index on Raspberry Pi OS) for | ||
| # prebuilt ARM wheels, so nothing is compiled from source. No Docker, no uv. |
Author
There was a problem hiding this comment.
This isn't true as it looked like zeroconf needed to be compiled. It took a good 20 minutes to install.
…le mpris-proxy Investigated a Pi Zero W pegged at ~100% CPU running the proxy. Root cause: Home Assistant requests PASSIVE BLE scanning, but bluetoothd was not started with --experimental, so the proxy silently fell back to ACTIVE scanning -- ~5x the advertisements (110 vs 20/s) and ~2x the CPU (79% vs 39%), with the shared WiFi/BT antenna's coexistence also pegging NetworkManager. Measured on the device: enabling --experimental (so passive actually works) plus disabling mpris-proxy took idle from ~0% to ~40%. install.sh now installs a bluetooth.service --experimental drop-in and disables mpris-proxy (a Bluetooth media bridge, useless on a headless proxy that was woken by every advertisement). deploy/README documents how to confirm passive scanning and the realistic hardware ceiling. Adds tools/measure_adverts.py (advert rate + duplicate ratio) and tools/scan_cpu_test.py (active vs passive CPU), the diagnostics used to find it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgFr3P6634eCsTShr5Vbf3
…t sources - Deprecation wording: 'opt-in in 2.0' (drop 'be removed'), in the startup warning and README, per review. - noise.py: link the ESPHome/aioesphomeapi sources the wire constants come from. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgFr3P6634eCsTShr5Vbf3
- Note Trixie (Debian 13) as supported. - Correct the 'nothing compiles' claim: some deps (zeroconf) build from source and can take tens of minutes on a Pi Zero; say so in install.sh and README. - Clone URL -> denvera/bt-proxy. - rfkill unblock + bluetoothctl power on after enabling bluetooth. - Drop the stale 'push to GitHub first' note from the deploy README. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgFr3P6634eCsTShr5Vbf3
Behaviour-preserving quality pass over the branch; all 57 tests still pass. - api_server: collapse the 3-layer handler-table machinery (_HandlerEntry / _RAW_MESSAGE_HANDLERS / _normalize_handler / comprehension) into one explicit (handler, State) dict; make State an IntEnum so the gate reads `state < req` without `.value`; route drain()/close() through self._transport so the frame helper actually owns the socket (its _closed guard was dead); catch the new FrameTooLargeError instead of a bare ValueError when splitting adv batches. - noise: add FrameTooLargeError(ValueError) so the size cap is a named contract; use int.to_bytes/from_bytes instead of hand-rolled shift/mask for the uint16 framing; derive API_ENCRYPTION_NAME from NOISE_PROTOCOL_NAME (one source). - __main__: drop main()'s redundant None branch; warn_if_unencrypted self-guards. - bt-proxy.service: sync the [Unit] ordering that had drifted from deploy/. - tests: hoist the copy-pasted fixtures (make_ble/start_server/read helpers/PSK) into tests/conftest.py. Skipped (noted): tests keep an independent plaintext decoder rather than reusing PlaintextFrameHelper (wire tests should not share the code under test); the frame-helper's async byte-at-a-time varint is not proto.decode_varint (different I/O model); the per-frame buffer-copy micro-opts (~10 frames/sec, not worth it). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgFr3P6634eCsTShr5Vbf3
deviantintegral
commented
Jul 12, 2026
Comment on lines
+47
to
+49
| #: Value advertised in the mDNS ``api_encryption`` TXT record (one source of | ||
| #: truth with the bytes form above). | ||
| API_ENCRYPTION_NAME = NOISE_PROTOCOL_NAME.decode() |
Author
There was a problem hiding this comment.
My setup is firewalled in such a way I can't easily test mdns.
deviantintegral
marked this pull request as ready for review
July 12, 2026 16:49
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds support for encryption and authentication. By default, it doesn't change the behaviour for existing users. However, it does mark no authentication by default as deprecated, with the idea being the defaults flip for a future 2.0.0 release.
I've been running this on an old Pi Zero W and it's completely fixed marginal signal issues I've had with a bluetooth water timer. It uses around 60% of the CPU, so it certainly makes this a single-task use of the Pi. But, at least it lets me reuse it!
It would be possible to split out the Pi Zero commits into a separate PR, but there's some overlap in the edits so I've left it together for now. Let me know if you would rather them split, with a second one dependent on this PR.
Thanks!