Library rewrite

.gitignore

@@ -26,3 +26,5 @@ share/python-wheels/
 .installed.cfg
 *.egg
 MANIFEST
+
+frames/

.ruff.toml

@@ -0,0 +1,41 @@
+# The contents of this file is based on https://github.com/home-assistant/core/blob/dev/pyproject.toml
+
+target-version = "py313"
+
+[lint]
+select = [
+    "ALL",
+]
+
+ignore = [
+    "ANN401", # Dynamically typed expressions (typing.Any) are disallowed
+    "ANN001",
+    "ANN201",
+    "ANN202",
+    "D203", # no-blank-line-before-class (incompatible with formatter)
+    "D212", # multi-line-summary-first-line (incompatible with formatter)
+    "COM812", # incompatible with formatter
+    "ISC001", # incompatible with formatter
+    "E501", # Line length is not important
+    "EM101",  
+    "TRY003", # Allow exception messages to be used directly
+    "ERA001", # Commented out code is allowed for examples
+    "T201", # Prints are allowed in examples
+    "PLR0913",
+    "FBT001", # Boolean positional args are part of the existing public API
+    "FBT002",
+    "N999",
+    "CPY001", # disable copyright check
+    "D104", # disable file docstring
+    "D100", # disable file docstring
+    "S311"
+]
+
+[lint.flake8-pytest-style]
+fixture-parentheses = false
+
+[lint.pyupgrade]
+keep-runtime-typing = true
+
+[lint.mccabe]
+max-complexity = 25

DOC.md

@@ -0,0 +1,121 @@
+# Reverse-Engineered Protocol Notes
+
+This document explains the technical details of the protocol used by the cameras, as discovered through the decompilation of SDKs found on the internet ([here](https://github.com/jameshilliard/android-p2p-sdk3.0) and [here](https://github.com/jameshilliard/HKiPhoneSDKDemo20160621)) and the analysis of network packets using Wireshark. It may therefore contain approximations resulting from reverse engineering, whilst the manufacturer has undoubtedly defined its protocol more precisely.  
+All the data presented here is implemented in the files `p2pcam/lan_scanner.py` and `p2pcam/lan_video.py`.
+
+## Discovery Process
+
+Camera discovery uses a UDP broadcast exchange on port `2627`.
+
+The scanner emits a discovery packet with this exact layout:
+
+```text
+[0-1]   00 00
+[2-3]   struct.pack("<H", (len(inner) + 4) << 4)
+[4-12]  inner packet header
+[13+]   ASCII dictionary body
+```
+
+The refresh body is built by `_encode_old_dict()` from these fields:
+
+```text
+TIME=3600;
+endTime=<unix_time_plus_3600>;
+MainCmd=LocalData;
+userType=hkclient;
+status=1;
+Prot=<listen_port>;
+MacIP=<random_marker>;
+```
+
+The code sends that packet to `255.255.255.255` and to each interface broadcast address derived from the host IPv4 interfaces. The exact packet template is:
+
+```text
+00 00 <len_lo> <len_hi>
+<command<<4 | 0x02> 0C 1D <inner_len_lo> <inner_len_hi> 00 00 00 00>
+TIME=3600;endTime=...;MainCmd=LocalData;userType=hkclient;status=1;Prot=<listen_port>;MacIP=<random_marker>;
+```
+
+Responses are collected on the source port and, optionally, a separate listen port. The decoder accepts two packet families:
+
+1. framed replies that start with `00 00` and store the packet length in the shifted outer-length field
+2. inner discovery packets where `data[0] >> 4 == COMMAND_LAN_REFRESH` and the dictionary body begins at byte 9
+
+Parsed fields are normalized into `LanDevice` objects by reading aliases for the same concept. For example, `HKID`, `hkid`, `DevID`, and `DSTHKID` are treated as the device identifier family, while `Prot`, `UDPPort`, `Port`, and `port` are treated as the network port family.
+
+The scanner ignores its own discovery echo by checking the `MacIP` field against the locally generated marker. It also recognizes 13-byte ACK packets using `_decode_ack()` and turns them into a simple online/status marker:
+
+```text
+00 00 d0 00 <cmd_lo> <flag> 20 09 00 <pipe_lo> <pipe_hi> 00 00
+```
+
+The result of discovery is a list of devices sorted by device ID and HKID.
+
+## Streaming
+
+Video streaming uses UDP port `5000` and follows a strict handshake/state machine before MJPEG data starts flowing.
+
+Every framed packet in this phase uses `_build_packet()`:
+
+```text
+[0-1]   packet counter (uint16 LE)
+[2-3]   outer length = total_packet_len << 4
+[4]     inner_cmd
+[5]     inner_flag1
+[6]     inner_flag2
+[7-8]   inner payload length
+[9-12]  inner_extra (4 bytes)
+[13+]   payload body
+```
+
+The streaming sequence is exactly:
+
+1. send the connection ping packets several times until the camera acknowledges
+2. send `HK_RES_REQ` to request the video session
+3. poll with `ICMD2` until the camera answers with `SessionCreate`
+4. send `SessionStart`
+5. receive MJPEG chunks, ACK camera `ICMD1` polls, and periodically send continue packets
+6. stop cleanly with `SessionDelete`
+
+The exact packet builders are:
+
+```text
+Ping 1:
+00 00 d0 00 82 0c 00 09 00 d1 07 00 00
+
+Ping 2:
+00 00 d0 00 a2 0c 40 09 00 d1 07 00 00
+
+HK_RES_REQ body:
+id=<hkid>;ftN0=video.vbVideo.MPEG4;ftN1=net.0;ftN2=HKPCPresent.HKPCPresent;opN2=<sid>;Callid=<callid>;sidN=<sid>;AsCode=337;MainCmd=HK_RES_REQ;user=Lan user;
+
+ICMD2 poll body:
+d4:ICMD2:293:SEQ1:<hkid>:GUARDSEQ1:<seq>
+
+SessionStart body:
+MainCmd=SessionStart;sidN=<sid>;ftN0=HKPCPresent.HKPCPresent;FD0=4;ftN1=net.1024;FD1=1024;
+
+ICMD1 ACK body:
+d4:ICMD1:293:lastreq1:<hkid>:SEQ3:<seq>e
+
+SessionDelete body:
+sidN=<sid>;MainCmd=SessionDelete;coz=;
+```
+
+Those bodies are encoded exactly as follows:
+
+- dictionary-style packets keep the first 2 bytes in plain ASCII and XOR the remaining bytes with `0xE9`
+- `ICMD`-style packets XOR every byte with `0xE9`
+
+The `HK_RES_REQ`, `SessionStart`, `ICMD1` ACK, `ICMD2` poll, and `SessionDelete` builders all follow those rules. The code uses a fixed session identifier and call identifier because those values are hard-coded defaults in the implementation.
+
+After `SessionCreate`, the camera begins sending MJPEG fragments. The assembler takes `data[4:]` as the payload, looks for `FF D8` to start a frame, and looks for `FF D9` to close it. If `_in_frame` is already true, the payload is appended before searching for EOI.
+
+Two additional keepalive mechanisms are required while streaming:
+
+- the client replies to camera `ICMD1` polls with `_build_icmd1_ack(hkid, seq, session_id)`
+- the client sends `cont_state.next_packet()` every 5 received fragments
+
+The continue packet builder is intentionally stateful because the camera expects the exact `_ContinueState` sequence: `nb_digits`, `idx`, `base_index`, and `_fragment_index` are mutated across calls, bytes `[2]` and `[7]` are rewritten for the current digit width, and the packet ends with `_CONT_END`.
+
+When the stream ends, the client sends `SessionDelete` so the camera releases the session and the next connection attempt starts from a clean state.

README.md

@@ -1,57 +1,71 @@
+<p align="center">
+  <img src="github/logo.png" width="100" />
+</p>
+
 # P2PCam
-Class to retrieve camera images from cameras using the p2p protocol
+
+Classes to retrieve camera images from cameras using the p2p protocol
 
 First of all i just wrote it to work as a class, the original connection and retrieval process has been made by [Jheyman](https://github.com/jheyman/) in his [videosurveillance script](https://github.com/jheyman/videosurveillance/).
 I rewrote it to run as a class instead of an application.
 
-So i had this [chinese camera](https://nl.aliexpress.com/item/Phone-monitor-P2P-Free-DDNS-Ontop-RT8633-HD-1-4-CMOS-1-0MP-Network-IP-Camera/990524792.html) laying around, it had this feature that you could access it from outside your home without the need for port forwarding. However after a couple of years this brand dissappeared and with it their services so i couldn't connect to it outside of my own network using [this app](https://play.google.com/store/apps/details?id=x.p2p.cam).
+So i had this [chinese camera](https://nl.aliexpress.com/item/Phone-monitor-P2P-Free-DDNS-Ontop-RT8633-HD-1-4-CMOS-1-0MP-Network-IP-Camera/990524792.html) laying around, it had this feature that you could access it from outside your home without the need for port forwarding. However after a couple of years this brand dissappeared and with it their services so i couldn't connect to it outside of my own network using [this app](https://apkpure.com/p2pcamviewer/x.p2p.cam).
 
 Which made owning this camera quite useless. But i had since gotten into Home Asssistant and got the idea to get it working in there since my instance ran locally so it should be able to access the camera.
 
-## Usage
-```
-import p2pcam
-import cv2
-import numpy as np
+It has been confirmed that this script works with cameras labelled MD81 and MD81S.
+
+## Quick start
+
+You can use the cli.py script to quickly test your camera.
 
-def saveFile(cam, jpeg):
-    RGBImage = cv2.imdecode(np.fromstring(jpeg, dtype=np.uint8), cv2.IMREAD_COLOR)
-    cv2.imwrite('image.jpg', RGBImage)
+```bash
+# Detect cameras on your local network
+python3 cli.py
 
-camera = p2pcam.P2PCam(<own ip>, <camera ip>)
-saveFile(camera, camera.retrieveImage())
+# Detect a camera on your network, connect to it and save 10 JPEG frames in the frames folder
+python3 cli.py --video --max-frames 10 --outdir frames/
+
+# Detect a camera on your network, connect to it and start an HTTP MJPEG server on port 8080
+python3 cli.py --video --serve --port 8080
+
+# If you want to use image transformations first install pillow
+pip install pillow
+# Then you can append --vertical-flip, --horizontal-flip or --add-timestamp to any command
 ```
-## Methods and Variables
-### Methods
-Any methods that may be useful.
 
-`camera.initialize()` Set some variables and attempt to connect to the camera for the first time.
+## API
+
+### LanScanner
+
+#### `refresh(timeout: float = 3.0) -> list[LanDevice]`
+
+Broadcasts a LAN refresh packet, waits for camera responses, and returns the discovered devices sorted by device ID and HKID.
 
-`camera.retrieveImage()` Retrieve a jpeg string from the camera.
+### LanVideoClient
 
-`camera.start()` Start a while true loop staying connected, this will not do anything if `onJpegReceived` isn't set.
+#### `stream(timeout: float = 60.0) -> Iterator[bytes]`
 
-`camera.loop()` Start a while loop doing `retrieveImage()` until a socket error occurs.
-### Variables
-Some variables you may want to set.
+Opens the UDP session, performs the full camera handshake, and yields complete JPEG frames as they become available.
 
-`camera.horizontal_flip` Flip camera horizontally. (if true requires numpy and cv2)
+#### `close() -> None`
 
-`camera.vertical_flip` Flip camera vertically. (if true requires numpy and cv2)
+Stops the stream and closes the UDP socket. Call this when you want to end the session without waiting for the generator to finish.
 
-`camera.addTimeStamp` Add a timestamp to the image. (if true requires numpy and cv2)
+### MJPEGServer
 
-`camera.debug` If true prints out some debugging information.
+#### `update_frame(frame: bytes) -> None`
 
+Replaces the currently broadcast frame and notifies all connected HTTP clients waiting on the next image.
 
-The port information will have to be set before initialisation.
+#### `start() -> None`
 
-`camera.UDP_PORT_HOST` Host udp port default: 5123
+Starts the threaded HTTP server and exposes the MJPEG stream on `/stream`.
 
-`camera.UDP_PORT_TARGET` Target udp port default: 5000
+#### `stop() -> None`
 
-`camera.SOCKET_TIMEOUT` Sets the socket timeout in seconds.
+Stops the HTTP server cleanly and closes its socket.
 
-`camera.NB_FRAGMENTS_TO_ACCUMULATE` How many packets to get a full image. If you put this number high you will get a higher quality image but it will take longer to retrieve. Default: 80
+## Protocol documentation
 
-`camera.onJpegReceived` Callback that will be executed if a jpeg image is retrieved. first argument will be the camera class, the second argument will be the jpeg image string.
+You can find more documentation about the protocol [in the DOC file](DOC.md).