Skip to content

feat(ccproxy): v2.0.0 — inspector architecture, lightllm, DAG pipeline, compliance#16

Open
starbaser wants to merge 405 commits into
mainfrom
dev
Open

feat(ccproxy): v2.0.0 — inspector architecture, lightllm, DAG pipeline, compliance#16
starbaser wants to merge 405 commits into
mainfrom
dev

Conversation

@starbaser

@starbaser starbaser commented Apr 16, 2026

Copy link
Copy Markdown
Owner

AI Summary

Complete rewrite of ccproxy from a LiteLLM proxy subprocess model to an in-process mitmproxy-based transparent LLM API interceptor. This is the v2.0.0 release (tagged v2.0.0-rc1).

  • Inspector architecture: mitmweb runs in-process via WebMaster API with dual listeners — reverse proxy + WireGuard namespace jail. No subprocess, no gateway server.
  • lightllm: Surgical nerve connector into LiteLLM's BaseConfig transformation pipeline, bypassing cost tracking and callback machinery entirely.
  • DAG-based hook pipeline: @hook(reads=..., writes=...) decorator-declared data dependencies, topologically sorted via Kahn's algorithm. Per-request overrides via x-ccproxy-hooks header.
  • SSE streaming: SseTransformer stateful stream callable — parses, transforms per-chunk via LiteLLM's provider iterators, re-serializes as OpenAI-format SSE.
  • Compliance profile learning: Provider-agnostic system that observes legitimate request shapes from WireGuard traffic and stamps compliance profiles onto proxied requests.
  • Gemini/Vertex AI support: Full routing, OAuth handling, context caching via cachedContents API, path rewriting for cloudcode-pa.googleapis.com.
  • Flows CLI: ccproxy flows list/dump/diff/compare/clear with multi-page HAR 1.2 output, jq filtering, and sliding-window diff across flow sets.
  • MCP notification endpoint: POST /mcp/notify for terminal event ingestion, buffered and injected as synthetic tool_use/tool_result pairs.
  • XDG config directory: Default config moved to ~/.config/ccproxy/ (breaking change).
  • init replaces install: CLI rename (breaking change).
  • Rich pipeline visualization: render_pipeline() builds a full DAG display with parallel groups via rich.columns.Columns.

Breaking Changes

  • Config directory: ~/.ccproxy/~/.config/ccproxy/
  • CLI: ccproxy installccproxy init
  • --debug flag replaced by --log-level / -v
  • forward_port / reverse_port replaced by unified port config
  • mitm config section renamed to inspect
  • Prisma/database infrastructure removed entirely
  • LiteLLM proxy subprocess removed
  • to_mermaid / to_ascii removed from HookDAG

Test plan

  • just test passes with ≥90% coverage
  • just lint / just typecheck clean
  • Smoke test: ccproxy run --inspect -- claude --model haiku -p "what's 2+2"
  • Verify ccproxy init creates config at ~/.config/ccproxy/
  • Verify flows CLI: ccproxy flows list, ccproxy flows dump
  • Verify Gemini routing through inspector

…uth fixes

Add `ccproxy flows` subcommand for querying mitmweb flows API (list, req,
res, client, diff, clear). Built on ccproxy config infrastructure with
httpx and rich output.

Fix forward_oauth: remove UA override from _inject_token() so compliance
profile handles it, replace unconditional x-goog-api-key clear with
conditional sentinel loop to prevent double-clear when auth_header targets
that header.

Add Gemini SDK path rewriting in redirect handler: strip routing prefix
(/gemini/) and map standard genai SDK paths to cloudcode-pa's /v1internal
endpoint. Add cloudcode-pa response envelope unwrapping in InspectorAddon
so the genai SDK receives standard Gemini format.

Update nix/defaults.nix with transforms, compliance config, and gemini
oat_sources destinations.
… skill

Add two-part architectural doc (docs/inspector-and-compliance.md) covering
the inspector MITM system and compliance learning system.

Add using-ccproxy-inspector skill with Python helper scripts:
- list_flows.py: enriched flow listing with provider/model/status filters
- inspect_flow.py: client-vs-forwarded request diff with change summary
- compliance_status.py: profile/accumulator status from on-disk store

Update using-ccproxy-api skill to reflect current defaults: compliance-based
headers/identity instead of explicit add_beta_headers/inject_claude_code_identity
hooks, redirect mode in transform routes, 401-triggered token refresh.
…plate

Rewrite SKILL.md with proper installation guide covering Home Manager
module, standalone setup, and per-project mkConfig instances. Add
configuration reference section with full ccproxy.yaml example.

Rewrite troubleshooting.md to match current inspector architecture:
remove stale references to rule_evaluator, model_router, --detach,
--mitm, TTL-based refresh, flat hooks list. Replace with compliance-
based diagnostics, flow inspection commands, and correct hook pipeline.

Update template ccproxy.yaml to match nix/defaults.nix: outbound hooks
now use inject_mcp_notifications, verbose_mode, apply_compliance
instead of add_beta_headers and inject_claude_code_identity. Add
compliance and gemini oat_sources sections.
The directory-exists guard blocked `ccproxy install` in any pre-existing
config dir (e.g. ~/.ccproxy created by a previous start) even when the
yaml file wasn't there yet. Now only checks per-file existence — creates
the directory silently and skips individual files that already exist
unless --force is passed.
Introduces a `mode` field to distinguish between redirect and transform
operations. Updates `InspectorAddon.responseheaders` and transform route
handlers to check this mode before processing streaming transforms.
Remove debug artifacts (check_auth.py, .env.example), untrack .mcp.json,
fill LICENSE placeholder, fix GitHub URLs, sync dev dependency pins,
strip ~700 lines of redundant docstrings/comments across 52 files,
rewrite change-history comments, fix dead code (mid-file imports,
vestigial case variants, empty TYPE_CHECKING block).
Tools inside the namespace (e.g. PAL MCP server) configured with
localhost base URLs couldn't reach host services — 127.0.0.1 is the
namespace's own isolated loopback. This caused connection refused for
any tool hardcoded to http://127.0.0.1:4000.

- Enable route_localnet sysctl so iptables OUTPUT DNAT works on loopback
- Add OUTPUT DNAT rule: 127.0.0.1 → 10.0.2.2 (slirp4netns gateway)
- Add port remap rule when running port differs from default (4000→4001)
- Pass proxy_port from cli.py to create_namespace()
- Atomic write for combined CA bundle via tempfile+rename
…uting

- Document namespace localhost→host DNAT routing and network topology
- Add ccproxy flows CLI commands to CLI reference
- Document tools/flows.py MitmwebClient subsystem
- Add Gemini-through-inspector routing notes (PAL + Gemini CLI paths)
- Fix inspector UI port (8083→8084), note dual-instance dev/prod setup
- test_cli.py: update TestInstallConfig tests to match new install behavior
  (no SystemExit on existing files, "Installed" not "Copied", "Configuration
  installed to:" not "Installation complete!")
- test_response_transform.py: add mode="transform" to TransformMeta fixtures
  so cross-provider and non-streaming response tests actually exercise the
  transform code path instead of silently hitting the redirect default
…ts API

Adds provider-side KV caching for Gemini transforms. When messages contain
Anthropic-style cache_control annotations, resolve_cached_content() creates
or finds existing cached content resources via Google's cachedContents API,
then passes the resource name into the generateContent request body.

fix(lightllm): strip bogus Authorization header for Gemini API key auth —
validate_environment() injects Bearer {api_key} which Google rejects.
Add 237 new tests across 12 files covering previously untested modules
and critical code paths: OAuth sentinel substitution, beta header
injection, Claude Code identity, pipeline guards, contentview rendering,
inspector pipeline wiring, credential loading, transform route redirect
mode, Gemini context cache integration, SSE response iteration, mitmweb
REST client, and flows CLI dispatcher.
…form route

sign_request() no longer accepts api_key and returns dict (headers only)
instead of tuple[headers, signed_body]. Run optional_params through
map_openai_params() before transform_request() to convert tool_choice
and other OpenAI-format params to provider-native format. Fall back to
request body model when dest_model is not set in the transform rule.
…D3, D5-D7

Immediate resolutions (from prior session):
- Delete dead `add_beta_headers` hook (superseded by compliance seed)
- Fix `x-ccproxy-oauth-injected` header leak to upstream (moved to flow.metadata)
- Add `api-key` header to forward_oauth guard (Azure OpenAI)
- Remove UA truncation in compliance logs
- Wire provider_map from config to InspectorTracer
- Fix `_serialize_for_comparison` import placement
- Make namespace install messages non-Nix-specific

Deferred item resolutions (this session):
- D1: Provider-agnostic ProfileStore — replace `seed_anthropic: bool` with
  `seed_profiles: list[ComplianceProfile] | None`, extract
  `_build_anthropic_seed_profile()` to module level
- D2: Format version mismatch degraded flag — `is_degraded` property on
  ProfileStore, set when version mismatch discards existing data, WARNING
  in apply_compliance hook
- D3: Config-exposed classifier sets — `additional_header_exclusions` and
  `additional_body_content_fields` on ComplianceConfig, threaded through
  classifier → extractor → observe_flow
- D5: verbose_mode — replaced DEFER with NOTE (requires live verification)
- D6: MCP notifications — expanded module docstring with integration flow
- D7: SSE transformer correctness — spec-correct multi-line data collection,
  silent drop on JSON errors, synthetic OpenAI error events on chunk_parser
  failure, model_dump(mode="json", exclude_none=True)
…ble merge operations

Refactors 5 private merge functions into public methods on a ComplianceMerger
class that users can subclass to override, skip, or reorder individual operations.
Adds compliance.merger_class config field and resolve_merger_class() resolver.
Implements CCH billing header hash verification against live mitmweb
flows. Extracts billing headers and user messages to validate the hash
algorithm used by Claude Code.
Adds humanize dependency for natural time formatting. Removes .mcp.json
from source (sentinel key config belongs in user environment, not repo).
Clarifies that consumers should merge with
`ccproxy.defaultSettings.settings` (top-level, no system selector) to
inherit all defaults. Adds port assignment table and process-compose
readiness probe example.
Catches broken routes, DNS, CA bundles, or namespace egress problems
before accepting traffic. Probes a single canary URL at startup and
refuses to start if unreachable, avoiding silent hangs on real requests.
Rename upstream_timeout_seconds to provider_timeout and flip the
default from 600.0 to None, matching Portkey AI's upstream behavior
(null timeout routes through plain fetch() with no wrapper). The
OAuth 401 retry client now branches on truthiness: explicit opt-in
builds an httpx.Timeout applied uniformly across phases; the default
path passes timeout=None directly.

Also change readiness_probe_url default to https://1.1.1.1/ — direct
IP avoids DNS dependency for the startup canary.
Route _run_inspect() startup banners (mitmweb URL, WireGuard/TLS keylog
paths, Inspector UI) and run_preflight_checks() port-conflict errors
through the logging system instead of builtin print/builtin_print, so
`ccproxy run` and `ccproxy run --inspect` reserve stdout exclusively for
the child process's output. Previously these bypassed setup_logging()'s
stderr handler and polluted stdout.

Add CCProxyConfig.use_journal (default False). When set AND the command
is `ccproxy start`, setup_logging() installs
systemd.journal.JournalHandler(SYSLOG_IDENTIFIER="ccproxy") with graceful
fallback to stderr on missing systemd-python or unavailable journald
socket. Exposed as the `journal` optional extra.
…uild

stdenv sets SSL_CERT_FILE=/no-cert-file.crt to block network access during
sandbox builds; uv warns on the missing path even though the install is
--offline --no-cache. Point it at pkgs.cacert in preInstall.
1.82.7 and 1.82.8 were compromised via stolen PyPI credentials (TeamPCP,
March 24 2026). BerriAI released v1.83.0 on March 30 via a hardened
CI/CD v2 pipeline; all prior versions through 1.82.6 were independently
audited clean. Resolves to 1.83.7.
Migrates ClientRequestContentview from deprecated mitmproxy Contentview
API (name/syntax_highlight/prettify properties) to the new View base
class with __call__ and render_priority methods. Also applies ruff
linting fixes across test files.
Works around google-gemini/gemini-cli#21691 where the CLI wipes
refresh_token during access_token refresh, causing auth failures after
~1hr. The hook stashes the refresh_token before triggering CLI refresh
and restores it if wiped.
Replaces table-based formatting with HAR-compliant JSON structure. Adds
_parse_client_request_text to parse mitmproxy's rendered output into
structured fields for downstream HAR generation.
Replaces legacy mitmproxy View base class with modern Contentview
interface, splitting __call__ into name/syntax_highlight/prettify
properties and methods. Updates test helpers to use Metadata objects
instead of raw flows.

BREAKING CHANGE: requires Python >=3.13
pyperclip 1.9.0 has no pyproject.toml (setup.py only), so uv2nix
attempts a source build without setuptools in scope, failing with
ModuleNotFoundError. Add it via nativeBuildInputs in the wheelFixes
overlay; pyproject.toml already covers the uv sync path via
[tool.uv.extra-build-dependencies].
Expand the `Flows` class docstring so `ccproxy flows --help` explains each
subcommand, the HAR 1.2 output format, and usage examples (jq, HAR viewers).
Update CLAUDE.md to describe the HAR output semantics, body handling, and
consumption patterns for `tools/flows.py`.
starbaser added 30 commits May 26, 2026 14:43
Previously load_hooks mutated singleton HookSpec objects without
resetting params, causing stale configuration to persist across repeated
loads. Now explicitly clears spec.params before validation to ensure
clean state.
Enables automated WSL2 validation by spinning up a disposable Windows 11
KVM VM, importing the .wsl artifact, running the PowerShell test harness
inside the guest, and collecting results via HTTP.
Simplifies kitstore.nix by removing unused repository configurations for
mitmproxy, slirp4netns, xepor, fastmcp, glom, and litellm.
Consolidates shape management documentation around the new `ccproxy
shapes` subcommand and removes obsolete WSL2 strategy document.
…est coverage gate

MCP surface (the headline): the documented POST /mcp/notify endpoint was
never mounted — the whole notification-injection feature was unreachable.
inspector/routes/mcp.py now serves it on the proxy listener (fire-and-forget,
200-always, lazy TTL expiry — expire() previously had no callers) and
rewrites /mcp to the in-process FastMCP server, so one socket serves SDK
traffic, MCP protocol, and notification ingestion. The unmounted FastAPI
router is deleted and the fastapi dependency dropped.
inject_mcp_notifications now emits the documented tasks_get return envelope.

Error handling: inject_auth config failures raise AuthConfigError instead of
silently forwarding unauthenticated requests; failed response transforms
return an OpenAI-shape 500 instead of passing the untransformed provider
body through.

Hygiene: delete orphan shaping/responses.py; delete dead
test_shell_integration.py (imported a removed symbol, hidden by a global
--ignore); conftest now resets the transport dispatch and gemini project
caches; static uv dependency-metadata for systemd-python so uv lock works
without systemd build tooling.

Docs made true: readiness probe documented at its real flat keys,
gemini_capacity.enabled default corrected to true, provider_map fixed,
ccproxy logs claim corrected, addon-chain diagram completed (13 addons),
docs/mcp.md upgraded to implemented status with real config keys/defaults,
new mcp section in configuration.md, AGENTS.md unmounted-note replaced.

Tests: +191 (pplx hooks 25%→96% / 20%→99%, routes/pplx 0%→98%, fingerprint
34%→100%, shapes 51%→100%, utils 55%→99%); total coverage 80.61% → 86.13%.
Coverage gate ratcheted to an honest 86 — just test passes truthfully for
the first time. CI gains unit-tests and lint+typecheck jobs (none existed).

CHANGELOG.md added with the 2.0.0 entry, 1.x→2.0 migration table, and known
limitations (citations drop, buffered-Gemini legacy path).
Introduces type aliases (_RoutedEvent, _QueueEvent, _WireObject, etc.)
across intake/render modules to eliminate generic Any annotations and
improve type safety without changing runtime behavior.
- run mypy over src/ccproxy and tests with no incremental cache

- add ty to the local gate for src and tests

- annotate test fixtures and helpers so the full suite typechecks

Validation: just lint; just typecheck; uv run ruff format --check .; just test
Moves template sync from pre-commit-config.yaml into a custom git hook
that calls nix run .#sync-ccproxy-template, simplifying the pre-commit
framework integration and making the sync step explicit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants