Conversation
Documented findings from probing SRP crypto, mycelium channels, CC endpoints, WASM module analysis, JS bundle crypto patterns, and API endpoint mapping against bugbounty-ctf.1password.com. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add full Anthropic Claude Code OAuth flow (PKCE browser login, token exchange, dual-endpoint refresh, credential storage with Claude CLI fallback) - Fix OpenAI Codex 404: bypass genai-pyo3's reqwest/HTTP2 transport with aiohttp-based Responses API client (Cloudflare rejects HTTP/2) - Fix Anthropic OAuth auth: use genai-pyo3 Client.with_request_override to send Bearer token directly instead of the adapter's ANTHROPIC_API_KEY env lookup - Upgrade genai-pyo3 0.1.16 → 0.5.2 for with_request_override support - Add both OAuth providers to top of setup menu - Fix Codex headers: ChatGPT-Account-ID casing, originator=codex_cli_rs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…iants Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When the browser callback server receives the auth code, any text the user pasted into stdin (the redirect URL) remains buffered. Subsequent Rich prompts (model selection, config confirmation) consume these leftover lines as empty input, causing the wizard to silently hang or skip steps. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…penClaw patterns OpenAI Codex flow: - Never read stdin while callback server is running (root cause of hangs) - Open browser silently, show "Waiting for sign-in..." — no confusing multi-step instructions - Only show paste fallback AFTER callback server times out (120s) - Detect SSH/remote sessions and use paste-only flow automatically - Handle port conflicts by sending /cancel to stale callback servers - Flush stdin with termios.tcflush after callback (not select loop) - Styled HTML success page in browser Anthropic flow: - No localhost callback server — use hosted callback page at console.anthropic.com (paste-only, matching Claude Code/Hermes/ReCursor) - Clean messaging: "Opening browser..." then "Paste code:" prompt - SSH/remote detection Setup wizard: - Cleaner credential reuse prompts - Remove redundant "Starting OAuth..." preamble Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Provider and model selection now use simple-term-menu for interactive arrow-key navigation instead of IntPrompt/Prompt number entry. Flushes stdin via termios.tcflush after menu returns to prevent escape-sequence leakage into subsequent prompts. Also bumps genai-pyo3 minimum to >=0.5.0 (for with_request_override). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The callback server was silently failing to bind when a previous stuck clearwing process held port 1455. The browser callback hit the OLD server (showing success HTML) while the new process waited forever on an empty queue. - Kill stale port holders via lsof + SIGTERM before binding - Use allow_reuse_address=True to handle TIME_WAIT sockets - Narrow except clause on put_nowait to queue.Full only Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Setup and doctor test invokes for openai_codex and anthropic_oauth were using the LangChain ChatModel path, which goes through genai-pyo3's reqwest/HTTP2 transport — blocked by Cloudflare with 403. Now uses AsyncLLMClient directly, which routes through the aiohttp transport that works. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…self Setup was storing base_url=https://chatgpt.com/backend-api in config, which made the request hit /backend-api/responses (403) instead of /backend-api/codex/responses (200). The AsyncLLMClient already appends /codex/ when base_url is None, so don't override it. Confirmed: /backend-api/responses → 403, /backend-api/codex/responses → 200. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The ChatGPT Codex backend rejects max_output_tokens with HTTP 400. The test invoke doesn't need a token limit — it's just a PONG check. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…properties genai-pyo3 0.5.2 changed ChatResponse.first_text, .texts, and .tool_calls from methods to properties. Updated all call sites across the codebase (15 files). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three root causes prevented sourcehunt from using the configured OpenAI Codex (gpt-5.5) provider: 1. CLEARWING_HOME override (blind-test sandbox) bypassed ~/.clearwing/ config.yaml — Config now falls back to user's home config for provider credentials even when CLEARWING_HOME points elsewhere. 2. Multiple SourceHuntRunner construction sites (agent tool, campaign, commit monitor) omitted provider_manager, causing silent fallthrough to hardcoded Anthropic defaults. Runner's _get_llm/_get_native_client now propagate provider_manager errors instead of swallowing them. 3. Codex Responses API compatibility: /codex/ URL path not appended when base_url was set by endpoint resolver; text.format schema used Chat Completions structure instead of Responses API structure (name at top level); chunk_size=150 too large for Codex throughput (now 30). Also fixes test FakeResponse mocks for genai-pyo3 0.5.2 (properties not methods, adds reasoning_content), updates docs/FFmpeg.md to current CLI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… nested in function Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Check Docker runtimes at startup and warn once instead of failing every sandbox creation. The container layer also retries without the runtime if Docker rejects it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Responses API uses function_call_output/function_call item types instead of role:"tool" and role:"assistant" with tool_calls. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
scapy is GPL-2.0; libpnet-pyo3 (https://pypi.org/project/libpnet-pyo3) is a thin pyo3 binding over libpnet, dual MIT/Apache-2.0. The two sr1-based callsites in scanning/{port,os}_scanner have direct equivalents in libpnet_pyo3.tcp_sr1 / tcp_send, so the swap is a mechanical one — no behavior change other than the underlying implementation. Changes: - pyproject.toml: replace scapy>=2.5.0 with libpnet-pyo3>=0.1.1 - scanning/port_scanner.py: tcp_sr1 + tcp_send (RST) instead of sr1 - scanning/os_scanner.py: tcp_sr1 for both _passive_detect and _active_detect; resp.ttl / resp.window / resp.flags replace the scapy layer accessors - core/config.py, agent/tools/__init__.py, tests/test_scanners.py: follow-on comment updates Notes: - Raw-socket privilege requirements are unchanged (root or CAP_NET_RAW). The "raw-socket" warning in port_scanner._syn_scan still fires for the same scan_type values. - Historical changelog references to scapy in docs/CHANGELOG-v1.0.md are left intact — they describe past behavior accurately.
genai-pyo3 0.6.0 moves its underlying rust-genai dep from a fork pin (carrying jeremychone/rust-genai#227) to upstream main, which has since shipped 0.5.x → 0.6.x. The Python API surface is unchanged; this is a floor bump to pull in the newer underlying genai crate.
0.6.1rc0 emits the adaptive thinking API (thinking.type.adaptive + output_config.effort) for Anthropic Opus 4.7/4.8 instead of the legacy thinking.type.enabled shape those models now reject, fixing the 400s reported in #49. Also relocks uv.lock, which was stale: #46 swapped scapy->libpnet-pyo3 in pyproject without relocking. This regen picks up that swap plus the genai-pyo3 bump (lockfile format also advances revision 1->3).
…chema (#54) Anthropic's structured-output validator uses constrained decoding and rejects numeric-range keywords (minimum/maximum/exclusiveMinimum/ exclusiveMaximum/multipleOf), 400ing every ranker call. Pydantic emits those from Field(ge=1, le=5). Encode the constraint in the type instead: a Literal[1,2,3,4,5] alias serializes to {"type": "integer", "enum": [1,2,3,4,5]} — which the validator supports and enforces via constrained decoding — while Pydantic still validates the range on parse. No schema post-processing needed. Fixes #50.
- genai-pyo3 0.7.0b6: adds cp314 wheels; tracks rust-genai v0.7.0-beta.6. - libpnet-pyo3 0.1.2: declares + CI-tests 3.14 (its abi3 wheel already ran on 3.14; this is the metadata/CI catch-up). uv.lock regenerated with uv 0.9.10. Beyond the two version bumps it also normalizes the lock to revision 3 and rewrites platform_system->sys_platform markers — the same output setup-uv@v7 (unpinned, latest) produces in CI, so this just front-runs an inevitable normalization.
…stop() SandboxContainer.stop() removes the container but does not close the docker SDK client, leaving its urllib3 HTTPConnectionPool with ~10 keep-alive unix-socket connections open. Each Hunter session creates a new SandboxContainer, so this leaks ~15 fd per session and exhausts the host ulimit on long scans (observed: ulimit -n 4096 hits its limit after ~270 Hunter sessions). Add a self._client.close() call after the container is removed. Validated by re-running a 2-file Hunter scan with the fix applied: unix socket fd count stays steady across multiple Hunter sessions (drops by ~7 right after each session.stop()), whereas the same run without the fix accumulates +15 unix fd per session. Fixes #68 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three sourcehunt sites call `await self._llm.aask(user, system=...)`: - clearwing/sourcehunt/nday_filter.py:154 - clearwing/sourcehunt/reveng_reconstructor.py:128 - clearwing/bench/crash_classifier.py:175 But neither AsyncLLMClient nor ChatModel defines `.aask` — only `.aask_text` and `.aask_json`. Each call site has a broad `except Exception` that silently swallows the AttributeError, degrading: - n-day: every CVE forced to `POSSIBLY_EXPLOITABLE` (the hard-coded except branch at nday_filter.py:166-169) - reveng: falls back to `_fallback_reconstruction` (raw `FUN_xxxx` names) - bench/crash_classifier: `classification.tier` retains the pre-existing `automated_tier`; LLM input never improves it Unit tests don't catch this because they inject `AsyncMock` which auto-vivifies any attribute access. Add a thin `.aask(user, system=None)` shim on both classes that delegates to `aask_text(system=, user=)`. Purely additive — no behavior change for any caller that uses `aask_text`/`aask_json` directly. Fixes #72
…alias Follow-up to the #73 compat shim. The three callers (nday_filter, reveng_reconstructor, bench.crash_classifier) receive a ChatModel (via provider_manager.get_llm), which had no aask_text — only ainvoke (returns a LangChain AIMessage, not the ChatResponse these callers read via .first_text). Rather than a positional .aask alias on both classes, give ChatModel real aask_text/aask_json that delegate to its underlying AsyncLLMClient (this also fixes ChatModel being unable to do structured output at all), and point the three callers at aask_text(system=, user=). aask_text is now the common method on both ChatModel and AsyncLLMClient, so callers are forward-compatible if the ChatModel facade is ever removed. Supersedes tsunoda-kazuya's shim (kept in history as the prior commit). Fixes #72. Verified live (Codex): ChatModel.aask_text -> 'OK', aask_json -> {'ok': True}.
Request ChatOptions(capture_reasoning_content / normalize_reasoning_content) for every model by default, but skip it for models that reject it. Currently only gpt-5.3-codex-spark errors when reasoning capture is requested; a module-level substring blacklist (_REASONING_CAPTURE_UNSUPPORTED_PATTERNS) + _model_supports_reasoning_capture() resolve a per-client self.capture_reasoning_content flag used at both ChatOptions sites. Note gpt-5.3-codex (non-spark) is unaffected — only the -spark variant is blacklisted.
Stage 1 of removing the ChatModel LangChain-compat facade. Repoint the two genuine standalone (non-tool-calling) ainvoke callers onto the native aask_text + response_text surface: - data/memory/summarizer.py: summarize() now calls llm.aask_text(system, user) and reads response_text(resp) instead of llm.ainvoke([...]).content. - agent/operator.py _adecide_next(): same, dropping the AIMessage list-content unwrapping. Both sites are forward-compatible: ChatModel already delegates aask_text to its underlying AsyncLLMClient, so they work unchanged today and after the callers are repointed to a native client. No bind_tools / graph tool-calling core is touched; ChatModel is not yet deleted. Tests updated to stub aask_text() -> ChatResponse(first_text=...) instead of ainvoke() -> AIMessage(content=...).
Production migrated these three callers off the removed .aask alias to ChatModel.aask_text (commits fb85490/b3a2817), but their test mocks still stubbed .aask and treated response.first_text as a callable (.return_value). The tests passed only by accident: AsyncMock auto-created .aask, so the code path never hit the real client. Against a live client (surfaced by the sourcehunt regression run) these paths would break. Mock aask_text instead of aask, and set first_text as an attribute (it is a property on ChatResponse, accessed as response.first_text).
Same class of stale mock as the nday/reveng/crash tests: production reads response.first_text as an attribute (it is a property on ChatResponse), but these tests stubbed it as a callable via .first_text.return_value. The MagicMock returned by attribute access was truthy, so code paths like (response.first_text or '') and .first_text.strip() operated on a MagicMock instead of the intended string, yielding 'unknown' primitive_type and MagicMock hardened_poc/validation payloads. Set first_text as an attribute so the mocked strings flow through.
The ranker computes a per-file priority score but returns files in enumeration (roughly alphabetical) order — it never sorts. HunterPool grouped files into A/B/C tiers but likewise preserved that order, so within a tier the most suspicious files were not hunted first. Under a tier budget the highest-priority files could be starved by lower-priority ones that happened to sort earlier; even unbudgeted, time-to-first-finding on the genuinely suspicious files was needlessly long. Sort each tier by priority descending (stable, so equal-priority files keep enumeration order) before expanding to work items, so the ranker's scores actually drive hunt order. Adds a regression test asserting a higher-priority tier-A file is dispatched before a lower-priority one listed earlier.
Contributor
Author
|
Superseded by the reconciliation branch — a plain master→main can't fast-forward (bidirectional divergence). Replaced by #NEW which rebases master's unique commits onto main (keeping both sides' good changes) so it does fast-forward. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bring
mainup to date withmaster(canonical branch).mainandmasterhave diverged bidirectionally:masteris 30 commits ahead (aask compat/native surface, capture_reasoning blacklist, and the ChatModel-migration-prep work).mainhas commitsmasterlacks — the merged-PR versions of refactor(llm): use genai-pyo3 native transport for Codex; drop ~175-line aiohttp shim #66, fix(llm): make reasoning_effort model-aware (denylist + retry-on-400) #65, feat(sourcehunt): respect gitignore during scans #45, Express ranker 1..5 scores as Literal so Anthropic accepts the schema #54, chore: bump genai-pyo3 to >=0.6.1rc0 #53, chore: bump genai-pyo3 pin to >=0.6.0 #48, CODEOWNERS, checkout-v7, dependabot-removal — some with real content not on master (verified: master is missing the fix(llm): make reasoning_effort model-aware (denylist + retry-on-400) #65reasoning_effortdenylist and.github/CODEOWNERS).So this can't fast-forward and shouldn't be a plain merge commit. See reconciliation notes in the thread.