Skip to content
risu729Public

About

Passive Chrome network capture with raw bodies, NetLog, and a first-class local plugin system.

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Repository files navigation

Kuebiko

Kuebiko is a passive-by-default, extensible network capture tool for browsers that expose the Chrome DevTools Protocol (CDP). It can launch a dedicated browser profile or attach to an existing local CDP endpoint, then saves request/response bodies plus metadata while you browse manually. Trusted local plugins can react to completed captures without changing browser traffic.

This tool is intentionally narrow. It does not use mitmproxy, SSLKEYLOGFILE, packet capture, request interception, browser automation, login automation, analytics, parsers, dashboards, or HAR viewers.

What This Is For

Use this when you want raw local capture files from normal manual browsing in a throwaway browser profile:

  • response bodies from CDP Network.getResponseBody, or Network.streamResourceContent with --stream-bodies
  • request payloads that the browser exposes through CDP
  • files the browser downloads, with --capture-downloads
  • an end-of-run snapshot of cookies, web storage, and IndexedDB, with --snapshot-storage
  • every web storage and IndexedDB change as it happens, with --track-storage
  • request/response metadata in append-only NDJSON
  • Chromium NetLog in the same run directory when using browser launch mode

The logger is written in TypeScript for Bun and runs on Windows, Linux, and macOS. Run the logger on the same OS as the browser whenever possible so it can connect to http://127.0.0.1:9222 without cross-VM networking assumptions.

Why Websites Usually Cannot Notice It

The logger observes the browser locally. The website does not receive a header, cookie, JavaScript variable, or protocol message saying that CDP logging is enabled.

Launch mode and the logger are passive by default:

  • CDP is bound to 127.0.0.1.
  • The logger uses the CDP Network domain to observe completed browser network activity and fetch stored bodies.
  • NetLog writes a local debugging file from the Chromium network stack.
  • There is no Fetch.enable, request pausing, request rewriting, or response rewriting.
  • There is no Runtime.evaluate, script injection, extension injection, or Debugger-domain attachment.
  • Launch mode does not use --headless, --enable-automation, or --remote-debugging-port=0.
  • Launch mode does not use --disable-quic; browser network behavior is kept close to normal.
  • After enabling Network on an attached popup, iframe, or worker target, the logger sends Runtime.runIfWaitingForDebugger for that target session; it does not otherwise use the Runtime domain.

--snapshot-storage adds three domains, and it is opt-in. It executes no page script and changes no browser setting:

  • It enables DOMStorage and IndexedDB on the page and iframe target sessions it reads, and only for the moment it reads them, at the very end of the run. Without the flag neither domain is ever enabled.
  • It calls Storage.getCookies once on the browser connection, plus DOMStorage.getDOMStorageItems, IndexedDB.requestDatabaseNames, IndexedDB.requestDatabase, and IndexedDB.requestData per origin.
  • All of these read from the browser process. No page script is executed for them: there is still no Runtime.enable and no Runtime.evaluate, which is exactly why the snapshot is done this way rather than by evaluating localStorage or an IndexedDB cursor in the page.

Two things about it are nevertheless visible from inside the browser, and both are accepted rather than worked around:

  • Reading IndexedDB opens a real connection and a readonly transaction on the page's own database. A versionchange upgrade the page starts at the same moment can therefore see a transient blocked event, for the moment at the end of the run in which the store is read.
  • Each value CDP hands back is wrapped in a remote object that stays pinned in the renderer's inspector object group. Releasing it would take Runtime.releaseObjectGroup, which this tool deliberately does not call, so the group is left un-released and the browser frees it when the connection goes away. That is a documented gap, not an oversight: no Runtime method is worth adding for it.

--track-storage reads the same three domains and is opt-in for the same reason, but it keeps DOMStorage and IndexedDB enabled for the run instead of opening them once at the end. It executes no page script and changes no browser setting either:

  • It enables DOMStorage on a page or iframe session the first time that target reaches an http(s) origin, and leaves it enabled for the rest of the run.
  • It enables IndexedDB and calls Storage.trackIndexedDBForOrigin for an origin on one session, and holds that tracking until the session detaches.
  • Per origin it sends the same reads the snapshot does, as a baseline: DOMStorage.getDOMStorageItems for both areas, then IndexedDB.requestDatabaseNames, IndexedDB.requestDatabase, and IndexedDB.requestData.
  • It sends IndexedDB.requestData again each time Chrome reports that a store changed. The transient blocked event and the un-released object group described above therefore apply once per baseline read and once per change, rather than once at the end of the run.
  • It sends no Storage.getCookies: cookies are not part of this stream.
  • Without the flag none of this happens and neither domain is ever enabled.

--capture-downloads is the one exception, and it is opt-in. That flag does change browser behavior rather than only observing it:

  • It calls Browser.setDownloadBehavior with allowAndName, so downloads are written into the run directory and named after their download GUID instead of going to your normal download folder under their suggested names.
  • allowAndName also allows every download request. Downloads Chrome would normally block, hold behind the multiple-downloads prompt, route through "Ask where to save each file", or gate behind the dangerous-file confirmation are saved silently instead: the download is not marked dangerous and nothing is asked.
  • The same call turns on the browser-wide Browser download events. No CDP domain is enabled per target for this, and the Network domain never reports downloads at all.

Where a download is written and what it is named on disk is not observable from page script. Allowing a download Chrome would otherwise have blocked or prompted about is: a page that starts one can see it go through. The flag is off by default, and without it the logger makes no such call and keeps downloads exactly where the browser would put them.

That means a destination site should see ordinary browser requests from the dedicated profile, not an explicit "logger enabled" signal.

This is not a stealth or evasion guarantee. A site may still notice ordinary environment differences, such as a fresh profile, missing old cookies, different cache state, different permissions, no usual extensions, or local timing changes from heavy logging. Some sites also use broad anti-debugging or automation heuristics. This project avoids the obvious automation and interception signals; it does not promise undetectability.

Install

Install the latest release with mise:

mise use -g github:risu729/kuebiko

Quick Start

Launch mode is the recommended path. It starts a browser with the required CDP and NetLog flags, uses a dedicated profile, and keeps capture files in one run directory.

Run the logger and let it launch your browser:

kuebiko \
  --launch-browser \
  --browser-command google-chrome

Use another browser command or executable path when needed:

kuebiko.exe `
  --launch-browser `
  --browser-command chrome.exe
kuebiko --launch-browser \
  --browser-path "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"

The browser opens with a dedicated profile, CDP bound to 127.0.0.1:9222, and NetLog writing to the same capture directory. Log in manually inside that profile and browse normally.

Dedicated Profile

Launch mode uses a dedicated profile under the platform default base directory:

  • Windows: %LOCALAPPDATA%\Kuebiko\browser-profile
  • macOS: ~/Library/Application Support/Kuebiko/browser-profile
  • Linux: ${XDG_STATE_HOME:-~/.local/state}/Kuebiko/browser-profile

The tool does not attach to your default browser profile and does not depend on it. Treat this profile as a separate browser identity. If a website needs login, log in manually inside this browser window.

Output Layout

Each run creates a timestamped directory under the platform capture root:

Kuebiko/captures/2026-07-06T12-34-56

The run directory contains:

run.json
metadata.ndjson
errors.ndjson
websocket.ndjson
eventsource.ndjson
downloads.ndjson
storage.ndjson
storage-snapshot.json
bodies/
requests/
downloads/
plugins/
netlog.json

bodies/ contains saved response bodies. requests/ contains request payloads that the browser exposes through CDP. Filenames are generated from timestamp, SHA-256, counter, and MIME-derived extension; URLs are not placed into filenames.

downloads/ is created only with --capture-downloads, because only the browser writes into it.

plugins/ is created when configured plugins write per-plugin output. The core logger still only writes raw capture files; plugins are trusted local extension code you opt into with --config.

run.json records how the run was started. --label <label> (repeatable) and --note <text> add labels and note fields so the run also records why it was made. Both fields are omitted when the flags are not used. A captureCookies boolean is always present so a capture directory says whether --capture-cookies was on and it may therefore hold plaintext session cookies. A streamBodies boolean is always present too, so the directory also says how its bodies were retrieved, a captureDownloads boolean says whether browser download behavior was changed for the run, and a snapshotStorage boolean says whether the directory may hold a whole stored session. A trackStorage boolean says the same about storage.ndjson, which holds a live session of its own.

run.json also records how complete the capture is: include and exclude hold the filter sources, maxBodyBytes the cap above which a body was never retrieved, and a netlog boolean says whether a NetLog was written beside the capture. The three filter fields are omitted when the flags are not used, so nothing distinguishes an unfiltered run by their absence alone.

metadata.ndjson contains one JSON object per completed response that passed the filters. When available, the same metadata line links to both a saved request payload and a saved response body.

--capture-cookies adds the raw wire headers to each metadata line as rawRequestHeaders and rawResponseHeaders, taken from the CDP Network.requestWillBeSentExtraInfo and Network.responseReceivedExtraInfo events. They sit next to the refined requestHeaders and responseHeaders, never in place of them. The raw sets are the only place Cookie appears at all, and the only place Set-Cookie appears verbatim for every response, so a capture made with the flag holds live session cookies in plaintext. A response that could not store a cookie also records blockedCookies with the reason Chrome rejected each one, which is usually the real answer when a session breaks, plus exemptedCookies for cookies stored despite third-party restrictions and cookiePartitionKey when partitioned cookies apply. All are omitted when there is nothing to report. The flag is off by default, and without it the logger never subscribes the ExtraInfo events. Plugin events keep the refined headers only, so raw cookies stay inside the capture directory, and run.json records captureCookies so the directory says whether it may hold them.

--stream-bodies is experimental. By default a body is read back with Network.getResponseBody once a request finishes, which depends on Chromium still holding it in its resource buffer. With this flag the logger instead enables Network.streamResourceContent on each response that passes the filters, then assembles the body from the buffered prefix the call returns followed by the streamed Network.dataReceived chunks. This recovers bodies the buffer path can miss: streaming responses, service-worker and cached responses, single responses above maxResourceBufferSize, and bodies dropped when a navigation or target teardown clears the buffer mid-fetch. Enabling the stream also moves the --include and --exclude decision earlier, to the response event, to bound the overhead. If a stream cannot be enabled or the method is unsupported, that request falls back to Network.getResponseBody, so nothing is lost relative to the default. The first such failure of the run is recorded in errors.ndjson as Network.streamResourceContent, and later ones are suppressed so a Chromium without the method cannot flood the file. A stream that finishes with no bytes at all, which happens on some service-worker and cached paths, also falls back rather than saving an empty body.

Server-Sent Events responses are never streamed: the connection normally never finishes, so its bytes would accumulate for the life of the page, and its messages are already captured in eventsource.ndjson.

--max-body-bytes still guards the size, and it still compares the encoded wire bytes: the running total is summed from the encodedDataLength of every Network.dataReceived event of the request, which is what the buffer path's Network.loadingFinished total counts as well. Past the limit the partial buffer is freed and the body is recorded as a skip, marked in errors.ndjson with the same Network.getResponseBody.skipped event the buffer path uses, even though nothing was fetched with that call. Unlike the buffer path, the guard cannot avoid the transfer, because an enabled stream cannot be turned off again; it only avoids writing and hashing the body. Without --max-body-bytes a default limit of maxResourceBufferSize applies to a streamed body, so a response whose Network.loadingFinished never arrives cannot grow without bound.

Every streamed body records base64Encoded: true in metadata.ndjson and in plugin events, where the buffer path reports false for a text response. That is only how CDP delivered the bytes; the saved file is byte-identical either way, and run.json records streamBodies. The flag is off by default while the CDP method is experimental.

A redirect chain reuses one CDP request ID, so it produces several metadata lines. Each 3xx hop is written when the browser follows it, with redirect: true and a 0-based redirectIndex; the terminal response of the same chain takes the next redirectIndex and carries no redirect field. Requests that never redirected have neither field. Hops keep their own status, location, and set-cookie response headers, and an inline request payload when the browser reported one, but they never have a response body: redirects carry none, and CDP would only return the final hop's body. Filters apply to each hop URL, exactly as they apply to a terminal response. With --capture-cookies, each hop keeps the raw headers of its own hop. Because the whole chain shares one request ID, a hop's responseReceivedExtraInfo that arrives only after the next hop started is dropped instead of being credited to the wrong hop, leaving that hop with its refined headers and no rawResponseHeaders.

websocket.ndjson contains one JSON object per WebSocket frame, in both directions: direction is "received" for a server-to-browser frame and "sent" for a client-to-server one. Each frame carries the socket url, taken from the Network.webSocketCreated event of the socket its requestId belongs to. A frame the logger sees without that event, because the socket was opened before it attached or had already closed, is still recorded but has no url.

eventsource.ndjson contains one JSON object per Server-Sent Events message, with the eventName, eventId, and data the browser parsed out of the stream. An SSE connection normally stays open for the life of the page, so it usually never reaches Network.loadingFinished and has no saved response body; the messages are the capture. A stream the server ends, or the page closes, does reach it and adds a normal metadata.ndjson record for the connection itself, while the messages stay in eventsource.ndjson either way. Each message carries the stream url, which is recoverable because an EventSource connection does produce Network.requestWillBeSent, unlike a WebSocket handshake. A message recorded after the logger lost that request state, because the target detached or the stream started before it attached, is still written but has no url.

--capture-downloads records files the browser downloads, which CDP never returns a response body for. With the flag the logger calls Browser.setDownloadBehavior once, on the browser connection rather than per target, so it covers every target of the default browser context including ones that attach later. The behavior is allowAndName with downloads/ inside the run directory as the download path, and eventsEnabled turns on the browser-wide Browser.downloadWillBegin and Browser.downloadProgress events the logger subscribes to. The deprecated Page equivalents are not used, so nothing is enabled per target and a download whose tab closes before it finishes is still recorded. Both are off without the flag: no download behavior change, no download tracking. When the run ends the logger sets the behavior back to default, which matters in attach mode where the browser outlives the logger.

Under allowAndName the browser names each saved file after the download GUID and nothing else. The file keeps that name: renaming it would race the browser's own writes, so downloads.ndjson is what maps the GUID back to a human-readable name. It contains one JSON object per finished download with the guid, the url and suggestedFilename the browser reported when the download began, the state, totalBytes and receivedBytes, the frameId that started it, the startedAt and timestamp times, and, for a completed download, the saved file path relative to the run directory plus its sha256. Downloads are content-addressed the same way response bodies are: the hash is over the file the browser wrote, computed in chunks so a large download does not have to be held in memory. The record is written after that hash, and shutdown waits for a hash still in flight so no finished download is lost to it.

A canceled download is recorded too, with its state and no file, so the loss is visible rather than silent. A completed download whose file the logger cannot read is recorded with an error instead of a hash, and also lands in errors.ndjson as a Browser.downloadProgress event. Chrome reports an interrupted download as canceled, and a resumed one completes afterwards: that appends a second line for the same guid, the completion superseding the earlier loss. Repeats of a state already recorded are ignored, so a guid never gets the same outcome twice.

--track-storage writes storage.ndjson, one JSON object per observed storage change, for the whole life of the run. It exists because the end-of-run snapshot below structurally loses data: it can only read the target sessions still attached when the run ends, so the web storage and IndexedDB of every origin whose tab was closed earlier is never captured at all. Following the changes as they happen is what closes that gap.

Each line has an area of "localStorage", "sessionStorage", or "indexedDB", and a change:

  • "baseline" is what an area already held when the logger first reached it. Chrome only reports what changes after a domain is enabled, so without this a consumer folding the file forward would start from an assumed-empty area. Web storage is baselined once per session and origin. IndexedDB tracking is held by one session per origin, so it is baselined once per owning session: when that session detaches, another target still open on the origin takes the tracking over and a fresh set of "baseline" lines follows. A "baseline" line therefore replaces what a consumer holds for that area rather than only ever appearing first.
  • "added", "updated", "removed", and "cleared" are the DOMStorage events, carrying key, newValue, and oldValue as Chrome reported them. A cleared area names no key.
  • "updated" on indexedDB is a Storage.indexedDBContentUpdated event. Chrome names only the store that changed, so the logger reads that one store back and records its databaseName, objectStoreName, entries, and hasMore. A read CDP refused sets error and leaves entries empty, so a failed read is not the same line as a store that was emptied.

Change-driven reads of one object store are serialized, and at most one further read is queued behind the one in flight, so a burst of updates to one store costs two reads rather than one per update and their pages cannot land out of order. The baseline reads do not go through that queue, and nothing is ordered against anything else: a change event can land before the baseline read of the same area finishes, which is what the change field and the timestamps are for.

--track-storage and --snapshot-storage are independent, and using both is the useful combination: the stream says how the session got to where it is, and the snapshot is the consolidated final state.

--snapshot-storage writes storage-snapshot.json once, at the end of the run, before the browser and the CDP connection are closed. It is the answer to a site that renders from data already in IndexedDB: a value on screen may never appear on the network again, so a later capture of the same page looks empty even though the page looks identical. It is also what makes a stored session replayable without logging in again.

It is one JSON file rather than another NDJSON file because it is a single point-in-time document, not a stream of events: one run produces exactly one, and the whole thing is written at once with the run's other writers already holding everything they will hold.

It contains:

  • cookies: the result of one Storage.getCookies call on the browser connection. No browserContextId is passed, which means the default browser context, so this is that context's whole cookie jar and not only the origins the run visited. That is deliberate: a stored session usually needs cookies of an API host that was never navigated to and therefore has no target of its own. In launch mode the jar is the dedicated profile's; in attach mode it is the jar of the browser you attached to.
  • origins: one entry per security origin, with localStorage and sessionStorage from DOMStorage.getDOMStorageItems, and databases from IndexedDB.requestDatabaseNames, IndexedDB.requestDatabase, and IndexedDB.requestData.

The origins are the http(s) origins of the page and iframe targets the run was actually attached to, resolved from the live target list at shutdown rather than from the URL each target had when it attached, which for a page is usually about:blank. Origins are not guessed from request URLs: a third-party API host has no target to ask on, and its web storage is not reachable from the pages the run observed. Each origin is read on its own target session, which is also what makes sessionStorage belong to the tab it came from. An origin open in several tabs is read once, so only the first of those tabs contributes its sessionStorage. Every read addresses storage by security origin, which Chrome resolves to a first-party storage key, so partitioned third-party storage is not what comes back. Worker targets are skipped, along with about:, chrome:, devtools:, and extension targets, none of which have web storage the storage domains can address by origin.

IndexedDB values are recorded exactly as CDP hands them over, which is a Runtime.RemoteObject per key, primary key, and value. Primitives carry their value directly. Objects and arrays carry only CDP's bounded preview, with its own overflow flag, plus className and description. They are not deeply materialized, and they will not round-trip back into a database. Deep values would require resolving the live object handle through the Runtime domain, which this tool does not use, so the handle is dropped instead and never written to the file. Treat IndexedDB contents in the snapshot as an inspectable record of what was stored, not as an exact copy of it.

The snapshot is bounded so shutdown cannot hang or run out of memory on a large database: at most 20 origins, 20 databases per origin, 20 object stores per database, 500 entries per object store, 5000 entries in total, and a 15 second deadline for the whole snapshot. Every cap that drops something sets truncated on the file, an object store that had more sets its own hasMore, an origin that had more databases records truncatedDatabases, and a database that had more object stores records truncatedObjectStores, so a short read is visible rather than looking like empty storage. There is no paging past the first page of an object store.

A snapshot is best effort. Any storage call the browser refuses is recorded in errors.ndjson under its CDP method name and leaves the rest of the file intact, so one origin cannot lose the others. If the browser is already gone when the run ends, which is the normal launch-mode race when you close the window yourself, no snapshot is written and errors.ndjson records a Storage.snapshot event saying why. Neither case stops the rest of shutdown or the run summary.

--include and --exclude apply to response URLs only. They never gate websocket.ndjson, because a frame without a URL could not be matched anyway, and they never gate eventsource.ndjson either, even though its messages do carry a URL: that is a deliberate choice, because filtering part of a live stream would be more confusing than not filtering it at all. They do not gate downloads.ndjson either: the browser has already written the file by then, so filtering the record would only hide it. They do not gate storage.ndjson, whose records are keyed by origin rather than by response URL. They do not gate storage-snapshot.json, which is keyed by origin rather than by response URL.

errors.ndjson contains per-request capture failures. Individual CDP failures do not stop the logger. WebSocket failures land here too: a frame the browser could not decode or send is recorded as a Network.webSocketFrameError event, since no frame line is written for it. As with frame records, it carries the socket URL only while the logger has a mapping for that socket.

Two of its events are about shutdown rather than about a request. Cdp.drainTimeout says how many event handlers were still running when the shutdown drain gave up on them. Such a handler still records if it finishes before the writers close, and is refused if it finishes after, so what it was holding may or may not be in the files. Plugin.shutdownTimeout says how many queued events a plugin never received because the plugin shutdown budget ran out first. Both are written while the writers are still open.

netlog.json is Chromium NetLog for network-stack debugging.

Shutdown

The run ends on Ctrl-C, on SIGTERM, on a shutdown IPC message, or when the browser closes. Shutdown then takes the storage snapshot, stops the plugins, closes the browser, closes the CDP connection, closes the writers, and prints the run summary, in that order. Every step has a deadline of its own, but the whole sequence still takes as long as the browser and the plugins make it take.

A second Ctrl-C or SIGTERM during that sequence exits immediately with status 130 instead of waiting. Nothing after it runs: no summary is printed, a --capture-downloads browser keeps saving downloads into the run directory, a browser started by launch mode is left running, and the writes still in flight are aborted, so the last line of an NDJSON file can be a partial one. Writes are serialized per file, so at most one trailing line per file is truncated and every line before it is intact. The second signal is there for a browser that will not close; prefer letting the first shutdown finish.

Run Summary

When a run ends, the logger prints a summary to stdout so silent losses are visible without opening errors.ndjson:

summary responses=482 response_bytes=19203112 requests=37 request_bytes=8241
summary websocket_frames=126 eventsource_messages=18 downloads=2 redirects=54
summary storage_changes=37
summary snapshot_origins=3 cookies=41 items=27 databases=2 entries=88
summary errors=4
summary_errors host=example.test total=2 Network.getResponseBody=2
summary_errors host=cdn.example.test total=1 Network.loadingFailed=1
summary_errors host=plugin:json-api-mirror total=1 Plugin.onEvent=1

responses and requests count saved body files, and the byte totals are the bytes written for them. websocket_frames counts every frame recorded, sent and received together, and eventsource_messages counts every recorded SSE message. downloads counts every download record written with --capture-downloads, canceled ones included, because a lost download is exactly what the summary should surface; the record's state says which it was. redirects counts recorded redirect hops, which have no body of their own and would otherwise be invisible in the totals. storage_changes counts every line written to storage.ndjson, baselines included, so it says how much the run watched rather than how many areas it ended up holding. It has a line of its own because the line above is already as wide as a terminal holds, and unlike the summary snapshot_ line below it is printed for every run: a run without --track-storage prints storage_changes=0. The summary snapshot_ line is printed only when --snapshot-storage wrote a snapshot, and counts what landed in it: origins, cookies, localStorage and sessionStorage items together, IndexedDB databases, and object store entries. A run without the flag prints no summary snapshot_ line at all.

summary_writers is printed only when a record file lost something, so a healthy run does not have the line. Each name is one of the NDJSON files without its extension, and says which of two things happened to it. metadata=lost means at least one record never reached the file and the file kept recording afterwards. metadata=stopped means its last write failed, so the file recorded nothing after that point and the counts above cover records it never received. Each one also writes a line to stderr naming the file and the first error behind it. A record appended after the writers close is refused rather than written behind the summary that already reported the run; Cdp.drainTimeout in errors.ndjson is what says handlers were still running at that point.

One summary_errors line is printed per host with the errors.ndjson event counts behind it, ordered by failure count. Only the top 20 hosts get a line; the rest are collapsed into a final remainder line.

Plugin failures have no URL, so they are grouped under plugin:<id> instead of a host. Host unknown collects failures whose URL the browser never delivered, such as Network.loadingFailed for a request that started before the logger attached to its target. Bodies dropped by a size limit are counted as Network.getResponseBody.skipped so a size policy is not mistaken for a capture failure, including a streamed body that never reached that call. Error counts cover all observed traffic, while the saved-body counts only cover responses that passed --include and --exclude.

What Gets Saved

For completed responses, metadata includes request and response fields such as:

  • URL, method, request ID, loader ID, target/session identifiers
  • request headers and response headers
  • status, status text, MIME type, protocol, remote IP/port
  • cache/service-worker/prefetch flags where the browser provides them
  • encoded data length
  • response body path, byte length, SHA-256, and CDP base64Encoded
  • request body path, byte length, SHA-256, and source when available
  • redirect and redirectIndex for requests that went through a redirect chain
  • rawRequestHeaders, rawResponseHeaders, blockedCookies, exemptedCookies, and cookiePartitionKey when --capture-cookies is used
  • any capture error for body retrieval

Response bodies are saved exactly from CDP's body result:

  • base64Encoded: true is decoded and written as bytes.
  • base64Encoded: false is written as UTF-8 bytes.
  • A --stream-bodies body is written from the assembled chunks as they are, and always reports base64Encoded: true.

Request payloads are written as UTF-8 bytes from CDP strings. The logger first uses inline request.postData from Network.requestWillBeSent when present. If the browser only reports hasPostData, the logger tries Network.getRequestPostData. This is suitable for JSON, forms, GraphQL, and other text request bodies. It is not raw upload byte capture, and arbitrary non-UTF-8 uploads may not round-trip exactly.

CDP bodies are not raw wire bytes. For exact network-stack debugging, use the companion netlog.json.

Plugins

The plugin system is a core part of Kuebiko. It lets trusted local TypeScript or JavaScript modules react to completed captures in real time without duplicating the CDP logger. The logger saves request/response files and metadata first. Plugins then receive small immutable events containing metadata and relative file paths.

Plugins run in the logger process. They are not sandboxed third-party code. A bad plugin cannot mutate requests through the logger API, but it can still use normal local runtime APIs, CPU, and memory. Plugin failures and queue overflows are written to errors.ndjson; capture continues.

Create a config file:

import { defineConfig } from "kuebiko";

export default defineConfig({
	plugins: [
		{
			module: "./plugins/json-api-mirror.ts",
			enabled: true,
			timeoutMs: 5000,
			queueSize: 1000,
		},
	],
});

defineConfig provides TypeScript context and validates plugin config entries when the config module is evaluated.

Plugin module paths are resolved relative to the config file.

Example plugin:

import { mkdir } from "node:fs/promises";
import { dirname } from "node:path";

import type { LoggerPlugin } from "kuebiko";

export default {
	id: "json-api-mirror",
	name: "JSON API Mirror",
	version: "0.1.0",
	events: ["response.completed"],

	async setup(ctx) {
		await mkdir(ctx.pluginDirectory, { recursive: true });
	},

	async onEvent(event, ctx) {
		if (event.event !== "response.completed") return;
		if (!event.response.bodyFile) return;
		if (!event.response.mimeType?.includes("json")) return;

		const source = ctx.resolveRunPath(event.response.bodyFile);
		const requestIdPattern = /[^A-Za-z0-9._-]/gu;
		const safeRequestId = event.request.requestId.replace(requestIdPattern, "_");
		const output = ctx.resolvePluginPath(`${safeRequestId}.json`);

		await mkdir(dirname(output), { recursive: true });
		await Bun.write(output, Bun.file(source));
	},
} satisfies LoggerPlugin;

Run with plugins:

kuebiko --config C:\path\logger.config.ts --out <capture-dir>

Disable configured plugins for a run:

kuebiko --config C:\path\logger.config.ts `
  --no-plugins --out <capture-dir>

Supported plugin events are:

  • run.started
  • run.stopping
  • run.stopped
  • response.completed
  • websocket.frame.received
  • websocket.frame.sent
  • eventsource.message
  • download.completed
  • storage.snapshot
  • capture.error

storage.ndjson is the one record file with no hook event of its own. A storage change is a value, not a path, and the values are exactly the credentials storage.snapshot withholds by publishing only a path and totals. There is therefore nothing to publish for it that would keep hook events path-based, so nothing is published and --track-storage reaches plugins only through the file.

Hook events do not contain inline request or response bodies. Read saved files with ctx.resolveRunPath(event.response.bodyFile) or the request-body path when present. WebSocket frames and SSE messages have no saved body file of their own: websocket.frame.* carries the frame in event.frame and eventsource.message carries the message in event.message, exactly as written to the matching NDJSON file.

download.completed is published for a saved download only, with the record in event.download exactly as written to downloads.ndjson. Like every other hook event it is path-based: read the file with ctx.resolveRunPath(event.download.file). A canceled download, or a completed one whose file could not be read, publishes nothing, because there is no file to hand a plugin; both are still written to downloads.ndjson.

storage.snapshot is published once, at the end of a run that used --snapshot-storage, after the file is written. Like every other hook event it is path-based: event.file is the run-relative path, event.counts holds the same totals the run summary prints, and event.truncated says whether a cap or the deadline cut the read short. The snapshot contents are never inlined, because they are the most sensitive thing the tool writes. Read the file with ctx.resolveRunPath(event.file) if the plugin genuinely needs it.

Redirect hops publish response.completed like any other recorded response, so plugins see the whole chain. Such an event has response.redirect set to true and response.bodyFile unset. Plugins that only care about bodies already skip it through the usual event.response.bodyFile check; plugins that follow login flows can use response.redirect and response.redirectIndex instead.

Verify A Capture

After browsing, check the latest run directory:

state_home="${XDG_STATE_HOME:-$HOME/.local/state}"
base="${KUEBIKO_BASE_DIR:-$state_home/Kuebiko}"
capture="$(find "$base/captures" -mindepth 1 -maxdepth 1 -type d |
  sort |
  tail -1)"
printf '%s\n' "$capture"
find "$capture/bodies" -type f | wc -l
find "$capture/requests" -type f | wc -l
wc -c "$capture/metadata.ndjson" "$capture/netlog.json"

On Windows PowerShell:

$capture = Get-ChildItem "$env:LOCALAPPDATA\Kuebiko\captures" |
  Sort-Object LastWriteTime -Descending |
  Select-Object -First 1

$capture.FullName
(Get-ChildItem "$($capture.FullName)\bodies" -File -Recurse).Count
(Get-ChildItem "$($capture.FullName)\requests" -File -Recurse).Count
(Get-Item "$($capture.FullName)\metadata.ndjson").Length
(Get-Item "$($capture.FullName)\netlog.json").Length

You should see metadata.ndjson grow while the logger is running. Normal CDP misses are recorded in errors.ndjson.

Browser Modes

Use launch mode first. It is easier to get right because the logger owns the browser process and supplies the CDP, dedicated-profile, and NetLog flags.

Use attach mode only when you specifically need to connect to a browser you started yourself, such as an existing profile. That is harder in practice: the browser must already have been started with --remote-debugging-port, and Chromium-family browsers often reuse an existing profile process instead of applying new flags. You may need to fully close that profile first or use a separate profile directory.

Launch Mode

Launch mode starts the browser, owns the dedicated profile for that run, enables CDP, writes NetLog by default, starts capture, and closes the browser when the logger exits.

Use a browser command from PATH:

kuebiko --launch-browser --browser-command google-chrome

Or use an explicit browser executable:

kuebiko --launch-browser \
  --browser-path "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
kuebiko --launch-browser `
  --browser-path "C:\Program Files\Google\Chrome\Application\chrome.exe"

Use an explicit capture directory:

kuebiko --launch-browser --browser-command google-chrome \
  --out "$HOME/captures/manual-run"

Disable NetLog for a run:

kuebiko --launch-browser --browser-command google-chrome --no-netlog

The logger intentionally does not auto-discover browsers. Pass either --browser-command or --browser-path. Launch mode uses these browser flags by default:

--user-data-dir=<profile-dir>
--remote-debugging-address=127.0.0.1
--remote-debugging-port=<port>
--log-net-log=<capture-dir>\netlog.json
--net-log-capture-mode=Everything

The NetLog flags are omitted when --no-netlog is set.

Pass repeated --browser-arg=<arg> values only when your local browser environment requires them. For example, CI Chrome sometimes requires --browser-arg=--no-sandbox. These extra args are explicit and are not added by default.

Attach Mode

Attach mode is for externally launched browsers. Start the browser yourself with CDP enabled, then run:

kuebiko --cdp http://127.0.0.1:9222 --out <capture-dir>

Attach mode does not launch a browser or write NetLog by itself. It only connects to the CDP endpoint you provide. If you also need NetLog in attach mode, the browser must have been started with --log-net-log=<path> before the logger connects.

NetLog Warning

Chrome and other Chromium-family browsers may show this banner after startup:

You are using an unsupported command-line flag: --log-net-log=<path>. Stability
and security will suffer.

This is expected when NetLog is enabled from the command line. --log-net-log is the Chromium-documented startup flag for writing a NetLog file, but browser security warning UI can still flag diagnostic command-line switches as potentially dangerous.

The warning does not mean that NetLog failed or that the browser ignored the flag. Verify capture by checking that netlog.json exists and grows in the run folder. The warning is meaningful: NetLog captures sensitive network metadata, and --net-log-capture-mode=Everything can include more private debugging detail than the default browser behavior.

Debugger Paused Banner

Chromium-family browsers may otherwise show this banner when a page opens a popup or another window while CDP is attached:

Debugger paused in another tab, click to switch to that tab.

The logger does not enable the CDP Debugger or Fetch domains and does not intentionally pause scripts. The browser can still create an auto-attached popup, iframe, or worker target in a debugger-waiting state. For each attached inspectable target, the logger enables Network first, then calls Runtime.runIfWaitingForDebugger for that target session.

This Runtime call only tells a target to continue if the browser has it waiting for a debugger. It is not request interception, script injection, browser automation, or general Runtime evaluation.

CLI

kuebiko [options]

Options:
  --cdp <url>              CDP endpoint (default: http://127.0.0.1:9222)
  --out <capture-dir>      Capture directory
  --verbose                Print verbose status logs
  --include <regex>        Only persist matching response URLs
  --exclude <regex>        Do not persist matching response URLs
  --max-body-bytes <num>   Skip body retrieval above encoded byte length
  --capture-cookies        Also record raw wire headers, including live cookies
  --capture-downloads      Save browser downloads into the run directory
  --snapshot-storage       Snapshot cookies and web storage when the run ends
  --stream-bodies          Assemble bodies from Network.streamResourceContent
                           (experimental)
  --track-storage          Record web storage and IndexedDB changes to
                           storage.ndjson
  --label <label>          Label recorded in run.json
  --note <text>            Free-form note recorded in run.json
  --config <path>          TS/JS logger config with plugin modules
  --no-plugins             Disable plugin loading from --config
  --launch-browser         Launch and own a local CDP browser process
  --browser-command <cmd>  Browser command for --launch-browser
  --browser-path <path>    Browser executable path for --launch-browser
  --browser-profile <dir>  Browser profile directory for --launch-browser
  --browser-arg <arg>      Extra browser arg for --launch-browser
  --cdp-port <port>        Local CDP port for --launch-browser
  --no-netlog              Disable netlog.json in --launch-browser mode
  --help, -h               Show help
  --version, -v            Show version

Releases

Successful pushes to main are released automatically from conventional commit messages after CI passes. GitHub Releases contain archives for Windows x64, Linux x64, and macOS arm64.

If --out is omitted, the logger creates a new timestamped capture folder under the platform default capture root:

  • Windows: %LOCALAPPDATA%\Kuebiko\captures
  • macOS: ~/Library/Application Support/Kuebiko/captures
  • Linux: ${XDG_STATE_HOME:-~/.local/state}/Kuebiko/captures

Set KUEBIKO_BASE_DIR to override the base directory on any platform.

Default Folders

  • Windows: %LOCALAPPDATA%\Kuebiko
  • macOS: ~/Library/Application Support/Kuebiko
  • Linux: ${XDG_STATE_HOME:-~/.local/state}/Kuebiko

Each base directory contains browser-profile, captures, and plugin output created by configured plugins. Nothing is intentionally written under %TEMP%, /tmp, or WSL /tmp during normal capture.

Development

mise install
mise run test
E2E_BROWSER_PATH=/path/to/chrome-or-chromium mise run test-e2e
mise run check --lint
mise run compile

mise run compile builds the configured Bun executables into dist/. Use mise run compile --target <target> to build only one target.

Known Limitations

  • CDP may fail to return bodies for downloads, streaming responses, very large responses, cached responses, service-worker cases, or after navigation races, because the default path reads them back from Chromium's retained resource buffer. --stream-bodies recovers many of these by assembling the body from Network.streamResourceContent instead, and falls back to the buffer path when a stream cannot be enabled, fails, or assembles nothing. It never streams a Server-Sent Events response, which eventsource.ndjson already covers.
  • Downloads are never recovered as response bodies. --capture-downloads saves the file the browser wrote instead, which is a different path with different costs: it changes where downloads go and what they are named on disk, and it allows downloads Chrome would otherwise have blocked or prompted about.
  • --capture-downloads covers the default browser context only. Browser.setDownloadBehavior is sent without a browserContextId, which means the default context, so downloads started in an incognito window or in any context created through Target.createBrowserContext keep the browser's normal behavior: they are not redirected into the run directory, emit no download events, and get no record.
  • Redirect hops are recorded as metadata only. Their status, location, and set-cookie headers are saved, but no response body exists to save, and a hop the logger did not observe from its start is skipped. redirectIndex counts from the first hop the logger saw, so a chain joined mid-flight is numbered relative to attach time and may have no index-0 record.
  • A redirect hop keeps only the request payload Chrome inlined in the event. When Chrome reports hasPostData without the data, usually for a large body, the hop records a request body error instead: refetching it would return the payload of the request that is already in flight.
  • CDP may not expose every request payload. Network.getRequestPostData can fail after navigation races and does not include uploaded files for multipart form data.
  • --max-body-bytes compares against CDP encodedDataLength; it is a skip guard, not a perfect final decoded-size predictor. With --stream-bodies it compares the same encoded bytes, summed per chunk, but it can only skip the write, not the transfer: an enabled stream cannot be turned off again.
  • WebSocket messages are not normal HTTP response bodies. Both directions are written to websocket.ndjson as individual frames, not reassembled messages, and a frame on a socket the logger never saw open has no url.
  • Server-Sent Events are captured as individual messages in eventsource.ndjson, not as a response body. The connection normally stays open for the life of the page, so CDP usually has no body to return for it, and a message recorded after the logger lost the stream's request state has no url.
  • --snapshot-storage records IndexedDB values as CDP's bounded previews, not as deep copies. Objects and arrays keep only the properties the preview carries, with an overflow flag when there were more. Restoring a database from the snapshot is not a supported use; reading what was stored is.
  • The snapshot covers the origins of attached page and iframe targets only, and it is a single point in time at shutdown. Storage of an origin whose tab was closed earlier in the run is not in it, and neither is storage of an origin the run only made requests to. A tab that navigated across origins contributes only the origin it ended on. --track-storage is the answer to this one: what a closed tab's storage held, and what it did on the way, is in storage.ndjson even though the snapshot cannot see it.
  • An origin open in several tabs is read once, on the first session seen for it. localStorage and IndexedDB are shared by those tabs anyway, but sessionStorage is not: the other tabs' sessionStorage is silently absent from the file, with nothing marking it as missing.
  • Every read addresses storage by security origin, which Chrome resolves to a first-party storage key. Partitioned third-party storage is therefore not covered: a cross-site iframe's partitioned localStorage, sessionStorage, and IndexedDB are not what comes back, even when that frame attached and was read on its own session.
  • --track-storage follows a target only once it reaches an http(s) origin, which it learns from Target.targetInfoChanged. Storage a page wrote before that event is in the baseline read, but a target Chrome never reports a URL change for is never followed.
  • --track-storage reads an object store with a flat cap of 500 entries and reports hasMore when the store holds more; it does not page through the rest. The snapshot's cap is the same 500, but shrinks as its own 5000-entry budget is spent, so the two do not always read the same number of entries.
  • Its baseline reads at most 20 databases per origin and 20 object stores per database, and a store past either cap reaches no record at all. That cap reports itself in errors.ndjson rather than in storage.ndjson, because there is no record for a store that was never read. Unlike the snapshot, the stream has no 20-origin cap, no run-wide entry budget, and no deadline.
  • --track-storage records no cookies. Cookie changes are observable on the wire with --capture-cookies and as a final jar with --snapshot-storage; CDP has no cookie-change event to follow, so a cookie set by script and never sent is in neither stream.
  • The snapshot caps what it reads: 20 origins, 20 databases per origin, 20 object stores per database, 500 entries per object store, 5000 entries in total, and a 15 second deadline. It never pages past the first page of an object store. Everything dropped is flagged with truncated, hasMore, truncatedDatabases, or truncatedObjectStores rather than being silently missing.
  • This tool does not parse, analyze, classify, or display responses.
  • Plugins are trusted local code running in the logger process. They are useful for local real-time consumers, but they are not sandboxed.
  • Logs can contain sensitive data, including private API requests, private API responses, submitted form content, and cookies-adjacent content. Treat every capture directory as secret.
  • With --capture-cookies, metadata.ndjson holds the raw Cookie and Set-Cookie headers exactly as they went over the wire. Such a capture contains live session cookies in plaintext and is enough to resume the sessions it recorded. The flag is off by default; turn it on only when the raw headers, or the blockedCookies reasons behind a broken session, are what you are debugging, and delete those runs as soon as you are done.
  • With --snapshot-storage, storage-snapshot.json is the single most sensitive file this tool writes. It holds the browser's live cookie jar for the default browser context together with the localStorage, sessionStorage, and IndexedDB contents of the origins the run visited. That is a whole stored session, not just the traffic of one: tokens, refresh tokens, and cached private data in one file, enough to resume the sessions it captured without logging in again. It is worse than --capture-cookies, which only holds the cookies that actually crossed the wire during the run. In attach mode the cookies are your own browser's, not a throwaway profile's. The flag is off by default; turn it on only when a stored session or IndexedDB-cached data is what you are after, keep such runs off shared and synced storage, and delete them as soon as you are done.
  • With --track-storage, storage.ndjson is as sensitive as that snapshot and in one way worse: it holds every value web storage took during the run, not only the last one. A token the site rotated is in the file along with each token that replaced it, and a value the page later deleted is still there. It holds no cookies, which is the one thing the snapshot has that it does not. The flag is off by default; treat a run that used it exactly as you would one that used --snapshot-storage.
  • Store capture directories somewhere private, avoid syncing them to cloud drives by default, delete runs you no longer need, and share only minimized redacted samples.

About

Passive Chrome network capture with raw bodies, NetLog, and a first-class local plugin system.

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Used by

Contributors

Languages