Report prerequisite exposures and add an opt-in deferred tracking buffer

CHANGELOG.md

@@ -11,9 +11,23 @@
   * Malformed bandit definitions or leaves degrade to the rule's aggregate weights instead of raising during evaluation; reported `variationWeights` always match the weights bucketing actually used, and a single, total validity rule governs every weight vector: `getBucketRanges` normalizes vectors with negative, non-finite, boolean, non-numeric, or float-overflowing entries (not just wrong length/sum) to equal weights — never raising, even on arbitrary-precision integers, so bucket ranges can never be inverted, and bandit leaf/aggregate/override propensities always describe the vector actually used. Bandit identifiers get the same treatment: a leaf with a non-integer `leafId` is malformed (aggregate-weights fallback), and a non-integer `banditVersion` is omitted, so exposure metadata never carries invalid attribution ids into bandit training. A rule that pairs `contextualBanditRef` with explicit `ranges` buckets on the ranges (unchanged, matching the JS SDK) but drops the bandit metadata, since leaf propensities cannot describe a ranges-governed assignment.
   * Contextual bandit exposure metadata survives remote evaluation: `rule.tracks` results from the proxy keep `leafId`, `variationWeights`, and `banditVersion` when replayed through the tracking callback, held to the same validity rules as locally evaluated exposures (invalid identifiers or weight vectors are dropped, not forwarded).
   * Async evaluations that perform I/O (remote eval or a sticky bucket service) freeze every mutable evaluation input — attributes, groups, overrides, forced variations and features, nested containers included — before their first await, so mutating a `UserContext` mid-flight cannot change leaf routing, remote payloads, or forced assignments. Plain CDN evaluations never yield and skip the copy entirely; deferred callbacks get fire-time snapshots in both clients. `preload_remote_eval` takes the same call-time snapshot, so mutating the context after preloading can no longer cache one attribute state's response under another's key via the stale-while-revalidate background refresh.
+* Deferred tracking: buffer experiment exposures for forwarding to a client SDK (which fires them via `setDeferredTrackingCalls` + `fireDeferredTrackingCalls`). Opt-in and independent of `on_experiment_viewed` — the buffer records first, the callback still fires. Entries use the JS SDK's `TrackingData` shape, deduped per unique assignment and JSON round-tripped at exposure time, so buffered payloads are always `json.dumps`-ready — an exposure carrying a non-JSON value (e.g. a `datetime` attribute) is dropped and logged, never the batch. Firing the forwarded `user` context on the receiving side requires JS SDK 1.7.0+.
+  * Sync client: `GrowthBook(defer_tracking=True)` with `get_deferred_tracking_calls()` / `clear_deferred_tracking_calls()`.
+  * Async client: pass a per-request `TrackingBuffer` to `eval_feature` / `run` / `is_on` / `is_off` / `get_feature_value` and read it with `buffer.get_calls()`; the caller owns the buffer, so concurrent requests never mix exposures.
+
+### Performance
+
+* Telemetry plumbing overhead on evaluations with no callbacks configured, measured with `tests/scripts/benchmark_eval_overhead.py` (500k sequential in-memory evals, best of 5): the default-value path costs ~0.13 microseconds more per evaluation than 3.0 (~1.29 -> ~1.42 us/eval, ~10%); a realistic experiment-rule path is within ~1% (~6.3 us/eval). The residual is the `EvaluationContext` carrying the callback fields — the mechanism that fixes silently-dropped prerequisite exposures — and is accepted deliberately; usage bookkeeping and the deprecated-kwarg shim are already skipped entirely when unused.
 
 ### Bug Fixes
 
+* Experiments that decide a prerequisite feature now report telemetry: previously `eval_prereqs` dropped every callback, so those exposures were silently lost. Fixed structurally by carrying `tracking_cb` / `feature_usage_cb` / `callback_subscription` on the `EvaluationContext` (like the JS SDK's global context) so nested evaluations inherit them. As a result:
+  * `on_experiment_viewed` fires for prerequisite experiment assignments (including when a gate ultimately fails, matching the JS SDK).
+  * `on_feature_usage` fires for every feature evaluated — prerequisites included — once per key per evaluation (previously it fired only for the top-level key, on every call, with no dedupe).
+  * The sync client's `subscribe()` callbacks see prerequisite experiments, and its `get_all_results()` includes them. The async client's subscriptions still fire only from `run()` — like the JS multi-user client, it has no per-user assignment change-detection, so per-eval firing would repeat every subscriber callback on every request.
+  * `core.eval_feature` / `core.run_experiment` now read callbacks from the `EvaluationContext`; the old `tracking_cb` / `callback_subscription` keyword arguments still work but are deprecated: they are installed on the context only for the duration of that call (prerequisites inherit them) and the previous context fields are restored afterwards, matching their pre-3.1 invocation-scoped behavior.
+* The tracking dedupe key is now a field tuple instead of a concatenated string, so distinct exposures cannot collide on field boundaries regardless of what the values contain.
+* `Experiment.to_dict()` no longer coerces an explicit `coverage` of `0` to `1`.
 * `savedGroups` from a feature refresh were applied to the evaluation context one refresh late in the synchronous client.
 * The built-in tracking plugin now sends the exposure-time user context attributes with experiment events (previously async client events had none).
 

README.md

@@ -498,6 +498,46 @@ is_enabled = gb.is_on("my-feature")  # -> Tracked automatically
 
 The tracking plugin provides batching, error handling, and works alongside your existing tracking callbacks. See the [plugin documentation](https://docs.growthbook.io/lib/python#tracking-plugins) for more details.
 
+#### Deferred Tracking
+
+When your server evaluates features but the analytics context lives on the client (server-side rendering, an API returning evaluated flags), buffer the exposures instead of tracking them server-side and forward them to a client SDK, which fires them through its own tracking callback.
+
+On the sync client, opt in with `defer_tracking=True`:
+
+```python
+gb = GrowthBook(attributes={"id": "user-1"}, features=features, defer_tracking=True)
+gb.is_on("my-feature")
+
+# JSON-ready entries in the JS SDK's TrackingData shape:
+# [{"experiment": {...}, "result": {...}, "user": {"attributes": {...}, "url": ""}}]
+calls = gb.get_deferred_tracking_calls()
+# ...send json.dumps(calls) to the client...
+gb.clear_deferred_tracking_calls()
+```
+
+The buffer lives as long as the instance: when one instance serves many users, read and clear it per request (or use one instance per request) so exposures are never forwarded to the wrong client.
+
+The receiving JavaScript SDK hydrates and fires them:
+
+```js
+gb.setDeferredTrackingCalls(callsFromServer);
+gb.fireDeferredTrackingCalls();
+```
+
+Forwarding the exposure-time `user` context to the tracking callback requires JS SDK 1.7.0+; older SDKs fire the exposures without the user argument.
+
+On the async client, pass a per-request `TrackingBuffer` — the explicit buffer is the opt-in, and because the caller owns it, concurrent requests never mix exposures:
+
+```python
+from growthbook import TrackingBuffer
+
+buffer = TrackingBuffer()
+await client.eval_feature("my-feature", user_context, tracking_buffer=buffer)
+calls = buffer.get_calls()
+```
+
+Buffering is independent of `on_experiment_viewed`: when both are configured, the buffer records first and the callback still fires. Entries are deduped per unique assignment and snapshotted at exposure time, and include prerequisite and passthrough exposures.
+
 ## Using Features
 
 There are 3 main methods for interacting with features.

growthbook/__init__.py

@@ -13,6 +13,7 @@
     JSONValue,
     Options,
     Result,
+    TrackingBuffer,
     TrackingCallback,
     UserContext,
 )
@@ -62,6 +63,7 @@
     "Options",
     "UserContext",
     "FeatureRefreshStrategy",
+    "TrackingBuffer",
     # Data model
     "Experiment",
     "Result",

growthbook/codegen.py

@@ -32,7 +32,7 @@
 
 from typing import {typing_imports}
 
-from growthbook import FeatureResult, GrowthBook, GrowthBookClient, UserContext
+from growthbook import FeatureResult, GrowthBook, GrowthBookClient, TrackingBuffer, UserContext
 
 '''
 
@@ -151,7 +151,13 @@ def _client_class(feature_types: Dict[str, str], is_async: bool) -> str:
     name = "TypedGrowthBookClient" if is_async else "TypedGrowthBook"
     base = "GrowthBookClient" if is_async else "GrowthBook"
     prefix = "async " if is_async else ""
-    ctx = ", user_context: UserContext" if is_async else ""
+    # The async overrides must stay signature-compatible with GrowthBookClient,
+    # whose eval methods take a keyword-only per-request tracking_buffer.
+    ctx = (
+        ", user_context: UserContext, *, tracking_buffer: Optional[TrackingBuffer] = None"
+        if is_async
+        else ""
+    )
     lines = [
         f"class {name}({base}):",
         f'    """{base} with strictly-typed feature keys (checker-only, runtime no-op)."""',
@@ -256,8 +262,8 @@ def generate(payload: Dict[str, Any], payload_format: str = "auto") -> str:
     if any("List[" in t for t in feature_types.values()):
         typing_imports.append("List")
     typing_imports.append("Literal")
-    if typed_count:
-        typing_imports.append("Optional")
+    # Always needed: the async overrides' tracking_buffer is Optional.
+    typing_imports.append("Optional")
     if any("Union[" in t for t in feature_types.values()):
         typing_imports.append("Union")
     # Overloads are emitted whenever get_feature_value or eval_feature has

growthbook/common_types.py

@@ -1,5 +1,9 @@
 #!/usr/bin/env python
 
+import json
+import logging
+import threading
+from copy import deepcopy
 from dataclasses import dataclass, field, replace
 from typing import (
     TYPE_CHECKING,
@@ -31,6 +35,8 @@
 # runtime, so this is cycle-free.
 from .plugins.base import PluginLike
 
+logger = logging.getLogger("growthbook")
+
 # Generic feature/experiment value type. Deliberately unbounded: a JSONValue
 # bound would reject TypedDict/dataclass-shaped fallbacks (see JS SDK issue #1729,
 # where the equivalent bound was shipped and then reverted).
@@ -159,7 +165,9 @@ def to_dict(self) -> Dict[str, Any]:
             "variations": self.variations,
             "weights": self.weights,
             "active": self.active,
-            "coverage": self.coverage or 1,
+            # None means "no coverage set" and reads as full coverage, but an
+            # explicit 0 must survive serialization (`or` would coerce it to 1).
+            "coverage": self.coverage if self.coverage is not None else 1,
             "condition": self.condition,
             "namespace": self.namespace,
             "force": self.force,
@@ -622,6 +630,23 @@ def snapshot_user_context(user: "UserContext") -> "UserContext":
     )
 
 
+# Identity of one exposure: (hashAttribute, hashValue, experiment key,
+# variation id). Same fields as JS getExperimentDedupeKey / Go
+# TrackingData.DedupeKey, but a tuple instead of a joined string so field
+# values containing a would-be delimiter can never make two distinct
+# exposures collide.
+TrackingDedupeKey = Tuple[str, str, str, str]
+
+
+def tracking_dedupe_key(experiment: "Experiment[Any]", result: "Result[Any]") -> TrackingDedupeKey:
+    return (
+        result.hashAttribute,
+        str(result.hashValue),
+        experiment.key,
+        str(result.variationId),
+    )
+
+
 def tracking_user_context(user: "UserContext") -> "UserContext":
     """Exposure-time snapshot of a user context for tracking and
     feature-usage callbacks (JS SDK: getTrackingUserContext).
@@ -632,6 +657,66 @@ def tracking_user_context(user: "UserContext") -> "UserContext":
     return replace(user, attributes=snapshot_attributes(user.attributes))
 
 
+class TrackingBuffer:
+    """Collector for deferred tracking calls: experiment exposures buffered
+    during evaluation so a server can forward them to a client SDK (which
+    fires them through its own tracking callback — JS setDeferredTrackingCalls
+    / fireDeferredTrackingCalls). Buffering is independent of the tracking
+    callback: when both are wired, the buffer is written first and the
+    callback still fires (Go SDK semantics).
+
+    Entries use the JS SDK's TrackingData shape —
+    ``{"experiment": {...}, "result": {...}, "user": {"attributes", "url"}}``
+    — JSON round-tripped at record time, so nothing the caller mutates
+    afterwards can reach the buffer and every stored entry is guaranteed
+    json.dumps-ready (an exposure carrying a non-JSON value is dropped and
+    logged, never the batch). Deduped by tracking_dedupe_key, first exposure
+    wins, insertion-ordered. Thread-safe: the async client may share one
+    buffer across concurrent evaluations of the same request."""
+
+    def __init__(self) -> None:
+        self._lock = threading.Lock()
+        self._calls: Dict[TrackingDedupeKey, Dict[str, Any]] = {}
+
+    def record(self, experiment: "Experiment[Any]", result: "Result[Any]", user: "UserContext") -> None:
+        key = tracking_dedupe_key(experiment, result)
+        if key in self._calls:
+            # Unlocked read is GIL-safe; the setdefault below closes the race
+            # (first exposure wins). This only skips building the entry twice.
+            return
+        try:
+            # JSON round-trip at record time (Go SDK: detachTrackingData).
+            # It snapshots — to_dict() aliases nested mutables (variations,
+            # condition, attribute values), so a shallow entry would let later
+            # caller mutations reach the buffer — AND it guarantees the
+            # json.dumps-ready contract per entry, so one exposure carrying a
+            # non-JSON value (datetime, NaN, custom object) is dropped here
+            # with a log instead of poisoning the whole forwarded batch at the
+            # caller's json.dumps. Telemetry must never break evaluation.
+            entry = json.loads(json.dumps({
+                "experiment": experiment.to_dict(),
+                "result": result.to_dict(),
+                "user": {"attributes": user.attributes, "url": user.url},
+            }, allow_nan=False))
+        except Exception:
+            logger.exception("Dropped deferred tracking call that cannot be JSON-serialized")
+            return
+        with self._lock:
+            self._calls.setdefault(key, entry)
+
+    def get_calls(self) -> List[Dict[str, Any]]:
+        """Buffered exposures as detached copies (mutating them cannot corrupt
+        the buffer, which get_calls leaves intact for a later read), ready for
+        json.dumps. Non-destructive; pair with clear()."""
+        with self._lock:
+            snapshot = list(self._calls.values())
+        return deepcopy(snapshot)
+
+    def clear(self) -> None:
+        with self._lock:
+            self._calls.clear()
+
+
 @dataclass
 class Options:
     url: Optional[str] = None
@@ -690,6 +775,24 @@ class EvaluationContext:
     # directly, letting the async client schedule persistence off the event loop.
     # None (the default) preserves the sync client's direct-call behavior.
     save_sticky_bucket_doc: Optional[Callable[[Dict[str, Any]], None]] = None
+    # Client callbacks carried on the context rather than threaded through as
+    # function parameters, so nested evaluations — prerequisites especially —
+    # inherit them automatically (JS SDK: ctx.global.trackingCallback). A
+    # recursive call site that forgets a parameter silently drops telemetry;
+    # a shared context field cannot be forgotten.
+    tracking_cb: Optional[Callable[["Experiment[Any]", "Result[Any]", UserContext], None]] = None
+    callback_subscription: Optional[Callable[["Experiment[Any]", "Result[Any]"], None]] = None
+    feature_usage_cb: Optional[Callable[[str, "FeatureResult[Any]", UserContext], None]] = None
+    # Feature keys already reported through feature_usage_cb during this
+    # evaluation — one usage event per key per top-level eval call, however
+    # many times a prerequisite chain re-visits it. Allocated lazily by
+    # _report_feature_usage: contexts are built on every evaluation, and evals
+    # with no usage callback must not pay for the set.
+    reported_features: Optional[Set[str]] = None
+    # When set, every exposure produced by this evaluation (prerequisites and
+    # passthrough included) is recorded here, before and independent of
+    # tracking_cb.
+    tracking_buffer: Optional[TrackingBuffer] = None
 
 
 # ---------------------------------------------------------------------------