Incremental state step 2: central recompute and diff

[codebot-robot]

Implements Step 2 of docs/incremental-state.md ("Central recompute and diff").

Key Changes

1. All inputs live in State:
   • GatewayClass and GatewayAddresses are recorded into State.
   • Stamped with a monotonically increasing revision counter (uint64) only on actual input mutations (spec, relevant metadata labels, secret data, configmap data, service ports/IP, etc.).
   • Replaced len(gw.Status.Addresses) > 0 in CompileModel so status computation never reads the status we wrote.
2. Pure Output Computation (ComputeOutputs):
   • Pure function computing desired statuses per Gateway, HTTPRoute, ListenerSet, BackendTLSPolicy, GatewayClass, proxy config, certificates, and resolved gateways.
   • Status merging (preserving LastTransitionTime and foreign controller entries) happens purely at write time.
3. Central Recompute & Diff:
   • State centrally recomputes and diffs desired statuses against previous outputs, emitting change events on per-kind channels.
   • Background coalescing/debouncing worker batches rapid bursts of watch events.
   • Updates the proxy and calls OnGatewaysUpdate centrally once per recompute when outputs change.
4. Simplified Reconcilers:
   • Reconcilers now strictly record inputs into State and write merged status via WatchesRawSource(source.Channel(...)).
   • Deleted all 19 cross-object EnqueueRequestsFromMapFunc mapping functions across all reconcilers.
5. Tests & Benchmarks:
   • Added unit tests in pkg/state/state_test.go verifying diffing and dependency propagation without mapping functions (e.g. Namespace label change, ReferenceGrant deletion, ConfigMap corruption).
   • Added BenchmarkState_Recompute in pkg/state/state_benchmark_test.go testing 100 Gateways and 1000 HTTPRoutes (~6.7ms per recompute).
   • All unit tests and conformance/e2e suites pass.

Fixes #628

[justinsb]

Thanks for this, it's a big step forward. The structure is what docs/incremental-state.md describes: every input lives in State with a revision counter, a pure ComputeOutputs produces everything, status is merged at write time, one central diff feeds per-kind channels, and all 19 mapping functions are gone. The benchmark looks good too.

Before we merge, please fix the following.

Correctness bugs

1. Status gets written on other controllers' Gateways. GatewayReconciler no longer checks gc.Spec.ControllerName. For a Gateway owned by another controller, GetDesiredGatewayStatus misses, the fallback calls ComputeDesiredGatewayStatus(gw, nil, addrs), and the result is written to that Gateway. It also calls AddressProvider for those Gateways, which would provision Deployments for them once Fast-path provisioning: per-Gateway LoadBalancer Services and per-Gateway ports #624 lands. Please skip Gateways whose class isn't ours (and remove them from State), and add a test for this case.
2. No GatewayClasses means every Gateway is compiled. In CompileModel, managedGatewayClasses stays nil when no GatewayClasses have been recorded, and a nil map means "compile every Gateway". It should be an empty set, so nothing is compiled until we know which classes are ours.
3. Events are silently dropped. sendEvent does a non-blocking send into a 1024-entry buffer, so with many objects or a burst of changes, events beyond that are lost and those statuses are never written. Please don't drop. Either send blocking from outside the lock, or (preferred) use a small custom source.Source that adds requests directly to the controller's workqueue, which deduplicates and never drops.

Design problems

4. "No output until synced" doesn't hold. RegisterReconcilers calls SetSynced(false), but State.Start sets synced = true as soon as the manager starts it. At that point the informer caches have synced, but the reconcilers haven't loaded those objects into State yet, so the first recomputes work from a partial world. Statuses flap (for example, a route is marked as having no matching parent before its Gateway has been recorded), and the proxy gets a partial config, so a restarting data-plane pod (Fast-path provisioning: per-Gateway LoadBalancer Services and per-Gateway ports #624) would briefly serve 404s.
   Suggestion: feed State straight from informer event handlers (mgr.GetCache().GetInformer(...) plus AddEventHandler), and mark it synced only when every handler registration reports HasSynced(). That also means Secret, Namespace, ConfigMap, Service and ReferenceGrant don't need reconcilers at all. The remaining reconcilers then only write status.
5. Reconcilers write stale status or recompile everything. A reconciler records its input and then immediately reads previousOutputs, which predates that input because recompute runs about 10 ms later. So it can write stale status (for example, an old observedGeneration). When no snapshot exists yet, it falls back to a synchronous full CompileModel, which is O(n²) during initial sync and works against the central design. Please remove the fallback and write only from computed outputs; the recompute already emits an event for new keys. Ideally, also skip the write until the outputs' revision is at least the revision of the input that was just recorded.
6. Callbacks run while holding the lock. proxy.UpdateConfig, UpdateCertificates and onGatewaysUpdate are called while holding s.mu. onGatewaysUpdate is an embedder hook: snigateway talks to the frontend over the network inside it, which blocks every upsert, and the hook would deadlock if it ever called a State getter. Please capture what you need under the lock and call these after unlocking (still serialized, from the recompute loop).

Cleanup

• UpsertHTTPRoute still compiles each route when it's recorded. That's wasted work under the lock, since CompileModel recompiles every route. If the returned ValidationCondition is still needed, it should come from the outputs.
• Unrelated comments were deleted: the HTTPS+TLS port-sharing TODOs in areProtocolsCompatible, the precedence-order comments in GetHTTPRoutes and GetListenerSets, and others. Please restore them.

Rebase

• This needs a rebase. The conflicts in compiled.go, conditions.go and conditions_test.go come from Keep route status.parents in a deterministic sorted order #627 (deterministic status.parents ordering). Please keep Keep route status.parents in a deterministic sorted order #627's behaviour in MergeHTTPRouteStatus: our entries sorted, other controllers' entries preserved after ours, and an order-only difference not counted as an update. A test that goes through the new merge path would confirm it.
• We plan to merge this before Fast-path provisioning: per-Gateway LoadBalancer Services and per-Gateway ports #624, which will then rebase. Its Deployment readiness signal fits naturally as another State input, next to the addresses.

[justinsb]

Thanks, this revision addresses nearly everything. The no-drop EventSource, informer-fed State with HasSynced gating, removing the fallback, running callbacks outside the lock, the foreign-Gateway fix and preserving #627's ordering all look good. A few things remain:

Bugs

1. Inputs should come from one place. Informer handlers now write into State, but reconcilers do too:

   • GatewayReconciler upserts Gateways and GatewayClasses, and deletes Gateways that are out of scope or not ours.
   • HTTPRouteReconciler and ListenerSetReconciler Get parent Gateways and ListenerSets (and their classes) and upsert them.

   This causes two problems:

   • GatewayFilter isn't really applied. The informer handler upserts every Gateway, so out-of-scope Gateways are compiled into the proxy config at startup and whenever they change, until a reconcile removes them again. With Fast-path provisioning: per-Gateway LoadBalancer Services and per-Gateway ports #624's per-Gateway data-plane pods, this means each pod would serve other Gateways' routes.
   • The writers fight. The reconciler deletes a Gateway, the next informer update re-adds it, and every round bumps the revision and triggers a recompute.

   Please make informer handlers the only source of Kubernetes inputs. Apply GatewayFilter there or inside State (for example, a State input filter checked in UpsertGateway). Reconcilers should only write status, plus record provider addresses, which are an external input. Unit tests that relied on reconcilers upserting parents should feed State directly. Please add a test that an out-of-scope Gateway never appears in the proxy config.

2. Events can be lost at startup. EventSource.Enqueue does nothing if no controller has called Start on it yet. State.Start and the controllers start concurrently, so the first recompute can emit before the queues exist, and those keys are lost. Please remember pending keys in EventSource (a set) and flush them into the queue in Start.

Cleanup

3. Please remove EventSource.testCh. It's a 10k-entry test hook written on every production enqueue. Tests can call Start with a fake or real workqueue and read the queue instead.
4. recomputeLocked duplicates Recompute, and it's still the path used before Start, where it calls the proxy and onGatewaysUpdate while holding the lock. Please keep a single recompute path that releases the lock before running callbacks. If it can be reached from more than one goroutine, serialize recomputes so an older config is never applied after a newer one.
5. GatewayState.BuildInternalState (in gateway.go) fakes a GatewayClass so that CompileModel accepts the Gateway. If this legacy path has no callers outside tests, please delete it; otherwise, pass the real class.

Items 1 and 2 should be fixed in this PR, since #624 will build on it; 3 to 5 are small.