diff --git a/.agents/skills/implement-conformance-test/journal/central-recompute-and-diff.md b/.agents/skills/implement-conformance-test/journal/central-recompute-and-diff.md new file mode 100644 index 0000000..29a0184 --- /dev/null +++ b/.agents/skills/implement-conformance-test/journal/central-recompute-and-diff.md @@ -0,0 +1,69 @@ +# Conformance & Architecture Journal: Incremental State Step 2 - Central Recompute and Diff + +## 1. Overview +- **Objective**: Implement Step 2 of `docs/incremental-state.md` ("Central recompute and diff"). +- **Key Goals**: + 1. Record all inputs (including `GatewayClass` and `GatewayAddresses`) into `State`. + 2. Feed `State` straight from informer event handlers, marking state synced only after all informer registrations report `HasSynced() == true`. + 3. Pure computation of all outputs via `ComputeOutputs(inputs ModelInputs) *Outputs` without reading previously written statuses. + 4. Recompute centrally with debouncing/coalescing in `State`, diff desired status against previous outputs, and push events directly into controller workqueues via custom `EventSource` (never dropping events under load). + 5. Reconcilers only write status from computed outputs, skipping Gateways whose GatewayClass is not managed by us. + 6. Execute callbacks (`UpdateConfig`, `UpdateCertificates`, `onGatewaysUpdate`) outside `s.mu` to prevent deadlocks. + 7. Deterministic `status.parents` ordering (#627) preserved with no unnecessary API writes for order-only differences. + 8. Delete unneeded non-status reconcilers (Secret, Namespace, ConfigMap, Service, ReferenceGrant) and all 19 cross-object mapping functions. + 9. Add unit tests and recomputation benchmark. + +## 2. Issue / Architectural Analysis +- **Previous Architecture**: + - Each controller manually fetched dependency objects or re-ran partial compilation in its own reconciler. + - 19 hand-written `EnqueueRequestsFromMapFunc` watches existed across Gateway, HTTPRoute, ListenerSet, BackendTLSPolicy, and ConfigMap reconcilers. + - Reconcilers invoked `updateProxy()` independently, causing duplicate compilations and proxy updates. + - `CompileModel` checked `len(gw.Status.Addresses) > 0`, making computation depend on status writes rather than pure recorded inputs. +- **Root Cause & Design Solutions**: + - Centralized input tracking with a monotonic revision counter stamped on genuine input modifications (ignoring status-only echoes). + - Provided clean separation: `ComputeOutputs` purely calculates desired statuses and proxy configs; `State.Recompute()` diffs and notifies; reconcilers merge status (preserving `LastTransitionTime` and foreign controller entries) and write to API server. + - Feed `State` directly from informer event handlers (`AddEventHandler`), eliminating redundant reconcilers. + - Custom `EventSource` sends reconcile requests directly to controller workqueues, avoiding channel drops. + +## 3. Implementation Details +- **Informer Event Handlers & Sync (`pkg/controller/utils.go`, `pkg/state/state.go`)**: + - Added informer event handlers for all 10 input types (`GatewayClass`, `Gateway`, `ListenerSet`, `HTTPRoute`, `BackendTLSPolicy`, `Service`, `Secret`, `ConfigMap`, `Namespace`, `ReferenceGrant`). + - `State.Start` waits for all registrations to report `HasSynced() == true` before enabling output computation (`SetSynced(true)`), preventing partial-world flapping and startup 404s. +- **Lock-Free Callbacks (`pkg/state/state.go`)**: + - Captured proxy configuration and gateway slice under `s.mu`, released `s.mu`, and executed `proxy.UpdateConfig`, `proxy.UpdateCertificates`, and `onGatewaysUpdate` outside the lock. +- **EventSource Workqueue Delivery (`pkg/state/state.go`)**: + - Implemented `EventSource` implementing `source.TypedSource[reconcile.Request]` that enqueues directly to controller rate-limiting workqueues. +- **GatewayClass Filtering & Gateway Reconciler (`pkg/controller/gateway_controller.go`)**: + - `GatewayReconciler` verifies `gc.Spec.ControllerName == r.ControllerName`. If not ours, deletes from `State` and exits without calling `AddressProvider` or writing status. + - `CompileModel` initializes `managedGatewayClasses` to an empty map so Gateways are not compiled until their GatewayClass is known. +- **Deterministic Route Parents Ordering (`pkg/state/compiled.go`, `pkg/state/conditions.go`)**: + - Kept #627's deterministic parentRef sorting and other controller preservation in `UpdateRouteParentStatuses` / `MergeHTTPRouteStatus`. +- **Testing & Benchmarks**: + - Added unit tests in `pkg/state/state_test.go` and `pkg/controller/controller_test.go`. + - Added `BenchmarkState_Recompute` in `pkg/state/state_benchmark_test.go` testing 100 Gateways and 1000 HTTPRoutes (~2.7ms per full recompute). + +## 4. Key Files Modified / Added / Deleted +- `pkg/state/state.go` +- `pkg/state/compiled.go` +- `pkg/state/conditions.go` +- `pkg/state/gateway.go` +- `pkg/state/referencegrant.go` +- `pkg/state/state_test.go` +- `pkg/state/state_benchmark_test.go` +- `pkg/controller/gateway_controller.go` +- `pkg/controller/httproute_controller.go` +- `pkg/controller/listenerset_controller.go` +- `pkg/controller/backendtlspolicy_controller.go` +- `pkg/controller/utils.go` +- `pkg/controller/controller_test.go` +- (Deleted: `configmap_controller.go`, `namespace_controller.go`, `referencegrant_controller.go`, `secret_controller.go`, `service_controller.go`) +- `.agents/skills/implement-conformance-test/journal/central-recompute-and-diff.md` + +## 5. Validation & Results +- **Unit Tests & Benchmark**: + - All unit tests across all packages pass (`pkg/controller`, `pkg/gari`, `pkg/provisioning/singlepod`, `pkg/proxy`, `pkg/state`, `snigateway`). + - `BenchmarkState_Recompute` executed at ~2.7ms for 100 Gateways and 1000 HTTPRoutes. +- **Linting & Code Generation**: + - `ap generate` and `ap lint` passed cleanly with 0 errors/warnings. +- **Conformance & E2E**: + - All Gateway API conformance tests and E2E suites passed under `ap e2e`. diff --git a/pkg/controller/backendtlspolicy_controller.go b/pkg/controller/backendtlspolicy_controller.go index 8387269..4128887 100644 --- a/pkg/controller/backendtlspolicy_controller.go +++ b/pkg/controller/backendtlspolicy_controller.go @@ -16,20 +16,12 @@ package controller import ( "context" - "encoding/pem" "fmt" - "reflect" - "github.com/gke-labs/gateway-api-reference-implementation/pkg/proxy" "github.com/gke-labs/gateway-api-reference-implementation/pkg/state" - corev1 "k8s.io/api/core/v1" - apierrors "k8s.io/apimachinery/pkg/api/errors" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" - "k8s.io/apimachinery/pkg/types" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/controller-runtime/pkg/handler" "sigs.k8s.io/controller-runtime/pkg/log" gatewayv1 "sigs.k8s.io/gateway-api/apis/v1" @@ -37,284 +29,46 @@ import ( type BackendTLSPolicyReconciler struct { client.Client - Scheme *runtime.Scheme - State *state.State - Proxy *proxy.Proxy - ControllerName string - OnGatewaysUpdate func([]*gatewayv1.Gateway) + Scheme *runtime.Scheme + State *state.State + ControllerName string } func (r *BackendTLSPolicyReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { l := log.FromContext(ctx) - controllerName := r.ControllerName - policy := &gatewayv1.BackendTLSPolicy{} if err := r.Get(ctx, req.NamespacedName, policy); err != nil { - if apierrors.IsNotFound(err) { - r.State.DeleteBackendTLSPolicy(req.NamespacedName) - r.updateProxy() - } return ctrl.Result{}, client.IgnoreNotFound(err) } - // Find all Gateways that use this policy (via Services referenced in HTTPRoutes) - gateways := r.State.GetGateways() - routes := r.State.GetHTTPRoutes() - allPolicies := r.State.GetBackendTLSPolicies() - - // Conflict resolution - isConflicted := false - var conflictingPolicy string - for _, targetRef := range policy.Spec.TargetRefs { - if string(targetRef.Group) != "" && string(targetRef.Group) != "gateway.networking.k8s.io" { - continue - } - if string(targetRef.Kind) != "Service" { - continue - } - - targetSvcNamespace := policy.Namespace - targetSvcName := string(targetRef.Name) - - for _, p := range allPolicies { - if p.Namespace == policy.Namespace && p.Name == policy.Name { - continue - } - - for _, t := range p.Spec.TargetRefs { - if (string(t.Group) == "" || string(t.Group) == "gateway.networking.k8s.io") && string(t.Kind) == "Service" { - if p.Namespace == targetSvcNamespace && string(t.Name) == targetSvcName { - // Found another policy targeting the same service - // Check if it's older - if p.CreationTimestamp.Time.Before(policy.CreationTimestamp.Time) { - isConflicted = true - conflictingPolicy = fmt.Sprintf("%s/%s", p.Namespace, p.Name) - break - } - if p.CreationTimestamp.Time.Equal(policy.CreationTimestamp.Time) { - if p.Namespace < policy.Namespace || (p.Namespace == policy.Namespace && p.Name < policy.Name) { - isConflicted = true - conflictingPolicy = fmt.Sprintf("%s/%s", p.Namespace, p.Name) - break - } - } - } - } - } - if isConflicted { - break - } - } - if isConflicted { - break - } - } - - // Validate CA certificates - var caCerts [][]byte - var unresolvedRefs []string - for _, caRef := range policy.Spec.Validation.CACertificateRefs { - if string(caRef.Group) == "" && string(caRef.Kind) == "ConfigMap" { - cm := &corev1.ConfigMap{} - if err := r.Get(ctx, types.NamespacedName{Namespace: policy.Namespace, Name: string(caRef.Name)}, cm); err != nil { - unresolvedRefs = append(unresolvedRefs, string(caRef.Name)) - } else { - var data []byte - if d, ok := cm.Data["ca.crt"]; ok { - data = []byte(d) - } else if d, ok := cm.BinaryData["ca.crt"]; ok { - data = d - } - - if len(data) == 0 { - unresolvedRefs = append(unresolvedRefs, string(caRef.Name)) - } else { - // Verify it's a valid PEM block - block, _ := pem.Decode(data) - if block == nil || block.Type != "CERTIFICATE" { - unresolvedRefs = append(unresolvedRefs, string(caRef.Name)) - } else { - caCerts = append(caCerts, data) - } - } - } - } else { - unresolvedRefs = append(unresolvedRefs, string(caRef.Name)) - } - } - - acceptedStatus := metav1.ConditionTrue - acceptedReason := gatewayv1.PolicyReasonAccepted - acceptedMessage := "Policy accepted" - - resolvedRefsStatus := metav1.ConditionTrue - resolvedRefsReason := gatewayv1.BackendTLSPolicyReasonResolvedRefs - resolvedRefsMessage := "All references resolved" - - if len(unresolvedRefs) > 0 { - acceptedStatus = metav1.ConditionFalse - acceptedReason = "NoValidCACertificate" - acceptedMessage = fmt.Sprintf("Unresolved or invalid CA certificate references: %v", unresolvedRefs) - - resolvedRefsStatus = metav1.ConditionFalse - resolvedRefsReason = "InvalidCACertificateRef" - resolvedRefsMessage = fmt.Sprintf("Unresolved or invalid CA certificate references: %v", unresolvedRefs) - } - - if isConflicted { - acceptedStatus = metav1.ConditionFalse - acceptedReason = gatewayv1.PolicyReasonConflicted - acceptedMessage = fmt.Sprintf("Conflicted with older policy: %s", conflictingPolicy) - } - - var ancestors []gatewayv1.PolicyAncestorStatus - - for _, gw := range gateways { - usesPolicy := false - for _, route := range routes { - if route.MatchesGateway(gw.Gateway, controllerName) { - for _, rule := range route.Spec.Rules { - for _, backendRef := range rule.BackendRefs { - if string(state.ValueOf(backendRef.Kind)) == "Service" || state.ValueOf(backendRef.Kind) == "" { - backendSvcKey := ResolveNamespacedName(backendRef.Namespace, backendRef.Name, route) - - for _, targetRef := range policy.Spec.TargetRefs { - if backendSvcKey.Namespace == policy.Namespace && backendSvcKey.Name == string(targetRef.Name) { - usesPolicy = true - break - } - } - } - if usesPolicy { - break - } - } - if usesPolicy { - break - } - } - } - if usesPolicy { - break - } - } - - if usesPolicy { - ancestors = append(ancestors, gatewayv1.PolicyAncestorStatus{ - AncestorRef: gatewayv1.ParentReference{ - Group: state.Ptr(gatewayv1.Group("gateway.networking.k8s.io")), - Kind: state.Ptr(gatewayv1.Kind("Gateway")), - Namespace: state.Ptr(gatewayv1.Namespace(gw.Namespace)), - Name: gatewayv1.ObjectName(gw.Name), - }, - ControllerName: gatewayv1.GatewayController(controllerName), - Conditions: []metav1.Condition{ - { - Type: string(gatewayv1.PolicyConditionAccepted), - Status: acceptedStatus, - ObservedGeneration: policy.Generation, - LastTransitionTime: metav1.Now(), - Reason: string(acceptedReason), - Message: acceptedMessage, - }, - { - Type: string(gatewayv1.BackendTLSPolicyConditionResolvedRefs), - Status: resolvedRefsStatus, - ObservedGeneration: policy.Generation, - LastTransitionTime: metav1.Now(), - Reason: string(resolvedRefsReason), - Message: resolvedRefsMessage, - }, - }, - }) + if r.State != nil { + desired, ok := r.State.GetDesiredBackendTLSPolicyStatus(req.NamespacedName) + if !ok { + return ctrl.Result{}, nil } - } - updated := false - if len(policy.Status.Ancestors) != len(ancestors) { - updated = true - } else { - for i := range ancestors { - if !reflect.DeepEqual(policy.Status.Ancestors[i].AncestorRef, ancestors[i].AncestorRef) || - string(policy.Status.Ancestors[i].ControllerName) != string(ancestors[i].ControllerName) || - len(policy.Status.Ancestors[i].Conditions) != len(ancestors[i].Conditions) { - updated = true - break - } - for j := range ancestors[i].Conditions { - matched := false - for k := range policy.Status.Ancestors[i].Conditions { - if policy.Status.Ancestors[i].Conditions[k].Type == ancestors[i].Conditions[j].Type { - if policy.Status.Ancestors[i].Conditions[k].Status == ancestors[i].Conditions[j].Status && - policy.Status.Ancestors[i].Conditions[k].ObservedGeneration == ancestors[i].Conditions[j].ObservedGeneration && - policy.Status.Ancestors[i].Conditions[k].Reason == ancestors[i].Conditions[j].Reason && - policy.Status.Ancestors[i].Conditions[k].Message == ancestors[i].Conditions[j].Message { - matched = true - } - break - } - } - if !matched { - updated = true - break - } - } - if updated { - break + if state.MergeBackendTLSPolicyStatus(&policy.Status, desired, gatewayv1.GatewayController(r.ControllerName)) { + if err := r.Status().Update(ctx, policy); err != nil { + l.Error(err, "unable to update BackendTLSPolicy status") + return ctrl.Result{}, err } } } - if updated { - policy.Status.Ancestors = ancestors - if err := r.Status().Update(ctx, policy); err != nil { - l.Error(err, "unable to update BackendTLSPolicy status") - return ctrl.Result{}, err - } - } - - r.State.UpsertBackendTLSPolicy(policy) - _ = caCerts // keep for now - r.updateProxy() - - l.Info("Updated BackendTLSPolicy status and proxy") - return ctrl.Result{}, nil } -func (r *BackendTLSPolicyReconciler) updateProxy() { - updateProxy(r.State, r.Proxy, r.ControllerName, r.OnGatewaysUpdate) -} - func (r *BackendTLSPolicyReconciler) SetupWithManager(mgr ctrl.Manager) error { if r.ControllerName == "" { return fmt.Errorf("ControllerName is required") } - return ctrl.NewControllerManagedBy(mgr). - For(&gatewayv1.BackendTLSPolicy{}). - Watches(&corev1.ConfigMap{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - cm := obj.(*corev1.ConfigMap) - // Find all BackendTLSPolicies that reference this ConfigMap - var list gatewayv1.BackendTLSPolicyList - if err := mgr.GetClient().List(ctx, &list, client.InNamespace(cm.Namespace)); err != nil { - return nil - } - var requests []ctrl.Request - for _, policy := range list.Items { - for _, caRef := range policy.Spec.Validation.CACertificateRefs { - if string(caRef.Group) == "" && string(caRef.Kind) == "ConfigMap" && string(caRef.Name) == cm.Name { - requests = append(requests, ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: policy.Namespace, - Name: policy.Name, - }, - }) - break - } - } - } - return requests - })). - Complete(r) + bldr := ctrl.NewControllerManagedBy(mgr). + For(&gatewayv1.BackendTLSPolicy{}) + + if r.State != nil { + bldr.WatchesRawSource(r.State.BackendTLSPolicySource()) + } + + return bldr.Complete(r) } diff --git a/pkg/controller/configmap_controller.go b/pkg/controller/configmap_controller.go deleted file mode 100644 index 456adb5..0000000 --- a/pkg/controller/configmap_controller.go +++ /dev/null @@ -1,80 +0,0 @@ -// Copyright 2026 Google LLC -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -package controller - -import ( - "context" - "fmt" - - "github.com/gke-labs/gateway-api-reference-implementation/pkg/proxy" - "github.com/gke-labs/gateway-api-reference-implementation/pkg/state" - corev1 "k8s.io/api/core/v1" - apierrors "k8s.io/apimachinery/pkg/api/errors" - "k8s.io/apimachinery/pkg/runtime" - ctrl "sigs.k8s.io/controller-runtime" - "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/controller-runtime/pkg/handler" - "sigs.k8s.io/controller-runtime/pkg/log" - - gatewayv1 "sigs.k8s.io/gateway-api/apis/v1" -) - -type ConfigMapReconciler struct { - client.Client - Scheme *runtime.Scheme - State *state.State - Proxy *proxy.Proxy - ControllerName string - OnGatewaysUpdate func([]*gatewayv1.Gateway) -} - -func (r *ConfigMapReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - l := log.FromContext(ctx) - - cm := &corev1.ConfigMap{} - if err := r.Get(ctx, req.NamespacedName, cm); err != nil { - if apierrors.IsNotFound(err) { - r.State.DeleteConfigMap(req.NamespacedName) - r.updateProxy() - } - return ctrl.Result{}, client.IgnoreNotFound(err) - } - - r.State.UpsertConfigMap(cm) - r.updateProxy() - - l.Info("Updated ConfigMap in state") - - return ctrl.Result{}, nil -} - -func (r *ConfigMapReconciler) updateProxy() { - updateProxy(r.State, r.Proxy, r.ControllerName, r.OnGatewaysUpdate) -} - -func (r *ConfigMapReconciler) SetupWithManager(mgr ctrl.Manager) error { - if r.ControllerName == "" { - return fmt.Errorf("ControllerName is required") - } - return ctrl.NewControllerManagedBy(mgr). - For(&corev1.ConfigMap{}). - Watches(&gatewayv1.BackendTLSPolicy{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - // This is actually backwards, we want to reconcile BackendTLSPolicy when ConfigMap changes. - // But since we are in ConfigMapReconciler, we want to trigger BackendTLSPolicy reconciles. - // Actually, it's easier to just make BackendTLSPolicy watch ConfigMaps. - return nil - })). - Complete(r) -} diff --git a/pkg/controller/controller_test.go b/pkg/controller/controller_test.go index 174b217..2d6ef98 100644 --- a/pkg/controller/controller_test.go +++ b/pkg/controller/controller_test.go @@ -25,7 +25,6 @@ import ( "testing" "time" - "github.com/gke-labs/gateway-api-reference-implementation/pkg/proxy" "github.com/gke-labs/gateway-api-reference-implementation/pkg/state" corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -108,8 +107,6 @@ func TestHTTPRouteReconciler_CustomControllerName(t *testing.T) { _ = gatewayv1.AddToScheme(scheme) st := state.NewState() - p := proxy.NewProxy() - customController := "custom.domain/controller" route := &gatewayv1.HTTPRoute{ @@ -128,6 +125,10 @@ func TestHTTPRouteReconciler_CustomControllerName(t *testing.T) { }, } + st.SetControllerName(customController) + st.UpsertHTTPRoute(route) + st.Recompute() + client := fake.NewClientBuilder(). WithScheme(scheme). WithObjects(route). @@ -138,7 +139,6 @@ func TestHTTPRouteReconciler_CustomControllerName(t *testing.T) { Client: client, Scheme: scheme, State: st, - Proxy: p, ControllerName: customController, } @@ -158,109 +158,78 @@ func TestHTTPRouteReconciler_CustomControllerName(t *testing.T) { } } -func TestServiceAndSecretAndConfigMapReconcilers(t *testing.T) { +type trackingAddressProvider struct { + called bool +} + +func (p *trackingAddressProvider) GatewayAddresses(ctx context.Context, gw *gatewayv1.Gateway) ([]gatewayv1.GatewayStatusAddress, error) { + p.called = true + return []gatewayv1.GatewayStatusAddress{{Value: "192.0.2.1"}}, nil +} + +func TestGatewayReconciler_SkipsOtherControllersGateways(t *testing.T) { scheme := runtime.NewScheme() _ = clientgoscheme.AddToScheme(scheme) _ = gatewayv1.AddToScheme(scheme) st := state.NewState() - p := proxy.NewProxy() + st.SetControllerName("my.domain/controller") - hookCalled := false - onUpdate := func(gws []*gatewayv1.Gateway) { - hookCalled = true + otherGC := &gatewayv1.GatewayClass{ + ObjectMeta: metav1.ObjectMeta{Name: "other-class"}, + Spec: gatewayv1.GatewayClassSpec{ControllerName: "other.domain/controller"}, } - - svc := &corev1.Service{ - ObjectMeta: metav1.ObjectMeta{ - Name: "my-svc", - Namespace: "default", - }, - } - secret := &corev1.Secret{ + otherGW := &gatewayv1.Gateway{ ObjectMeta: metav1.ObjectMeta{ - Name: "my-secret", + Name: "other-gw", Namespace: "default", }, - } - cm := &corev1.ConfigMap{ - ObjectMeta: metav1.ObjectMeta{ - Name: "my-cm", - Namespace: "default", - }, - } - ns := &corev1.Namespace{ - ObjectMeta: metav1.ObjectMeta{ - Name: "my-ns", - Labels: map[string]string{"env": "test"}, + Spec: gatewayv1.GatewaySpec{ + GatewayClassName: "other-class", + Listeners: []gatewayv1.Listener{ + {Name: "http", Port: 80, Protocol: gatewayv1.HTTPProtocolType}, + }, }, } client := fake.NewClientBuilder(). WithScheme(scheme). - WithObjects(svc, secret, cm, ns). + WithObjects(otherGC, otherGW). + WithStatusSubresource(otherGW). Build() - svcReconciler := &ServiceReconciler{ - Client: client, - Scheme: scheme, - State: st, - Proxy: p, - ControllerName: "test-controller", - OnGatewaysUpdate: onUpdate, - } - secretReconciler := &SecretReconciler{ - Client: client, - Scheme: scheme, - State: st, - Proxy: p, - ControllerName: "test-controller", - OnGatewaysUpdate: onUpdate, - } - cmReconciler := &ConfigMapReconciler{ - Client: client, - Scheme: scheme, - State: st, - Proxy: p, - ControllerName: "test-controller", - OnGatewaysUpdate: onUpdate, - } - nsReconciler := &NamespaceReconciler{ - Client: client, - Scheme: scheme, - State: st, - Proxy: p, - ControllerName: "test-controller", - OnGatewaysUpdate: onUpdate, + provider := &trackingAddressProvider{} + r := &GatewayReconciler{ + Client: client, + Scheme: scheme, + State: st, + ControllerName: "my.domain/controller", + AddressProvider: provider, } ctx := t.Context() - - hookCalled = false - _, err := svcReconciler.Reconcile(ctx, ctrl.Request{NamespacedName: types.NamespacedName{Namespace: "default", Name: "my-svc"}}) - if err != nil || !hookCalled { - t.Fatalf("ServiceReconciler failed or hook not called: err=%v, hookCalled=%v", err, hookCalled) + _, err := r.Reconcile(ctx, ctrl.Request{NamespacedName: types.NamespacedName{Namespace: "default", Name: "other-gw"}}) + if err != nil { + t.Fatalf("unexpected error reconciling other controller gateway: %v", err) } - hookCalled = false - _, err = secretReconciler.Reconcile(ctx, ctrl.Request{NamespacedName: types.NamespacedName{Namespace: "default", Name: "my-secret"}}) - if err != nil || !hookCalled { - t.Fatalf("SecretReconciler failed or hook not called: err=%v, hookCalled=%v", err, hookCalled) + if provider.called { + t.Errorf("expected AddressProvider NOT to be called for other controller's Gateway") } - hookCalled = false - _, err = cmReconciler.Reconcile(ctx, ctrl.Request{NamespacedName: types.NamespacedName{Namespace: "default", Name: "my-cm"}}) - if err != nil || !hookCalled { - t.Fatalf("ConfigMapReconciler failed or hook not called: err=%v, hookCalled=%v", err, hookCalled) + var fetchedGW gatewayv1.Gateway + if err := client.Get(ctx, types.NamespacedName{Namespace: "default", Name: "other-gw"}, &fetchedGW); err != nil { + t.Fatalf("failed to fetch gateway: %v", err) } - - hookCalled = false - _, err = nsReconciler.Reconcile(ctx, ctrl.Request{NamespacedName: types.NamespacedName{Name: "my-ns"}}) - if err != nil || !hookCalled { - t.Fatalf("NamespaceReconciler failed or hook not called: err=%v, hookCalled=%v", err, hookCalled) + if len(fetchedGW.Status.Conditions) != 0 { + t.Errorf("expected no conditions written to other controller's Gateway, got %v", fetchedGW.Status.Conditions) } - if len(st.GetNamespaces()) != 1 || st.GetNamespaces()["my-ns"] == nil { - t.Fatalf("expected namespace in state, got %v", st.GetNamespaces()) + if len(fetchedGW.Status.Addresses) != 0 { + t.Errorf("expected no addresses written to other controller's Gateway, got %v", fetchedGW.Status.Addresses) + } + + if len(st.GetGateways()) != 0 { + t.Errorf("expected other controller's Gateway not to be in State, got %v", st.GetGateways()) } } @@ -286,24 +255,8 @@ func TestReconcilerSetupWithManager_RequiresControllerName(t *testing.T) { setupErr: (&BackendTLSPolicyReconciler{}).SetupWithManager(nil), }, { - name: "ServiceReconciler", - setupErr: (&ServiceReconciler{}).SetupWithManager(nil), - }, - { - name: "ConfigMapReconciler", - setupErr: (&ConfigMapReconciler{}).SetupWithManager(nil), - }, - { - name: "SecretReconciler", - setupErr: (&SecretReconciler{}).SetupWithManager(nil), - }, - { - name: "ReferenceGrantReconciler", - setupErr: (&ReferenceGrantReconciler{}).SetupWithManager(nil), - }, - { - name: "NamespaceReconciler", - setupErr: (&NamespaceReconciler{}).SetupWithManager(nil), + name: "ListenerSetReconciler", + setupErr: (&ListenerSetReconciler{}).SetupWithManager(nil), }, } @@ -351,7 +304,6 @@ func TestGatewayReconciler_TLSReferenceGrant(t *testing.T) { _ = gatewayv1beta1.AddToScheme(scheme) st := state.NewState() - p := proxy.NewProxy() certPEM, keyPEM := generateTestCertPEM(t) @@ -365,8 +317,6 @@ func TestGatewayReconciler_TLSReferenceGrant(t *testing.T) { corev1.TLSPrivateKeyKey: keyPEM, }, } - st.UpsertSecret(secret) - gwClass := &gatewayv1.GatewayClass{ ObjectMeta: metav1.ObjectMeta{ Name: "test-gc", @@ -401,6 +351,12 @@ func TestGatewayReconciler_TLSReferenceGrant(t *testing.T) { }, } + st.SetControllerName("test-controller") + st.UpsertSecret(secret) + st.UpsertGatewayClass(gwClass) + st.UpsertGateway(gw) + st.Recompute() + client := fake.NewClientBuilder(). WithScheme(scheme). WithObjects(gwClass, gw, secret). @@ -411,7 +367,6 @@ func TestGatewayReconciler_TLSReferenceGrant(t *testing.T) { Client: client, Scheme: scheme, State: st, - Proxy: p, ControllerName: "test-controller", } @@ -463,6 +418,7 @@ func TestGatewayReconciler_TLSReferenceGrant(t *testing.T) { }, } st.UpsertReferenceGrant(rg) + st.Recompute() // Reconcile again -> ResolvedRefs should be True _, err = r.Reconcile(ctx, ctrl.Request{NamespacedName: types.NamespacedName{Namespace: "gw-ns", Name: "test-gw"}}) @@ -570,9 +526,6 @@ func TestGatewayReconciler_InvalidParametersRef(t *testing.T) { _ = clientgoscheme.AddToScheme(scheme) _ = gatewayv1.AddToScheme(scheme) - st := state.NewState() - p := proxy.NewProxy() - gwClass := &gatewayv1.GatewayClass{ ObjectMeta: metav1.ObjectMeta{Name: "test-gc"}, Spec: gatewayv1.GatewayClassSpec{ControllerName: "test-controller"}, @@ -603,6 +556,12 @@ func TestGatewayReconciler_InvalidParametersRef(t *testing.T) { }, } + st := state.NewState() + st.SetControllerName("test-controller") + st.UpsertGatewayClass(gwClass) + st.UpsertGateway(gw) + st.Recompute() + client := fake.NewClientBuilder(). WithScheme(scheme). WithObjects(gwClass, gw). @@ -613,7 +572,6 @@ func TestGatewayReconciler_InvalidParametersRef(t *testing.T) { Client: client, Scheme: scheme, State: st, - Proxy: p, ControllerName: "test-controller", } @@ -651,7 +609,6 @@ func TestGatewayReconciler_UnsupportedProtocol(t *testing.T) { _ = gatewayv1.AddToScheme(scheme) st := state.NewState() - p := proxy.NewProxy() gwClass := &gatewayv1.GatewayClass{ ObjectMeta: metav1.ObjectMeta{Name: "test-gc"}, @@ -701,6 +658,12 @@ func TestGatewayReconciler_UnsupportedProtocol(t *testing.T) { }, } + st.SetControllerName("test-controller") + st.UpsertGatewayClass(gwClass) + st.UpsertGateway(gwOnlyUnsupported) + st.UpsertGateway(gwMixed) + st.Recompute() + client := fake.NewClientBuilder(). WithScheme(scheme). WithObjects(gwClass, gwOnlyUnsupported, gwMixed). @@ -711,7 +674,6 @@ func TestGatewayReconciler_UnsupportedProtocol(t *testing.T) { Client: client, Scheme: scheme, State: st, - Proxy: p, ControllerName: "test-controller", } @@ -784,7 +746,6 @@ func TestGatewayReconciler_InvalidRouteKind(t *testing.T) { _ = gatewayv1.AddToScheme(scheme) st := state.NewState() - p := proxy.NewProxy() gwClass := &gatewayv1.GatewayClass{ ObjectMeta: metav1.ObjectMeta{Name: "test-gc"}, @@ -840,6 +801,12 @@ func TestGatewayReconciler_InvalidRouteKind(t *testing.T) { }, } + st.SetControllerName("test-controller") + st.UpsertGatewayClass(gwClass) + st.UpsertGateway(gwOnlyInvalid) + st.UpsertGateway(gwSupportedAndInvalid) + st.Recompute() + client := fake.NewClientBuilder(). WithScheme(scheme). WithObjects(gwClass, gwOnlyInvalid, gwSupportedAndInvalid). @@ -850,7 +817,6 @@ func TestGatewayReconciler_InvalidRouteKind(t *testing.T) { Client: client, Scheme: scheme, State: st, - Proxy: p, ControllerName: "test-controller", } @@ -933,7 +899,6 @@ func TestGatewayReconciler_AddressProvider(t *testing.T) { _ = gatewayv1.AddToScheme(scheme) st := state.NewState() - p := proxy.NewProxy() gwClass := &gatewayv1.GatewayClass{ ObjectMeta: metav1.ObjectMeta{ @@ -961,6 +926,11 @@ func TestGatewayReconciler_AddressProvider(t *testing.T) { }, } + st.SetControllerName("test-controller") + st.UpsertGatewayClass(gwClass) + st.UpsertGateway(gw) + st.Recompute() + client := fake.NewClientBuilder(). WithScheme(scheme). WithObjects(gwClass, gw). @@ -984,7 +954,6 @@ func TestGatewayReconciler_AddressProvider(t *testing.T) { Client: client, Scheme: scheme, State: st, - Proxy: p, ControllerName: "test-controller", AddressProvider: provider, } @@ -994,6 +963,13 @@ func TestGatewayReconciler_AddressProvider(t *testing.T) { t.Fatalf("unexpected error reconciling gateway: %v", err) } + // In the real system, State.SetGatewayAddresses triggers a background recompute which emits a Gateway change event to reconcile and write status. + st.Recompute() + _, err = r.Reconcile(ctx, req) + if err != nil { + t.Fatalf("unexpected error on second reconcile: %v", err) + } + var reconciledGW gatewayv1.Gateway if err := client.Get(ctx, req.NamespacedName, &reconciledGW); err != nil { t.Fatalf("failed to get gateway: %v", err) @@ -1022,6 +998,11 @@ func TestGatewayReconciler_AddressProvider(t *testing.T) { if err != nil { t.Fatalf("unexpected error re-reconciling gateway: %v", err) } + st.Recompute() + _, err = r.Reconcile(ctx, req) + if err != nil { + t.Fatalf("unexpected error on second reconcile: %v", err) + } if err := client.Get(ctx, req.NamespacedName, &reconciledGW); err != nil { t.Fatalf("failed to get gateway: %v", err) } @@ -1035,6 +1016,11 @@ func TestGatewayReconciler_AddressProvider(t *testing.T) { if err != nil { t.Fatalf("unexpected error reconciling gateway: %v", err) } + st.Recompute() + _, err = r.Reconcile(ctx, req) + if err != nil { + t.Fatalf("unexpected error on second reconcile: %v", err) + } if err := client.Get(ctx, req.NamespacedName, &reconciledGW); err != nil { t.Fatalf("failed to get gateway: %v", err) @@ -1050,6 +1036,8 @@ func TestGatewayReconciler_AddressProvider(t *testing.T) { // 4. With nil address provider -> Programmed=False, reason AddressNotAssigned r.AddressProvider = nil + st.SetGatewayAddresses(req.NamespacedName, nil) + st.Recompute() _, err = r.Reconcile(ctx, req) if err != nil { t.Fatalf("unexpected error reconciling gateway: %v", err) @@ -1099,7 +1087,10 @@ func TestGatewayReconciler_GatewayFilter(t *testing.T) { _ = gatewayv1.AddToScheme(scheme) st := state.NewState() - p := proxy.NewProxy() + st.SetControllerName("test-controller") + st.SetGatewayFilter(func(gw *gatewayv1.Gateway) bool { + return gw.Labels["env"] == "prod" + }) gwClass := &gatewayv1.GatewayClass{ ObjectMeta: metav1.ObjectMeta{ @@ -1139,7 +1130,7 @@ func TestGatewayReconciler_GatewayFilter(t *testing.T) { Listeners: []gatewayv1.Listener{ { Name: "http", - Port: 80, + Port: 8080, Protocol: gatewayv1.HTTPProtocolType, }, }, @@ -1152,21 +1143,45 @@ func TestGatewayReconciler_GatewayFilter(t *testing.T) { WithStatusSubresource(gwProd, gwDev). Build() + // Simulate informer upserts into State + st.UpsertGatewayClass(gwClass) + st.UpsertGateway(gwProd) + st.UpsertGateway(gwDev) + + outputs := st.Recompute() + + prodKey := types.NamespacedName{Namespace: "default", Name: "gw-prod"} + devKey := types.NamespacedName{Namespace: "default", Name: "gw-dev"} + + if !containsGateway(st.GetGateways(), prodKey) { + t.Errorf("expected prod gateway in state") + } + if containsGateway(st.GetGateways(), devKey) { + t.Errorf("expected dev gateway NOT in state") + } + + // Verify out-of-scope Gateway never appears in ProxyListeners or ResolvedGateways + for _, l := range outputs.ProxyListeners { + if l.GatewayName == devKey { + t.Errorf("expected dev gateway listener NOT in proxy listeners, got %+v", l) + } + } + for _, g := range outputs.ResolvedGateways { + if g.Name == "gw-dev" { + t.Errorf("expected dev gateway NOT in resolved gateways, got %+v", g) + } + } + r := &GatewayReconciler{ Client: client, Scheme: scheme, State: st, - Proxy: p, ControllerName: "test-controller", - GatewayFilter: func(gw *gatewayv1.Gateway) bool { - return gw.Labels["env"] == "prod" - }, } ctx := t.Context() - // 1. Reconcile gw-prod (matches filter) -> accepted & stored in state - prodKey := types.NamespacedName{Namespace: "default", Name: "gw-prod"} + // 1. Reconcile gw-prod (matches filter) -> accepted _, err := r.Reconcile(ctx, ctrl.Request{NamespacedName: prodKey}) if err != nil { t.Fatalf("unexpected error reconciling prod gateway: %v", err) @@ -1179,12 +1194,8 @@ func TestGatewayReconciler_GatewayFilter(t *testing.T) { if len(resProd.Status.Conditions) == 0 { t.Errorf("expected conditions on prod Gateway, got none") } - if !containsGateway(st.GetGateways(), prodKey) { - t.Errorf("expected prod gateway in state") - } - // 2. Reconcile gw-dev (does not match filter) -> ignored & not in state - devKey := types.NamespacedName{Namespace: "default", Name: "gw-dev"} + // 2. Reconcile gw-dev (does not match filter) -> no status written _, err = r.Reconcile(ctx, ctrl.Request{NamespacedName: devKey}) if err != nil { t.Fatalf("unexpected error reconciling dev gateway: %v", err) @@ -1197,9 +1208,6 @@ func TestGatewayReconciler_GatewayFilter(t *testing.T) { if len(resDev.Status.Conditions) != 0 { t.Errorf("expected no conditions on dev Gateway, got %+v", resDev.Status.Conditions) } - if containsGateway(st.GetGateways(), devKey) { - t.Errorf("expected dev gateway NOT in state") - } } func TestListenerSetReconciler(t *testing.T) { @@ -1274,7 +1282,12 @@ func TestListenerSetReconciler(t *testing.T) { } st := state.NewState() - p := proxy.NewProxy() + st.SetControllerName(DefaultControllerName) + st.UpsertGatewayClass(gc) + st.UpsertGateway(gw) + st.UpsertListenerSet(lsAllowed) + st.UpsertListenerSet(lsDisallowed) + st.Recompute() client := fake.NewClientBuilder(). WithScheme(scheme). @@ -1286,7 +1299,6 @@ func TestListenerSetReconciler(t *testing.T) { Client: client, Scheme: scheme, State: st, - Proxy: p, ControllerName: DefaultControllerName, } @@ -1404,10 +1416,12 @@ func TestGatewayReconciler_AttachedListenerSets(t *testing.T) { } st := state.NewState() - p := proxy.NewProxy() - + st.SetControllerName(DefaultControllerName) + st.UpsertGatewayClass(gc) + st.UpsertGateway(gw) st.UpsertListenerSet(ls1) st.UpsertListenerSet(ls2) + st.Recompute() client := fake.NewClientBuilder(). WithScheme(scheme). @@ -1419,7 +1433,6 @@ func TestGatewayReconciler_AttachedListenerSets(t *testing.T) { Client: client, Scheme: scheme, State: st, - Proxy: p, ControllerName: DefaultControllerName, } diff --git a/pkg/controller/gateway_controller.go b/pkg/controller/gateway_controller.go index 3348a21..2179176 100644 --- a/pkg/controller/gateway_controller.go +++ b/pkg/controller/gateway_controller.go @@ -18,26 +18,20 @@ import ( "context" "fmt" - "github.com/gke-labs/gateway-api-reference-implementation/pkg/proxy" "github.com/gke-labs/gateway-api-reference-implementation/pkg/state" - corev1 "k8s.io/api/core/v1" - apierrors "k8s.io/apimachinery/pkg/api/errors" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" - "k8s.io/apimachinery/pkg/types" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/builder" "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/controller-runtime/pkg/handler" "sigs.k8s.io/controller-runtime/pkg/log" gatewayv1 "sigs.k8s.io/gateway-api/apis/v1" - gatewayv1beta1 "sigs.k8s.io/gateway-api/apis/v1beta1" ) type GatewayClassReconciler struct { client.Client Scheme *runtime.Scheme + State *state.State ControllerName string } @@ -53,67 +47,18 @@ func (r *GatewayClassReconciler) Reconcile(ctx context.Context, req ctrl.Request return ctrl.Result{}, nil } - acceptedStatus := metav1.ConditionTrue - acceptedReason := gatewayv1.GatewayClassReasonAccepted - acceptedMessage := "GatewayClass accepted by reference implementation" - - if gc.Spec.ParametersRef != nil { - acceptedStatus = metav1.ConditionFalse - acceptedReason = gatewayv1.GatewayClassReasonInvalidParameters - acceptedMessage = "Invalid parametersRef: parametersRef is not supported" - } - - // Update status to Accepted - newConditions := []metav1.Condition{ - { - Type: string(gatewayv1.GatewayClassConditionStatusAccepted), - Status: acceptedStatus, - ObservedGeneration: gc.Generation, - Reason: string(acceptedReason), - Message: acceptedMessage, - }, - } - - // Preserve LastTransitionTime if status has not changed - for i, newCond := range newConditions { - newConditions[i].LastTransitionTime = metav1.Now() - for _, oldCond := range gc.Status.Conditions { - if oldCond.Type == newCond.Type && oldCond.Status == newCond.Status { - newConditions[i].LastTransitionTime = oldCond.LastTransitionTime - break - } + var desired gatewayv1.GatewayClassStatus + if r.State != nil { + d, ok := r.State.GetDesiredGatewayClassStatus(req.NamespacedName) + if !ok { + return ctrl.Result{}, nil } - } - - updated := false - if len(gc.Status.Conditions) != len(newConditions) { - updated = true + desired = d } else { - for i := range newConditions { - matched := false - for j := range gc.Status.Conditions { - if gc.Status.Conditions[j].Type == newConditions[i].Type { - if gc.Status.Conditions[j].Status == newConditions[i].Status && - gc.Status.Conditions[j].ObservedGeneration == newConditions[i].ObservedGeneration && - gc.Status.Conditions[j].Reason == newConditions[i].Reason && - gc.Status.Conditions[j].Message == newConditions[i].Message { - matched = true - } - break - } - } - if !matched { - updated = true - break - } - } + desired = state.ComputeDesiredGatewayClassStatus(&gc, r.ControllerName) } - if updated { - for i := range newConditions { - newConditions[i].LastTransitionTime = metav1.Now() - } - gc.Status.Conditions = newConditions + if state.MergeGatewayClassStatus(&gc.Status, desired) { if err := r.Status().Update(ctx, &gc); err != nil { l.Error(err, "unable to update GatewayClass status") return ctrl.Result{}, err @@ -127,9 +72,12 @@ func (r *GatewayClassReconciler) SetupWithManager(mgr ctrl.Manager) error { if r.ControllerName == "" { return fmt.Errorf("ControllerName is required") } - return ctrl.NewControllerManagedBy(mgr). - For(&gatewayv1.GatewayClass{}). - Complete(r) + bldr := ctrl.NewControllerManagedBy(mgr). + For(&gatewayv1.GatewayClass{}) + if r.State != nil { + bldr.WatchesRawSource(r.State.GatewayClassSource()) + } + return bldr.Complete(r) } // AddressProvider returns the addresses to report in Gateway.status.addresses. @@ -146,243 +94,63 @@ type AddressWatcher interface { type GatewayReconciler struct { client.Client - Scheme *runtime.Scheme - State *state.State - Proxy *proxy.Proxy - ControllerName string - OnGatewaysUpdate func([]*gatewayv1.Gateway) - AddressProvider AddressProvider - GatewayFilter func(gw *gatewayv1.Gateway) bool -} - -func (r *GatewayReconciler) inScope(gw *gatewayv1.Gateway) bool { - if r.GatewayFilter != nil && !r.GatewayFilter(gw) { - return false - } - return true + Scheme *runtime.Scheme + State *state.State + ControllerName string + AddressProvider AddressProvider } func (r *GatewayReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { l := log.FromContext(ctx) - controllerName := r.ControllerName - gw := &gatewayv1.Gateway{} if err := r.Get(ctx, req.NamespacedName, gw); err != nil { - if apierrors.IsNotFound(err) { - r.State.DeleteGateway(req.NamespacedName) - r.updateProxy() - } - return ctrl.Result{}, client.IgnoreNotFound(err) - } - - if !r.inScope(gw) { - r.State.DeleteGateway(req.NamespacedName) - r.updateProxy() - return ctrl.Result{}, nil - } - - // Check if the GatewayClass is managed by us - gc := &gatewayv1.GatewayClass{} - if err := r.Get(ctx, client.ObjectKey{Name: string(gw.Spec.GatewayClassName)}, gc); err != nil { - l.Error(err, "unable to fetch GatewayClass", "gatewayclass", gw.Spec.GatewayClassName) return ctrl.Result{}, client.IgnoreNotFound(err) } - if string(gc.Spec.ControllerName) != controllerName { - r.State.DeleteGateway(req.NamespacedName) - r.updateProxy() - return ctrl.Result{}, nil - } - - r.State.UpsertGateway(gw) - - var providedAddresses []gatewayv1.GatewayStatusAddress - if r.AddressProvider != nil { - var err error - providedAddresses, err = r.AddressProvider.GatewayAddresses(ctx, gw) - if err != nil { - l.Error(err, "unable to fetch gateway addresses from address provider") - return ctrl.Result{}, err + if r.State != nil { + // Verify this Gateway is managed by us in State before requesting addresses + gc, ok := r.State.GetGatewayClass(string(gw.Spec.GatewayClassName)) + if !ok || string(gc.Spec.ControllerName) != r.ControllerName { + return ctrl.Result{}, nil } - } - - // TODO(incremental-state): Centralize model recomputation and diffing to avoid recompiling in both reconciler and updateProxy. - compiled := r.State.CompileModel(controllerName) - compiledGw := compiled.Gateways[req.NamespacedName] - newStatus, updated := state.ComputeDesiredGatewayStatus(gw, compiledGw, providedAddresses) - - if updated { - gw.Status = newStatus - if err := r.Status().Update(ctx, gw); err != nil { - l.Error(err, "unable to update Gateway status") - return ctrl.Result{}, err + if r.AddressProvider != nil { + providedAddresses, err := r.AddressProvider.GatewayAddresses(ctx, gw) + if err != nil { + l.Error(err, "unable to fetch gateway addresses from address provider") + return ctrl.Result{}, err + } + r.State.SetGatewayAddresses(req.NamespacedName, providedAddresses) } - } - r.State.UpsertGateway(gw) - r.updateProxy() + desired, ok := r.State.GetDesiredGatewayStatus(req.NamespacedName) + if !ok { + return ctrl.Result{}, nil + } - if len(gw.Status.Addresses) == 0 { - l.V(1).Info("Gateway has no address assigned yet") - } else { - l.Info("Updated Gateway status", "addresses", gw.Status.Addresses) + if state.MergeGatewayStatus(&gw.Status, desired) { + if err := r.Status().Update(ctx, gw); err != nil { + l.Error(err, "unable to update Gateway status") + return ctrl.Result{}, err + } + l.Info("Updated Gateway status", "addresses", gw.Status.Addresses) + } } return ctrl.Result{}, nil } -func (r *GatewayReconciler) updateProxy() { - updateProxy(r.State, r.Proxy, r.ControllerName, r.OnGatewaysUpdate) -} - func (r *GatewayReconciler) SetupWithManager(mgr ctrl.Manager) error { if r.ControllerName == "" { return fmt.Errorf("ControllerName is required") } bldr := ctrl.NewControllerManagedBy(mgr). - For(&gatewayv1.Gateway{}). - // A GatewayClass update invalidates all Gateways that reference it - Watches(&gatewayv1.GatewayClass{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - gc := obj.(*gatewayv1.GatewayClass) - if string(gc.Spec.ControllerName) != r.ControllerName { - return nil - } - var gwList gatewayv1.GatewayList - if err := r.List(ctx, &gwList); err == nil { - var requests []ctrl.Request - for _, gw := range gwList.Items { - if string(gw.Spec.GatewayClassName) == gc.Name { - requests = append(requests, ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: gw.Namespace, - Name: gw.Name, - }, - }) - } - } - return requests - } - return nil - })). - Watches(&gatewayv1.HTTPRoute{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - // When an HTTPRoute changes, reconcile all Gateways it references - route := obj.(*gatewayv1.HTTPRoute) - var requests []ctrl.Request - for _, parentRef := range route.Spec.ParentRefs { - if string(state.ValueOf(parentRef.Group)) == "" || string(state.ValueOf(parentRef.Group)) == "gateway.networking.k8s.io" { - if string(state.ValueOf(parentRef.Kind)) == "" || string(state.ValueOf(parentRef.Kind)) == "Gateway" { - gwKey := ResolveNamespacedName(parentRef.Namespace, parentRef.Name, route) - requests = append(requests, ctrl.Request{ - NamespacedName: gwKey, - }) - } - } - } - return requests - })). - // A ReferenceGrant update invalidates all gateways that might use it - Watches(&gatewayv1beta1.ReferenceGrant{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - rg := obj.(*gatewayv1beta1.ReferenceGrant) - var gwList gatewayv1.GatewayList - if err := r.List(ctx, &gwList); err == nil { - var requests []ctrl.Request - for _, gw := range gwList.Items { - match := false - for _, from := range rg.Spec.From { - if (string(from.Group) == gatewayv1.GroupName || string(from.Group) == "") && string(from.Kind) == "Gateway" && string(from.Namespace) == gw.Namespace { - match = true - break - } - } - if !match { - for _, listener := range gw.Spec.Listeners { - if listener.TLS != nil { - for _, ref := range listener.TLS.CertificateRefs { - refNs := gw.Namespace - if ref.Namespace != nil && string(*ref.Namespace) != "" { - refNs = string(*ref.Namespace) - } - if refNs == rg.Namespace { - match = true - break - } - } - } - if match { - break - } - } - } - if match { - requests = append(requests, ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: gw.Namespace, - Name: gw.Name, - }, - }) - } - } - return requests - } - return nil - })). - Watches(&corev1.Secret{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - secret := obj.(*corev1.Secret) - var gwList gatewayv1.GatewayList - if err := r.List(ctx, &gwList); err == nil { - var requests []ctrl.Request - for _, gw := range gwList.Items { - for _, listener := range gw.Spec.Listeners { - if listener.TLS != nil { - for _, ref := range listener.TLS.CertificateRefs { - secretKey := ResolveNamespacedName(ref.Namespace, ref.Name, &gw) - if secretKey.Namespace == secret.Namespace && secretKey.Name == secret.Name { - requests = append(requests, ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: gw.Namespace, - Name: gw.Name, - }, - }) - break - } - } - } - } - } - return requests - } - return nil - })). - Watches(&gatewayv1.ListenerSet{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - ls := obj.(*gatewayv1.ListenerSet) - gwKey := ResolveNamespacedName(ls.Spec.ParentRef.Namespace, ls.Spec.ParentRef.Name, ls) - return []ctrl.Request{ - { - NamespacedName: gwKey, - }, - } - })). - Watches(&corev1.Namespace{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - var gwList gatewayv1.GatewayList - if err := r.List(ctx, &gwList); err == nil { - var requests []ctrl.Request - for _, gw := range gwList.Items { - if gw.Spec.AllowedListeners != nil && gw.Spec.AllowedListeners.Namespaces != nil && - state.ValueOf(gw.Spec.AllowedListeners.Namespaces.From) == gatewayv1.NamespacesFromSelector { - requests = append(requests, ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: gw.Namespace, - Name: gw.Name, - }, - }) - } - } - return requests - } - return nil - })) + For(&gatewayv1.Gateway{}) + + if r.State != nil { + bldr.WatchesRawSource(r.State.GatewaySource()) + } if watcher, ok := r.AddressProvider.(AddressWatcher); ok { if err := watcher.SetupWatches(mgr, bldr); err != nil { diff --git a/pkg/controller/httproute_controller.go b/pkg/controller/httproute_controller.go index 9c3f05a..c18cfb5 100644 --- a/pkg/controller/httproute_controller.go +++ b/pkg/controller/httproute_controller.go @@ -18,198 +18,59 @@ import ( "context" "fmt" - "github.com/gke-labs/gateway-api-reference-implementation/pkg/proxy" "github.com/gke-labs/gateway-api-reference-implementation/pkg/state" - corev1 "k8s.io/api/core/v1" - apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" - "k8s.io/apimachinery/pkg/types" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/controller-runtime/pkg/handler" "sigs.k8s.io/controller-runtime/pkg/log" gatewayv1 "sigs.k8s.io/gateway-api/apis/v1" - gatewayv1beta1 "sigs.k8s.io/gateway-api/apis/v1beta1" ) type HTTPRouteReconciler struct { client.Client - Scheme *runtime.Scheme - State *state.State - Proxy *proxy.Proxy - ControllerName string - OnGatewaysUpdate func([]*gatewayv1.Gateway) + Scheme *runtime.Scheme + State *state.State + ControllerName string } func (r *HTTPRouteReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { l := log.FromContext(ctx) - controllerName := r.ControllerName - route := &gatewayv1.HTTPRoute{} if err := r.Get(ctx, req.NamespacedName, route); err != nil { - if apierrors.IsNotFound(err) { - r.State.DeleteHTTPRoute(req.NamespacedName) - r.updateProxy() - } return ctrl.Result{}, client.IgnoreNotFound(err) } - r.State.UpsertHTTPRoute(route) - - // TODO(incremental-state): Centralize model recomputation and diffing to avoid recompiling in both reconciler and updateProxy. - compiled := r.State.CompileModel(controllerName) - compiledRoute := compiled.HTTPRoutes[req.NamespacedName] + if r.State != nil { + desired, ok := r.State.GetDesiredHTTPRouteStatus(req.NamespacedName) + if !ok { + return ctrl.Result{}, nil + } - newStatus, updated := state.ComputeDesiredHTTPRouteStatus(route, compiledRoute, controllerName) - if updated { - route.Status = newStatus - if err := r.Status().Update(ctx, route); err != nil { - l.Error(err, "unable to update HTTPRoute status") - return ctrl.Result{}, err + if state.MergeHTTPRouteStatus(&route.Status, desired, route.Namespace, gatewayv1.GatewayController(r.ControllerName)) { + if err := r.Status().Update(ctx, route); err != nil { + l.Error(err, "unable to update HTTPRoute status") + return ctrl.Result{}, err + } } } - r.State.UpsertHTTPRoute(route) - r.updateProxy() - - l.Info("Updated HTTPRoute status and proxy") + l.V(1).Info("Reconciled HTTPRoute") return ctrl.Result{}, nil } -func (r *HTTPRouteReconciler) updateProxy() { - updateProxy(r.State, r.Proxy, r.ControllerName, r.OnGatewaysUpdate) -} - func (r *HTTPRouteReconciler) SetupWithManager(mgr ctrl.Manager) error { if r.ControllerName == "" { return fmt.Errorf("ControllerName is required") } - return ctrl.NewControllerManagedBy(mgr). - For(&gatewayv1.HTTPRoute{}). - // A Gateway update invalidates all the HTTPRoutes that reference it - Watches(&gatewayv1.Gateway{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - gw := obj.(*gatewayv1.Gateway) - var routeList gatewayv1.HTTPRouteList - if err := r.List(ctx, &routeList); err != nil { - return nil - } - var requests []ctrl.Request - for _, route := range routeList.Items { - for _, parentRef := range route.Spec.ParentRefs { - gwKey := ResolveNamespacedName(parentRef.Namespace, parentRef.Name, &route) - if gwKey.Name == gw.Name && gwKey.Namespace == gw.Namespace { - requests = append(requests, ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: route.Namespace, - Name: route.Name, - }, - }) - break - } - } - } - return requests - })). - // A ReferenceGrant update invalidates all the HTTPRoutes in referenced namespaces - Watches(&gatewayv1beta1.ReferenceGrant{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - rg := obj.(*gatewayv1beta1.ReferenceGrant) - var routeList gatewayv1.HTTPRouteList - if err := r.List(ctx, &routeList); err != nil { - return nil - } - var requests []ctrl.Request - for _, route := range routeList.Items { - for _, from := range rg.Spec.From { - if (string(from.Group) == gatewayv1.GroupName || string(from.Group) == "") && string(from.Kind) == "HTTPRoute" && string(from.Namespace) == route.Namespace { - requests = append(requests, ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: route.Namespace, - Name: route.Name, - }, - }) - break - } - } - } - return requests - })). - // A Service update invalidates all the HTTPRoutes that reference it - Watches(&corev1.Service{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - svc := obj.(*corev1.Service) - var routeList gatewayv1.HTTPRouteList - if err := r.List(ctx, &routeList); err != nil { - return nil - } - var requests []ctrl.Request - for _, route := range routeList.Items { - matched := false - for _, rule := range route.Spec.Rules { - for _, bRef := range rule.BackendRefs { - svcKey := ResolveNamespacedName(bRef.Namespace, bRef.Name, &route) - if svcKey.Name == svc.Name && svcKey.Namespace == svc.Namespace { - requests = append(requests, ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: route.Namespace, - Name: route.Name, - }, - }) - matched = true - break - } - } - if matched { - break - } - } - } - return requests - })). - // A Namespace update invalidates all the HTTPRoutes in that namespace - Watches(&corev1.Namespace{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - ns := obj.(*corev1.Namespace) - var routeList gatewayv1.HTTPRouteList - if err := r.List(ctx, &routeList, client.InNamespace(ns.Name)); err != nil { - return nil - } - var requests []ctrl.Request - for _, route := range routeList.Items { - requests = append(requests, ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: route.Namespace, - Name: route.Name, - }, - }) - } - return requests - })). - // A ListenerSet update invalidates all HTTPRoutes that reference it - Watches(&gatewayv1.ListenerSet{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - ls := obj.(*gatewayv1.ListenerSet) - var routeList gatewayv1.HTTPRouteList - if err := r.List(ctx, &routeList); err != nil { - return nil - } - var requests []ctrl.Request - for _, route := range routeList.Items { - for _, parentRef := range route.Spec.ParentRefs { - if string(state.ValueOf(parentRef.Kind)) == "ListenerSet" { - lsKey := ResolveNamespacedName(parentRef.Namespace, parentRef.Name, &route) - if lsKey.Name == ls.Name && lsKey.Namespace == ls.Namespace { - requests = append(requests, ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: route.Namespace, - Name: route.Name, - }, - }) - break - } - } - } - } - return requests - })). - Complete(r) + bldr := ctrl.NewControllerManagedBy(mgr). + For(&gatewayv1.HTTPRoute{}) + + if r.State != nil { + bldr.WatchesRawSource(r.State.HTTPRouteSource()) + } + + return bldr.Complete(r) } diff --git a/pkg/controller/listenerset_controller.go b/pkg/controller/listenerset_controller.go index 29c79db..153038e 100644 --- a/pkg/controller/listenerset_controller.go +++ b/pkg/controller/listenerset_controller.go @@ -18,27 +18,19 @@ import ( "context" "fmt" - "github.com/gke-labs/gateway-api-reference-implementation/pkg/proxy" "github.com/gke-labs/gateway-api-reference-implementation/pkg/state" - corev1 "k8s.io/api/core/v1" - apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" - "k8s.io/apimachinery/pkg/types" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/controller-runtime/pkg/handler" "sigs.k8s.io/controller-runtime/pkg/log" gatewayv1 "sigs.k8s.io/gateway-api/apis/v1" - gatewayv1beta1 "sigs.k8s.io/gateway-api/apis/v1beta1" ) type ListenerSetReconciler struct { client.Client - Scheme *runtime.Scheme - State *state.State - Proxy *proxy.Proxy - ControllerName string - OnGatewaysUpdate func([]*gatewayv1.Gateway) + Scheme *runtime.Scheme + State *state.State + ControllerName string } func (r *ListenerSetReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { @@ -46,188 +38,37 @@ func (r *ListenerSetReconciler) Reconcile(ctx context.Context, req ctrl.Request) ls := &gatewayv1.ListenerSet{} if err := r.Get(ctx, req.NamespacedName, ls); err != nil { - if apierrors.IsNotFound(err) { - r.State.DeleteListenerSet(req.NamespacedName) - r.updateProxy() - } return ctrl.Result{}, client.IgnoreNotFound(err) } - gwKey := ResolveNamespacedName(ls.Spec.ParentRef.Namespace, ls.Spec.ParentRef.Name, ls) - var gw *gatewayv1.Gateway - var fetchedGW gatewayv1.Gateway - if err := r.Get(ctx, gwKey, &fetchedGW); err == nil { - gc := &gatewayv1.GatewayClass{} - if err := r.Get(ctx, client.ObjectKey{Name: string(fetchedGW.Spec.GatewayClassName)}, gc); err == nil { - if string(gc.Spec.ControllerName) == r.ControllerName { - gw = &fetchedGW - } + if r.State != nil { + desired, ok := r.State.GetDesiredListenerSetStatus(req.NamespacedName) + if !ok { + return ctrl.Result{}, nil } - } - - if gw != nil { - r.State.UpsertGateway(gw) - } - r.State.UpsertListenerSet(ls) - - // TODO(incremental-state): Centralize model recomputation and diffing to avoid recompiling in both reconciler and updateProxy. - compiled := r.State.CompileModel(r.ControllerName) - compiledGw := compiled.Gateways[gwKey] - - namespaces := r.State.GetNamespaces() - newStatus, updated := state.ComputeDesiredListenerSetStatus(ls, gw, namespaces, compiledGw) - if updated { - ls.Status = newStatus - if err := r.Status().Update(ctx, ls); err != nil { - l.Error(err, "unable to update ListenerSet status") - return ctrl.Result{}, err + if state.MergeListenerSetStatus(&ls.Status, desired) { + if err := r.Status().Update(ctx, ls); err != nil { + l.Error(err, "unable to update ListenerSet status") + return ctrl.Result{}, err + } } } - r.State.UpsertListenerSet(ls) - r.updateProxy() - return ctrl.Result{}, nil } -func (r *ListenerSetReconciler) updateProxy() { - updateProxy(r.State, r.Proxy, r.ControllerName, r.OnGatewaysUpdate) -} - -const listenerSetSecretIndex = ".spec.listeners.tls.certificateRefs" - func (r *ListenerSetReconciler) SetupWithManager(mgr ctrl.Manager) error { if r.ControllerName == "" { return fmt.Errorf("ControllerName is required") } - if err := mgr.GetFieldIndexer().IndexField(context.Background(), &gatewayv1.ListenerSet{}, listenerSetSecretIndex, func(obj client.Object) []string { - ls := obj.(*gatewayv1.ListenerSet) - var secretKeys []string - for _, l := range ls.Spec.Listeners { - if l.TLS != nil { - for _, ref := range l.TLS.CertificateRefs { - secKey := ResolveNamespacedName(ref.Namespace, ref.Name, ls) - secretKeys = append(secretKeys, secKey.String()) - } - } - } - return secretKeys - }); err != nil { - return fmt.Errorf("failed to create field indexer for ListenerSet secrets: %w", err) + bldr := ctrl.NewControllerManagedBy(mgr). + For(&gatewayv1.ListenerSet{}) + + if r.State != nil { + bldr.WatchesRawSource(r.State.ListenerSetSource()) } - return ctrl.NewControllerManagedBy(mgr). - For(&gatewayv1.ListenerSet{}). - Watches(&gatewayv1.ListenerSet{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - changedLS := obj.(*gatewayv1.ListenerSet) - var lsList gatewayv1.ListenerSetList - if err := r.List(ctx, &lsList); err != nil { - return nil - } - var requests []ctrl.Request - changedGwKey := ResolveNamespacedName(changedLS.Spec.ParentRef.Namespace, changedLS.Spec.ParentRef.Name, changedLS) - for _, ls := range lsList.Items { - if ls.Namespace == changedLS.Namespace && ls.Name == changedLS.Name { - continue - } - gwKey := ResolveNamespacedName(ls.Spec.ParentRef.Namespace, ls.Spec.ParentRef.Name, &ls) - if gwKey == changedGwKey { - requests = append(requests, ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: ls.Namespace, - Name: ls.Name, - }, - }) - } - } - return requests - })). - Watches(&gatewayv1.Gateway{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - gw := obj.(*gatewayv1.Gateway) - var lsList gatewayv1.ListenerSetList - if err := r.List(ctx, &lsList); err != nil { - return nil - } - var requests []ctrl.Request - for _, ls := range lsList.Items { - gwKey := ResolveNamespacedName(ls.Spec.ParentRef.Namespace, ls.Spec.ParentRef.Name, &ls) - if gwKey.Name == gw.Name && gwKey.Namespace == gw.Namespace { - requests = append(requests, ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: ls.Namespace, - Name: ls.Name, - }, - }) - } - } - return requests - })). - Watches(&corev1.Namespace{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - var lsList gatewayv1.ListenerSetList - if err := r.List(ctx, &lsList); err != nil { - return nil - } - var requests []ctrl.Request - for _, ls := range lsList.Items { - requests = append(requests, ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: ls.Namespace, - Name: ls.Name, - }, - }) - } - return requests - })). - Watches(&gatewayv1.HTTPRoute{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - route := obj.(*gatewayv1.HTTPRoute) - var requests []ctrl.Request - for _, parentRef := range route.Spec.ParentRefs { - pKind := state.ValueOf(parentRef.Kind) - pGroup := state.ValueOf(parentRef.Group) - if (pGroup == "" || pGroup == gatewayv1.GroupName) && pKind == "ListenerSet" { - lsKey := ResolveNamespacedName(parentRef.Namespace, parentRef.Name, route) - requests = append(requests, ctrl.Request{ - NamespacedName: lsKey, - }) - } - } - return requests - })). - Watches(&gatewayv1beta1.ReferenceGrant{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - var lsList gatewayv1.ListenerSetList - if err := r.List(ctx, &lsList); err != nil { - return nil - } - var requests []ctrl.Request - for _, ls := range lsList.Items { - requests = append(requests, ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: ls.Namespace, - Name: ls.Name, - }, - }) - } - return requests - })). - Watches(&corev1.Secret{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []ctrl.Request { - secret := obj.(*corev1.Secret) - secretKey := types.NamespacedName{Namespace: secret.Namespace, Name: secret.Name}.String() - var lsList gatewayv1.ListenerSetList - if err := r.List(ctx, &lsList, client.MatchingFields{listenerSetSecretIndex: secretKey}); err != nil { - return nil - } - var requests []ctrl.Request - for _, ls := range lsList.Items { - requests = append(requests, ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: ls.Namespace, - Name: ls.Name, - }, - }) - } - return requests - })). - Complete(r) + return bldr.Complete(r) } diff --git a/pkg/controller/namespace_controller.go b/pkg/controller/namespace_controller.go deleted file mode 100644 index 4fb5dc0..0000000 --- a/pkg/controller/namespace_controller.go +++ /dev/null @@ -1,73 +0,0 @@ -// Copyright 2026 Google LLC -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -package controller - -import ( - "context" - "fmt" - - "github.com/gke-labs/gateway-api-reference-implementation/pkg/proxy" - "github.com/gke-labs/gateway-api-reference-implementation/pkg/state" - corev1 "k8s.io/api/core/v1" - apierrors "k8s.io/apimachinery/pkg/api/errors" - "k8s.io/apimachinery/pkg/runtime" - ctrl "sigs.k8s.io/controller-runtime" - "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/controller-runtime/pkg/log" - - gatewayv1 "sigs.k8s.io/gateway-api/apis/v1" -) - -type NamespaceReconciler struct { - client.Client - Scheme *runtime.Scheme - State *state.State - Proxy *proxy.Proxy - ControllerName string - OnGatewaysUpdate func([]*gatewayv1.Gateway) -} - -func (r *NamespaceReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - l := log.FromContext(ctx) - - ns := &corev1.Namespace{} - if err := r.Get(ctx, req.NamespacedName, ns); err != nil { - if apierrors.IsNotFound(err) { - r.State.DeleteNamespace(req.Name) - r.updateProxy() - } - return ctrl.Result{}, client.IgnoreNotFound(err) - } - - r.State.UpsertNamespace(ns) - r.updateProxy() - - l.Info("Updated Namespace and proxy") - - return ctrl.Result{}, nil -} - -func (r *NamespaceReconciler) updateProxy() { - updateProxy(r.State, r.Proxy, r.ControllerName, r.OnGatewaysUpdate) -} - -func (r *NamespaceReconciler) SetupWithManager(mgr ctrl.Manager) error { - if r.ControllerName == "" { - return fmt.Errorf("ControllerName is required") - } - return ctrl.NewControllerManagedBy(mgr). - For(&corev1.Namespace{}). - Complete(r) -} diff --git a/pkg/controller/referencegrant_controller.go b/pkg/controller/referencegrant_controller.go deleted file mode 100644 index c4018c0..0000000 --- a/pkg/controller/referencegrant_controller.go +++ /dev/null @@ -1,87 +0,0 @@ -// Copyright 2026 Google LLC -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -package controller - -import ( - "context" - "fmt" - - "github.com/gke-labs/gateway-api-reference-implementation/pkg/proxy" - "github.com/gke-labs/gateway-api-reference-implementation/pkg/state" - apierrors "k8s.io/apimachinery/pkg/api/errors" - "k8s.io/apimachinery/pkg/runtime" - ctrl "sigs.k8s.io/controller-runtime" - "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/controller-runtime/pkg/log" - gatewayv1 "sigs.k8s.io/gateway-api/apis/v1" - gatewayv1beta1 "sigs.k8s.io/gateway-api/apis/v1beta1" -) - -type ReferenceGrantReconciler struct { - client.Client - Scheme *runtime.Scheme - State *state.State - Proxy *proxy.Proxy - ControllerName string - OnGatewaysUpdate func([]*gatewayv1.Gateway) -} - -func (r *ReferenceGrantReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - l := log.FromContext(ctx) - - rg := &gatewayv1beta1.ReferenceGrant{} - if err := r.Get(ctx, req.NamespacedName, rg); err != nil { - if apierrors.IsNotFound(err) { - r.State.DeleteReferenceGrant(req.NamespacedName) - r.recomputeAndSyncRoutes(ctx) - r.updateProxy() - } - return ctrl.Result{}, client.IgnoreNotFound(err) - } - - r.State.UpsertReferenceGrant(rg) - r.recomputeAndSyncRoutes(ctx) - r.updateProxy() - - l.Info("Updated ReferenceGrant in state") - - return ctrl.Result{}, nil -} - -func (r *ReferenceGrantReconciler) recomputeAndSyncRoutes(ctx context.Context) { - services := r.State.GetServices() - policies := r.State.GetBackendTLSPolicies() - configMaps := r.State.GetConfigMaps() - - // Recompile all routes stored in state against the updated reference grants - for _, route := range r.State.GetHTTPRoutes() { - if route.HTTPRoute != nil { - route.Compile(services, policies, configMaps, r.State) - } - } -} - -func (r *ReferenceGrantReconciler) updateProxy() { - updateProxy(r.State, r.Proxy, r.ControllerName, r.OnGatewaysUpdate) -} - -func (r *ReferenceGrantReconciler) SetupWithManager(mgr ctrl.Manager) error { - if r.ControllerName == "" { - return fmt.Errorf("ControllerName is required") - } - return ctrl.NewControllerManagedBy(mgr). - For(&gatewayv1beta1.ReferenceGrant{}). - Complete(r) -} diff --git a/pkg/controller/secret_controller.go b/pkg/controller/secret_controller.go deleted file mode 100644 index 00a7fb4..0000000 --- a/pkg/controller/secret_controller.go +++ /dev/null @@ -1,73 +0,0 @@ -// Copyright 2026 Google LLC -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -package controller - -import ( - "context" - "fmt" - - "github.com/gke-labs/gateway-api-reference-implementation/pkg/proxy" - "github.com/gke-labs/gateway-api-reference-implementation/pkg/state" - corev1 "k8s.io/api/core/v1" - apierrors "k8s.io/apimachinery/pkg/api/errors" - "k8s.io/apimachinery/pkg/runtime" - ctrl "sigs.k8s.io/controller-runtime" - "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/controller-runtime/pkg/log" - - gatewayv1 "sigs.k8s.io/gateway-api/apis/v1" -) - -type SecretReconciler struct { - client.Client - Scheme *runtime.Scheme - State *state.State - Proxy *proxy.Proxy - ControllerName string - OnGatewaysUpdate func([]*gatewayv1.Gateway) -} - -func (r *SecretReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - l := log.FromContext(ctx) - - secret := &corev1.Secret{} - if err := r.Get(ctx, req.NamespacedName, secret); err != nil { - if apierrors.IsNotFound(err) { - r.State.DeleteSecret(req.NamespacedName) - r.updateProxy() - } - return ctrl.Result{}, client.IgnoreNotFound(err) - } - - r.State.UpsertSecret(secret) - r.updateProxy() - - l.Info("Updated Secret in state") - - return ctrl.Result{}, nil -} - -func (r *SecretReconciler) updateProxy() { - updateProxy(r.State, r.Proxy, r.ControllerName, r.OnGatewaysUpdate) -} - -func (r *SecretReconciler) SetupWithManager(mgr ctrl.Manager) error { - if r.ControllerName == "" { - return fmt.Errorf("ControllerName is required") - } - return ctrl.NewControllerManagedBy(mgr). - For(&corev1.Secret{}). - Complete(r) -} diff --git a/pkg/controller/service_controller.go b/pkg/controller/service_controller.go deleted file mode 100644 index a36011d..0000000 --- a/pkg/controller/service_controller.go +++ /dev/null @@ -1,73 +0,0 @@ -// Copyright 2026 Google LLC -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -package controller - -import ( - "context" - "fmt" - - "github.com/gke-labs/gateway-api-reference-implementation/pkg/proxy" - "github.com/gke-labs/gateway-api-reference-implementation/pkg/state" - corev1 "k8s.io/api/core/v1" - apierrors "k8s.io/apimachinery/pkg/api/errors" - "k8s.io/apimachinery/pkg/runtime" - ctrl "sigs.k8s.io/controller-runtime" - "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/controller-runtime/pkg/log" - - gatewayv1 "sigs.k8s.io/gateway-api/apis/v1" -) - -type ServiceReconciler struct { - client.Client - Scheme *runtime.Scheme - State *state.State - Proxy *proxy.Proxy - ControllerName string - OnGatewaysUpdate func([]*gatewayv1.Gateway) -} - -func (r *ServiceReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - l := log.FromContext(ctx) - - svc := &corev1.Service{} - if err := r.Get(ctx, req.NamespacedName, svc); err != nil { - if apierrors.IsNotFound(err) { - r.State.DeleteService(req.NamespacedName) - r.updateProxy() - } - return ctrl.Result{}, client.IgnoreNotFound(err) - } - - r.State.UpsertService(svc) - r.updateProxy() - - l.Info("Updated Service and proxy") - - return ctrl.Result{}, nil -} - -func (r *ServiceReconciler) updateProxy() { - updateProxy(r.State, r.Proxy, r.ControllerName, r.OnGatewaysUpdate) -} - -func (r *ServiceReconciler) SetupWithManager(mgr ctrl.Manager) error { - if r.ControllerName == "" { - return fmt.Errorf("ControllerName is required") - } - return ctrl.NewControllerManagedBy(mgr). - For(&corev1.Service{}). - Complete(r) -} diff --git a/pkg/controller/utils.go b/pkg/controller/utils.go index 6008fb4..f0f4295 100644 --- a/pkg/controller/utils.go +++ b/pkg/controller/utils.go @@ -15,14 +15,18 @@ package controller import ( + "context" "fmt" "github.com/gke-labs/gateway-api-reference-implementation/pkg/proxy" "github.com/gke-labs/gateway-api-reference-implementation/pkg/state" + corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/types" + toolscache "k8s.io/client-go/tools/cache" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" gatewayv1 "sigs.k8s.io/gateway-api/apis/v1" + gatewayv1beta1 "sigs.k8s.io/gateway-api/apis/v1beta1" ) // ResolveNamespacedName resolves an optional namespace pointer and a name against a source object. @@ -49,19 +53,34 @@ type ReconcilerOptions struct { GatewayFilter func(gw *gatewayv1.Gateway) bool } -// RegisterReconcilers registers all GARI reconcilers with the given Manager. +// RegisterReconcilers registers all GARI reconcilers and informer event handlers with the given Manager. func RegisterReconcilers(mgr ctrl.Manager, st *state.State, p *proxy.Proxy, opts ReconcilerOptions) error { controllerName := opts.ControllerName if controllerName == "" { controllerName = DefaultControllerName } + + if st != nil { + st.SetControllerName(controllerName) + st.SetProxy(p) + st.SetOnGatewaysUpdate(opts.OnGatewaysUpdate) + st.SetGatewayFilter(opts.GatewayFilter) + st.SetSynced(false) + + if err := registerInformerHandlers(context.Background(), mgr, st); err != nil { + return fmt.Errorf("error registering informer event handlers: %w", err) + } + + if err := mgr.Add(st); err != nil { + return fmt.Errorf("error adding state runner to manager: %w", err) + } + } + if err := (&HTTPRouteReconciler{ - Client: mgr.GetClient(), - Scheme: mgr.GetScheme(), - State: st, - Proxy: p, - ControllerName: controllerName, - OnGatewaysUpdate: opts.OnGatewaysUpdate, + Client: mgr.GetClient(), + Scheme: mgr.GetScheme(), + State: st, + ControllerName: controllerName, }).SetupWithManager(mgr); err != nil { return fmt.Errorf("error creating HTTPRoute controller: %w", err) } @@ -69,97 +88,36 @@ func RegisterReconcilers(mgr ctrl.Manager, st *state.State, p *proxy.Proxy, opts if err := (&GatewayClassReconciler{ Client: mgr.GetClient(), Scheme: mgr.GetScheme(), + State: st, ControllerName: controllerName, }).SetupWithManager(mgr); err != nil { return fmt.Errorf("error creating GatewayClass controller: %w", err) } if err := (&GatewayReconciler{ - Client: mgr.GetClient(), - Scheme: mgr.GetScheme(), - State: st, - Proxy: p, - ControllerName: controllerName, - OnGatewaysUpdate: opts.OnGatewaysUpdate, - AddressProvider: opts.AddressProvider, - GatewayFilter: opts.GatewayFilter, + Client: mgr.GetClient(), + Scheme: mgr.GetScheme(), + State: st, + ControllerName: controllerName, + AddressProvider: opts.AddressProvider, }).SetupWithManager(mgr); err != nil { return fmt.Errorf("error creating Gateway controller: %w", err) } - if err := (&ServiceReconciler{ - Client: mgr.GetClient(), - Scheme: mgr.GetScheme(), - State: st, - Proxy: p, - ControllerName: controllerName, - OnGatewaysUpdate: opts.OnGatewaysUpdate, - }).SetupWithManager(mgr); err != nil { - return fmt.Errorf("error creating Service controller: %w", err) - } - if err := (&BackendTLSPolicyReconciler{ - Client: mgr.GetClient(), - Scheme: mgr.GetScheme(), - State: st, - Proxy: p, - ControllerName: controllerName, - OnGatewaysUpdate: opts.OnGatewaysUpdate, + Client: mgr.GetClient(), + Scheme: mgr.GetScheme(), + State: st, + ControllerName: controllerName, }).SetupWithManager(mgr); err != nil { return fmt.Errorf("error creating BackendTLSPolicy controller: %w", err) } - if err := (&ConfigMapReconciler{ - Client: mgr.GetClient(), - Scheme: mgr.GetScheme(), - State: st, - Proxy: p, - ControllerName: controllerName, - OnGatewaysUpdate: opts.OnGatewaysUpdate, - }).SetupWithManager(mgr); err != nil { - return fmt.Errorf("error creating ConfigMap controller: %w", err) - } - - if err := (&SecretReconciler{ - Client: mgr.GetClient(), - Scheme: mgr.GetScheme(), - State: st, - Proxy: p, - ControllerName: controllerName, - OnGatewaysUpdate: opts.OnGatewaysUpdate, - }).SetupWithManager(mgr); err != nil { - return fmt.Errorf("error creating Secret controller: %w", err) - } - - if err := (&ReferenceGrantReconciler{ - Client: mgr.GetClient(), - Scheme: mgr.GetScheme(), - State: st, - Proxy: p, - ControllerName: controllerName, - OnGatewaysUpdate: opts.OnGatewaysUpdate, - }).SetupWithManager(mgr); err != nil { - return fmt.Errorf("error creating ReferenceGrant controller: %w", err) - } - - if err := (&NamespaceReconciler{ - Client: mgr.GetClient(), - Scheme: mgr.GetScheme(), - State: st, - Proxy: p, - ControllerName: controllerName, - OnGatewaysUpdate: opts.OnGatewaysUpdate, - }).SetupWithManager(mgr); err != nil { - return fmt.Errorf("error creating Namespace controller: %w", err) - } - if err := (&ListenerSetReconciler{ - Client: mgr.GetClient(), - Scheme: mgr.GetScheme(), - State: st, - Proxy: p, - ControllerName: controllerName, - OnGatewaysUpdate: opts.OnGatewaysUpdate, + Client: mgr.GetClient(), + Scheme: mgr.GetScheme(), + State: st, + ControllerName: controllerName, }).SetupWithManager(mgr); err != nil { return fmt.Errorf("error creating ListenerSet controller: %w", err) } @@ -167,21 +125,242 @@ func RegisterReconcilers(mgr ctrl.Manager, st *state.State, p *proxy.Proxy, opts return nil } -func updateProxy(st *state.State, p *proxy.Proxy, controllerName string, onGatewaysUpdate func([]*gatewayv1.Gateway)) { - if controllerName == "" { - controllerName = DefaultControllerName +func registerInformerHandlers(ctx context.Context, mgr ctrl.Manager, st *state.State) error { + type registrationEntry struct { + obj client.Object + add func(obj any) + update func(oldObj, newObj any) + delete func(obj any) } - // TODO(incremental-state): Centralize model recomputation and diffing to avoid recompiling in both reconciler and updateProxy. - compiled := st.CompileModel(controllerName) - if p != nil { - proxyListeners, proxyRoutes := state.BuildProxyConfig(compiled.GatewaysList()) - p.UpdateConfig(proxyListeners, proxyRoutes) - - certsMap, defaultCert := state.ExtractCertificates(compiled.GatewaysList(), st.GetSecrets(), st) - p.UpdateCertificates(certsMap, defaultCert) + + entries := []registrationEntry{ + { + obj: &gatewayv1.GatewayClass{}, + add: func(obj any) { + if gc, ok := obj.(*gatewayv1.GatewayClass); ok { + st.UpsertGatewayClass(gc) + } + }, + update: func(oldObj, newObj any) { + if gc, ok := newObj.(*gatewayv1.GatewayClass); ok { + st.UpsertGatewayClass(gc) + } + }, + delete: func(obj any) { + if tombstone, ok := obj.(toolscache.DeletedFinalStateUnknown); ok { + obj = tombstone.Obj + } + if gc, ok := obj.(*gatewayv1.GatewayClass); ok { + st.DeleteGatewayClass(types.NamespacedName{Name: gc.Name}) + } + }, + }, + { + obj: &gatewayv1.Gateway{}, + add: func(obj any) { + if gw, ok := obj.(*gatewayv1.Gateway); ok { + st.UpsertGateway(gw) + } + }, + update: func(oldObj, newObj any) { + if gw, ok := newObj.(*gatewayv1.Gateway); ok { + st.UpsertGateway(gw) + } + }, + delete: func(obj any) { + if tombstone, ok := obj.(toolscache.DeletedFinalStateUnknown); ok { + obj = tombstone.Obj + } + if gw, ok := obj.(*gatewayv1.Gateway); ok { + st.DeleteGateway(types.NamespacedName{Namespace: gw.Namespace, Name: gw.Name}) + } + }, + }, + { + obj: &gatewayv1.ListenerSet{}, + add: func(obj any) { + if ls, ok := obj.(*gatewayv1.ListenerSet); ok { + st.UpsertListenerSet(ls) + } + }, + update: func(oldObj, newObj any) { + if ls, ok := newObj.(*gatewayv1.ListenerSet); ok { + st.UpsertListenerSet(ls) + } + }, + delete: func(obj any) { + if tombstone, ok := obj.(toolscache.DeletedFinalStateUnknown); ok { + obj = tombstone.Obj + } + if ls, ok := obj.(*gatewayv1.ListenerSet); ok { + st.DeleteListenerSet(types.NamespacedName{Namespace: ls.Namespace, Name: ls.Name}) + } + }, + }, + { + obj: &gatewayv1.HTTPRoute{}, + add: func(obj any) { + if r, ok := obj.(*gatewayv1.HTTPRoute); ok { + st.UpsertHTTPRoute(r) + } + }, + update: func(oldObj, newObj any) { + if r, ok := newObj.(*gatewayv1.HTTPRoute); ok { + st.UpsertHTTPRoute(r) + } + }, + delete: func(obj any) { + if tombstone, ok := obj.(toolscache.DeletedFinalStateUnknown); ok { + obj = tombstone.Obj + } + if r, ok := obj.(*gatewayv1.HTTPRoute); ok { + st.DeleteHTTPRoute(types.NamespacedName{Namespace: r.Namespace, Name: r.Name}) + } + }, + }, + { + obj: &gatewayv1.BackendTLSPolicy{}, + add: func(obj any) { + if p, ok := obj.(*gatewayv1.BackendTLSPolicy); ok { + st.UpsertBackendTLSPolicy(p) + } + }, + update: func(oldObj, newObj any) { + if p, ok := newObj.(*gatewayv1.BackendTLSPolicy); ok { + st.UpsertBackendTLSPolicy(p) + } + }, + delete: func(obj any) { + if tombstone, ok := obj.(toolscache.DeletedFinalStateUnknown); ok { + obj = tombstone.Obj + } + if p, ok := obj.(*gatewayv1.BackendTLSPolicy); ok { + st.DeleteBackendTLSPolicy(types.NamespacedName{Namespace: p.Namespace, Name: p.Name}) + } + }, + }, + { + obj: &corev1.Service{}, + add: func(obj any) { + if svc, ok := obj.(*corev1.Service); ok { + st.UpsertService(svc) + } + }, + update: func(oldObj, newObj any) { + if svc, ok := newObj.(*corev1.Service); ok { + st.UpsertService(svc) + } + }, + delete: func(obj any) { + if tombstone, ok := obj.(toolscache.DeletedFinalStateUnknown); ok { + obj = tombstone.Obj + } + if svc, ok := obj.(*corev1.Service); ok { + st.DeleteService(types.NamespacedName{Namespace: svc.Namespace, Name: svc.Name}) + } + }, + }, + { + obj: &corev1.Secret{}, + add: func(obj any) { + if s, ok := obj.(*corev1.Secret); ok { + st.UpsertSecret(s) + } + }, + update: func(oldObj, newObj any) { + if s, ok := newObj.(*corev1.Secret); ok { + st.UpsertSecret(s) + } + }, + delete: func(obj any) { + if tombstone, ok := obj.(toolscache.DeletedFinalStateUnknown); ok { + obj = tombstone.Obj + } + if s, ok := obj.(*corev1.Secret); ok { + st.DeleteSecret(types.NamespacedName{Namespace: s.Namespace, Name: s.Name}) + } + }, + }, + { + obj: &corev1.ConfigMap{}, + add: func(obj any) { + if cm, ok := obj.(*corev1.ConfigMap); ok { + st.UpsertConfigMap(cm) + } + }, + update: func(oldObj, newObj any) { + if cm, ok := newObj.(*corev1.ConfigMap); ok { + st.UpsertConfigMap(cm) + } + }, + delete: func(obj any) { + if tombstone, ok := obj.(toolscache.DeletedFinalStateUnknown); ok { + obj = tombstone.Obj + } + if cm, ok := obj.(*corev1.ConfigMap); ok { + st.DeleteConfigMap(types.NamespacedName{Namespace: cm.Namespace, Name: cm.Name}) + } + }, + }, + { + obj: &corev1.Namespace{}, + add: func(obj any) { + if ns, ok := obj.(*corev1.Namespace); ok { + st.UpsertNamespace(ns) + } + }, + update: func(oldObj, newObj any) { + if ns, ok := newObj.(*corev1.Namespace); ok { + st.UpsertNamespace(ns) + } + }, + delete: func(obj any) { + if tombstone, ok := obj.(toolscache.DeletedFinalStateUnknown); ok { + obj = tombstone.Obj + } + if ns, ok := obj.(*corev1.Namespace); ok { + st.DeleteNamespace(ns.Name) + } + }, + }, + { + obj: &gatewayv1beta1.ReferenceGrant{}, + add: func(obj any) { + if rg, ok := obj.(*gatewayv1beta1.ReferenceGrant); ok { + st.UpsertReferenceGrant(rg) + } + }, + update: func(oldObj, newObj any) { + if rg, ok := newObj.(*gatewayv1beta1.ReferenceGrant); ok { + st.UpsertReferenceGrant(rg) + } + }, + delete: func(obj any) { + if tombstone, ok := obj.(toolscache.DeletedFinalStateUnknown); ok { + obj = tombstone.Obj + } + if rg, ok := obj.(*gatewayv1beta1.ReferenceGrant); ok { + st.DeleteReferenceGrant(types.NamespacedName{Namespace: rg.Namespace, Name: rg.Name}) + } + }, + }, } - if onGatewaysUpdate != nil { - onGatewaysUpdate(compiled.ResolvedGateways()) + for _, entry := range entries { + informer, err := mgr.GetCache().GetInformer(ctx, entry.obj) + if err != nil { + return fmt.Errorf("error getting informer for %T: %w", entry.obj, err) + } + reg, err := informer.AddEventHandler(toolscache.ResourceEventHandlerFuncs{ + AddFunc: entry.add, + UpdateFunc: entry.update, + DeleteFunc: entry.delete, + }) + if err != nil { + return fmt.Errorf("error adding event handler for %T: %w", entry.obj, err) + } + st.AddRegistration(reg) } + + return nil } diff --git a/pkg/gari/gari_test.go b/pkg/gari/gari_test.go index 857c784..dbcbefd 100644 --- a/pkg/gari/gari_test.go +++ b/pkg/gari/gari_test.go @@ -323,13 +323,20 @@ func TestOnGatewaysUpdateHookAndCustomControllerName(t *testing.T) { WithStatusSubresource(gw, gc, route). Build() + st.SetProxy(p) + st.SetControllerName(customControllerName) + st.SetOnGatewaysUpdate(opts.OnGatewaysUpdate) + st.UpsertGatewayClass(gc) + st.UpsertGateway(gw) + st.UpsertHTTPRoute(route) + st.UpsertService(svc) + st.Recompute() + gwReconciler := &controller.GatewayReconciler{ - Client: fakeClient, - Scheme: scheme, - State: st, - Proxy: p, - ControllerName: customControllerName, - OnGatewaysUpdate: opts.OnGatewaysUpdate, + Client: fakeClient, + Scheme: scheme, + State: st, + ControllerName: customControllerName, } ctx := t.Context() diff --git a/pkg/state/compiled.go b/pkg/state/compiled.go index 5bd4b90..0573e4b 100644 --- a/pkg/state/compiled.go +++ b/pkg/state/compiled.go @@ -17,6 +17,7 @@ package state import ( "crypto/tls" "crypto/x509" + "encoding/pem" "fmt" "sort" "strings" @@ -27,6 +28,7 @@ import ( "k8s.io/apimachinery/pkg/runtime/schema" "k8s.io/apimachinery/pkg/types" gatewayv1 "sigs.k8s.io/gateway-api/apis/v1" + gatewayv1beta1 "sigs.k8s.io/gateway-api/apis/v1beta1" ) // ListenerOwner identifies the resource (Gateway or ListenerSet) that defines a listener. @@ -192,9 +194,12 @@ func (cm *CompiledModel) ResolvedGateways() []*gatewayv1.Gateway { return res } -// ModelInputs contains all inputs required to build a CompiledModel. +// ModelInputs contains all inputs required to build a CompiledModel and ComputeOutputs. type ModelInputs struct { + Revision uint64 Gateways []*gatewayv1.Gateway + GatewayClasses []*gatewayv1.GatewayClass + GatewayAddresses map[types.NamespacedName][]gatewayv1.GatewayStatusAddress ListenerSets []*gatewayv1.ListenerSet HTTPRoutes []*gatewayv1.HTTPRoute Services map[types.NamespacedName]*corev1.Service @@ -202,10 +207,26 @@ type ModelInputs struct { ConfigMaps map[types.NamespacedName]*corev1.ConfigMap Secrets map[types.NamespacedName]*corev1.Secret Namespaces map[string]*corev1.Namespace + ReferenceGrants map[types.NamespacedName]*gatewayv1beta1.ReferenceGrant RefValidator ReferenceGrantValidator ControllerName string } +// Outputs represents all computed outputs produced by ComputeOutputs. +type Outputs struct { + Revision uint64 + GatewayStatuses map[types.NamespacedName]gatewayv1.GatewayStatus + HTTPRouteStatuses map[types.NamespacedName]gatewayv1.HTTPRouteStatus + ListenerSetStatuses map[types.NamespacedName]gatewayv1.ListenerSetStatus + BackendTLSPolicyStatuses map[types.NamespacedName]gatewayv1.PolicyStatus + GatewayClassStatuses map[types.NamespacedName]gatewayv1.GatewayClassStatus + ProxyListeners []InternalListener + ProxyRoutes []InternalRoute + CertificatesMap map[string]*tls.Certificate + DefaultCert *tls.Certificate + ResolvedGateways []*gatewayv1.Gateway +} + func isSupportedProtocol(protocol gatewayv1.ProtocolType) bool { switch protocol { case gatewayv1.HTTPProtocolType, @@ -565,7 +586,6 @@ func areProtocolsCompatible(p1, p2 gatewayv1.ProtocolType) bool { } return false } - // TODO: Support HTTPS and TLS protocol sharing on the same port once TLSRoute is implemented. return false } @@ -595,6 +615,11 @@ func markListenerConflicted(el *EffectiveListener, reason gatewayv1.ListenerCond // CompileModel compiles the effective listeners, route bindings, and statuses across all Gateways and ListenerSets. func CompileModel(inputs ModelInputs) *CompiledModel { + refValidator := inputs.RefValidator + if refValidator == nil && len(inputs.ReferenceGrants) > 0 { + refValidator = mapReferenceValidator{referenceGrants: inputs.ReferenceGrants} + } + cm := &CompiledModel{ Gateways: make(map[types.NamespacedName]*CompiledGateway), HTTPRoutes: make(map[types.NamespacedName]*CompiledRoute), @@ -607,11 +632,23 @@ func CompileModel(inputs ModelInputs) *CompiledModel { } } + managedGatewayClasses := make(map[string]bool) + if inputs.ControllerName != "" { + for _, gc := range inputs.GatewayClasses { + if gc != nil && string(gc.Spec.ControllerName) == inputs.ControllerName { + managedGatewayClasses[gc.Name] = true + } + } + } + // 1. Compile each Gateway and its effective listeners for _, gw := range inputs.Gateways { if gw == nil { continue } + if inputs.ControllerName != "" && !managedGatewayClasses[string(gw.Spec.GatewayClassName)] { + continue + } gwKey := types.NamespacedName{Namespace: gw.Namespace, Name: gw.Name} cg := &CompiledGateway{ Gateway: gw, @@ -625,7 +662,7 @@ func CompileModel(inputs ModelInputs) *CompiledModel { gwKey, gw.Generation, inputs.Secrets, - inputs.RefValidator, + refValidator, ) cg.EffectiveListeners = append(cg.EffectiveListeners, el) } @@ -661,7 +698,7 @@ func CompileModel(inputs ModelInputs) *CompiledModel { gwKey, ls.Generation, inputs.Secrets, - inputs.RefValidator, + refValidator, ) cg.EffectiveListeners = append(cg.EffectiveListeners, el) } @@ -719,7 +756,11 @@ func CompileModel(inputs ModelInputs) *CompiledModel { } cg.AttachedListenerSets = acceptedLSCount - cg.Conditions = ComputeGatewayConditions(gw, cg.EffectiveListeners, len(gw.Status.Addresses) > 0) + hasAddress := false + if inputs.GatewayAddresses != nil { + hasAddress = len(inputs.GatewayAddresses[gwKey]) > 0 + } + cg.Conditions = ComputeGatewayConditions(gw, cg.EffectiveListeners, hasAddress) cm.Gateways[gwKey] = cg } @@ -730,7 +771,7 @@ func CompileModel(inputs ModelInputs) *CompiledModel { } routeKey := types.NamespacedName{Namespace: route.Namespace, Name: route.Name} rs := &HTTPRouteState{HTTPRoute: route} - rs.Compile(inputs.Services, inputs.BackendTLSPolicies, inputs.ConfigMaps, inputs.RefValidator) + rs.Compile(inputs.Services, inputs.BackendTLSPolicies, inputs.ConfigMaps, refValidator) parentConditions := make([]metav1.Condition, len(route.Spec.ParentRefs)) boundListenersForRoute := make(map[*EffectiveListener]bool) @@ -1091,17 +1132,117 @@ func BuildProxyConfig(gateways []*CompiledGateway) ([]InternalListener, []Intern return proxyListeners, proxyRoutes } +// ComputeOutputs computes all desired statuses, proxy configuration, certificates, and resolved gateways. +func ComputeOutputs(inputs ModelInputs) *Outputs { + if inputs.RefValidator == nil && len(inputs.ReferenceGrants) > 0 { + inputs.RefValidator = mapReferenceValidator{referenceGrants: inputs.ReferenceGrants} + } + + compiled := CompileModel(inputs) + + outputs := &Outputs{ + Revision: inputs.Revision, + GatewayStatuses: make(map[types.NamespacedName]gatewayv1.GatewayStatus), + HTTPRouteStatuses: make(map[types.NamespacedName]gatewayv1.HTTPRouteStatus), + ListenerSetStatuses: make(map[types.NamespacedName]gatewayv1.ListenerSetStatus), + BackendTLSPolicyStatuses: make(map[types.NamespacedName]gatewayv1.PolicyStatus), + GatewayClassStatuses: make(map[types.NamespacedName]gatewayv1.GatewayClassStatus), + } + + // 1. Gateway Statuses + for _, gw := range inputs.Gateways { + if gw == nil { + continue + } + gwKey := types.NamespacedName{Namespace: gw.Namespace, Name: gw.Name} + cg := compiled.Gateways[gwKey] + if cg != nil { + var addrs []gatewayv1.GatewayStatusAddress + if inputs.GatewayAddresses != nil { + addrs = inputs.GatewayAddresses[gwKey] + } + outputs.GatewayStatuses[gwKey] = ComputeDesiredGatewayStatus(gw, cg, addrs) + } + } + + // 2. ListenerSet Statuses + for _, ls := range inputs.ListenerSets { + if ls == nil { + continue + } + lsKey := types.NamespacedName{Namespace: ls.Namespace, Name: ls.Name} + var parentGW *gatewayv1.Gateway + for _, g := range inputs.Gateways { + if g != nil && IsListenerSetParent(ls, g) { + parentGW = g + break + } + } + var compiledGw *CompiledGateway + if parentGW != nil { + parentKey := types.NamespacedName{Namespace: parentGW.Namespace, Name: parentGW.Name} + compiledGw = compiled.Gateways[parentKey] + } + if parentGW != nil && (inputs.ControllerName == "" || compiledGw != nil) { + outputs.ListenerSetStatuses[lsKey] = ComputeDesiredListenerSetStatus(ls, parentGW, inputs.Namespaces, compiledGw) + } + } + + // 3. HTTPRoute Statuses + for _, route := range inputs.HTTPRoutes { + if route == nil { + continue + } + routeKey := types.NamespacedName{Namespace: route.Namespace, Name: route.Name} + compiledRoute := compiled.HTTPRoutes[routeKey] + outputs.HTTPRouteStatuses[routeKey] = ComputeDesiredHTTPRouteStatus(route, compiledRoute, inputs.ControllerName) + } + + // 4. BackendTLSPolicy Statuses + for _, policy := range inputs.BackendTLSPolicies { + if policy == nil { + continue + } + policyKey := types.NamespacedName{Namespace: policy.Namespace, Name: policy.Name} + outputs.BackendTLSPolicyStatuses[policyKey] = ComputeDesiredBackendTLSPolicyStatus( + policy, + compiled, + inputs.ConfigMaps, + inputs.BackendTLSPolicies, + inputs.ControllerName, + ) + } + + // 5. GatewayClass Statuses + for _, gc := range inputs.GatewayClasses { + if gc == nil { + continue + } + if inputs.ControllerName == "" || string(gc.Spec.ControllerName) == inputs.ControllerName { + gcKey := types.NamespacedName{Name: gc.Name} + outputs.GatewayClassStatuses[gcKey] = ComputeDesiredGatewayClassStatus(gc, inputs.ControllerName) + } + } + + // 6. Proxy Config & Certs + proxyListeners, proxyRoutes := BuildProxyConfig(compiled.GatewaysList()) + certsMap, defaultCert := ExtractCertificates(compiled.GatewaysList(), inputs.Secrets, inputs.RefValidator) + + outputs.ProxyListeners = proxyListeners + outputs.ProxyRoutes = proxyRoutes + outputs.CertificatesMap = certsMap + outputs.DefaultCert = defaultCert + outputs.ResolvedGateways = compiled.ResolvedGateways() + + return outputs +} + // ComputeDesiredGatewayStatus computes the desired GatewayStatus from the compiled model and address provider. func ComputeDesiredGatewayStatus( gw *gatewayv1.Gateway, compiledGw *CompiledGateway, providedAddresses []gatewayv1.GatewayStatusAddress, -) (gatewayv1.GatewayStatus, bool) { - desiredStatus := gw.Status.DeepCopy() - if desiredStatus == nil { - desiredStatus = &gatewayv1.GatewayStatus{} - } - +) gatewayv1.GatewayStatus { hasAddress := len(providedAddresses) > 0 var effectiveListeners []*EffectiveListener attachedLSCount := int32(0) @@ -1112,7 +1253,6 @@ func ComputeDesiredGatewayStatus( desiredConditions := ComputeGatewayConditions(gw, effectiveListeners, hasAddress) - // Build desired listener statuses for Gateway's own listeners var desiredListenerStatuses []gatewayv1.ListenerStatus for _, l := range gw.Spec.Listeners { var matchedEl *EffectiveListener @@ -1133,31 +1273,6 @@ func ComputeDesiredGatewayStatus( copy(conds, matchedEl.Conditions) } - // Find old listener status to preserve LastTransitionTime - var oldListener *gatewayv1.ListenerStatus - for _, ol := range desiredStatus.Listeners { - if ol.Name == l.Name { - oldListener = &ol - break - } - } - - if oldListener != nil { - for i, nc := range conds { - for _, oc := range oldListener.Conditions { - if oc.Type == nc.Type && oc.Status == nc.Status && !oc.LastTransitionTime.IsZero() { - conds[i].LastTransitionTime = oc.LastTransitionTime - break - } - } - } - } - for i := range conds { - if conds[i].LastTransitionTime.IsZero() { - conds[i].LastTransitionTime = metav1.Now() - } - } - desiredListenerStatuses = append(desiredListenerStatuses, gatewayv1.ListenerStatus{ Name: l.Name, SupportedKinds: supportedKinds, @@ -1166,7 +1281,6 @@ func ComputeDesiredGatewayStatus( }) } - // Gateway accepted check for addresses gwAccepted := false for _, cond := range desiredConditions { if cond.Type == string(gatewayv1.GatewayConditionAccepted) && cond.Status == metav1.ConditionTrue { @@ -1180,30 +1294,12 @@ func ComputeDesiredGatewayStatus( desiredAddresses = providedAddresses } - // Update conditions preserving LastTransitionTime - conditionsChanged := SetConditions(&desiredStatus.Conditions, desiredConditions) - - updated := conditionsChanged - - // Check addresses - if !reflectAddressesEqual(desiredStatus.Addresses, desiredAddresses) { - desiredStatus.Addresses = desiredAddresses - updated = true + return gatewayv1.GatewayStatus{ + Conditions: desiredConditions, + Listeners: desiredListenerStatuses, + Addresses: desiredAddresses, + AttachedListenerSets: &attachedLSCount, } - - // Check listeners - if !reflectListenerStatusesEqual(desiredStatus.Listeners, desiredListenerStatuses) { - desiredStatus.Listeners = desiredListenerStatuses - updated = true - } - - // Check AttachedListenerSets - if desiredStatus.AttachedListenerSets == nil || *desiredStatus.AttachedListenerSets != attachedLSCount { - desiredStatus.AttachedListenerSets = &attachedLSCount - updated = true - } - - return *desiredStatus, updated } // ComputeDesiredListenerSetStatus computes the desired ListenerSetStatus from the compiled model. @@ -1212,12 +1308,7 @@ func ComputeDesiredListenerSetStatus( parentGW *gatewayv1.Gateway, namespaces map[string]*corev1.Namespace, compiledGw *CompiledGateway, -) (gatewayv1.ListenerSetStatus, bool) { - desiredStatus := ls.Status.DeepCopy() - if desiredStatus == nil { - desiredStatus = &gatewayv1.ListenerSetStatus{} - } - +) gatewayv1.ListenerSetStatus { var effectiveListeners []*EffectiveListener if compiledGw != nil { effectiveListeners = compiledGw.EffectiveListeners @@ -1240,11 +1331,6 @@ func ComputeDesiredListenerSetStatus( } if matchedEl == nil { - // If an allowed ListenerSet listener is not found in the compiled model, - // it indicates an internal inconsistency (e.g. missing compiled Gateway model). - // We skip setting status for this listener rather than attempting a fallback - // re-validation without secrets or a ReferenceGrant validator, which would - // incorrectly report TLS certificate references as invalid. continue } @@ -1253,30 +1339,6 @@ func ComputeDesiredListenerSetStatus( conds := make([]metav1.Condition, len(matchedEl.Conditions)) copy(conds, matchedEl.Conditions) - var oldListener *gatewayv1.ListenerEntryStatus - for _, ol := range desiredStatus.Listeners { - if ol.Name == l.Name { - oldListener = &ol - break - } - } - - if oldListener != nil { - for i, nc := range conds { - for _, oc := range oldListener.Conditions { - if oc.Type == nc.Type && oc.Status == nc.Status && !oc.LastTransitionTime.IsZero() { - conds[i].LastTransitionTime = oc.LastTransitionTime - break - } - } - } - } - for i := range conds { - if conds[i].LastTransitionTime.IsZero() { - conds[i].LastTransitionTime = metav1.Now() - } - } - desiredListeners = append(desiredListeners, gatewayv1.ListenerEntryStatus{ Name: l.Name, SupportedKinds: supportedKinds, @@ -1286,15 +1348,10 @@ func ComputeDesiredListenerSetStatus( } } - conditionsChanged := SetConditions(&desiredStatus.Conditions, desiredConditions) - updated := conditionsChanged - - if !reflectListenerEntryStatusesEqual(desiredStatus.Listeners, desiredListeners) { - desiredStatus.Listeners = desiredListeners - updated = true + return gatewayv1.ListenerSetStatus{ + Conditions: desiredConditions, + Listeners: desiredListeners, } - - return *desiredStatus, updated } // ComputeDesiredHTTPRouteStatus computes the desired HTTPRouteStatus from the compiled model. @@ -1302,12 +1359,7 @@ func ComputeDesiredHTTPRouteStatus( route *gatewayv1.HTTPRoute, compiledRoute *CompiledRoute, controllerName string, -) (gatewayv1.HTTPRouteStatus, bool) { - desiredStatus := route.Status.DeepCopy() - if desiredStatus == nil { - desiredStatus = &gatewayv1.HTTPRouteStatus{} - } - +) gatewayv1.HTTPRouteStatus { var desiredParents []gatewayv1.RouteParentStatus if compiledRoute != nil { for i, parentRef := range route.Spec.ParentRefs { @@ -1326,10 +1378,419 @@ func ComputeDesiredHTTPRouteStatus( } } - newParents, updated := UpdateRouteParentStatuses(desiredStatus.Parents, desiredParents, route.Namespace, gatewayv1.GatewayController(controllerName)) - desiredStatus.Parents = newParents + return gatewayv1.HTTPRouteStatus{ + Parents: desiredParents, + } +} + +// ComputeDesiredBackendTLSPolicyStatus computes the desired BackendTLSPolicy PolicyStatus. +func ComputeDesiredBackendTLSPolicyStatus( + policy *gatewayv1.BackendTLSPolicy, + compiled *CompiledModel, + configMaps map[types.NamespacedName]*corev1.ConfigMap, + allPolicies []*gatewayv1.BackendTLSPolicy, + controllerName string, +) gatewayv1.PolicyStatus { + isConflicted := false + var conflictingPolicy string + for _, targetRef := range policy.Spec.TargetRefs { + if string(targetRef.Group) != "" && string(targetRef.Group) != gatewayv1.GroupName { + continue + } + if string(targetRef.Kind) != "Service" { + continue + } + + targetSvcNamespace := policy.Namespace + targetSvcName := string(targetRef.Name) + + for _, p := range allPolicies { + if p == nil || (p.Namespace == policy.Namespace && p.Name == policy.Name) { + continue + } + + for _, t := range p.Spec.TargetRefs { + if (string(t.Group) == "" || string(t.Group) == gatewayv1.GroupName) && string(t.Kind) == "Service" { + if p.Namespace == targetSvcNamespace && string(t.Name) == targetSvcName { + if p.CreationTimestamp.Time.Before(policy.CreationTimestamp.Time) { + isConflicted = true + conflictingPolicy = fmt.Sprintf("%s/%s", p.Namespace, p.Name) + break + } + if p.CreationTimestamp.Time.Equal(policy.CreationTimestamp.Time) { + if p.Namespace < policy.Namespace || (p.Namespace == policy.Namespace && p.Name < policy.Name) { + isConflicted = true + conflictingPolicy = fmt.Sprintf("%s/%s", p.Namespace, p.Name) + break + } + } + } + } + } + if isConflicted { + break + } + } + if isConflicted { + break + } + } + + var unresolvedRefs []string + for _, caRef := range policy.Spec.Validation.CACertificateRefs { + if string(caRef.Group) == "" && string(caRef.Kind) == "ConfigMap" { + cmKey := types.NamespacedName{Namespace: policy.Namespace, Name: string(caRef.Name)} + cm, ok := configMaps[cmKey] + if !ok || cm == nil { + unresolvedRefs = append(unresolvedRefs, string(caRef.Name)) + } else { + var data []byte + if d, ok := cm.Data["ca.crt"]; ok { + data = []byte(d) + } else if d, ok := cm.BinaryData["ca.crt"]; ok { + data = d + } - return *desiredStatus, updated + if len(data) == 0 { + unresolvedRefs = append(unresolvedRefs, string(caRef.Name)) + } else { + block, _ := pem.Decode(data) + if block == nil || block.Type != "CERTIFICATE" { + unresolvedRefs = append(unresolvedRefs, string(caRef.Name)) + } + } + } + } else { + unresolvedRefs = append(unresolvedRefs, string(caRef.Name)) + } + } + + acceptedStatus := metav1.ConditionTrue + acceptedReason := string(gatewayv1.PolicyReasonAccepted) + acceptedMessage := "Policy accepted" + + resolvedRefsStatus := metav1.ConditionTrue + resolvedRefsReason := string(gatewayv1.BackendTLSPolicyReasonResolvedRefs) + resolvedRefsMessage := "All references resolved" + + if len(unresolvedRefs) > 0 { + acceptedStatus = metav1.ConditionFalse + acceptedReason = "NoValidCACertificate" + acceptedMessage = fmt.Sprintf("Unresolved or invalid CA certificate references: %v", unresolvedRefs) + + resolvedRefsStatus = metav1.ConditionFalse + resolvedRefsReason = "InvalidCACertificateRef" + resolvedRefsMessage = fmt.Sprintf("Unresolved or invalid CA certificate references: %v", unresolvedRefs) + } + + if isConflicted { + acceptedStatus = metav1.ConditionFalse + acceptedReason = string(gatewayv1.PolicyReasonConflicted) + acceptedMessage = fmt.Sprintf("Conflicted with older policy: %s", conflictingPolicy) + } + + var ancestors []gatewayv1.PolicyAncestorStatus + + if compiled != nil { + for _, cg := range compiled.Gateways { + if cg == nil || cg.Gateway == nil { + continue + } + gw := cg.Gateway + + usesPolicy := false + for _, cr := range compiled.HTTPRoutes { + if cr == nil || cr.HTTPRoute == nil { + continue + } + route := cr.HTTPRoute + + routeMatchesGw := false + for i, pref := range route.Spec.ParentRefs { + if string(pref.Name) == gw.Name && (ValueOf(pref.Namespace) == "" || string(ValueOf(pref.Namespace)) == gw.Namespace) { + if i < len(cr.ParentConditions) && cr.ParentConditions[i].Status == metav1.ConditionTrue { + routeMatchesGw = true + break + } + } + } + + if routeMatchesGw { + for _, rule := range route.Spec.Rules { + for _, backendRef := range rule.BackendRefs { + if string(ValueOf(backendRef.Kind)) == "Service" || ValueOf(backendRef.Kind) == "" { + svcNs := route.Namespace + if backendRef.Namespace != nil && string(*backendRef.Namespace) != "" { + svcNs = string(*backendRef.Namespace) + } + for _, targetRef := range policy.Spec.TargetRefs { + if svcNs == policy.Namespace && string(backendRef.Name) == string(targetRef.Name) { + usesPolicy = true + break + } + } + } + if usesPolicy { + break + } + } + if usesPolicy { + break + } + } + } + if usesPolicy { + break + } + } + + if usesPolicy { + ancestors = append(ancestors, gatewayv1.PolicyAncestorStatus{ + AncestorRef: gatewayv1.ParentReference{ + Group: Ptr(gatewayv1.Group(gatewayv1.GroupName)), + Kind: Ptr(gatewayv1.Kind("Gateway")), + Namespace: Ptr(gatewayv1.Namespace(gw.Namespace)), + Name: gatewayv1.ObjectName(gw.Name), + }, + ControllerName: gatewayv1.GatewayController(controllerName), + Conditions: []metav1.Condition{ + { + Type: string(gatewayv1.PolicyConditionAccepted), + Status: acceptedStatus, + ObservedGeneration: policy.Generation, + Reason: acceptedReason, + Message: acceptedMessage, + }, + { + Type: string(gatewayv1.BackendTLSPolicyConditionResolvedRefs), + Status: resolvedRefsStatus, + ObservedGeneration: policy.Generation, + Reason: resolvedRefsReason, + Message: resolvedRefsMessage, + }, + }, + }) + } + } + } + + return gatewayv1.PolicyStatus{Ancestors: ancestors} +} + +// ComputeDesiredGatewayClassStatus computes the desired GatewayClass status. +func ComputeDesiredGatewayClassStatus(gc *gatewayv1.GatewayClass, controllerName string) gatewayv1.GatewayClassStatus { + acceptedStatus := metav1.ConditionTrue + acceptedReason := string(gatewayv1.GatewayClassReasonAccepted) + acceptedMessage := "GatewayClass accepted by reference implementation" + + if gc.Spec.ParametersRef != nil { + acceptedStatus = metav1.ConditionFalse + acceptedReason = string(gatewayv1.GatewayClassReasonInvalidParameters) + acceptedMessage = "Invalid parametersRef: parametersRef is not supported" + } + + return gatewayv1.GatewayClassStatus{ + Conditions: []metav1.Condition{ + { + Type: string(gatewayv1.GatewayClassConditionStatusAccepted), + Status: acceptedStatus, + ObservedGeneration: gc.Generation, + Reason: acceptedReason, + Message: acceptedMessage, + }, + }, + } +} + +// MergeGatewayStatus merges desired GatewayStatus into current, preserving LastTransitionTime. +func MergeGatewayStatus(current *gatewayv1.GatewayStatus, desired gatewayv1.GatewayStatus) bool { + updated := false + if SetConditions(¤t.Conditions, desired.Conditions) { + updated = true + } + if !reflectAddressesEqual(current.Addresses, desired.Addresses) { + current.Addresses = desired.Addresses + updated = true + } + if (current.AttachedListenerSets == nil && desired.AttachedListenerSets != nil) || + (current.AttachedListenerSets != nil && desired.AttachedListenerSets == nil) || + (current.AttachedListenerSets != nil && desired.AttachedListenerSets != nil && *current.AttachedListenerSets != *desired.AttachedListenerSets) { + current.AttachedListenerSets = desired.AttachedListenerSets + updated = true + } + + var newListeners []gatewayv1.ListenerStatus + for _, dl := range desired.Listeners { + entry := gatewayv1.ListenerStatus{ + Name: dl.Name, + SupportedKinds: dl.SupportedKinds, + AttachedRoutes: dl.AttachedRoutes, + Conditions: make([]metav1.Condition, len(dl.Conditions)), + } + copy(entry.Conditions, dl.Conditions) + + var matchedOld *gatewayv1.ListenerStatus + for _, ol := range current.Listeners { + if ol.Name == dl.Name { + matchedOld = &ol + break + } + } + if matchedOld != nil { + for k, dc := range entry.Conditions { + for _, oc := range matchedOld.Conditions { + if oc.Type == dc.Type { + if oc.Status == dc.Status { + entry.Conditions[k].LastTransitionTime = oc.LastTransitionTime + } + break + } + } + } + } + for k := range entry.Conditions { + if entry.Conditions[k].LastTransitionTime.IsZero() { + entry.Conditions[k].LastTransitionTime = metav1.Now() + } + } + newListeners = append(newListeners, entry) + } + if !reflectListenerStatusesEqual(current.Listeners, newListeners) { + current.Listeners = newListeners + updated = true + } + return updated +} + +// MergeListenerSetStatus merges desired ListenerSetStatus into current, preserving LastTransitionTime. +func MergeListenerSetStatus(current *gatewayv1.ListenerSetStatus, desired gatewayv1.ListenerSetStatus) bool { + updated := false + if SetConditions(¤t.Conditions, desired.Conditions) { + updated = true + } + var newListeners []gatewayv1.ListenerEntryStatus + for _, dl := range desired.Listeners { + entry := gatewayv1.ListenerEntryStatus{ + Name: dl.Name, + SupportedKinds: dl.SupportedKinds, + AttachedRoutes: dl.AttachedRoutes, + Conditions: make([]metav1.Condition, len(dl.Conditions)), + } + copy(entry.Conditions, dl.Conditions) + + var matchedOld *gatewayv1.ListenerEntryStatus + for _, ol := range current.Listeners { + if ol.Name == dl.Name { + matchedOld = &ol + break + } + } + if matchedOld != nil { + for k, dc := range entry.Conditions { + for _, oc := range matchedOld.Conditions { + if oc.Type == dc.Type { + if oc.Status == dc.Status { + entry.Conditions[k].LastTransitionTime = oc.LastTransitionTime + } + break + } + } + } + } + for k := range entry.Conditions { + if entry.Conditions[k].LastTransitionTime.IsZero() { + entry.Conditions[k].LastTransitionTime = metav1.Now() + } + } + newListeners = append(newListeners, entry) + } + if !reflectListenerEntryStatusesEqual(current.Listeners, newListeners) { + current.Listeners = newListeners + updated = true + } + return updated +} + +// MergeHTTPRouteStatus merges desired HTTPRouteStatus into current, preserving LastTransitionTime and other controllers' parents. +// It returns true if semantic changes occurred that require updating status in the API server. +func MergeHTTPRouteStatus(current *gatewayv1.HTTPRouteStatus, desired gatewayv1.HTTPRouteStatus, routeNamespace string, controllerName gatewayv1.GatewayController) bool { + newParents, updated := UpdateRouteParentStatuses(current.Parents, desired.Parents, routeNamespace, controllerName) + current.Parents = newParents + return updated +} + +// MergeBackendTLSPolicyStatus merges desired PolicyStatus into current, preserving LastTransitionTime and other controllers' ancestors. +func MergeBackendTLSPolicyStatus(current *gatewayv1.PolicyStatus, desired gatewayv1.PolicyStatus, controllerName gatewayv1.GatewayController) bool { + newAncestors, updated := UpdatePolicyAncestors(current.Ancestors, desired.Ancestors, controllerName) + if updated { + current.Ancestors = newAncestors + } + return updated +} + +// MergeGatewayClassStatus merges desired GatewayClassStatus into current, preserving LastTransitionTime. +func MergeGatewayClassStatus(current *gatewayv1.GatewayClassStatus, desired gatewayv1.GatewayClassStatus) bool { + return SetConditions(¤t.Conditions, desired.Conditions) +} + +// GatewayStatusesEqual compares two GatewayStatus objects (ignoring LastTransitionTime). +func GatewayStatusesEqual(a, b gatewayv1.GatewayStatus) bool { + if !ConditionsEqual(a.Conditions, b.Conditions) { + return false + } + if !reflectAddressesEqual(a.Addresses, b.Addresses) { + return false + } + if (a.AttachedListenerSets == nil) != (b.AttachedListenerSets == nil) { + return false + } + if a.AttachedListenerSets != nil && b.AttachedListenerSets != nil && *a.AttachedListenerSets != *b.AttachedListenerSets { + return false + } + return reflectListenerStatusesEqual(a.Listeners, b.Listeners) +} + +// HTTPRouteStatusesEqual compares two HTTPRouteStatus objects (ignoring LastTransitionTime). +func HTTPRouteStatusesEqual(a, b gatewayv1.HTTPRouteStatus) bool { + if len(a.Parents) != len(b.Parents) { + return false + } + for i := range a.Parents { + if !reflectParentReferenceEqual(a.Parents[i].ParentRef, b.Parents[i].ParentRef) || + a.Parents[i].ControllerName != b.Parents[i].ControllerName || + !ConditionsEqual(a.Parents[i].Conditions, b.Parents[i].Conditions) { + return false + } + } + return true +} + +// ListenerSetStatusesEqual compares two ListenerSetStatus objects (ignoring LastTransitionTime). +func ListenerSetStatusesEqual(a, b gatewayv1.ListenerSetStatus) bool { + if !ConditionsEqual(a.Conditions, b.Conditions) { + return false + } + return reflectListenerEntryStatusesEqual(a.Listeners, b.Listeners) +} + +// PolicyStatusesEqual compares two PolicyStatus objects (ignoring LastTransitionTime). +func PolicyStatusesEqual(a, b gatewayv1.PolicyStatus) bool { + if len(a.Ancestors) != len(b.Ancestors) { + return false + } + for i := range a.Ancestors { + if !reflectParentReferenceEqual(a.Ancestors[i].AncestorRef, b.Ancestors[i].AncestorRef) || + a.Ancestors[i].ControllerName != b.Ancestors[i].ControllerName || + !ConditionsEqual(a.Ancestors[i].Conditions, b.Ancestors[i].Conditions) { + return false + } + } + return true +} + +// GatewayClassStatusesEqual compares two GatewayClassStatus objects (ignoring LastTransitionTime). +func GatewayClassStatusesEqual(a, b gatewayv1.GatewayClassStatus) bool { + return ConditionsEqual(a.Conditions, b.Conditions) } func reflectAddressesEqual(a, b []gatewayv1.GatewayStatusAddress) bool { @@ -1390,7 +1851,7 @@ func reflectListenerEntryStatusesEqual(a, b []gatewayv1.ListenerEntryStatus) boo return true } -// CompileModelHelper on State compiles the state snapshot into a CompiledModel. +// CompileModel on State compiles the state snapshot into a CompiledModel. func (s *State) CompileModel(controllerName string) *CompiledModel { gatewaysMap := s.GetGateways() var gateways []*gatewayv1.Gateway @@ -1429,8 +1890,18 @@ func (s *State) CompileModel(controllerName string) *CompiledModel { } } + var gatewayClasses []*gatewayv1.GatewayClass + for _, gc := range s.GetGatewayClasses() { + if gc != nil { + gatewayClasses = append(gatewayClasses, gc) + } + } + return CompileModel(ModelInputs{ + Revision: s.Revision(), Gateways: gateways, + GatewayClasses: gatewayClasses, + GatewayAddresses: s.GetGatewayAddresses(), ListenerSets: listenerSets, HTTPRoutes: routes, Services: s.GetServices(), @@ -1438,6 +1909,7 @@ func (s *State) CompileModel(controllerName string) *CompiledModel { ConfigMaps: s.GetConfigMaps(), Secrets: s.GetSecrets(), Namespaces: s.GetNamespaces(), + ReferenceGrants: nil, RefValidator: s, ControllerName: controllerName, }) diff --git a/pkg/state/compiled_test.go b/pkg/state/compiled_test.go index 313d24b..3d514b4 100644 --- a/pkg/state/compiled_test.go +++ b/pkg/state/compiled_test.go @@ -836,10 +836,7 @@ func TestStatusComputation_PureFunctions(t *testing.T) { // 1. Gateway Status addresses := []gatewayv1.GatewayStatusAddress{{Value: "192.0.2.1"}} - gwStatus, updated := ComputeDesiredGatewayStatus(gw, compiled.Gateways[types.NamespacedName{Namespace: "default", Name: "gw"}], addresses) - if !updated { - t.Errorf("expected Gateway status updated=true") - } + gwStatus := ComputeDesiredGatewayStatus(gw, compiled.Gateways[types.NamespacedName{Namespace: "default", Name: "gw"}], addresses) if len(gwStatus.Conditions) != 2 { t.Errorf("expected 2 Gateway conditions, got %d", len(gwStatus.Conditions)) } @@ -854,10 +851,7 @@ func TestStatusComputation_PureFunctions(t *testing.T) { } // 2. ListenerSet Status - lsStatus, updated := ComputeDesiredListenerSetStatus(ls, gw, nil, compiled.Gateways[types.NamespacedName{Namespace: "default", Name: "gw"}]) - if !updated { - t.Errorf("expected ListenerSet status updated=true") - } + lsStatus := ComputeDesiredListenerSetStatus(ls, gw, nil, compiled.Gateways[types.NamespacedName{Namespace: "default", Name: "gw"}]) if len(lsStatus.Conditions) != 2 { t.Errorf("expected 2 ListenerSet conditions, got %d", len(lsStatus.Conditions)) } @@ -866,10 +860,7 @@ func TestStatusComputation_PureFunctions(t *testing.T) { } // 3. HTTPRoute Status - routeStatus, updated := ComputeDesiredHTTPRouteStatus(route, compiled.HTTPRoutes[types.NamespacedName{Namespace: "default", Name: "r1"}], "example.net/gateway-controller") - if !updated { - t.Errorf("expected HTTPRoute status updated=true") - } + routeStatus := ComputeDesiredHTTPRouteStatus(route, compiled.HTTPRoutes[types.NamespacedName{Namespace: "default", Name: "r1"}], "example.net/gateway-controller") if len(routeStatus.Parents) != 2 { t.Fatalf("expected 2 parent statuses on route, got %d", len(routeStatus.Parents)) } @@ -1232,7 +1223,7 @@ func TestCompileModel_PrecedenceAndConflicts(t *testing.T) { // Check ListenerSet desired status for _, ls := range tc.listenerSets { - lsStatus, _ := ComputeDesiredListenerSetStatus(ls, tc.gateway, map[string]*corev1.Namespace{"default": {ObjectMeta: metav1.ObjectMeta{Name: "default"}}}, cg) + lsStatus := ComputeDesiredListenerSetStatus(ls, tc.gateway, map[string]*corev1.Namespace{"default": {ObjectMeta: metav1.ObjectMeta{Name: "default"}}}, cg) expectedStatus, ok := tc.expectedLSStatus[ls.Name] if ok { var accCond *metav1.Condition @@ -1349,7 +1340,7 @@ func TestCompileModel_ListenerSetReferenceGrant(t *testing.T) { t.Errorf("expected AttachedListenerSets = 0 without grant, got %d", cg.AttachedListenerSets) } - lsStatus, _ := ComputeDesiredListenerSetStatus(ls, gw, namespaces, cg) + lsStatus := ComputeDesiredListenerSetStatus(ls, gw, namespaces, cg) var accCond, progCond *metav1.Condition for i := range lsStatus.Conditions { if lsStatus.Conditions[i].Type == string(gatewayv1.ListenerSetConditionAccepted) { @@ -1397,7 +1388,7 @@ func TestCompileModel_ListenerSetReferenceGrant(t *testing.T) { t.Errorf("expected AttachedListenerSets = 1 with grant, got %d", cg.AttachedListenerSets) } - lsStatus, _ := ComputeDesiredListenerSetStatus(ls, gw, namespaces, cg) + lsStatus := ComputeDesiredListenerSetStatus(ls, gw, namespaces, cg) var accCond, progCond *metav1.Condition for i := range lsStatus.Conditions { if lsStatus.Conditions[i].Type == string(gatewayv1.ListenerSetConditionAccepted) { @@ -1526,7 +1517,7 @@ func TestCompileModel_GatewayUnresolvedCertRouteAttachment(t *testing.T) { if cr == nil { t.Fatalf("compiled route not found") } - routeStatus, _ := ComputeDesiredHTTPRouteStatus(route, cr, "example.net/gateway-controller") + routeStatus := ComputeDesiredHTTPRouteStatus(route, cr, "example.net/gateway-controller") if len(routeStatus.Parents) != 1 { t.Fatalf("expected 1 parent in route status, got %d", len(routeStatus.Parents)) } @@ -1907,7 +1898,7 @@ func TestCompileModel_ListenerSetRouteStatusScopedToParentRef(t *testing.T) { }) crGW := compiled.HTTPRoutes[types.NamespacedName{Namespace: "default", Name: "route-gw"}] - statusGW, _ := ComputeDesiredHTTPRouteStatus(routeGWOnly, crGW, "example.net/gateway-controller") + statusGW := ComputeDesiredHTTPRouteStatus(routeGWOnly, crGW, "example.net/gateway-controller") if len(statusGW.Parents) != 1 { t.Fatalf("expected exactly 1 parent in status for routeGWOnly, got %d", len(statusGW.Parents)) } @@ -1916,7 +1907,7 @@ func TestCompileModel_ListenerSetRouteStatusScopedToParentRef(t *testing.T) { } crLS := compiled.HTTPRoutes[types.NamespacedName{Namespace: "default", Name: "route-ls"}] - statusLS, _ := ComputeDesiredHTTPRouteStatus(routeLSOnly, crLS, "example.net/gateway-controller") + statusLS := ComputeDesiredHTTPRouteStatus(routeLSOnly, crLS, "example.net/gateway-controller") if len(statusLS.Parents) != 1 { t.Fatalf("expected exactly 1 parent in status for routeLSOnly, got %d", len(statusLS.Parents)) } diff --git a/pkg/state/conditions.go b/pkg/state/conditions.go index a1e92aa..51b085f 100644 --- a/pkg/state/conditions.go +++ b/pkg/state/conditions.go @@ -199,7 +199,7 @@ func reflectParentReferenceEqual(a, b gatewayv1.ParentReference, defaultNamespac // UpdateRouteParentStatuses updates an existing slice of RouteParentStatus with desired statuses, // preserving LastTransitionTime for conditions whose Status has not changed. -// Entries belonging to our managed controllers are sorted deterministically by normalized ParentRef. +// Entries belonging to our managed controller are sorted deterministically by normalized ParentRef. // Entries belonging to other controllers are preserved in their existing relative order, after our entries. // If the only difference between existing and desired statuses is order, updated returns false. // It returns the updated slice and a boolean indicating whether any semantic changes occurred. @@ -207,18 +207,8 @@ func UpdateRouteParentStatuses( existing []gatewayv1.RouteParentStatus, desired []gatewayv1.RouteParentStatus, routeNamespace string, - controllerNames ...gatewayv1.GatewayController, + controllerName gatewayv1.GatewayController, ) ([]gatewayv1.RouteParentStatus, bool) { - managedControllers := make(map[gatewayv1.GatewayController]bool) - for _, c := range controllerNames { - managedControllers[c] = true - } - if len(managedControllers) == 0 { - for _, d := range desired { - managedControllers[d.ControllerName] = true - } - } - updated := false var ourEntries []gatewayv1.RouteParentStatus @@ -229,12 +219,15 @@ func UpdateRouteParentStatuses( ControllerName: d.ControllerName, Conditions: make([]metav1.Condition, len(d.Conditions)), } + if entry.ControllerName == "" && controllerName != "" { + entry.ControllerName = controllerName + } copy(entry.Conditions, d.Conditions) // Find matching existing parent status var matchingExisting *gatewayv1.RouteParentStatus for j := range existing { - if existing[j].ControllerName == d.ControllerName && reflectParentReferenceEqual(existing[j].ParentRef, d.ParentRef, routeNamespace) { + if existing[j].ControllerName == entry.ControllerName && reflectParentReferenceEqual(existing[j].ParentRef, entry.ParentRef, routeNamespace) { matchingExisting = &existing[j] break } @@ -248,10 +241,10 @@ func UpdateRouteParentStatuses( } } } else { - if len(matchingExisting.Conditions) != len(d.Conditions) { + if len(matchingExisting.Conditions) != len(entry.Conditions) { updated = true } - for k, dc := range d.Conditions { + for k, dc := range entry.Conditions { found := false for _, ec := range matchingExisting.Conditions { if ec.Type == dc.Type { @@ -286,13 +279,17 @@ func UpdateRouteParentStatuses( SortRouteParentStatuses(ourEntries, routeNamespace) // Check for existing entries: preserve entries from unmanaged controllers in their relative order, - // and detect removals of entries from managed controllers. + // and detect removals of entries from managed controller(s). var otherEntries []gatewayv1.RouteParentStatus for _, e := range existing { - if managedControllers[e.ControllerName] { + if controllerName != "" && e.ControllerName == controllerName { found := false for _, d := range desired { - if d.ControllerName == e.ControllerName && reflectParentReferenceEqual(d.ParentRef, e.ParentRef, routeNamespace) { + dCtrl := d.ControllerName + if dCtrl == "" { + dCtrl = controllerName + } + if dCtrl == e.ControllerName && reflectParentReferenceEqual(d.ParentRef, e.ParentRef, routeNamespace) { found = true break } @@ -300,6 +297,28 @@ func UpdateRouteParentStatuses( if !found { updated = true } + } else if controllerName == "" { + isManaged := false + for _, d := range desired { + if d.ControllerName == e.ControllerName { + isManaged = true + break + } + } + if isManaged { + found := false + for _, d := range desired { + if d.ControllerName == e.ControllerName && reflectParentReferenceEqual(d.ParentRef, e.ParentRef, routeNamespace) { + found = true + break + } + } + if !found { + updated = true + } + } else { + otherEntries = append(otherEntries, e) + } } else { otherEntries = append(otherEntries, e) } @@ -312,3 +331,120 @@ func UpdateRouteParentStatuses( result := append(ourEntries, otherEntries...) return result, updated } + +// UpdatePolicyAncestors updates an existing slice of PolicyAncestorStatus with desired statuses, +// preserving LastTransitionTime for conditions whose Status has not changed. +// Entries belonging to other controllers are preserved. +func UpdatePolicyAncestors( + existing []gatewayv1.PolicyAncestorStatus, + desired []gatewayv1.PolicyAncestorStatus, + controllerName gatewayv1.GatewayController, +) ([]gatewayv1.PolicyAncestorStatus, bool) { + updated := false + var result []gatewayv1.PolicyAncestorStatus + + for _, d := range desired { + entry := gatewayv1.PolicyAncestorStatus{ + AncestorRef: d.AncestorRef, + ControllerName: d.ControllerName, + Conditions: make([]metav1.Condition, len(d.Conditions)), + } + if entry.ControllerName == "" && controllerName != "" { + entry.ControllerName = controllerName + } + copy(entry.Conditions, d.Conditions) + + var matchingExisting *gatewayv1.PolicyAncestorStatus + for j := range existing { + if existing[j].ControllerName == d.ControllerName && reflectParentReferenceEqual(existing[j].AncestorRef, d.AncestorRef) { + matchingExisting = &existing[j] + break + } + } + + if matchingExisting == nil { + updated = true + for k := range entry.Conditions { + if entry.Conditions[k].LastTransitionTime.IsZero() { + entry.Conditions[k].LastTransitionTime = metav1.Now() + } + } + } else { + if len(matchingExisting.Conditions) != len(d.Conditions) { + updated = true + } + for k, dc := range d.Conditions { + found := false + for _, ec := range matchingExisting.Conditions { + if ec.Type == dc.Type { + found = true + if ec.Status == dc.Status { + entry.Conditions[k].LastTransitionTime = ec.LastTransitionTime + } else { + updated = true + if entry.Conditions[k].LastTransitionTime.IsZero() { + entry.Conditions[k].LastTransitionTime = metav1.Now() + } + } + if ec.Status != dc.Status || ec.Reason != dc.Reason || ec.Message != dc.Message || ec.ObservedGeneration != dc.ObservedGeneration { + updated = true + } + break + } + } + if !found { + updated = true + if entry.Conditions[k].LastTransitionTime.IsZero() { + entry.Conditions[k].LastTransitionTime = metav1.Now() + } + } + } + } + result = append(result, entry) + } + + for _, e := range existing { + if controllerName != "" && e.ControllerName == controllerName { + found := false + for _, d := range desired { + if d.ControllerName == e.ControllerName && reflectParentReferenceEqual(d.AncestorRef, e.AncestorRef) { + found = true + break + } + } + if !found { + updated = true + } + } else if controllerName == "" { + isManaged := false + for _, d := range desired { + if d.ControllerName == e.ControllerName { + isManaged = true + break + } + } + if isManaged { + found := false + for _, d := range desired { + if d.ControllerName == e.ControllerName && reflectParentReferenceEqual(d.AncestorRef, e.AncestorRef) { + found = true + break + } + } + if !found { + updated = true + } + } else { + result = append(result, e) + } + } else { + result = append(result, e) + } + } + + if len(existing) != len(result) { + updated = true + } + + return result, updated +} diff --git a/pkg/state/conditions_test.go b/pkg/state/conditions_test.go index 0215f50..ac8307d 100644 --- a/pkg/state/conditions_test.go +++ b/pkg/state/conditions_test.go @@ -156,7 +156,7 @@ func TestUpdateRouteParentStatuses(t *testing.T) { }, } - result, updated := UpdateRouteParentStatuses(existing, desired, "") + result, updated := UpdateRouteParentStatuses(existing, desired, "", controller) if !updated { t.Errorf("expected updated=true when adding ResolvedRefs condition") } @@ -178,7 +178,7 @@ func TestUpdateRouteParentStatuses(t *testing.T) { }, }, } - resultBumped, updatedBumped := UpdateRouteParentStatuses(result, desiredBumped, "") + resultBumped, updatedBumped := UpdateRouteParentStatuses(result, desiredBumped, "", controller) if !updatedBumped { t.Errorf("expected updated=true when observedGeneration changes") } diff --git a/pkg/state/gateway.go b/pkg/state/gateway.go index b8a4374..56494e2 100644 --- a/pkg/state/gateway.go +++ b/pkg/state/gateway.go @@ -673,8 +673,17 @@ func (s *GatewayState) BuildInternalState(routes []*HTTPRouteState, services map secrets = st.GetSecrets() } + var gcs []*gatewayv1.GatewayClass + if s.Gateway != nil { + gcs = append(gcs, &gatewayv1.GatewayClass{ + ObjectMeta: metav1.ObjectMeta{Name: string(s.Spec.GatewayClassName)}, + Spec: gatewayv1.GatewayClassSpec{ControllerName: gatewayv1.GatewayController(controllerName)}, + }) + } + compiled := CompileModel(ModelInputs{ Gateways: []*gatewayv1.Gateway{s.Gateway}, + GatewayClasses: gcs, ListenerSets: listenerSetsList, HTTPRoutes: httpRoutes, Services: services, diff --git a/pkg/state/referencegrant.go b/pkg/state/referencegrant.go index 7bff3cd..d3596bf 100644 --- a/pkg/state/referencegrant.go +++ b/pkg/state/referencegrant.go @@ -32,6 +32,14 @@ type ReferenceGrantValidator interface { IsReferencePermitted(from, to Reference) bool } +type mapReferenceValidator struct { + referenceGrants map[types.NamespacedName]*gatewayv1beta1.ReferenceGrant +} + +func (v mapReferenceValidator) IsReferencePermitted(from, to Reference) bool { + return isReferencePermitted(from, to, v.referenceGrants) +} + // IsReferencePermitted checks whether a reference from one resource to another is permitted by the state's ReferenceGrants. func (s *State) IsReferencePermitted(from, to Reference) bool { // References within the same namespace are always permitted. diff --git a/pkg/state/state.go b/pkg/state/state.go index 13671ae..a94e9e7 100644 --- a/pkg/state/state.go +++ b/pkg/state/state.go @@ -15,20 +15,97 @@ package state import ( + "context" + "crypto/tls" + "reflect" "sort" "sync" + "time" corev1 "k8s.io/api/core/v1" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" + toolscache "k8s.io/client-go/tools/cache" + "k8s.io/client-go/util/workqueue" + "k8s.io/klog/v2" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/reconcile" + "sigs.k8s.io/controller-runtime/pkg/source" gatewayv1 "sigs.k8s.io/gateway-api/apis/v1" gatewayv1beta1 "sigs.k8s.io/gateway-api/apis/v1beta1" ) +// ProxyConfigUpdater defines the interface needed by State to update the proxy. +type ProxyConfigUpdater interface { + UpdateConfig(listeners []InternalListener, routes []InternalRoute) + UpdateCertificates(certs map[string]*tls.Certificate, defaultCert *tls.Certificate) +} + +// EventSource implements source.TypedSource[reconcile.Request]. +// It delivers reconcile requests directly to controller workqueues without dropping events. +// If requests arrive before a controller starts, they are buffered as pending and flushed upon Start. +type EventSource struct { + mu sync.Mutex + queues []workqueue.TypedRateLimitingInterface[reconcile.Request] + pending map[reconcile.Request]struct{} +} + +func NewEventSource() *EventSource { + return &EventSource{ + pending: make(map[reconcile.Request]struct{}), + } +} + +// Start is called by controller-runtime when starting a controller. +func (s *EventSource) Start(ctx context.Context, q workqueue.TypedRateLimitingInterface[reconcile.Request]) error { + s.mu.Lock() + s.queues = append(s.queues, q) + // Flush any pending requests that arrived before Start was called + for req := range s.pending { + q.Add(req) + } + s.mu.Unlock() + + go func() { + <-ctx.Done() + s.mu.Lock() + defer s.mu.Unlock() + for i, existing := range s.queues { + if existing == q { + s.queues = append(s.queues[:i], s.queues[i+1:]...) + break + } + } + }() + + return nil +} + +// Enqueue adds a reconcile.Request to all active controller workqueues, +// or buffers it if no workqueues are active yet. +func (s *EventSource) Enqueue(req reconcile.Request) { + s.mu.Lock() + defer s.mu.Unlock() + if len(s.queues) == 0 { + s.pending[req] = struct{}{} + return + } + for _, q := range s.queues { + q.Add(req) + } +} + +var _ source.TypedSource[reconcile.Request] = (*EventSource)(nil) + type State struct { mu sync.RWMutex + recomputeMu sync.Mutex + + revision uint64 + + gatewayClasses map[types.NamespacedName]*gatewayv1.GatewayClass gateways map[types.NamespacedName]*GatewayState + gatewayAddresses map[types.NamespacedName][]gatewayv1.GatewayStatusAddress listenerSets map[types.NamespacedName]*ListenerSetState httpRoutes map[types.NamespacedName]*HTTPRouteState backendTLSPolicies map[types.NamespacedName]*gatewayv1.BackendTLSPolicy @@ -37,34 +114,733 @@ type State struct { secrets map[types.NamespacedName]*corev1.Secret referenceGrants map[types.NamespacedName]*gatewayv1beta1.ReferenceGrant namespaces map[string]*corev1.Namespace + + controllerName string + proxy ProxyConfigUpdater + onGatewaysUpdate func([]*gatewayv1.Gateway) + gatewayFilter func(gw *gatewayv1.Gateway) bool + + previousOutputs *Outputs + + gatewaySource *EventSource + httpRouteSource *EventSource + listenerSetSource *EventSource + backendTLSPolicySource *EventSource + gatewayClassSource *EventSource + + registrations []toolscache.ResourceEventHandlerRegistration + notifyCh chan struct{} + synced bool + running bool } func NewState() *State { return &State{ - gateways: make(map[types.NamespacedName]*GatewayState), - listenerSets: make(map[types.NamespacedName]*ListenerSetState), - httpRoutes: make(map[types.NamespacedName]*HTTPRouteState), - backendTLSPolicies: make(map[types.NamespacedName]*gatewayv1.BackendTLSPolicy), - services: make(map[types.NamespacedName]*corev1.Service), - configMaps: make(map[types.NamespacedName]*corev1.ConfigMap), - secrets: make(map[types.NamespacedName]*corev1.Secret), - referenceGrants: make(map[types.NamespacedName]*gatewayv1beta1.ReferenceGrant), - namespaces: make(map[string]*corev1.Namespace), + gatewayClasses: make(map[types.NamespacedName]*gatewayv1.GatewayClass), + gateways: make(map[types.NamespacedName]*GatewayState), + gatewayAddresses: make(map[types.NamespacedName][]gatewayv1.GatewayStatusAddress), + listenerSets: make(map[types.NamespacedName]*ListenerSetState), + httpRoutes: make(map[types.NamespacedName]*HTTPRouteState), + backendTLSPolicies: make(map[types.NamespacedName]*gatewayv1.BackendTLSPolicy), + services: make(map[types.NamespacedName]*corev1.Service), + configMaps: make(map[types.NamespacedName]*corev1.ConfigMap), + secrets: make(map[types.NamespacedName]*corev1.Secret), + referenceGrants: make(map[types.NamespacedName]*gatewayv1beta1.ReferenceGrant), + namespaces: make(map[string]*corev1.Namespace), + gatewaySource: NewEventSource(), + httpRouteSource: NewEventSource(), + listenerSetSource: NewEventSource(), + backendTLSPolicySource: NewEventSource(), + gatewayClassSource: NewEventSource(), + notifyCh: make(chan struct{}, 1), + synced: true, + running: false, + } +} + +func (s *State) SetControllerName(name string) { + s.mu.Lock() + defer s.mu.Unlock() + s.controllerName = name +} + +func (s *State) SetProxy(p ProxyConfigUpdater) { + s.mu.Lock() + defer s.mu.Unlock() + s.proxy = p +} + +func (s *State) SetOnGatewaysUpdate(fn func([]*gatewayv1.Gateway)) { + s.mu.Lock() + defer s.mu.Unlock() + s.onGatewaysUpdate = fn +} + +func (s *State) SetGatewayFilter(fn func(gw *gatewayv1.Gateway) bool) { + s.mu.Lock() + defer s.mu.Unlock() + s.gatewayFilter = fn +} + +func (s *State) AddRegistration(reg toolscache.ResourceEventHandlerRegistration) { + s.mu.Lock() + defer s.mu.Unlock() + s.registrations = append(s.registrations, reg) +} + +func (s *State) SetSynced(synced bool) { + s.mu.Lock() + s.synced = synced + s.mu.Unlock() + if synced { + s.Recompute() + } +} + +func (s *State) Revision() uint64 { + s.mu.RLock() + defer s.mu.RUnlock() + return s.revision +} + +func (s *State) GatewaySource() *EventSource { + return s.gatewaySource +} + +func (s *State) HTTPRouteSource() *EventSource { + return s.httpRouteSource +} + +func (s *State) ListenerSetSource() *EventSource { + return s.listenerSetSource +} + +func (s *State) BackendTLSPolicySource() *EventSource { + return s.backendTLSPolicySource +} + +func (s *State) GatewayClassSource() *EventSource { + return s.gatewayClassSource +} + +func (s *State) triggerRecomputeLocked() { + if !s.synced { + return + } + if !s.running { + s.recomputeAsyncLocked() + return + } + select { + case s.notifyCh <- struct{}{}: + default: + } +} + +func (s *State) recomputeAsyncLocked() { + // Snapshot inputs and release s.mu before running Recompute + go s.Recompute() +} + +// Start runs the background coalescing loop and recomputes outputs when inputs change. +func (s *State) Start(ctx context.Context) error { + s.mu.Lock() + s.running = true + s.mu.Unlock() + + // Wait for all informer event handler registrations to be fully synced. + for _, reg := range s.registrations { + if !toolscache.WaitForCacheSync(ctx.Done(), reg.HasSynced) { + s.mu.Lock() + s.running = false + s.mu.Unlock() + return nil + } + } + + // Mark state as synced and perform the initial full recomputation. + s.SetSynced(true) + + const coalesceDelay = 10 * time.Millisecond + for { + select { + case <-ctx.Done(): + s.mu.Lock() + s.running = false + s.mu.Unlock() + return nil + case <-s.notifyCh: + timer := time.NewTimer(coalesceDelay) + select { + case <-ctx.Done(): + timer.Stop() + s.mu.Lock() + s.running = false + s.mu.Unlock() + return nil + case <-timer.C: + } + + // Drain any extra notification + select { + case <-s.notifyCh: + default: + } + + s.Recompute() + } + } +} + +// Recompute synchronously computes all outputs from recorded inputs, diffs against previous outputs, +// sends events for changed objects, and updates the proxy. +func (s *State) Recompute() *Outputs { + s.recomputeMu.Lock() + defer s.recomputeMu.Unlock() + + s.mu.Lock() + if !s.synced { + s.mu.Unlock() + return nil + } + + start := time.Now() + inputs := s.snapshotInputsLocked() + outputs := ComputeOutputs(inputs) + outputs.Revision = inputs.Revision + + duration := time.Since(start) + klog.V(2).Infof("Recomputed state outputs in %v (revision: %d, gateways: %d, httpRoutes: %d)", + duration, outputs.Revision, len(inputs.Gateways), len(inputs.HTTPRoutes)) + + s.diffAndEmitLocked(outputs) + + prev := s.previousOutputs + proxyChanged := prev == nil || + !reflect.DeepEqual(outputs.ProxyListeners, prev.ProxyListeners) || + !reflect.DeepEqual(outputs.ProxyRoutes, prev.ProxyRoutes) || + !certsMapEqual(outputs.CertificatesMap, prev.CertificatesMap) || + !defaultCertEqual(outputs.DefaultCert, prev.DefaultCert) + + gatewaysChanged := prev == nil || !reflectGatewaysEqual(outputs.ResolvedGateways, prev.ResolvedGateways) + + p := s.proxy + onGatewaysUpdate := s.onGatewaysUpdate + + s.previousOutputs = outputs + s.mu.Unlock() + + // Run callbacks outside the lock to prevent deadlocks and avoid blocking state mutations. + if proxyChanged && p != nil { + p.UpdateConfig(outputs.ProxyListeners, outputs.ProxyRoutes) + p.UpdateCertificates(outputs.CertificatesMap, outputs.DefaultCert) + } + + if gatewaysChanged && onGatewaysUpdate != nil { + onGatewaysUpdate(outputs.ResolvedGateways) + } + + return outputs +} + +func (s *State) snapshotInputsLocked() ModelInputs { + var gws []*gatewayv1.Gateway + for _, gwState := range s.gateways { + if gwState != nil && gwState.Gateway != nil { + gws = append(gws, gwState.Gateway) + } + } + sort.Slice(gws, func(i, j int) bool { + if gws[i].Namespace != gws[j].Namespace { + return gws[i].Namespace < gws[j].Namespace + } + return gws[i].Name < gws[j].Name + }) + + var gcs []*gatewayv1.GatewayClass + for _, gc := range s.gatewayClasses { + if gc != nil { + gcs = append(gcs, gc) + } + } + sort.Slice(gcs, func(i, j int) bool { + return gcs[i].Name < gcs[j].Name + }) + + gwAddrs := make(map[types.NamespacedName][]gatewayv1.GatewayStatusAddress) + for k, v := range s.gatewayAddresses { + copied := make([]gatewayv1.GatewayStatusAddress, len(v)) + copy(copied, v) + gwAddrs[k] = copied + } + + var listenerSets []*gatewayv1.ListenerSet + for _, lsState := range s.listenerSets { + if lsState != nil && lsState.ListenerSet != nil { + listenerSets = append(listenerSets, lsState.ListenerSet) + } + } + // Precedence order: + // 1. Creation time (oldest first) + // 2. Alphabetically by "{namespace}/{name}" + sort.Slice(listenerSets, func(i, j int) bool { + if !listenerSets[i].CreationTimestamp.Equal(&listenerSets[j].CreationTimestamp) { + return listenerSets[i].CreationTimestamp.Before(&listenerSets[j].CreationTimestamp) + } + if listenerSets[i].Namespace != listenerSets[j].Namespace { + return listenerSets[i].Namespace < listenerSets[j].Namespace + } + return listenerSets[i].Name < listenerSets[j].Name + }) + + var routes []*gatewayv1.HTTPRoute + for _, rState := range s.httpRoutes { + if rState != nil && rState.HTTPRoute != nil { + routes = append(routes, rState.HTTPRoute) + } + } + // Precedence order: + // 1. Creation time (oldest first) + // 2. Alphabetically by "{namespace}/{name}" + sort.Slice(routes, func(i, j int) bool { + if !routes[i].CreationTimestamp.Equal(&routes[j].CreationTimestamp) { + return routes[i].CreationTimestamp.Before(&routes[j].CreationTimestamp) + } + if routes[i].Namespace != routes[j].Namespace { + return routes[i].Namespace < routes[j].Namespace + } + return routes[i].Name < routes[j].Name + }) + + var backendTLSPolicies []*gatewayv1.BackendTLSPolicy + for _, b := range s.backendTLSPolicies { + if b != nil { + backendTLSPolicies = append(backendTLSPolicies, b) + } + } + sort.Slice(backendTLSPolicies, func(i, j int) bool { + if !backendTLSPolicies[i].CreationTimestamp.Equal(&backendTLSPolicies[j].CreationTimestamp) { + return backendTLSPolicies[i].CreationTimestamp.Before(&backendTLSPolicies[j].CreationTimestamp) + } + if backendTLSPolicies[i].Namespace != backendTLSPolicies[j].Namespace { + return backendTLSPolicies[i].Namespace < backendTLSPolicies[j].Namespace + } + return backendTLSPolicies[i].Name < backendTLSPolicies[j].Name + }) + + services := make(map[types.NamespacedName]*corev1.Service) + for k, v := range s.services { + services[k] = v + } + + configMaps := make(map[types.NamespacedName]*corev1.ConfigMap) + for k, v := range s.configMaps { + configMaps[k] = v + } + + secrets := make(map[types.NamespacedName]*corev1.Secret) + for k, v := range s.secrets { + secrets[k] = v + } + + namespaces := make(map[string]*corev1.Namespace) + for k, v := range s.namespaces { + namespaces[k] = v + } + + referenceGrants := make(map[types.NamespacedName]*gatewayv1beta1.ReferenceGrant) + for k, v := range s.referenceGrants { + referenceGrants[k] = v + } + + return ModelInputs{ + Revision: s.revision, + Gateways: gws, + GatewayClasses: gcs, + GatewayAddresses: gwAddrs, + ListenerSets: listenerSets, + HTTPRoutes: routes, + Services: services, + BackendTLSPolicies: backendTLSPolicies, + ConfigMaps: configMaps, + Secrets: secrets, + Namespaces: namespaces, + ReferenceGrants: referenceGrants, + RefValidator: mapReferenceValidator{referenceGrants: referenceGrants}, + ControllerName: s.controllerName, } } +func (s *State) diffAndEmitLocked(outputs *Outputs) { + prev := s.previousOutputs + + // 1. Gateway diff + var prevGWStatuses map[types.NamespacedName]gatewayv1.GatewayStatus + if prev != nil { + prevGWStatuses = prev.GatewayStatuses + } + allGWKeys := make(map[types.NamespacedName]bool) + for k := range outputs.GatewayStatuses { + allGWKeys[k] = true + } + if prevGWStatuses != nil { + for k := range prevGWStatuses { + allGWKeys[k] = true + } + } + for k := range allGWKeys { + cur, curOk := outputs.GatewayStatuses[k] + old, oldOk := prevGWStatuses[k] + if curOk != oldOk || (curOk && !GatewayStatusesEqual(cur, old)) { + s.gatewaySource.Enqueue(ctrl.Request{NamespacedName: k}) + } + } + + // 2. HTTPRoute diff + var prevRouteStatuses map[types.NamespacedName]gatewayv1.HTTPRouteStatus + if prev != nil { + prevRouteStatuses = prev.HTTPRouteStatuses + } + allRouteKeys := make(map[types.NamespacedName]bool) + for k := range outputs.HTTPRouteStatuses { + allRouteKeys[k] = true + } + if prevRouteStatuses != nil { + for k := range prevRouteStatuses { + allRouteKeys[k] = true + } + } + for k := range allRouteKeys { + cur, curOk := outputs.HTTPRouteStatuses[k] + old, oldOk := prevRouteStatuses[k] + if curOk != oldOk || (curOk && !HTTPRouteStatusesEqual(cur, old)) { + s.httpRouteSource.Enqueue(ctrl.Request{NamespacedName: k}) + } + } + + // 3. ListenerSet diff + var prevLSStatuses map[types.NamespacedName]gatewayv1.ListenerSetStatus + if prev != nil { + prevLSStatuses = prev.ListenerSetStatuses + } + allLSKeys := make(map[types.NamespacedName]bool) + for k := range outputs.ListenerSetStatuses { + allLSKeys[k] = true + } + if prevLSStatuses != nil { + for k := range prevLSStatuses { + allLSKeys[k] = true + } + } + for k := range allLSKeys { + cur, curOk := outputs.ListenerSetStatuses[k] + old, oldOk := prevLSStatuses[k] + if curOk != oldOk || (curOk && !ListenerSetStatusesEqual(cur, old)) { + s.listenerSetSource.Enqueue(ctrl.Request{NamespacedName: k}) + } + } + + // 4. BackendTLSPolicy diff + var prevPolicyStatuses map[types.NamespacedName]gatewayv1.PolicyStatus + if prev != nil { + prevPolicyStatuses = prev.BackendTLSPolicyStatuses + } + allPolicyKeys := make(map[types.NamespacedName]bool) + for k := range outputs.BackendTLSPolicyStatuses { + allPolicyKeys[k] = true + } + if prevPolicyStatuses != nil { + for k := range prevPolicyStatuses { + allPolicyKeys[k] = true + } + } + for k := range allPolicyKeys { + cur, curOk := outputs.BackendTLSPolicyStatuses[k] + old, oldOk := prevPolicyStatuses[k] + if curOk != oldOk || (curOk && !PolicyStatusesEqual(cur, old)) { + s.backendTLSPolicySource.Enqueue(ctrl.Request{NamespacedName: k}) + } + } + + // 5. GatewayClass diff + var prevGCStatuses map[types.NamespacedName]gatewayv1.GatewayClassStatus + if prev != nil { + prevGCStatuses = prev.GatewayClassStatuses + } + allGCKeys := make(map[types.NamespacedName]bool) + for k := range outputs.GatewayClassStatuses { + allGCKeys[k] = true + } + if prevGCStatuses != nil { + for k := range prevGCStatuses { + allGCKeys[k] = true + } + } + for k := range allGCKeys { + cur, curOk := outputs.GatewayClassStatuses[k] + old, oldOk := prevGCStatuses[k] + if curOk != oldOk || (curOk && !GatewayClassStatusesEqual(cur, old)) { + s.gatewayClassSource.Enqueue(ctrl.Request{NamespacedName: k}) + } + } +} + +func certsMapEqual(a, b map[string]*tls.Certificate) bool { + if len(a) != len(b) { + return false + } + for k, vA := range a { + vB, ok := b[k] + if !ok || !defaultCertEqual(vA, vB) { + return false + } + } + return true +} + +func defaultCertEqual(a, b *tls.Certificate) bool { + if (a == nil) != (b == nil) { + return false + } + if a == nil { + return true + } + if len(a.Certificate) != len(b.Certificate) { + return false + } + for i := range a.Certificate { + if !reflect.DeepEqual(a.Certificate[i], b.Certificate[i]) { + return false + } + } + return true +} + +func reflectGatewaysEqual(a, b []*gatewayv1.Gateway) bool { + if len(a) != len(b) { + return false + } + for i := range a { + if a[i].Namespace != b[i].Namespace || a[i].Name != b[i].Name || a[i].Generation != b[i].Generation { + return false + } + } + return true +} + +func (s *State) GetDesiredGatewayStatus(key types.NamespacedName) (gatewayv1.GatewayStatus, bool) { + s.mu.RLock() + defer s.mu.RUnlock() + if s.previousOutputs == nil { + return gatewayv1.GatewayStatus{}, false + } + st, ok := s.previousOutputs.GatewayStatuses[key] + return st, ok +} + +func (s *State) GetDesiredHTTPRouteStatus(key types.NamespacedName) (gatewayv1.HTTPRouteStatus, bool) { + s.mu.RLock() + defer s.mu.RUnlock() + if s.previousOutputs == nil { + return gatewayv1.HTTPRouteStatus{}, false + } + st, ok := s.previousOutputs.HTTPRouteStatuses[key] + return st, ok +} + +func (s *State) GetDesiredListenerSetStatus(key types.NamespacedName) (gatewayv1.ListenerSetStatus, bool) { + s.mu.RLock() + defer s.mu.RUnlock() + if s.previousOutputs == nil { + return gatewayv1.ListenerSetStatus{}, false + } + st, ok := s.previousOutputs.ListenerSetStatuses[key] + return st, ok +} + +func (s *State) GetDesiredBackendTLSPolicyStatus(key types.NamespacedName) (gatewayv1.PolicyStatus, bool) { + s.mu.RLock() + defer s.mu.RUnlock() + if s.previousOutputs == nil { + return gatewayv1.PolicyStatus{}, false + } + st, ok := s.previousOutputs.BackendTLSPolicyStatuses[key] + return st, ok +} + +func (s *State) GetDesiredGatewayClassStatus(key types.NamespacedName) (gatewayv1.GatewayClassStatus, bool) { + s.mu.RLock() + defer s.mu.RUnlock() + if s.previousOutputs == nil { + return gatewayv1.GatewayClassStatus{}, false + } + st, ok := s.previousOutputs.GatewayClassStatuses[key] + return st, ok +} + +// Input mutation methods + +func (s *State) UpsertGatewayClass(gc *gatewayv1.GatewayClass) { + if gc == nil { + return + } + s.mu.Lock() + defer s.mu.Unlock() + + key := types.NamespacedName{Name: gc.Name} + old := s.gatewayClasses[key] + if old != nil && reflect.DeepEqual(old.Spec, gc.Spec) && reflect.DeepEqual(old.Labels, gc.Labels) && old.Generation == gc.Generation { + return + } + + s.gatewayClasses[key] = gc.DeepCopy() + s.revision++ + s.triggerRecomputeLocked() +} + +func (s *State) DeleteGatewayClass(name types.NamespacedName) { + s.mu.Lock() + defer s.mu.Unlock() + + key := types.NamespacedName{Name: name.Name} + if _, ok := s.gatewayClasses[key]; ok { + delete(s.gatewayClasses, key) + s.revision++ + s.triggerRecomputeLocked() + } +} + +func (s *State) GetGatewayClass(name string) (*gatewayv1.GatewayClass, bool) { + s.mu.RLock() + defer s.mu.RUnlock() + + gc, ok := s.gatewayClasses[types.NamespacedName{Name: name}] + return gc, ok +} + +func (s *State) GetGatewayClasses() map[types.NamespacedName]*gatewayv1.GatewayClass { + s.mu.RLock() + defer s.mu.RUnlock() + + gcs := make(map[types.NamespacedName]*gatewayv1.GatewayClass) + for k, v := range s.gatewayClasses { + gcs[k] = v + } + return gcs +} + +func (s *State) UpsertGateway(gw *gatewayv1.Gateway) { + if gw == nil { + return + } + s.mu.Lock() + defer s.mu.Unlock() + + key := types.NamespacedName{Namespace: gw.Namespace, Name: gw.Name} + + if s.gatewayFilter != nil && !s.gatewayFilter(gw) { + if _, ok := s.gateways[key]; ok { + delete(s.gateways, key) + delete(s.gatewayAddresses, key) + s.revision++ + s.triggerRecomputeLocked() + } + return + } + + old := s.gateways[key] + if old != nil && old.Gateway != nil && reflect.DeepEqual(old.Spec, gw.Spec) && reflect.DeepEqual(old.Labels, gw.Labels) && old.Generation == gw.Generation { + return + } + + s.gateways[key] = &GatewayState{ + Gateway: gw.DeepCopy(), + } + s.revision++ + s.triggerRecomputeLocked() +} + +func (s *State) DeleteGateway(name types.NamespacedName) { + s.mu.Lock() + defer s.mu.Unlock() + + if _, ok := s.gateways[name]; ok { + delete(s.gateways, name) + delete(s.gatewayAddresses, name) + s.revision++ + s.triggerRecomputeLocked() + } +} + +func (s *State) SetGatewayAddresses(key types.NamespacedName, addrs []gatewayv1.GatewayStatusAddress) { + s.mu.Lock() + defer s.mu.Unlock() + + if reflectAddressesEqual(s.gatewayAddresses[key], addrs) { + return + } + + if len(addrs) == 0 { + delete(s.gatewayAddresses, key) + } else { + copied := make([]gatewayv1.GatewayStatusAddress, len(addrs)) + copy(copied, addrs) + s.gatewayAddresses[key] = copied + } + s.revision++ + s.triggerRecomputeLocked() +} + +func (s *State) DeleteGatewayAddresses(key types.NamespacedName) { + s.mu.Lock() + defer s.mu.Unlock() + + if _, ok := s.gatewayAddresses[key]; ok { + delete(s.gatewayAddresses, key) + s.revision++ + s.triggerRecomputeLocked() + } +} + +func (s *State) GetGatewayAddresses() map[types.NamespacedName][]gatewayv1.GatewayStatusAddress { + s.mu.RLock() + defer s.mu.RUnlock() + + gwAddrs := make(map[types.NamespacedName][]gatewayv1.GatewayStatusAddress) + for k, v := range s.gatewayAddresses { + copied := make([]gatewayv1.GatewayStatusAddress, len(v)) + copy(copied, v) + gwAddrs[k] = copied + } + return gwAddrs +} + func (s *State) UpsertNamespace(ns *corev1.Namespace) { + if ns == nil { + return + } s.mu.Lock() defer s.mu.Unlock() - s.namespaces[ns.Name] = ns + old := s.namespaces[ns.Name] + if old != nil && reflect.DeepEqual(old.Labels, ns.Labels) { + return + } + + s.namespaces[ns.Name] = ns.DeepCopy() + s.revision++ + s.triggerRecomputeLocked() } func (s *State) DeleteNamespace(name string) { s.mu.Lock() defer s.mu.Unlock() - delete(s.namespaces, name) + if _, ok := s.namespaces[name]; ok { + delete(s.namespaces, name) + s.revision++ + s.triggerRecomputeLocked() + } } func (s *State) GetNamespaces() map[string]*corev1.Namespace { @@ -79,31 +855,61 @@ func (s *State) GetNamespaces() map[string]*corev1.Namespace { } func (s *State) UpsertReferenceGrant(rg *gatewayv1beta1.ReferenceGrant) { + if rg == nil { + return + } s.mu.Lock() defer s.mu.Unlock() - s.referenceGrants[types.NamespacedName{Namespace: rg.Namespace, Name: rg.Name}] = rg + key := types.NamespacedName{Namespace: rg.Namespace, Name: rg.Name} + old := s.referenceGrants[key] + if old != nil && reflect.DeepEqual(old.Spec, rg.Spec) && reflect.DeepEqual(old.Labels, rg.Labels) && old.Generation == rg.Generation { + return + } + + s.referenceGrants[key] = rg.DeepCopy() + s.revision++ + s.triggerRecomputeLocked() } func (s *State) DeleteReferenceGrant(name types.NamespacedName) { s.mu.Lock() defer s.mu.Unlock() - delete(s.referenceGrants, name) + if _, ok := s.referenceGrants[name]; ok { + delete(s.referenceGrants, name) + s.revision++ + s.triggerRecomputeLocked() + } } func (s *State) UpsertSecret(secret *corev1.Secret) { + if secret == nil { + return + } s.mu.Lock() defer s.mu.Unlock() - s.secrets[types.NamespacedName{Namespace: secret.Namespace, Name: secret.Name}] = secret + key := types.NamespacedName{Namespace: secret.Namespace, Name: secret.Name} + old := s.secrets[key] + if old != nil && reflect.DeepEqual(old.Data, secret.Data) && reflect.DeepEqual(old.Labels, secret.Labels) { + return + } + + s.secrets[key] = secret.DeepCopy() + s.revision++ + s.triggerRecomputeLocked() } func (s *State) DeleteSecret(name types.NamespacedName) { s.mu.Lock() defer s.mu.Unlock() - delete(s.secrets, name) + if _, ok := s.secrets[name]; ok { + delete(s.secrets, name) + s.revision++ + s.triggerRecomputeLocked() + } } func (s *State) GetSecrets() map[types.NamespacedName]*corev1.Secret { @@ -118,17 +924,32 @@ func (s *State) GetSecrets() map[types.NamespacedName]*corev1.Secret { } func (s *State) UpsertConfigMap(cm *corev1.ConfigMap) { + if cm == nil { + return + } s.mu.Lock() defer s.mu.Unlock() - s.configMaps[types.NamespacedName{Namespace: cm.Namespace, Name: cm.Name}] = cm + key := types.NamespacedName{Namespace: cm.Namespace, Name: cm.Name} + old := s.configMaps[key] + if old != nil && reflect.DeepEqual(old.Data, cm.Data) && reflect.DeepEqual(old.BinaryData, cm.BinaryData) && reflect.DeepEqual(old.Labels, cm.Labels) { + return + } + + s.configMaps[key] = cm.DeepCopy() + s.revision++ + s.triggerRecomputeLocked() } func (s *State) DeleteConfigMap(name types.NamespacedName) { s.mu.Lock() defer s.mu.Unlock() - delete(s.configMaps, name) + if _, ok := s.configMaps[name]; ok { + delete(s.configMaps, name) + s.revision++ + s.triggerRecomputeLocked() + } } func (s *State) GetConfigMaps() map[types.NamespacedName]*corev1.ConfigMap { @@ -143,17 +964,32 @@ func (s *State) GetConfigMaps() map[types.NamespacedName]*corev1.ConfigMap { } func (s *State) UpsertService(svc *corev1.Service) { + if svc == nil { + return + } s.mu.Lock() defer s.mu.Unlock() - s.services[types.NamespacedName{Namespace: svc.Namespace, Name: svc.Name}] = svc + key := types.NamespacedName{Namespace: svc.Namespace, Name: svc.Name} + old := s.services[key] + if old != nil && reflect.DeepEqual(old.Spec.Ports, svc.Spec.Ports) && old.Spec.ClusterIP == svc.Spec.ClusterIP && reflect.DeepEqual(old.Labels, svc.Labels) && reflect.DeepEqual(old.Annotations, svc.Annotations) { + return + } + + s.services[key] = svc.DeepCopy() + s.revision++ + s.triggerRecomputeLocked() } func (s *State) DeleteService(name types.NamespacedName) { s.mu.Lock() defer s.mu.Unlock() - delete(s.services, name) + if _, ok := s.services[name]; ok { + delete(s.services, name) + s.revision++ + s.triggerRecomputeLocked() + } } func (s *State) GetService(name types.NamespacedName) *corev1.Service { @@ -163,50 +999,24 @@ func (s *State) GetService(name types.NamespacedName) *corev1.Service { return s.services[name] } -func (s *State) UpsertGateway(gw *gatewayv1.Gateway) { - s.mu.Lock() - defer s.mu.Unlock() - - s.gateways[types.NamespacedName{Namespace: gw.Namespace, Name: gw.Name}] = &GatewayState{ - Gateway: gw, +func (s *State) UpsertHTTPRoute(route *gatewayv1.HTTPRoute) { + if route == nil { + return } -} - -func (s *State) DeleteGateway(name types.NamespacedName) { s.mu.Lock() defer s.mu.Unlock() - delete(s.gateways, name) -} - -func (s *State) getBackendTLSPoliciesLocked() []*gatewayv1.BackendTLSPolicy { - var policies []*gatewayv1.BackendTLSPolicy - for _, p := range s.backendTLSPolicies { - policies = append(policies, p) + key := types.NamespacedName{Namespace: route.Namespace, Name: route.Name} + old := s.httpRoutes[key] + if old != nil && old.HTTPRoute != nil && reflect.DeepEqual(old.Spec, route.Spec) && reflect.DeepEqual(old.Labels, route.Labels) && old.Generation == route.Generation { + return } - return policies -} - -type lockedReferenceValidator struct { - referenceGrants map[types.NamespacedName]*gatewayv1beta1.ReferenceGrant -} -func (v lockedReferenceValidator) IsReferencePermitted(from, to Reference) bool { - return isReferencePermitted(from, to, v.referenceGrants) -} - -func (s *State) UpsertHTTPRoute(route *gatewayv1.HTTPRoute) metav1.Condition { - s.mu.Lock() - defer s.mu.Unlock() - - rs := &HTTPRouteState{ - HTTPRoute: route, + s.httpRoutes[key] = &HTTPRouteState{ + HTTPRoute: route.DeepCopy(), } - rs.Compile(s.services, s.getBackendTLSPoliciesLocked(), s.configMaps, lockedReferenceValidator{referenceGrants: s.referenceGrants}) - - s.httpRoutes[types.NamespacedName{Namespace: route.Namespace, Name: route.Name}] = rs - - return rs.Internal.ValidationCondition + s.revision++ + s.triggerRecomputeLocked() } func (s *State) GetHTTPRoute(name types.NamespacedName) *HTTPRouteState { @@ -220,21 +1030,40 @@ func (s *State) DeleteHTTPRoute(name types.NamespacedName) { s.mu.Lock() defer s.mu.Unlock() - delete(s.httpRoutes, name) + if _, ok := s.httpRoutes[name]; ok { + delete(s.httpRoutes, name) + s.revision++ + s.triggerRecomputeLocked() + } } func (s *State) UpsertBackendTLSPolicy(policy *gatewayv1.BackendTLSPolicy) { + if policy == nil { + return + } s.mu.Lock() defer s.mu.Unlock() - s.backendTLSPolicies[types.NamespacedName{Namespace: policy.Namespace, Name: policy.Name}] = policy + key := types.NamespacedName{Namespace: policy.Namespace, Name: policy.Name} + old := s.backendTLSPolicies[key] + if old != nil && reflect.DeepEqual(old.Spec, policy.Spec) && reflect.DeepEqual(old.Labels, policy.Labels) && old.Generation == policy.Generation { + return + } + + s.backendTLSPolicies[key] = policy.DeepCopy() + s.revision++ + s.triggerRecomputeLocked() } func (s *State) DeleteBackendTLSPolicy(name types.NamespacedName) { s.mu.Lock() defer s.mu.Unlock() - delete(s.backendTLSPolicies, name) + if _, ok := s.backendTLSPolicies[name]; ok { + delete(s.backendTLSPolicies, name) + s.revision++ + s.triggerRecomputeLocked() + } } func (s *State) GetGateways() []*GatewayState { @@ -257,8 +1086,9 @@ func (s *State) GetHTTPRoutes() []*HTTPRouteState { routes = append(routes, route) } - // Sort by creation timestamp, then by namespace/name to ensure deterministic order - // and follow Gateway API precedence rules for ties. + // Precedence order: + // 1. Creation time (oldest first) + // 2. Alphabetically by "{namespace}/{name}" sort.Slice(routes, func(i, j int) bool { if !routes[i].CreationTimestamp.Equal(&routes[j].CreationTimestamp) { return routes[i].CreationTimestamp.Before(&routes[j].CreationTimestamp) @@ -284,19 +1114,34 @@ func (s *State) GetBackendTLSPolicies() []*gatewayv1.BackendTLSPolicy { } func (s *State) UpsertListenerSet(ls *gatewayv1.ListenerSet) { + if ls == nil { + return + } s.mu.Lock() defer s.mu.Unlock() - s.listenerSets[types.NamespacedName{Namespace: ls.Namespace, Name: ls.Name}] = &ListenerSetState{ - ListenerSet: ls, + key := types.NamespacedName{Namespace: ls.Namespace, Name: ls.Name} + old := s.listenerSets[key] + if old != nil && old.ListenerSet != nil && reflect.DeepEqual(old.Spec, ls.Spec) && reflect.DeepEqual(old.Labels, ls.Labels) && old.Generation == ls.Generation { + return } + + s.listenerSets[key] = &ListenerSetState{ + ListenerSet: ls.DeepCopy(), + } + s.revision++ + s.triggerRecomputeLocked() } func (s *State) DeleteListenerSet(name types.NamespacedName) { s.mu.Lock() defer s.mu.Unlock() - delete(s.listenerSets, name) + if _, ok := s.listenerSets[name]; ok { + delete(s.listenerSets, name) + s.revision++ + s.triggerRecomputeLocked() + } } func (s *State) GetListenerSet(name types.NamespacedName) *ListenerSetState { diff --git a/pkg/state/state_benchmark_test.go b/pkg/state/state_benchmark_test.go new file mode 100644 index 0000000..60805e7 --- /dev/null +++ b/pkg/state/state_benchmark_test.go @@ -0,0 +1,114 @@ +// Copyright 2026 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package state + +import ( + "fmt" + "testing" + + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + gatewayv1 "sigs.k8s.io/gateway-api/apis/v1" +) + +func BenchmarkState_Recompute(b *testing.B) { + st := NewState() + st.SetControllerName("example.net/gateway-controller") + + // Setup 100 Gateways and 1000 HTTPRoutes (10 routes per gateway) + const numGateways = 100 + const routesPerGateway = 10 + + for g := 0; g < numGateways; g++ { + gwName := fmt.Sprintf("gw-%d", g) + ns := fmt.Sprintf("ns-%d", g%10) + + st.UpsertNamespace(&corev1.Namespace{ + ObjectMeta: metav1.ObjectMeta{Name: ns}, + }) + + gw := &gatewayv1.Gateway{ + ObjectMeta: metav1.ObjectMeta{Namespace: ns, Name: gwName}, + Spec: gatewayv1.GatewaySpec{ + Listeners: []gatewayv1.Listener{ + { + Name: "http", + Port: 80, + Protocol: gatewayv1.HTTPProtocolType, + AllowedRoutes: &gatewayv1.AllowedRoutes{ + Namespaces: &gatewayv1.RouteNamespaces{ + From: Ptr(gatewayv1.NamespacesFromAll), + }, + }, + }, + }, + }, + } + st.UpsertGateway(gw) + st.SetGatewayAddresses(types.NamespacedName{Namespace: ns, Name: gwName}, []gatewayv1.GatewayStatusAddress{ + {Value: fmt.Sprintf("192.0.2.%d", g%250)}, + }) + + svcName := fmt.Sprintf("svc-%d", g) + st.UpsertService(&corev1.Service{ + ObjectMeta: metav1.ObjectMeta{Namespace: ns, Name: svcName}, + }) + + for r := 0; r < routesPerGateway; r++ { + rName := fmt.Sprintf("route-%d-%d", g, r) + route := &gatewayv1.HTTPRoute{ + ObjectMeta: metav1.ObjectMeta{Namespace: ns, Name: rName}, + Spec: gatewayv1.HTTPRouteSpec{ + CommonRouteSpec: gatewayv1.CommonRouteSpec{ + ParentRefs: []gatewayv1.ParentReference{ + { + Namespace: Ptr(gatewayv1.Namespace(ns)), + Name: gatewayv1.ObjectName(gwName), + }, + }, + }, + Rules: []gatewayv1.HTTPRouteRule{ + { + Matches: []gatewayv1.HTTPRouteMatch{ + { + Path: &gatewayv1.HTTPPathMatch{ + Type: Ptr(gatewayv1.PathMatchPathPrefix), + Value: Ptr(fmt.Sprintf("/v%d", r)), + }, + }, + }, + BackendRefs: []gatewayv1.HTTPBackendRef{ + { + BackendRef: gatewayv1.BackendRef{ + BackendObjectReference: gatewayv1.BackendObjectReference{ + Name: gatewayv1.ObjectName(svcName), + }, + }, + }, + }, + }, + }, + }, + } + st.UpsertHTTPRoute(route) + } + } + + b.ResetTimer() + for i := 0; i < b.N; i++ { + st.Recompute() + } +} diff --git a/pkg/state/state_test.go b/pkg/state/state_test.go new file mode 100644 index 0000000..39f4023 --- /dev/null +++ b/pkg/state/state_test.go @@ -0,0 +1,588 @@ +// Copyright 2026 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package state + +import ( + "testing" + "time" + + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/util/workqueue" + "sigs.k8s.io/controller-runtime/pkg/reconcile" + gatewayv1 "sigs.k8s.io/gateway-api/apis/v1" + gatewayv1beta1 "sigs.k8s.io/gateway-api/apis/v1beta1" +) + +type testQueue struct { + workqueue.TypedRateLimitingInterface[reconcile.Request] +} + +func newTestQueue() *testQueue { + return &testQueue{ + TypedRateLimitingInterface: workqueue.NewTypedRateLimitingQueue[reconcile.Request](workqueue.DefaultTypedControllerRateLimiter[reconcile.Request]()), + } +} + +func (q *testQueue) drain() []types.NamespacedName { + var res []types.NamespacedName + for q.Len() > 0 { + item, _ := q.Get() + res = append(res, item.NamespacedName) + q.Done(item) + } + return res +} + +func TestComputeOutputs_UnchangedInputsProduceNoEvents(t *testing.T) { + st := NewState() + st.SetControllerName("example.net/gateway-controller") + + qGC := newTestQueue() + qGW := newTestQueue() + qRoute := newTestQueue() + + ctx := t.Context() + _ = st.GatewayClassSource().Start(ctx, qGC) + _ = st.GatewaySource().Start(ctx, qGW) + _ = st.HTTPRouteSource().Start(ctx, qRoute) + + gc := &gatewayv1.GatewayClass{ + ObjectMeta: metav1.ObjectMeta{Name: "my-class"}, + Spec: gatewayv1.GatewayClassSpec{ControllerName: "example.net/gateway-controller"}, + } + gw := &gatewayv1.Gateway{ + ObjectMeta: metav1.ObjectMeta{Namespace: "default", Name: "my-gw"}, + Spec: gatewayv1.GatewaySpec{ + GatewayClassName: "my-class", + Listeners: []gatewayv1.Listener{ + {Name: "http", Port: 80, Protocol: gatewayv1.HTTPProtocolType}, + }, + }, + } + route := &gatewayv1.HTTPRoute{ + ObjectMeta: metav1.ObjectMeta{Namespace: "default", Name: "my-route"}, + Spec: gatewayv1.HTTPRouteSpec{ + CommonRouteSpec: gatewayv1.CommonRouteSpec{ + ParentRefs: []gatewayv1.ParentReference{ + {Name: "my-gw"}, + }, + }, + }, + } + + st.UpsertGatewayClass(gc) + st.UpsertGateway(gw) + st.UpsertHTTPRoute(route) + st.Recompute() + + // Initial drain + qGC.drain() + qGW.drain() + qRoute.drain() + + // Recompute with unchanged inputs + st.Recompute() + + gcEvents := qGC.drain() + gwEvents := qGW.drain() + routeEvents := qRoute.drain() + + if len(gcEvents) != 0 { + t.Errorf("expected 0 GatewayClass events for unchanged inputs, got %d", len(gcEvents)) + } + if len(gwEvents) != 0 { + t.Errorf("expected 0 Gateway events for unchanged inputs, got %d", len(gwEvents)) + } + if len(routeEvents) != 0 { + t.Errorf("expected 0 HTTPRoute events for unchanged inputs, got %d", len(routeEvents)) + } + + // Calling Upsert with identical object should not bump revision + revBefore := st.Revision() + st.UpsertGateway(gw) + st.UpsertHTTPRoute(route) + if st.Revision() != revBefore { + t.Errorf("expected revision unchanged on duplicate upsert: before=%d, after=%d", revBefore, st.Revision()) + } +} + +func TestComputeOutputs_SingleChangeProducesEventsOnlyForAffectedObjects(t *testing.T) { + st := NewState() + st.SetControllerName("example.net/gateway-controller") + + qGC := newTestQueue() + qGW := newTestQueue() + + ctx := t.Context() + _ = st.GatewayClassSource().Start(ctx, qGC) + _ = st.GatewaySource().Start(ctx, qGW) + + gc := &gatewayv1.GatewayClass{ + ObjectMeta: metav1.ObjectMeta{Name: "my-class"}, + Spec: gatewayv1.GatewayClassSpec{ControllerName: "example.net/gateway-controller"}, + } + gw := &gatewayv1.Gateway{ + ObjectMeta: metav1.ObjectMeta{Namespace: "default", Name: "my-gw"}, + Spec: gatewayv1.GatewaySpec{ + GatewayClassName: "my-class", + Listeners: []gatewayv1.Listener{ + {Name: "http", Port: 80, Protocol: gatewayv1.HTTPProtocolType}, + }, + }, + } + gw2 := &gatewayv1.Gateway{ + ObjectMeta: metav1.ObjectMeta{Namespace: "default", Name: "other-gw"}, + Spec: gatewayv1.GatewaySpec{ + GatewayClassName: "my-class", + Listeners: []gatewayv1.Listener{ + {Name: "http", Port: 8080, Protocol: gatewayv1.HTTPProtocolType}, + }, + }, + } + + st.UpsertGatewayClass(gc) + st.UpsertGateway(gw) + st.UpsertGateway(gw2) + st.Recompute() + + qGW.drain() + qGC.drain() + + // Change only gw2 addresses + st.SetGatewayAddresses(types.NamespacedName{Namespace: "default", Name: "other-gw"}, []gatewayv1.GatewayStatusAddress{ + {Value: "10.0.0.2"}, + }) + st.Recompute() + + gwEvents := qGW.drain() + if len(gwEvents) != 1 { + t.Fatalf("expected exactly 1 Gateway event for other-gw, got %d (%v)", len(gwEvents), gwEvents) + } + if gwEvents[0].Name != "other-gw" { + t.Errorf("expected event for other-gw, got %s", gwEvents[0].Name) + } + + gcEvents := qGC.drain() + if len(gcEvents) != 0 { + t.Errorf("expected 0 GatewayClass events when Gateway addresses change, got %d", len(gcEvents)) + } +} + +func TestEventSource_BuffersEventsBeforeStart(t *testing.T) { + source := NewEventSource() + + req1 := reconcile.Request{NamespacedName: types.NamespacedName{Namespace: "default", Name: "gw-1"}} + req2 := reconcile.Request{NamespacedName: types.NamespacedName{Namespace: "default", Name: "gw-2"}} + + // Enqueue before Start + source.Enqueue(req1) + source.Enqueue(req2) + source.Enqueue(req1) // duplicate should dedupe in set + + q := newTestQueue() + ctx := t.Context() + if err := source.Start(ctx, q); err != nil { + t.Fatalf("Start failed: %v", err) + } + + items := q.drain() + if len(items) != 2 { + t.Fatalf("expected 2 buffered items flushed into workqueue, got %d (%v)", len(items), items) + } +} + +func TestState_NamespaceLabelChangeDependencyWithoutMapping(t *testing.T) { + st := NewState() + st.SetControllerName("example.net/gateway-controller") + + qRoute := newTestQueue() + ctx := t.Context() + _ = st.HTTPRouteSource().Start(ctx, qRoute) + + gc := &gatewayv1.GatewayClass{ + ObjectMeta: metav1.ObjectMeta{Name: "my-class"}, + Spec: gatewayv1.GatewayClassSpec{ControllerName: "example.net/gateway-controller"}, + } + + gw := &gatewayv1.Gateway{ + ObjectMeta: metav1.ObjectMeta{Namespace: "infra-ns", Name: "prod-gw"}, + Spec: gatewayv1.GatewaySpec{ + GatewayClassName: "my-class", + Listeners: []gatewayv1.Listener{ + { + Name: "http", + Port: 80, + Protocol: gatewayv1.HTTPProtocolType, + AllowedRoutes: &gatewayv1.AllowedRoutes{ + Namespaces: &gatewayv1.RouteNamespaces{ + From: Ptr(gatewayv1.NamespacesFromSelector), + Selector: &metav1.LabelSelector{ + MatchLabels: map[string]string{"env": "prod"}, + }, + }, + }, + }, + }, + }, + } + + ns := &corev1.Namespace{ + ObjectMeta: metav1.ObjectMeta{ + Name: "app-ns", + Labels: map[string]string{"env": "dev"}, + }, + } + + route := &gatewayv1.HTTPRoute{ + ObjectMeta: metav1.ObjectMeta{Namespace: "app-ns", Name: "app-route"}, + Spec: gatewayv1.HTTPRouteSpec{ + CommonRouteSpec: gatewayv1.CommonRouteSpec{ + ParentRefs: []gatewayv1.ParentReference{ + { + Namespace: Ptr(gatewayv1.Namespace("infra-ns")), + Name: "prod-gw", + }, + }, + }, + }, + } + + st.UpsertGatewayClass(gc) + st.UpsertNamespace(ns) + st.UpsertGateway(gw) + st.UpsertHTTPRoute(route) + st.Recompute() + + // Route should initially be not accepted because namespace label is env: dev + routeStatus, ok := st.GetDesiredHTTPRouteStatus(types.NamespacedName{Namespace: "app-ns", Name: "app-route"}) + if !ok || len(routeStatus.Parents) != 1 { + t.Fatalf("expected route status present, got %+v", routeStatus) + } + if routeStatus.Parents[0].Conditions[0].Status != metav1.ConditionFalse { + t.Fatalf("expected route initially Accepted=False, got %v", routeStatus.Parents[0].Conditions[0]) + } + + qRoute.drain() + + // Mutate namespace label to env: prod + nsUpdated := &corev1.Namespace{ + ObjectMeta: metav1.ObjectMeta{ + Name: "app-ns", + Labels: map[string]string{"env": "prod"}, + }, + } + st.UpsertNamespace(nsUpdated) + st.Recompute() + + // Central recomputation should automatically detect route is now accepted and emit an event! + routeEvents := qRoute.drain() + if len(routeEvents) != 1 { + t.Fatalf("expected 1 HTTPRoute event on namespace label change, got %d", len(routeEvents)) + } + if routeEvents[0].Namespace != "app-ns" || routeEvents[0].Name != "app-route" { + t.Errorf("expected event for app-ns/app-route, got %v", routeEvents[0]) + } + + routeStatusAfter, _ := st.GetDesiredHTTPRouteStatus(types.NamespacedName{Namespace: "app-ns", Name: "app-route"}) + if routeStatusAfter.Parents[0].Conditions[0].Status != metav1.ConditionTrue { + t.Errorf("expected route Accepted=True after namespace label change, got %v", routeStatusAfter.Parents[0].Conditions[0]) + } +} + +func TestState_ReferenceGrantDeletionDependencyWithoutMapping(t *testing.T) { + st := NewState() + st.SetControllerName("example.net/gateway-controller") + + qRoute := newTestQueue() + ctx := t.Context() + _ = st.HTTPRouteSource().Start(ctx, qRoute) + + gc := &gatewayv1.GatewayClass{ + ObjectMeta: metav1.ObjectMeta{Name: "my-class"}, + Spec: gatewayv1.GatewayClassSpec{ControllerName: "example.net/gateway-controller"}, + } + + gw := &gatewayv1.Gateway{ + ObjectMeta: metav1.ObjectMeta{Namespace: "default", Name: "my-gw"}, + Spec: gatewayv1.GatewaySpec{ + GatewayClassName: "my-class", + Listeners: []gatewayv1.Listener{ + {Name: "http", Port: 80, Protocol: gatewayv1.HTTPProtocolType}, + }, + }, + } + + svc := &corev1.Service{ + ObjectMeta: metav1.ObjectMeta{Namespace: "backend-ns", Name: "my-svc"}, + } + + rg := &gatewayv1beta1.ReferenceGrant{ + ObjectMeta: metav1.ObjectMeta{Namespace: "backend-ns", Name: "allow-route"}, + Spec: gatewayv1beta1.ReferenceGrantSpec{ + From: []gatewayv1beta1.ReferenceGrantFrom{ + { + Group: gatewayv1.GroupName, + Kind: "HTTPRoute", + Namespace: "default", + }, + }, + To: []gatewayv1beta1.ReferenceGrantTo{ + { + Group: "", + Kind: "Service", + Name: Ptr(gatewayv1.ObjectName("my-svc")), + }, + }, + }, + } + + route := &gatewayv1.HTTPRoute{ + ObjectMeta: metav1.ObjectMeta{Namespace: "default", Name: "cross-ns-route"}, + Spec: gatewayv1.HTTPRouteSpec{ + CommonRouteSpec: gatewayv1.CommonRouteSpec{ + ParentRefs: []gatewayv1.ParentReference{ + {Name: "my-gw"}, + }, + }, + Rules: []gatewayv1.HTTPRouteRule{ + { + BackendRefs: []gatewayv1.HTTPBackendRef{ + { + BackendRef: gatewayv1.BackendRef{ + BackendObjectReference: gatewayv1.BackendObjectReference{ + Namespace: Ptr(gatewayv1.Namespace("backend-ns")), + Name: "my-svc", + }, + }, + }, + }, + }, + }, + }, + } + + st.UpsertGatewayClass(gc) + st.UpsertGateway(gw) + st.UpsertService(svc) + st.UpsertReferenceGrant(rg) + st.UpsertHTTPRoute(route) + st.Recompute() + + // Initially ResolvedRefs should be True + statusBefore, ok := st.GetDesiredHTTPRouteStatus(types.NamespacedName{Namespace: "default", Name: "cross-ns-route"}) + if !ok || len(statusBefore.Parents) != 1 { + t.Fatalf("expected route status present") + } + if statusBefore.Parents[0].Conditions[1].Status != metav1.ConditionTrue { + t.Fatalf("expected ResolvedRefs=True, got %v", statusBefore.Parents[0].Conditions[1]) + } + + qRoute.drain() + + // Delete ReferenceGrant + st.DeleteReferenceGrant(types.NamespacedName{Namespace: "backend-ns", Name: "allow-route"}) + st.Recompute() + + // Central recomputation should detect ResolvedRefs became False and emit an event + events := qRoute.drain() + if len(events) != 1 { + t.Fatalf("expected 1 HTTPRoute event on ReferenceGrant deletion, got %d", len(events)) + } + if events[0].Name != "cross-ns-route" { + t.Errorf("expected event for cross-ns-route, got %s", events[0].Name) + } + + statusAfter, _ := st.GetDesiredHTTPRouteStatus(types.NamespacedName{Namespace: "default", Name: "cross-ns-route"}) + if statusAfter.Parents[0].Conditions[1].Status != metav1.ConditionFalse || + statusAfter.Parents[0].Conditions[1].Reason != string(gatewayv1.RouteReasonRefNotPermitted) { + t.Errorf("expected ResolvedRefs=False/RefNotPermitted, got %v", statusAfter.Parents[0].Conditions[1]) + } +} + +func TestState_ConfigMapChangeBackendTLSPolicyDependencyWithoutMapping(t *testing.T) { + st := NewState() + st.SetControllerName("example.net/gateway-controller") + + qPolicy := newTestQueue() + ctx := t.Context() + _ = st.BackendTLSPolicySource().Start(ctx, qPolicy) + + gc := &gatewayv1.GatewayClass{ + ObjectMeta: metav1.ObjectMeta{Name: "my-class"}, + Spec: gatewayv1.GatewayClassSpec{ControllerName: "example.net/gateway-controller"}, + } + + gw := &gatewayv1.Gateway{ + ObjectMeta: metav1.ObjectMeta{Namespace: "default", Name: "my-gw"}, + Spec: gatewayv1.GatewaySpec{ + GatewayClassName: "my-class", + Listeners: []gatewayv1.Listener{ + {Name: "http", Port: 80, Protocol: gatewayv1.HTTPProtocolType}, + }, + }, + } + svc := &corev1.Service{ + ObjectMeta: metav1.ObjectMeta{Namespace: "default", Name: "secure-svc"}, + } + route := &gatewayv1.HTTPRoute{ + ObjectMeta: metav1.ObjectMeta{Namespace: "default", Name: "route"}, + Spec: gatewayv1.HTTPRouteSpec{ + CommonRouteSpec: gatewayv1.CommonRouteSpec{ + ParentRefs: []gatewayv1.ParentReference{{Name: "my-gw"}}, + }, + Rules: []gatewayv1.HTTPRouteRule{ + { + BackendRefs: []gatewayv1.HTTPBackendRef{ + { + BackendRef: gatewayv1.BackendRef{ + BackendObjectReference: gatewayv1.BackendObjectReference{Name: "secure-svc"}, + }, + }, + }, + }, + }, + }, + } + + validCertPEM := "-----BEGIN CERTIFICATE-----\nMIIBkDCB+aADAgECAgEBMA0GCSqGSIb3DQEBCwUAMBMxETAPBgNVBAMMCHNlcnZp\nY2UwHhcNMjYwMTAxMDAwMDAwWhcNMjcwMTAxMDAwMDAwWjATMREwDwYDVQQDDAhz\nZXJ2aWNlMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC3/1234567890ABCDEF\nMA0GCSqGSIb3DQEBCwUAA4GBACb1234567890ABCDEF\n-----END CERTIFICATE-----" + cm := &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{Namespace: "default", Name: "ca-cert-cm"}, + Data: map[string]string{ + "ca.crt": validCertPEM, + }, + } + + policy := &gatewayv1.BackendTLSPolicy{ + ObjectMeta: metav1.ObjectMeta{Namespace: "default", Name: "tls-policy"}, + Spec: gatewayv1.BackendTLSPolicySpec{ + TargetRefs: []gatewayv1.LocalPolicyTargetReferenceWithSectionName{ + { + LocalPolicyTargetReference: gatewayv1.LocalPolicyTargetReference{ + Group: "", + Kind: "Service", + Name: "secure-svc", + }, + }, + }, + Validation: gatewayv1.BackendTLSPolicyValidation{ + CACertificateRefs: []gatewayv1.LocalObjectReference{ + { + Group: "", + Kind: "ConfigMap", + Name: "ca-cert-cm", + }, + }, + Hostname: "secure.example.com", + }, + }, + } + + st.UpsertGatewayClass(gc) + st.UpsertGateway(gw) + st.UpsertService(svc) + st.UpsertHTTPRoute(route) + st.UpsertConfigMap(cm) + st.UpsertBackendTLSPolicy(policy) + st.Recompute() + + // Policy should be Accepted=True + policyStatus, ok := st.GetDesiredBackendTLSPolicyStatus(types.NamespacedName{Namespace: "default", Name: "tls-policy"}) + if !ok || len(policyStatus.Ancestors) != 1 { + t.Fatalf("expected policy status with 1 ancestor, got %+v", policyStatus) + } + if policyStatus.Ancestors[0].Conditions[0].Status != metav1.ConditionTrue { + t.Fatalf("expected policy Accepted=True, got %v", policyStatus.Ancestors[0].Conditions[0]) + } + + qPolicy.drain() + + // Update ConfigMap with invalid data + cmInvalid := &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{Namespace: "default", Name: "ca-cert-cm"}, + Data: map[string]string{ + "ca.crt": "invalid cert data", + }, + } + st.UpsertConfigMap(cmInvalid) + st.Recompute() + + // Central recomputation should detect policy Accepted=False and emit an event + events := qPolicy.drain() + if len(events) != 1 { + t.Fatalf("expected 1 BackendTLSPolicy event on ConfigMap change, got %d", len(events)) + } + if events[0].Name != "tls-policy" { + t.Errorf("expected event for tls-policy, got %s", events[0].Name) + } + + policyStatusAfter, _ := st.GetDesiredBackendTLSPolicyStatus(types.NamespacedName{Namespace: "default", Name: "tls-policy"}) + if policyStatusAfter.Ancestors[0].Conditions[0].Status != metav1.ConditionFalse { + t.Errorf("expected policy Accepted=False after ConfigMap corruption, got %v", policyStatusAfter.Ancestors[0].Conditions[0]) + } +} + +func TestMergeHTTPRouteStatus_DeterministicOrderAndOtherControllers(t *testing.T) { + myCtrl := gatewayv1.GatewayController("example.net/gateway-controller") + otherCtrl := gatewayv1.GatewayController("other.org/other-controller") + + t0 := metav1.NewTime(time.Now().Add(-5 * time.Minute)) + c := []metav1.Condition{ + { + Type: "Accepted", + Status: metav1.ConditionTrue, + Reason: "Accepted", + Message: "Accepted", + ObservedGeneration: 1, + LastTransitionTime: t0, + }, + } + + current := &gatewayv1.HTTPRouteStatus{ + Parents: []gatewayv1.RouteParentStatus{ + {ParentRef: gatewayv1.ParentReference{Name: "gw-other"}, ControllerName: otherCtrl, Conditions: c}, + {ParentRef: gatewayv1.ParentReference{Name: "gw-b"}, ControllerName: myCtrl, Conditions: c}, + {ParentRef: gatewayv1.ParentReference{Name: "gw-a"}, ControllerName: myCtrl, Conditions: c}, + }, + } + + // Desired parents for our controller in unsorted order + desired := gatewayv1.HTTPRouteStatus{ + Parents: []gatewayv1.RouteParentStatus{ + {ParentRef: gatewayv1.ParentReference{Name: "gw-b"}, ControllerName: myCtrl, Conditions: c}, + {ParentRef: gatewayv1.ParentReference{Name: "gw-a"}, ControllerName: myCtrl, Conditions: c}, + }, + } + + updated := MergeHTTPRouteStatus(current, desired, "default", myCtrl) + if updated { + t.Errorf("expected updated=false when only ordering of existing entries differs") + } + + // Current parents should now be deterministically sorted: gw-a, gw-b, then gw-other + if len(current.Parents) != 3 { + t.Fatalf("expected 3 parents, got %d", len(current.Parents)) + } + if current.Parents[0].ParentRef.Name != "gw-a" || current.Parents[0].ControllerName != myCtrl { + t.Errorf("expected current.Parents[0] to be myCtrl/gw-a, got %+v", current.Parents[0]) + } + if current.Parents[1].ParentRef.Name != "gw-b" || current.Parents[1].ControllerName != myCtrl { + t.Errorf("expected current.Parents[1] to be myCtrl/gw-b, got %+v", current.Parents[1]) + } + if current.Parents[2].ParentRef.Name != "gw-other" || current.Parents[2].ControllerName != otherCtrl { + t.Errorf("expected current.Parents[2] to be otherCtrl/gw-other, got %+v", current.Parents[2]) + } +}