Skip to content

Central config distribution: data-plane pods should not need cluster-wide read access #637

Closed

Description

@justinsb

Follow-up to #620 / #624.

With per-Gateway data-plane Deployments (#624), each data-plane pod runs the full GARI pipeline (informers, State, ComputeOutputs) filtered to its own Gateway, with status writes disabled. That means the gari-dataplane ServiceAccount needs cluster-wide get/list/watch on Secrets, Services, EndpointSlices, ConfigMaps, Namespaces and all Gateway API kinds.

We are moving the data-plane resources into the Gateway's namespace (see the comment on #624, needed for #635). Then a ServiceAccount bound to that ClusterRole sits in every namespace that has a Gateway. Anyone who can create pods there could mount it and read every Secret in the cluster.

Goal: state is computed only in the controller. Each data plane receives just its own compiled config and needs (almost) no RBAC.

Sketch, open to alternatives:

  • The controller serializes each Gateway's data-plane config from Outputs: proxy listeners and routes, only the certificates that Gateway references, and resolved backend endpoints. It writes this into a per-Gateway Secret in the Gateway's namespace, owned by the Gateway and labelled like the other per-Gateway resources.
  • The data-plane mode watches only that one Secret, via a namespaced Role with resourceNames, and applies it with the existing ProxyConfigUpdater UpdateConfig/UpdateCertificates hooks.
  • Endpoint changes flow through the controller. Mind the write rate: coalesce updates, and consider the 1MiB object size limit. If that becomes a problem, a streaming API from the controller (xDS-like) is the alternative.
  • Running the binary directly, with all Gateways in one process, keeps working unchanged.
  • Remove the gari-dataplane ClusterRole. Data-plane pods get only the namespaced Role.

Not assigned yet. It depends on #624 landing.

Activity

  1. added a commit that references this issue on Oct 10, 2026
    8a34cd1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions