Skip to content

Add extension support to local Kind setup #992

Description

@silvi-t

Problem

The testsuite's make local-setup currently deploys the Kuadrant operator via either components (kustomize from GitHub) or helm (official chart). Neither mode supports loading extensions — custom controller binaries injected into the operator container.

On OCP clusters, extensions are deployed via the helm-charts-olm repo, which patches the operator's OLM CSV to add an init container that copies extension binaries from a custom image into an emptyDir volume mounted at /extensions. This OLM-based approach doesn't apply to Kind, but the underlying mechanism is simple: mount the extension binary from a container image into the operator pod.

There is currently no way to test extensions on a local Kind cluster.

Goal

Patch the Kuadrant operator deployment on Kind to:

  1. Add an emptyDir volume (extensions-binary-volume)
  2. Add an init container (copy-extensions) that copies binaries from EXTENSIONS_IMAGE into the volume
  3. Mount the volume at /extensions in the manager container
  4. Apply extension manifests (CRDs, RBAC) from a local file

This replicates what the OCP helm-charts-olm extension patch does, but targeting the Kubernetes Deployment directly instead of an OLM CSV.

Proposed approach

Extensions are opt-in via INSTALL_EXTENSIONS=true, following the same pattern as INSTALL_PROMETHEUS and INSTALL_TRACING.

After the operator is deployed (deploy-kuadrant-operator), apply the extension manifests and patch the kuadrant-operator-controller-manager deployment:

kubectl apply -f $EXTENSIONS_MANIFESTS

kubectl patch deployment kuadrant-operator-controller-manager \
  -n $KUADRANT_NAMESPACE --type=json -p='[
    {
      "op": "add",
      "path": "/spec/template/spec/volumes/-",
      "value": {"name": "extensions-binary-volume", "emptyDir": {}}
    },
    {
      "op": "add",
      "path": "/spec/template/spec/containers/0/volumeMounts/-",
      "value": {"mountPath": "/extensions", "name": "extensions-binary-volume"}
    },
    {
      "op": "add",
      "path": "/spec/template/spec/initContainers",
      "value": [{
        "name": "copy-extensions",
        "command": ["cp", "-r", "/extensions/.", "/export"],
        "image": "$EXTENSIONS_IMAGE",
        "imagePullPolicy": "Always",
        "volumeMounts": [{"mountPath": "/export", "name": "extensions-binary-volume"}]
      }]
    }
  ]'

If INSTALL_EXTENSIONS=true but the file at EXTENSIONS_MANIFESTS doesn't exist, the setup should fail with an error.

New environment variables

Variable Default Description
INSTALL_EXTENSIONS false Enable extension deployment. Set to true to opt in.
EXTENSIONS_IMAGE quay.io/kuadrant/internal-extensions:latest Container image with extension binaries.
EXTENSIONS_MANIFESTS ./extensionsManifests.yaml Path to extension

Scope

  • Add a deploy-extensions target (e.g., in make/kuadrant.mk or a new make/extensions.mk)
  • Call it from local-setup after deploy-kuadrant-operator, conditional on INSTALL_EXTENSIONS=true
  • Validate that EXTENSIONS_MANIFESTS file exists when INSTALL_EXTENSIONS=true
  • Wait for the operator rollout to complete after patching
  • Add extensionsManifests.yaml to .gitignore (user-specific content)
  • Works with both components and helm deployment modes

Usage example

# Default: no extensions
make local-setup

# With extensions (manifests stored locally in repo root)
cp ~/my-extensions/manifests.yaml ./extensionsManifests.yaml
INSTALL_EXTENSIONS=true make local-setup

# With extensions, manifests stored elsewhere
INSTALL_EXTENSIONS=true EXTENSIONS_MANIFESTS=~/my-extensions/manifests.yaml make local-setup

# Custom extensions image
INSTALL_EXTENSIONS=true EXTENSIONS_IMAGE=quay.io/myorg/my-extensions:dev make local-setup

Why

  • Enables testing extensions locally on Kind without needing an OCP cluster or OLM
  • Uses the same init container + emptyDir pattern as the OCP deployment, just applied directly to the Deployment instead of the CSV
  • Follows existing conventions (INSTALL_PROMETHEUS, INSTALL_TRACING) for optional components

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

  • Status
    🆕 New
  • Status
    In Progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions