THREESCALE-12447 Add support for one-way SSL connections to external Redis instances by tkan145 · Pull Request #1154 · 3scale/3scale-operator

tkan145 · 2026-03-30T02:53:47Z

What

https://redhat.atlassian.net/browse/THREESCALE-12447

Verification steps

Prepare backend-redis DB

Details

Update backend-redis deployment as follow

$ cd config/dev-database/backend-redis

Update service

9,10c9,11
<     - port: 6379
<       targetPort: 6379
---
>     - port: 6380         # TLS port
>       targetPort: 6380
>       name: redis-tls

Update secrete

8c8
<   REDIS_QUEUES_URL: redis://backend-redis.$(NAMESPACE).svc.cluster.local:6379/1
---
>   REDIS_QUEUES_URL: rediss://backend-redis.$(NAMESPACE).svc.cluster.local:6380/1
11c11
<   REDIS_STORAGE_URL: redis://backend-redis.$(NAMESPACE).svc.cluster.local:6379/2
---
>   REDIS_STORAGE_URL: rediss://backend-redis.$(NAMESPACE).svc.cluster.local:6380/2

Update Deployment

21c21
<             - containerPort: 6379
---
>             - containerPort: 6380
31c31
<               port: 6379
---
>               port: 6380
36c36
<               port: 6379
---
>               port: 6380
41a42,48
>             - name: redis-config-volume
>               mountPath: /etc/redis/redis.conf
>               subPath: redis.conf
>             - name: redis-tls-volume
>               mountPath: /etc/redis/certs
>               readOnly: true
>           command: ["/bin/sh", "-c", "redis-server /etc/redis/redis.conf"]
45c52,58
<             claimName: backend-redis-pvc
---
>             claimName: system-redis-pvc
>         - name: redis-config-volume
>           configMap:
>             name: redis-config-redis
>         - name: redis-tls-volume
>           secret:
>             secretName: redis-tls-secret

Update kuztomization.yaml

9c9,22
<     newName: quay.io/fedora/redis-7
\ No newline at end of file
---
>     newName: quay.io/fedora/redis-7
>
> configMapGenerator:
> - name: redis-config-redis
>   behavior: create
>   files:
>   - redis.conf
>
> secretGenerator:
> - name: redis-tls-secret
>   files:
>   - ./certs/redis-server.crt
>   - ./certs/redis-server.key
>   - ./certs/ca.crt

Create a file called redis.conf with the following content

# redis.conf
bind 0.0.0.0
protected-mode no
port 0
tls-port 6380
tls-cert-file /etc/redis/certs/redis-server.crt
tls-key-file /etc/redis/certs/redis-server.key
tls-ca-cert-file /etc/redis/certs/ca.crt
tls-auth-clients no
stop-writes-on-bgsave-error no
save ""

Generate certs (update the domain to match your cluster)

mkdir certs
cd certs

openssl genpkey -algorithm RSA -out ca.key
openssl req -x509 -key ca.key -out ca.crt -days 365 -subj "/CN=backend-redis.3scale-atra.svc.cluster.local" -addext "subjectAltName=DNS:backend-redis.3scale-atra.svc.cluster.local"

openssl genpkey -algorithm RSA -out backend-redis-client.key
openssl req -new -key backend-redis-client.key -out backend-redis-client.csr -subj "/CN=backend-redis-client.3scale-test.svc.cluster.local"
openssl x509 -req -in backend-redis-client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out backend-redis-client.crt -days 365

openssl genpkey -algorithm RSA -out redis-server.key
openssl req -new -key redis-server.key -out redis-server.csr -subj "/CN=backend-redis.3scale-test.svc.cluster.local" -addext "subjectAltName=DNS:backend-redis.3scale-test.svc.cluster.local"
openssl x509 -req -extfile <(printf "subjectAltName=DNS:backend-redis.3scale-test.svc.cluster.local") -in redis-server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out redis-server.crt -days 365

Prepare system-redis DB

Details

Update system-redis deployment as follow

$ cd config/dev-database/system-redis

Update service

9,10c9,11
<     - port: 6379
<       targetPort: 6379
---
>     - port: 6380         # TLS port
>       targetPort: 6380
>       name: redis-tls

Update secret

8c8
<   URL: redis://system-redis.$(NAMESPACE).svc.cluster.local:6379/1
---
>   URL: rediss://system-redis.$(NAMESPACE).svc.cluster.local:6380/1

Update Deployment

21c21
<             - containerPort: 6379
---
>             - containerPort: 6380
31c31
<               port: 6379
---
>               port: 6380
36c36
<               port: 6379
---
>               port: 6380
41a42,48
>             - name: redis-config-volume
>               mountPath: /etc/redis/redis.conf
>               subPath: redis.conf
>             - name: redis-tls-volume
>               mountPath: /etc/redis/certs
>               readOnly: true
>           command: ["/bin/sh", "-c", "redis-server /etc/redis/redis.conf"]
45c52,58
<             claimName: backend-redis-pvc
---
>             claimName: system-redis-pvc
>         - name: redis-config-volume
>           configMap:
>             name: redis-config-redis
>         - name: redis-tls-volume
>           secret:
>             secretName: redis-tls-secret

Update kuztomization.yaml

9c9,22
<     newName: quay.io/fedora/redis-7
\ No newline at end of file
---
>     newName: quay.io/fedora/redis-7
>
> configMapGenerator:
> - name: redis-config-redis
>   behavior: create
>   files:
>   - redis.conf
>
> secretGenerator:
> - name: redis-tls-secret
>   files:
>   - ./certs/redis-server.crt
>   - ./certs/redis-server.key
>   - ./certs/ca.crt

Create a file called redis.conf with the following content

# redis.conf
bind 0.0.0.0
protected-mode no
port 0
tls-port 6380
tls-cert-file /etc/redis/certs/redis-server.crt
tls-key-file /etc/redis/certs/redis-server.key
tls-ca-cert-file /etc/redis/certs/ca.crt
tls-auth-clients no
stop-writes-on-bgsave-error no
save ""

Generate certs (update the domain to match your cluster)

mkdir certs
cd certs

openssl genpkey -algorithm RSA -out ca.key
openssl req -x509 -key ca.key -out ca.crt -days 365 -subj "/CN=system-redis.3scale-test.svc.cluster.local" -addext "subjectAltName=DNS:system-redis.3scale-test.svc.cluster.local"

openssl genpkey -algorithm RSA -out system-redis-client.key
openssl req -new -key system-redis-client.key -out system-redis-client.csr -subj "/CN=system-redis-client.3scale-test.svc.cluster.local"
openssl x509 -req -in system-redis-client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out system-redis-client.crt -days 365

openssl genpkey -algorithm RSA -out redis-server.key
openssl req -new -key redis-server.key -out redis-server.csr -subj "/CN=system-redis.3scale-test.svc.cluster.local" -addext "subjectAltName=DNS:system-redis.3scale-test.svc.cluster.local"
openssl x509 -req -extfile <(printf "subjectAltName=DNS:system-redis.3scale-test.svc.cluster.local") -in redis-server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out redis-server.crt -days 365

One way TLS

Prepare the namespace

make cluster/prepare/local

Prepare APIM

export NAMESPACE=3scale-test

cat << EOF | oc create -f -
kind: Secret
apiVersion: v1
metadata:
  name: s3-credentials
  namespace: $NAMESPACE
data:
  AWS_ACCESS_KEY_ID: c29tZXRoaW5nCg==
  AWS_BUCKET: c29tZXRoaW5nCg==
  AWS_REGION: dXMtd2VzdC0xCg==
  AWS_SECRET_ACCESS_KEY: c29tZXRoaW5nCg==
type: Opaque
EOF

DOMAIN=$(oc get routes console -n openshift-console -o json | jq -r '.status.ingress[0].routerCanonicalHostname' | sed 's/router-default.//')
cat << EOF | oc create -f -
kind: APIManager
apiVersion: apps.3scale.net/v1alpha1
metadata:
  name: 3scale
  namespace: $NAMESPACE
spec:
  wildcardDomain: $DOMAIN
  apicast:
    stagingSpec:
      replicas: 0
    productionSpec:
      replicas: 0
  backend:
    backendRedisTLSEnabled: true
    queuesRedisTLSEnabled: true
  system:
    systemRedisTLSEnabled: true
  externalComponents:
    backend:
      redis: true
    system:
      database: true
      redis: true
EOF

Start the operator

make run

Check the operator log, you should see the following
backend-redis

2026-04-29T15:02:51+10:00       ERROR   Reconciler error        {"controller": "apimanager", "controllerGroup": "apps.3scale.net", "controllerKind": "APIManager", "APIManager": {"name":"3scale","namespace":"3scale-test"}, "namespace": "3scale-test", "name": "3scale", "reconcileID": "40fe517f-6b9a-457a-a516-5c120619cdce", "error": "validation errors for Redis TLS configuration in 'backend-redis' secret: 'backendRedisTLSEnabled: true' is set in apimanager. Secret validation errors: [Secret field 'REDIS_SSL_CA' is required in secret 'backend-redis' Secret field 'REDIS_SSL_CERT' is required in secret 'backend-redis' Secret field 'REDIS_SSL_KEY' is required in secret 'backend-redis']\n'queuesRedisTLSEnabled: true' is set in apimanager. Secret validation errors: [Secret field 'REDIS_SSL_QUEUES_CA' is required in secret 'backend-redis' Secret field 'REDIS_SSL_QUEUES_CERT' is required in secret 'backend-redis' Secret field 'REDIS_SSL_QUEUES_KEY' is required in secret 'backend-redis']"}

system-redis

2026-04-29T15:06:55+10:00       ERROR   Reconciler error        {"controller": "apimanager", "controllerGroup": "apps.3scale.net", "controllerKind": "APIManager", "APIManager": {"name":"3scale","namespace":"3scale-test"}, "namespace": "3scale-test", "name": "3scale", "reconcileID": "e3900ddc-bd61-4a08-8bfc-22c122603610", "error": "validation errors for Redis TLS configuration in 'system-redis' secret: Secret field 'REDIS_SSL_CA' is required in secret 'system-redis'\nSecret field 'REDIS_SSL_CERT' is required in secret 'system-redis'\nSecret field 'REDIS_SSL_KEY' is required in secret 'system-redis'"}

Edit backend-redis secret and add following fields with empty value
- REDIS_SSL_CA
- REDIS_SSL_CERT
- REDIS_SSL_KEY
- REDIS_SSL_QUEUES_CA
- REDIS_SSL_QUEUES_CERT
- REDIS_SSL_QUEUES_KEY
Edit system-redis secret and add the following keys with empty value
- REDIS_SSL_CA
- REDIS_SSL_CERT
- REDIS_SSL_KEY
Restart the operator

CTRL-C
make run

Now you should see the following

backend-worker

Error connecting to Redis queue storage: Failed to load CA Certificate or CA Path
Error connecting to Redis queue storage: Failed to load CA Certificate or CA Path

backend-cron

Error connecting to Redis queue storage: Failed to load CA Certificate or CA Path
Error connecting to Redis queue storage: Failed to load CA Certificate or CA Path

system-app-pre

Failed to load CA Certificate or CA Path

Stop the operator

CTRL-A

Copy CA cert from system-redis folder and backend-redis folder then update the secret
- backend-redis
  - REDIS_SSL_CA
  - REDIS_SSL_QUEUES_CA
- system-redis
  - REDIS_SSL_CA
Delete system-app jobs
Start the operator again, this time you should see all pods running

make run

mTLS

Update redis.conf and set tls-auth-clients yes to force mTLS connect and repeat steps above
Now you should pods are failing again due to missing certs
Update secret to include client cert/key pair and you shall see all pods running again

Split the single RedisTLSEnabled flag into independent SystemRedisTLS and BackendRedisTLS configs. Conditionally set env var and mount only the required TLS secret items (CA only for one-way, CA+cert+key for mutual TLS).

tkan145 · 2026-03-30T05:57:37Z

/retest

tkan145 requested a review from a team as a code owner March 30, 2026 02:53

tkan145 added 2 commits March 30, 2026 13:08

Support one-way system Redis TLS connection

c3e7e6f

Split the single RedisTLSEnabled flag into independent SystemRedisTLS and BackendRedisTLS configs. Conditionally set env var and mount only the required TLS secret items (CA only for one-way, CA+cert+key for mutual TLS).

Support one-way backend Redis TLS connection

86d09c2

tkan145 force-pushed the THREESCALE-12447 branch from 0005316 to 86d09c2 Compare March 30, 2026 03:09

tkan145 requested a review from urbanikb April 29, 2026 06:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

THREESCALE-12447 Add support for one-way SSL connections to external Redis instances#1154

THREESCALE-12447 Add support for one-way SSL connections to external Redis instances#1154
tkan145 wants to merge 2 commits into3scale:masterfrom
tkan145:THREESCALE-12447

tkan145 commented Mar 30, 2026 •

edited

Loading

Uh oh!

tkan145 commented Mar 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

tkan145 commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Verification steps

Prepare backend-redis DB

Prepare system-redis DB

One way TLS

mTLS

Uh oh!

tkan145 commented Mar 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

tkan145 commented Mar 30, 2026 •

edited

Loading