Aks automatic garc blog - #5925
Steve Griffith (swgriffith) wants to merge 11 commits into
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR adds a new AKS blog post under website/blog/ that provides a hands-on walkthrough for setting up GitHub Actions Runner Controller (ARC) on AKS Automatic and validating an end-to-end GitHub Actions workflow using ephemeral runner pods.
Changes:
- Adds a new tutorial-style blog post covering AKS Automatic cluster creation, ARC install/config, workflow validation, troubleshooting, and cleanup.
- Includes example Helm values and a sample GitHub Actions workflow targeting an ARC runner scale set.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| authors: ["steve-griffith"] | ||
| tags: ["automatic", "github-actions", "arc", "devops"] |
|
One more resiliency note worth calling out somewhere in the post (e.g. near cluster creation or in a "production considerations" aside): avoid spot/preemptible node pools for ARC runner nodes. Spot reclamation can kill a runner pod mid-job; since ARC runners are ephemeral, the pod isn't rescheduled and the job fails and has to be resubmitted from scratch rather than gracefully recovering like a stateless workload would. |
Brian Redmond (chzbrgr71)
left a comment
Brian Redmond (chzbrgr71)
left a comment
There was a problem hiding this comment.
Looks ready to publish. Approved.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.
Suppressed comments (6)
Previously missed (5) — in code that hasn't changed since the last review.
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:11
- This Learn URL is locale-specific (
/en-us/). Repo blog guidelines prefer non-locale Learn URLs to avoid unnecessary redirects and keep links consistent.
The combination of ARC and AKS Automatic gives you the power of ARC on a production-ready AKS cluster with managed node pools, built-in monitoring, scaling, security settings, and other defaults that follow [AKS best practices](https://learn.microsoft.com/en-us/azure/well-architected/service-guides/azure-kubernetes-service). For runner workloads specifically, the [pod readiness SLA](https://learn.microsoft.com/azure/aks/intro-aks-automatic#pod-readiness-sla) is also a strong fit because CI/CD jobs depend on predictable pod startup.
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:4
- The front matter
descriptionlooks significantly shorter than the blog guideline target (150–160 characters), which can hurt SEO consistency across posts. Consider expanding it to a 150–160 character summary.
description: "A hands-on walkthrough for creating an AKS Automatic cluster and running GitHub Actions Runner Controller runner scale sets on it."
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:155
- Creating the secret with
--from-literalexpands the token into thekubectlprocess arguments, which can be observable on the machine (for example viaps) while the command runs. Prefer passing the token via stdin (and optionally unset it afterward).
kubectl create secret generic github-pat \
--namespace "${ARC_RUNNERS_NAMESPACE}" \
--from-literal=github_token="${GITHUB_TOKEN}"
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:17
- The truncate marker line has trailing whitespace, which can trip markdown linters and creates noisy diffs. Remove the trailing space.
<!-- truncate -->
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:19
- The blog post structure guidelines call for a hero image after the truncate marker with descriptive alt text (for accessibility and consistent previews). This post doesn’t include one yet.
<!-- truncate -->
Let’s walk through the full setup.
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:6
- The tag IDs in front matter should match keys in
website/blog/tags.yml.automatic,github-actions,arc, anddevopsdon’t appear to exist there, so the post will be inconsistent with the repo’s centralized tag metadata (and may break tag pages). Use existing tag keys (for example,aks-automatic) and/or add the missing tags totags.yml.
tags: ["automatic", "github-actions", "arc", "devops"]
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
Suppressed comments (2)
website/blog/tags.yml:174
- The
tags.ymlentries appear to be kept in alphabetical order in this section (for example:gateway-api,general, ...).github-actionsshould sort beforegrafana, but it’s currently placed after it, which makes the file harder to scan and maintain.
entra:
label: Entra
permalink: /entra
description: Microsoft Entra identity and access management for AKS clusters and workloads.
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:11
- This link uses a locale-specific Microsoft Learn URL (
/en-us/). Our blog guidelines recommend using locale-agnostic Learn URLs to avoid hard-coding locale in published content.
The combination of ARC and AKS Automatic gives you the power of ARC on a production-ready AKS cluster with managed node pools, built-in monitoring, scaling, security settings, and other defaults that follow [AKS best practices](https://learn.microsoft.com/en-us/azure/well-architected/service-guides/azure-kubernetes-service). For runner workloads specifically, the [pod readiness SLA](https://learn.microsoft.com/azure/aks/intro-aks-automatic#pod-readiness-sla) is also a strong fit because CI/CD jobs depend on predictable pod startup.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
Suppressed comments (3)
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:4
- The front matter
descriptionis ~129 characters. Our blog guidelines call for an SEO description in the 150–160 character range.
description: "A hands-on walkthrough for creating an AKS Automatic cluster and running GitHub Actions Runner Controller runner scale sets on it."
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:17
- The truncate marker has trailing whitespace. Trim it to avoid markdown lint/formatting issues.
<!-- truncate -->
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:19
- This post is missing the hero image that should appear right after
<!-- truncate -->(typically). Also, the truncate marker is after 4 intro paragraphs; guidelines suggest placing it after 2–3 paragraphs so the listing excerpt stays short.
AKS Automatic is secure by default and has production-minded safeguards enabled. That's a good thing, but it also means the default public ARC Helm chart values need a little tuning. In particular, we need to be explicit about resource requests and image tags.
<!-- truncate -->
Let’s walk through the full setup.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
Suppressed comments (2)
Previously missed (2) — in code that hasn't changed since the last review.
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:11
- This link uses a locale-specific Learn URL (
/en-us/). The repo blog guidelines recommend using the generic Learn URL to avoid regional redirects and keep links consistent.
The combination of ARC and AKS Automatic gives you the power of ARC on a production-ready AKS cluster with managed node pools, built-in monitoring, scaling, security settings, and other defaults that follow [AKS best practices](https://learn.microsoft.com/en-us/azure/well-architected/service-guides/azure-kubernetes-service). For runner workloads specifically, the [pod readiness SLA](https://learn.microsoft.com/azure/aks/intro-aks-automatic#pod-readiness-sla) is also a strong fit because CI/CD jobs depend on predictable pod startup.
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:153
- Creating the secret with
--from-literal=..."${GITHUB_TOKEN}"puts the token value on the command line after shell expansion, which can leak via process listing on multi-user systems. Prefer piping to stdin and using--from-fileso the token value isn’t an argv argument.
kubectl create secret generic github-pat \
--namespace "${ARC_RUNNERS_NAMESPACE}" \
--from-literal=github_token="${GITHUB_TOKEN}"
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical and moderate findings affect command correctness, authentication, secret handling, validation, and documentation quality.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (9)
Previously missed (6) — in code that hasn't changed since the last review.
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:103
- AKS Automatic disables local accounts and uses Azure RBAC, but this walkthrough neither grants the current principal an Azure Kubernetes Service RBAC role nor documents the
kubeloginexec-auth setup. On a newly created cluster,kubectl get nodesand the subsequent Helm/kubectl commands cannot authenticate. Add the role assignment andkubeloginprerequisite/configuration before continuing.
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:153 - Passing the PAT with
--from-literalputs the secret in thekubectlprocess arguments, where local users or diagnostic tooling can read it. Since this is presented as the token-storage path, use file/stdin-based input or explicitly warn about this exposure.
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:220 - This only lists
arc-systems, where the controller runs. The listener is created by the scale-set release inarc-runners, so the check does not show the listener even though the next paragraph says it should be running. Add a pod check for${ARC_RUNNERS_NAMESPACE}.
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:270 git pushpublishes the workflow from the current branch, but this dispatch is hard-coded tomain. It fails when the default branch differs or when the lab is run from a feature branch. Dispatch the same branch that was pushed, or define one branch variable and reuse it.
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:280- This immediately selects the first listed run after dispatch, which is racy: GitHub may not have indexed the new run yet, so a rerun can watch an older run and report the wrong result. Poll until a run newer than the dispatch is visible, or capture the dispatched run's ID before calling
gh run watch.
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:4 - The new
descriptionis 149 characters, below the site's required 150–160-character range. Add a meaningful term so the post meets the front-matter SEO requirement.
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:375
- The
gha-runner-scale-setrelease is installed inARC_RUNNERS_NAMESPACE, so its listener pod is there; usingARC_SYSTEMS_NAMESPACEyields no matching pods and hides the useful listener logs. Use the runner namespace for this command.
kubectl logs \
--namespace "${ARC_SYSTEMS_NAMESPACE}" \
-l app.kubernetes.io/component=runner-scale-set-listener \
--tail 200
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:11
- The link uses
/en-us/, which the blog guidance forbids for Microsoft Learn URLs. Use the locale-neutral URL so the post is not locked to one locale.
The combination of ARC and AKS Automatic gives you the power of ARC on a production-ready AKS cluster with managed node pools, built-in monitoring, scaling, security settings, and other defaults that follow [AKS best practices](https://learn.microsoft.com/en-us/azure/well-architected/service-guides/azure-kubernetes-service). For runner workloads specifically, the [pod readiness SLA](https://learn.microsoft.com/azure/aks/intro-aks-automatic#pod-readiness-sla) is also a strong fit because CI/CD jobs depend on predictable pod startup.
website/blog/2026-08-19-github-actions-runner-controller-aks-automatic/index.md:301
- These result bullets have subject–verb agreement errors (
job move,pod use, andlog print). Correct the verbs so the validation results read clearly.
- The GitHub Actions job move from `queued` to `in_progress` to `completed`.
- An ephemeral runner object and pod appear in `arc-runners`.
- The runner pod use your pinned `ghcr.io/actions/actions-runner` image.
- The workflow log print `ARC runner reached workflow execution`.
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Lite
| export GITHUB_OWNER=<github-owner> | ||
| export GITHUB_REPO=<github-repo> |
Paul Yu (pauldotyu)
left a comment
Paul Yu (pauldotyu)
left a comment
There was a problem hiding this comment.
Left some feedback. Also, you might want to bump the dates in the filename and frontmatter to a date when you think we'll be able to merge.
|
|
||
| AKS Automatic is secure by default and has production-minded safeguards enabled. That's a good thing, but it also means the default public ARC Helm chart values need a little tuning. In particular, we need to be explicit about resource requests and image tags. | ||
|
|
||
| <!-- truncate --> |
There was a problem hiding this comment.
I think we should move this truncate marker to just below the first paragraph.
|
|
||
| GitHub Actions Runner Controller, also known as GitHub ARC or ARC for short, is a popular way to run self-hosted GitHub Actions runners on Kubernetes. In this walkthrough, we’ll set up ARC on [AKS Automatic](https://learn.microsoft.com/azure/aks/intro-aks-automatic) and run a real GitHub Actions job on an ephemeral runner pod. | ||
|
|
||
| The combination of ARC and AKS Automatic gives you the power of ARC on a production-ready AKS cluster with managed node pools, built-in monitoring, scaling, security settings, and other defaults that follow [AKS best practices](https://learn.microsoft.com/en-us/azure/well-architected/service-guides/azure-kubernetes-service). For runner workloads specifically, the [pod readiness SLA](https://learn.microsoft.com/azure/aks/intro-aks-automatic#pod-readiness-sla) is also a strong fit because CI/CD jobs depend on predictable pod startup. |
There was a problem hiding this comment.
| The combination of ARC and AKS Automatic gives you the power of ARC on a production-ready AKS cluster with managed node pools, built-in monitoring, scaling, security settings, and other defaults that follow [AKS best practices](https://learn.microsoft.com/en-us/azure/well-architected/service-guides/azure-kubernetes-service). For runner workloads specifically, the [pod readiness SLA](https://learn.microsoft.com/azure/aks/intro-aks-automatic#pod-readiness-sla) is also a strong fit because CI/CD jobs depend on predictable pod startup. | |
| The combination of ARC and AKS Automatic gives you the power of ARC on a production-ready AKS cluster with managed node pools, built-in monitoring, scaling, security settings, and other defaults that follow [AKS best practices](https://learn.microsoft.com/azure/well-architected/service-guides/azure-kubernetes-service). For runner workloads specifically, the [pod readiness SLA](https://learn.microsoft.com/azure/aks/intro-aks-automatic#pod-readiness-sla) is also a strong fit because CI/CD jobs depend on predictable pod startup. |
|
|
||
| ## Prerequisites | ||
|
|
||
| First, make sure you have the Azure CLI, GitHub CLI, `kubectl`, and Helm installed and authenticated: |
There was a problem hiding this comment.
Can we add hyperlinks for the folks that don’t have these tools yet?
| gh auth status | ||
| ``` | ||
|
|
||
| You'll also need a GitHub token with permission to manage Actions runners for the target repository. For a quick repo-scoped lab, a classic PAT with `repo` scope is enough for a private repo. For anything production-ish, I'd use a GitHub App instead so the permissions and rotation story are cleaner. |
There was a problem hiding this comment.
Might be good to wrap this in a Docusaurus admonition and drop a link here for gh apps https://docs.github.com/en/apps/creating-github-apps/about-creating-github-apps/about-creating-github-apps
| - The runner pod use your pinned `ghcr.io/actions/actions-runner` image. | ||
| - The workflow log print `ARC runner reached workflow execution`. | ||
|
|
||
| In my validation run, the job ran on a pod named like `arc-auto-runners-<id>-runner-<id>`, reported runner version `2.336.0`, and completed successfully. |
There was a problem hiding this comment.
I'd lean toward "the validation run" over "my" - feels more like they're following along than watching you do it
| In my validation run, the job ran on a pod named like `arc-auto-runners-<id>-runner-<id>`, reported runner version `2.336.0`, and completed successfully. | |
| In the validation run, you'll see the job ran on a pod named `arc-auto-runners-<id>-runner-<id>`, reported runner version `2.336.0`, and completed successfully. |
| container <manager> has no resource requests | ||
| ``` | ||
|
|
||
| make sure the controller chart has `resources` set and the runner scale set chart has `listenerTemplate.spec.containers[].resources` set. |
There was a problem hiding this comment.
Might be good to iterate that Deployment Safeguards is blocking this for missing a best practice (which is also crucial piece of info to know for scheduling and scaling too)
|
|
||
| ### The `latest` runner image is blocked | ||
|
|
||
| AKS Automatic safeguards can reject the default runner image because it uses a floating `latest` tag: |
There was a problem hiding this comment.
| AKS Automatic safeguards can reject the default runner image because it uses a floating `latest` tag: | |
| AKS Automatic Deployment Safeguards can reject the default runner image because it uses a floating `latest` tag: |
| update the image tag to a current runner release and upgrade the Helm release: | ||
|
|
||
| ```bash | ||
| helm upgrade --install "${RUNNER_SET_NAME}" \ | ||
| oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set \ | ||
| --namespace "${ARC_RUNNERS_NAMESPACE}" \ | ||
| --wait \ | ||
| --timeout 10m \ | ||
| -f arc-runner-set-values.yaml | ||
| ``` |
There was a problem hiding this comment.
If this is really an upgrade, I think we should drop the --install flag, might still work with it too.
| update the image tag to a current runner release and upgrade the Helm release: | |
| ```bash | |
| helm upgrade --install "${RUNNER_SET_NAME}" \ | |
| oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set \ | |
| --namespace "${ARC_RUNNERS_NAMESPACE}" \ | |
| --wait \ | |
| --timeout 10m \ | |
| -f arc-runner-set-values.yaml | |
| ``` | |
| Update the image tag to a current runner release and upgrade the Helm release: | |
| ```bash | |
| helm upgrade "${RUNNER_SET_NAME}" \ | |
| oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set \ | |
| --namespace "${ARC_RUNNERS_NAMESPACE}" \ | |
| --wait \ | |
| --timeout 10m \ | |
| -f arc-runner-set-values.yaml |
| arc: | ||
| label: Actions Runner Controller | ||
| permalink: /arc | ||
| description: Running GitHub Actions Runner Controller on AKS. |
There was a problem hiding this comment.
Would this tag be colliding with Azure Arc or Arc-enabled K8s? maybe use gh-arc or something similar?
This PR adds a new AKS blog post that walks through setting up GitHub Actions Runner Controller on AKS Automatic and validating an end-to-end GitHub Actions workflow on an ephemeral runner pod.
What’s included: