docs(blog): add Prepared Image Specification cold-start post - #5934
docs(blog): add Prepared Image Specification cold-start post#5934Spencer Libbing (stl327) wants to merge 12 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a new AKS Engineering blog post explaining how Prepared Image Specification (PIS) can reduce node/workload cold-start latency, along with the required author metadata entry.
Changes:
- Added a new blog article detailing PIS benefits, boundaries, and benchmark results plus a “Try PIS” walkthrough.
- Added a new author profile entry for Spencer Libbing in the shared blog author registry.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md | New blog post content for the PIS cold-start/scale-out latency topic (includes preview notice, methodology, and how-to links). |
| website/blog/authors.yml | Adds the spencer-libbing author record used by the new post. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.
Suppressed comments (2)
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:4
- The front-matter
descriptionis shorter than the 150–160 character SEO target used for this blog. Expand it slightly so it meets the length requirement.
title: "Reducing AKS Cold Starts with Prepared Image Specification"
description: "Learn how Prepared Image Specification can reduce AKS scale-out latency by preparing container images and host customizations ahead of time."
date: 2026-08-27
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:24
- This post is missing a hero image immediately after
<!-- truncate -->, which is the consistent pattern in other recent posts (for example,website/blog/2026-06-29-llm-routing-on-aks/index.mdandwebsite/blog/2026-04-08-agent-skills-for-aks/index.md). Add./hero-image.png(or another suitable hero asset) and include meaningful alt text.
<!-- truncate -->
In our controlled comparisons, the strongest end-to-end result was a Windows burst that added three nodes: PIS reduced median time to start all three workloads by **74%**. PIS also reduced the median slowest image pull by **92% on Linux** and **98% on Windows**. For a T4 GPU model-serving pool, median model-ready and first-token latency improved by about **3%**, while p95 latency improved by about **14%** and **13%**, respectively.
> **Preview notice:** PIS is currently an AKS preview feature. Preview features are opt-in, provided as-is and as-available, and not intended for production use. Review the current prerequisites, regions, limitations, and support policy before testing it.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 3 changed files in this pull request and generated 1 comment.
Suppressed comments (1)
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:136
- The unquoted
<digest>is interpreted as input redirection by Bash, causing the sample to fail with a missing-file error instead of passing an image reference to Azure CLI. Quote the complete placeholder argument so readers can replace the digest without changing the command structure.
--container-images myacr.azurecr.io/model-server@sha256:<digest> \
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 3 changed files in this pull request and generated no new comments.
Suppressed comments (7)
Previously missed (3) — in code that hasn't changed since the last review.
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:2
- This title uses headline-style capitalization, but repository guidance requires sentence-style capitalization for titles and headings. Lowercase “cold starts” to match that requirement.
title: "Reducing AKS Cold Starts with Prepared Image Specification"
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:56
- This row combines model readiness and first-token latency but reports only the 14% p95 value. Line 95 states that these endpoints had different p95 reductions (14% and 13%), so the table currently makes the first-token result ambiguous.
| Serve a model on a T4 GPU node | Model healthy and first token returned | **3% at median; 14% at p95** |
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:24
- Spell out GPU on first use. The repository’s acronym guidance requires the expanded term before the abbreviation.
This issue also appears in the following locations of the same file:
- line 40
- line 67
- line 73
In our controlled comparisons, the strongest end-to-end result was a Windows burst that added three nodes: PIS reduced median time to start all three workloads by **74%**. PIS also reduced the median slowest image pull by **92% on Linux** and **98% on Windows**. For a T4 GPU model-serving pool, median model-ready and first-token latency improved by about **3%**, while p95 latency improved by about **14%** and **13%**, respectively.
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:40
- Spell out SAS on first use so the security guidance is clear and follows the repository’s acronym requirements.
| Protect private dependencies | Managed identity and data-plane role-based access control (RBAC) can retrieve private artifacts without embedding keys or SAS tokens in scripts. |
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:67
- Spell out CA on first use. As written, readers encounter the abbreviation before “CA certificates” later in the post.
Not every task should be baked. A small Linux CA-and-policy setup was **14% slower at the median** in this test, even though its tail favored PIS. For small tasks, normal scale-out variation and other provisioning stages can outweigh the preparation that PIS removes.
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:73
- Spell out SLO on first use to meet the repository’s acronym guidance.
PIS moves that repeated work out of the burst path. In the three-node Windows comparison, this reduced median workload-start latency by 74%. For services with a strict scale-out SLO, that changes the capacity conversation from "How many idle nodes do we need?" to "How quickly can prepared capacity serve traffic?"
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:136
- The unquoted
<digest>placeholder is parsed by Bash as input/output redirection, so copying this example causes a shell error instead of passing the image reference to Azure CLI. Quote the complete placeholder argument (or assign it to a quoted variable).
--container-images myacr.azurecr.io/model-server@sha256:<digest> \
|  | ||
|
|
||
| In our controlled comparisons, the strongest end-to-end result was a Windows burst that added three nodes: PIS reduced median time to start all three workloads by **74%**. PIS also reduced the median slowest image pull by **92% on Linux** and **98% on Windows**. For a T4 GPU model-serving pool, median model-ready and first-token latency improved by about **3%**, while p95 latency improved by about **14%** and **13%**, respectively. | ||
|
|
There was a problem hiding this comment.
Regarding 3%/14% model-ready improvement for P50/P95, could you explain how this A/B test is run? Does it cover the GPU node provisioning, bootstrapping, inferencing engine container image pulling, model download/loading?
From what I have observed, most customers reserve GPU nodes for inferencing and training workloads for two reasons: GPU capacity scarcity and cold start latency. In that case, GPU nodes scale out/in may not be the primary use case for those customers, while GPU node OS upgrade could be a challenge. Can we measure how PIS helps in this scenario?
There was a problem hiding this comment.
Allen Wen (@AllenWen-at-Azure) , didn't you receive so much customer feedback about GPU scale up lately? What you are saying above is a bit at odds with the recent feedback. Maybe I am confused?
There was a problem hiding this comment.
This query suggests that most of the top 20 AKS GPU clusters reserve GPU nodes by either turning off cluster autoscaling or setting the min and max node values to be the same.
So, I don't think GPU node cold start is on the critical path of customer AI workload deployment, although it is still important for AKS to keep improving it.
I think what we should focus on instead is how PIS can help reduce disruption when customers upgrade their existing GPU nodes.
Nikhil Kaul (nikhilkaul1234)
left a comment
Nikhil Kaul (nikhilkaul1234)
left a comment
There was a problem hiding this comment.
Really strong post. The structure works, the results are presented honestly, and the "How to read these results" section is especially helpful. I left three small comments from the perspective of someone new to PIS, mostly about making the benchmark framing easier to follow. Otherwise, this looks great.
|
|
||
|  | ||
|
|
||
| In our controlled comparisons, the strongest end-to-end result was a Windows burst that added three nodes: PIS reduced median time to start all three workloads by **74%**. PIS also reduced the median slowest image pull by **92% on Linux** and **98% on Windows**. For a T4 GPU model-serving pool, median model-ready and first-token latency improved by about **3%**, while p95 latency improved by about **14%** and **13%**, respectively. |
There was a problem hiding this comment.
This test result is confusing. Why do we see 92% improvement in image pull in Linux, but the time to first token only improves by 3%? It isn't impressive enough to convince people to use the feature.
Also, 3% is so low that someone may question whether it is within margin of error. How many runs did we have here?
|  | ||
|
|
||
| In our controlled comparisons, the strongest end-to-end result was a Windows burst that added three nodes: PIS reduced median time to start all three workloads by **74%**. PIS also reduced the median slowest image pull by **92% on Linux** and **98% on Windows**. For a T4 GPU model-serving pool, median model-ready and first-token latency improved by about **3%**, while p95 latency improved by about **14%** and **13%**, respectively. | ||
|
|
There was a problem hiding this comment.
Allen Wen (@AllenWen-at-Azure) , didn't you receive so much customer feedback about GPU scale up lately? What you are saying above is a bit at odds with the recent feedback. Maybe I am confused?
| | Large image pulls during a three-node burst | Slowest new-node pull completed | **92% at Linux median; 98% at Windows median** | | ||
| | Install a pinned portable runtime | Runtime verified and workload started | **10% at Linux median; 34% at Windows median** | | ||
| | Prepare a large dependency bundle | Dependencies verified and workload started | **20% at Linux median; 5% at Windows median** | | ||
| | Serve a model on a T4 GPU node | Model healthy and first token returned | **3% at median; 14% at p95** | |
There was a problem hiding this comment.
You would expect this last test to be kind of in line with 10 to 74%.. But 3% is oddly low. Why is it so different?
| - **Windows gained the most.** Large Windows image and startup costs created more repeated work for PIS to remove, especially during multi-node bursts. | ||
| - **Tail latency often improved more than the median.** PIS made the slowest scale-out events less severe even when the typical Linux event changed only slightly. | ||
| - **Large, stable preparation paid off.** Image layers, a portable runtime, and a large dependency bundle were useful candidates because every standard node otherwise repeated the work. | ||
| - **GPU gains were real but smaller at the median.** Preloading the model and server image helped, but GPU VM creation, node registration, accelerator readiness, scheduling, and model initialization still dominated much of the end-to-end path. |
There was a problem hiding this comment.
Given this is a blog post, should we talk about how more work is being done to cut these layers and customers should stay tuned?
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 3 changed files in this pull request and generated no new comments.
Suppressed comments (3)
Previously missed (2) — in code that hasn't changed since the last review.
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:119
postScriptActionisn't part of the current PIS customization-script schema; the CLI accepts onlyname,script,scriptType, andexecutionPointfor each item. Leaving this property in the copy-paste example can cause argument validation to fail. Remove it and makeexecutionPointthe final property.
"postScriptAction": "None"
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:143
az aks nodepool adddoesn't support--pis-id; the documented flag is--prepared-image-specification-id. As written, this copy-paste command fails with an unrecognized argument instead of creating the PIS-backed pool.
--pis-id "$PIS_VERSION_ID"
website/blog/2026-08-27-reducing-aks-cold-starts-with-prepared-image-specification/index.md:4
- The publish date is already stale: it is August 27, while this review is occurring on September 1. Because this front-matter value overrides the folder-derived date, the newly published post would display the drafting date; update it to the actual publication date before merge, or mark the post as a draft until then.
date: 2026-08-27
|
|
||
| > **Preview notice:** PIS is currently an AKS preview feature. Preview features are opt-in, provided as-is and as-available, and not intended for production use. Review the current prerequisites, regions, limitations, and support policy before testing it. | ||
|
|
||
| ## What PIS changes for customers |
There was a problem hiding this comment.
suggest full name on the first title appearance I know it's in the text at the beginning but there's no title level appearance first, so given this is short I think it's better visually to show full.
having said that about the first occurrence, are we sure we want to use the acronym at all?
There was a problem hiding this comment.
🟡 Changes recommended
The CLI example uses an invalid option, benchmark claims conflict with the PR description, and the required conclusion is missing.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 2/3 changed files
- Comments generated: 4
- Review effort level: Balanced
|
|
||
| The results below compare otherwise-equivalent standard and PIS-backed pools. A reduction means the PIS-backed path reached the measured outcome sooner. Mean saved is the average same-round difference, while p90 and p95 show tail latency. Times are rounded to whole seconds. | ||
|
|
||
| In our controlled comparisons, the strongest end-to-end result was a Windows burst that added three nodes: PIS reduced median time to start all three workloads by **74%**. PIS also reduced the median slowest image pull by **92% on Linux** and **98% on Windows**. For a T4 graphics processing unit (GPU) model-serving pool, PIS reached model readiness and returned the first token **14 seconds sooner on average** across 46 paired scale-outs. At p95, model readiness was **503 seconds with PIS versus 510 seconds without it**, and first token was **537 versus 543 seconds**. Once the GPU was allocatable, PIS returned the first token **27 seconds sooner on average**. |
|
|
||
| For Windows, use PowerShell and set `scriptType` to `PowerShell`. Keep secrets out of scripts and use managed identity with data-plane RBAC for private registries and artifacts. | ||
|
|
||
| ## Resources |
There was a problem hiding this comment.
🟡 Changes recommended
The article lacks the required conclusion and uses inconsistent title capitalization.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (1)
website/blog/2026-09-02-reducing-aks-cold-starts-with-prepared-image-specification/index.md:143
- Add the required conclusion before the resources section. The blog post structure calls for a conclusion that summarizes the key takeaway; ending directly with links leaves the article without that closing guidance.
## Resources
- Files reviewed: 2/3 changed files
- Comments generated: 1
- Review effort level: Balanced
There was a problem hiding this comment.
🟢 Approval recommended
The article meets repository publishing requirements, and its links, metadata, assets, and documented commands are consistent.
Review details
- Files reviewed: 2/3 changed files
- Comments generated: 0 new
- Review effort level: Balanced
Summary
Publishes an AKS Engineering Blog article about Prepared Image Specification (PIS), an AKS preview feature that moves stable, repeated node preparation into an AKS-managed, versioned node image.
This change:
Customer impact
PIS prepares pinned images and stable Bash or PowerShell customizations ahead of recurring scale-outs. Customers can spend preparation time when a version is released, then avoid repeating the same work on every new node.
Headline results in the post include:
The post keeps the boundary clear: PIS doesn't remove VM allocation, node registration, networking, scheduling, driver initialization, or application startup.
Benchmark context
The GPU A/B test concurrently scaled matched
Standard_NC4as_T4_v3standard and PIS-backed pools from one to two nodes. It covered GPU VM provisioning, node bootstrapping and registration, managed driver readiness, image availability, model download or prepared-asset verification, model loading, health, and first token. The standard node downloaded and hash-verified the model and pulled the pinned image if absent; the PIS-backed node verified the same preloaded model and used the preloaded pinned image.Image-caching tests used 100 rounds per arm, customization scenarios used 10 same-round pairs per cell, and the GPU scenario used 46 same-round pairs. PIS build and initial prepared-pool creation were excluded from recurring scale-out timing. The tests did not measure GPU node upgrades or replacement disruption, production traffic, autoscaler decision time, or billing impact.
Validation
npm run buildnpm run typecheck