Skip to content

[CI] Configure nightly CodeQL languages and tracing boundaries - #26891

Open
dalexsoto wants to merge 2 commits into
mainfrom
dev/alex/cql
Open

dalexsoto wants to merge 2 commits into
mainfrom
dev/alex/cql

Conversation

@dalexsoto

@dalexsoto dalexsoto commented Oct 15, 2026 •

Copy link
Copy Markdown
Member

Looks like codeQL runs on arm64 now

Explicitly configure the nightly CodeQL job to scan C#, C/C++, Swift, PowerShell, Actions, Python, and JavaScript instead of relying on language autodetection on macOS ARM64. Keep the language selection local to this job, preserving shared defaults, the main-only schedule, ARM64 pool requirements, and the existing CodeQL enablement setting.

Add an optional, default-empty postAnalysisSteps stepList to the shared build template. Run it after CodeQL finalization and before teardown so callers can perform non-compilation work outside the tracing window while certificates and other build resources are still available. Preserve the existing buildSteps hook between compilation and finalization.

Move the nightly package assembly and artifact publication steps into postAnalysisSteps. Keep their commands, conditions, timeouts, and artifact settings unchanged; the main build still runs make all and make install inside the tracing window.

Use always() for CodeQL finalization so cancellation can still attempt to finish analysis and clean up tracing. Retain the CodeQL-enabled and main-branch guards.

Explicitly configure the nightly CodeQL job to scan C#, C/C++, Swift,
PowerShell, Actions, Python, and JavaScript instead of relying on language
autodetection on macOS ARM64. Keep the language selection local to this
job, preserving shared defaults, the main-only schedule, ARM64 pool
requirements, and the existing CodeQL enablement setting.

Add an optional, default-empty postAnalysisSteps stepList to the shared
build template. Run it after CodeQL finalization and before teardown so
callers can perform non-compilation work outside the tracing window while
certificates and other build resources are still available. Preserve the
existing buildSteps hook between compilation and finalization.

Move the nightly package assembly and artifact publication steps into
postAnalysisSteps. Keep their commands, conditions, timeouts, and artifact
settings unchanged; the main build still runs make all and make install
inside the tracing window.

Use always() for CodeQL finalization so cancellation can still attempt to
finish analysis and clean up tracing. Retain the CodeQL-enabled and
main-branch guards.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 7ce9eddd-f371-46aa-8ddd-b7b1d3028375
@dalexsoto
dalexsoto requested a review from rolfbjarne as a code owner October 15, 2026 22:53
Copilot AI balanced review requested due to automatic review settings October 15, 2026 22:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The selected languages’ support on macOS ARM64 and cancellation cleanup need validation with the internal CodeQL tasks in Azure Pipelines.

0 open findings

What changed in this PR

Configures nightly CodeQL scanning explicitly and separates package publication from the analysis tracing window.

Changes:

  • Selects seven languages locally in the nightly job.
  • Adds a default-empty postAnalysisSteps hook and moves nightly packaging and publication into it.
  • Uses always() for finalization while retaining enablement and main-branch guards.
File Description
tools/​devops/​automation/​templates/​build/​build.yml Adds the post-analysis hook before teardown and updates finalization conditions.
tools/​devops/​automation/​run-nightly-codeql.yml Sets explicit languages and moves packaging and publication after analysis.

🧠 Review effort: Balanced

@dalexsoto dalexsoto added ready-to-review This PR is ready to review/merge. copilot labels Oct 15, 2026
@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

Comment thread tools/devops/automation/templates/build/build.yml Outdated
Comment on lines +199 to +201
- ${{ each step in parameters.postAnalysisSteps }}:
- ${{ each pair in step }}:
${{ pair.key }}: ${{ pair.value }}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do anything at all after analysis is done/finalized?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep you are right that is not really needed

@rolfbjarne

Copy link
Copy Markdown
Member

Looks like codeQL runs on arm64 now

It's in beta, which it was last time I checked, and it's using Rosetta, which is what was causing the problem in the first place:

Support for Apple Silicon is currently in beta. Building your code during analysis requires Rosetta 2; analyzing an interpreted language or using build mode none runs natively.

that said, it was Mono causing problems, and we're not using Mono anymore, so maybe everything will be fine now!

@dalexsoto
dalexsoto requested review from rolfbjarne and a balanced review from Copilot October 16, 2026 18:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The described post-analysis hook, packaging relocation, and cancellation-aware finalization are missing and must be implemented or removed from the stated scope.

1 open finding

🧠 Review effort: Balanced

displayName: CodeQL
timeoutInMinutes: 480
variables:
Codeql.Language: csharp,cpp,swift,powershell,actions,python,javascript

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updating the PR title + description before merging to make it accurate would be good.

@vs-mobiletools-engineering-service2

Copy link
Copy Markdown
Collaborator

✅ API diff for current PR / commit

NET (empty diffs)

✅ API diff vs stable

NET (empty diffs)

ℹ️ Generator diff

Generator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes)

Pipeline on Agent
Hash: 5819a5964a9ab2365ce2c31cc31f2b0daf54d08b [PR build]

@vs-mobiletools-engineering-service2

Copy link
Copy Markdown
Collaborator

🔥 [CI Build #5819a59] Test results 🔥

Test results

❌ Tests failed on VSTS: test results

0 tests crashed, 3 tests failed, 218 tests passed.

Failures

❌ dotnettests tests (macOS)

1 tests failed, 0 tests passed.

Failed tests

  • DotNet tests: Failed (Execution failed with exit code 1)
    • Xamarin.Tests.BundleStructureTest.Build(MacOSX,"osx-x64",All,"Re...: '/Users/cloudtest/vss/_work/1/s/macios/tests/dotnet/BundleStructure/macOS/bin/Release/net11.0-macos/osx-x64/BundleStructure.app/...

Html Report (VSDrops) Download

❌ monotouch tests (iOS)

2 tests failed, 18 tests passed.

Failed tests

  • monotouch-test/iOS - simulator/Release (trimmable static registrar, all optimizations): TimedOut
  • monotouch-test/iOS - simulator/Release (strict inline Class.GetHandle): TimedOut

Html Report (VSDrops) Download

Successes

✅ assembly-processing: All 1 tests passed. Html Report (VSDrops) Download
✅ cecil: All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (iOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (MacCatalyst): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (Multiple platforms): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (tvOS): All 1 tests passed. Html Report (VSDrops) Download
✅ framework: All 2 tests passed. Html Report (VSDrops) Download
✅ fsharp: All 4 tests passed. Html Report (VSDrops) Download
✅ generator: All 5 tests passed. Html Report (VSDrops) Download
✅ interdependent-binding-projects: All 4 tests passed. Html Report (VSDrops) Download
✅ introspection: All 4 tests passed. Html Report (VSDrops) Download
✅ linker (iOS): All 21 tests passed. Html Report (VSDrops) Download
✅ linker (MacCatalyst): All 21 tests passed. Html Report (VSDrops) Download
✅ linker (macOS): All 21 tests passed. Html Report (VSDrops) Download
✅ linker (tvOS): All 21 tests passed. Html Report (VSDrops) Download
✅ monotouch (MacCatalyst): All 20 tests passed. Html Report (VSDrops) Download
✅ monotouch (macOS): All 20 tests passed. Html Report (VSDrops) Download
✅ monotouch (tvOS): All 20 tests passed. Html Report (VSDrops) Download
✅ msbuild: All 2 tests passed. Html Report (VSDrops) Download
✅ sharpie: All 1 tests passed. Html Report (VSDrops) Download
✅ windows: All 3 tests passed. [attempt 2] Html Report (VSDrops) Download
✅ xcframework: All 4 tests passed. Html Report (VSDrops) Download
✅ xtro: All 1 tests passed. Html Report (VSDrops) Download

macOS tests

✅ Tests on macOS Sonoma (14): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Sequoia (15): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Tahoe (26): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Golden Gate (27): All 5 tests passed. Html Report (VSDrops) Download

Linux Build Verification

✅ Linux build succeeded

Pipeline on Agent
Hash: 5819a5964a9ab2365ce2c31cc31f2b0daf54d08b [PR build]

displayName: CodeQL
timeoutInMinutes: 480
variables:
Codeql.Language: csharp,cpp,swift,powershell,actions,python,javascript

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updating the PR title + description before merging to make it accurate would be good.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

copilot ready-to-review This PR is ready to review/merge.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants