Skip to content

update_spirv.md

Latest commit

 

History

History
250 lines (167 loc) · 8.12 KB

update_spirv.md

File metadata and controls

250 lines (167 loc) · 8.12 KB
Raw

Updating external spirv

There are three directories under external that are related to SPIR-V:

  • external/spirv-headers
  • external/spirv-tools
  • external/spirv-tools-generated

In order to use the latest or custom SPIR-V, they need to be updated.

Update SPIRV-Tools

On the Slang repo, you need to update to use the latest commit of SPIRV-Tools and SPIRV-Headers.

  1. Create a branch for the update.

    # This doc will use "update_spirv" as a branch name,
# but you can use a different name.
git checkout -b update_spirv

  2. Synchronize and update submodules.

    git submodule sync
git submodule update --init --recursive

  3. Update the SPIRV-Tools submodule to the latest version.

    git -C external/spirv-tools fetch
git -C external/spirv-tools checkout origin/main

Build spirv-tools

A directory, external/spirv-tools/generated, holds a set of files generated from spirv-tools directory. You need to build spirv-tools in order to generate them.

cd external
cd spirv-tools
python3.exe utils\git-sync-deps # this step may require you to register your ssh public key to gitlab.khronos.org
cmake.exe . -B build-generated
cmake.exe --build build-generated --config Release

# Go back to repository root
cd ../..

Do not name the CMake build directory build. SPIRV-Tools' DEPS file has a build entry that checks out Chromium's //build tree into external/spirv-tools/build/, so the git-sync-deps step above fills that directory with unrelated Chromium headers (build_config.h, buildflag.h, precompile.h). Configuring CMake into the same directory would mix those headers in with the generated ones, and the copy step below would pick them up — which the Verify SPIRV Generated Files CI job then reports as orphaned files.

Update SPIRV-Headers

  1. Update the SPIRV-Headers submodule to what SPIRV-Tools uses 
    grep spirv_headers_revision external/spirv-tools/DEPS
git -C external/spirv-headers fetch origin [commit hash from the previous command]
git -C external/spirv-headers checkout [commit hash from the previous command]

Note that the update of SPIRV-Headers should be done after running python3.exe utils\git-sync-deps, because the python script will update external/spirv-tools/external/spirv-headers to whichever commit the current SPIRV-Tools depends on.

Copy the generated files from spirv-tools/build-generated/ to spirv-tools-generated/

Copy the generated header files from external/spirv-tools/build-generated/ to external/spirv-tools-generated/.

rm external/spirv-tools-generated/*.h
rm external/spirv-tools-generated/*.inc
cp external/spirv-tools/build-generated/*.h external/spirv-tools-generated/
cp external/spirv-tools/build-generated/*.inc external/spirv-tools-generated/

Build Slang and run slang-test

After SPIRV submodules are updated, you need to build and test.

# Make sure to clean up data generated from the previous SPIRV
rm -fr build

There are many ways to build Slang executables. Refer to the document for more detail. For a quick reference, you can build with the following commands,

cmake.exe --preset vs2022
cmake.exe --build --preset release

After building Slang executables, run slang-test to see all tests are passing.

set SLANG_RUN_SPIRV_VALIDATION=1
build\Release\bin\slang-test.exe -use-test-server -server-count 8

It is often the case that some of tests fail, because of the changes on SPIRV-Header. You need to properly resolve them before proceed.

Commit and create a Pull Request

After testing is done, you need to stage and commit the updated submodule references and any generated files. Note that when you want to use new commit IDs of the submodules, you have to stage with git-add command for the directory of the submodule itself.

git add external/spirv-headers
git add external/spirv-tools
git add external/spirv-tools-generated

# Add any other changes needed to resolve test failures

git commit -m "Update SPIRV-Tools and SPIRV-Headers to latest versions"
git push origin update_spirv # Use your own branch name as needed

Once all changes are pushed to GitHub, you can create a Pull Request on the main Slang repository.

CI Validation

The Slang CI system includes an automated check that verifies the generated files in external/spirv-tools-generated/ are up-to-date whenever changes are made to external/spirv-tools or external/spirv-headers.

What the CI Check Does

When you create a Pull Request that modifies the SPIRV submodules, the CI will:

  1. Detect changes to external/spirv-tools or external/spirv-headers
  2. Verify that the spirv-headers commit matches what spirv-tools/DEPS expects
  3. Automatically regenerate the files that should be in external/spirv-tools-generated/
  4. Compare the regenerated files with the committed files
  5. Fail the CI if there are any discrepancies or mismatches

This ensures that:

  • The spirv-headers version is compatible with spirv-tools
  • All generated files are correctly synchronized with the SPIRV-Tools version

When Does the CI Check Run?

The check only runs when:

  • A pull request modifies external/spirv-tools/**
  • A pull request modifies external/spirv-headers/**
  • A pull request modifies external/spirv-tools-generated/**
  • The workflow or check script itself is modified

This path filtering prevents unnecessary builds and keeps CI fast.

What Files Are Validated?

The check verifies that the following generated files are present and up-to-date:

  • *.inc files (e.g., build-version.inc, core_tables_body.inc, etc.)
  • *.h files (e.g., DebugInfo.h, OpenCLDebugInfo100.h, etc.)

The README.md file in external/spirv-tools-generated/ is excluded from validation.

If the CI Check Fails

If you see a failure from the "Check SPIRV Generated Files" job, it means the generated files are out of sync. The CI output will show:

  • Which files are missing
  • Which files have differences
  • Which files are orphaned (should be removed)

To fix the issue, follow the instructions in the CI output, or re-run the steps in the Copy the generated files section above.

Common Failure Scenarios

Scenario 1: Missing Files

ERROR: Missing file in spirv-tools-generated: new_file.inc

This means a new file is now generated by SPIRV-Tools but hasn't been added to the repository.

Fix: Follow the copy steps in the error message to add the new file.

Scenario 2: Outdated Files

ERROR: File differs: build-version.inc

This means the file exists but its content doesn't match what SPIRV-Tools would generate.

Fix: Regenerate and replace the file following the instructions in the CI output.

Scenario 3: Orphaned Files

ERROR: Orphaned file in spirv-tools-generated: old_file.inc

This means a file exists in the repository but is no longer generated by SPIRV-Tools.

Fix: Remove the file with git rm external/spirv-tools-generated/old_file.inc

Scenario 4: spirv-headers Commit Mismatch

ERROR: spirv-headers commit mismatch!
ERROR:   Expected (from spirv-tools/DEPS): 6bb105b6c4b3a246e1e6bb96366fe14c6dbfde83
ERROR:   Actual (submodule):                1234567890abcdef1234567890abcdef12345678

This means the spirv-headers submodule commit doesn't match what spirv-tools expects in its DEPS file.

Fix: Update spirv-headers to the expected commit:

git -C external/spirv-headers fetch
git -C external/spirv-headers checkout 6bb105b6c4b3a246e1e6bb96366fe14c6dbfde83
git add external/spirv-headers

Scenario 5: Submodule Not Updated

If you updated generated files but forgot to update the submodule reference:

git add external/spirv-tools
git add external/spirv-tools-generated

Testing Locally

You can run the same check locally before pushing to catch issues early:

bash extras/check-spirv-generated.sh

This script will verify that your generated files match what would be produced by building SPIRV-Tools with the current submodule version.