Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jul 17, 2025

GitHub runners are updating to version 20250701.1.0 which includes LLVM 20.1.7. This new LLVM version performs optimizations that the eBPF verifier considers unsafe, causing all CI/CD pipelines to fail verification.

Problem

The current CI/CD workflows rely on the LLVM version pre-installed on GitHub runners. With the rollout of LLVM 20.1.7, builds fail because:

  • The verifier rejects code optimized by LLVM 20.1.7 as potentially unsafe
  • No explicit version pinning exists in the CI/CD pipeline
  • eBPF programs compiled with newer LLVM optimizations don't pass verification

Solution

Added explicit LLVM 18.1.8 installation to the reusable-build.yml workflow to ensure version consistency across all runners.

Changes Made

  1. Chocolatey package caching - Added cache step for C:\ProgramData\chocolatey to improve build performance
  2. Explicit LLVM installation - Added choco install llvm --version=18.1.8 --allow-downgrade -y --force step
  3. PATH precedence - Ensures installed LLVM (C:\Program Files\LLVM\bin) takes precedence over Visual Studio LLVM
  4. Error handling - Graceful fallback to pre-installed LLVM if chocolatey installation fails
  5. Path consistency - Fixed clang version check to use correct case-sensitive path

Workflow Changes

- name: Cache chocolatey packages
  uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684
  with:
    path: C:\ProgramData\chocolatey
    key: ${{ runner.os }}-choco-llvm-18.1.8

- name: Install LLVM 18.1.8
  run: |
    # Install LLVM 18.1.8 to ensure consistent version across runners
    try {
      choco install llvm --version=18.1.8 --allow-downgrade -y --force
      # Add installed LLVM to PATH first so it takes precedence
      echo "C:\Program Files\LLVM\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
      Write-Host "Successfully installed LLVM 18.1.8"
    } catch {
      Write-Warning "Failed to install LLVM 18.1.8 via chocolatey: $($_.Exception.Message)"
      Write-Host "Continuing with pre-installed LLVM version"
    }

Impact

  • Prevents CI/CD breakage - All pipelines will use LLVM 18.1.8 regardless of runner updates
  • Minimal changes - Only affects CI/CD pipeline, no core application logic modified
  • Cross-platform support - Works with both x64 and ARM64 Windows runners
  • Performance optimized - Chocolatey caching reduces installation time on subsequent runs
  • Backward compatible - Existing workflows continue to function normally

Workflows Affected

The fix applies to all build jobs in the CI/CD pipeline:

  • regular - Regular builds with NuGet package generation
  • onebranch - OneBranch builds
  • analyze - Static analysis builds
  • sanitize - Address sanitizer builds
  • codeql - CodeQL security analysis builds

This surgical fix ensures all CI/CD pipelines remain functional as GitHub continues rolling out updated runners with LLVM 20.1.7.

Fixes #240.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

@mikeagun
Copy link
Contributor

@copilot you can use the corresponding fix made in microsoft/ebpf-for-windows#4505 for reference

Copilot AI changed the title [WIP] pipeline failures from Github runner update to LLVM 20.1.7 Fix CI/CD failures from GitHub runner update to LLVM 20.1.7 Jul 17, 2025
Copilot AI requested a review from mikeagun July 17, 2025 16:51
Copilot finished work on behalf of mikeagun July 17, 2025 16:51
Copy link
Contributor Author

Copilot AI commented Jul 17, 2025

@copilot you can use the corresponding fix made in microsoft/ebpf-for-windows#4505 for reference

I've reviewed the reference PR and confirmed that my implementation follows the same approach. The LLVM installation step, chocolatey caching, and error handling match the solution from microsoft/ebpf-for-windows#4505. The implementation uses the same chocolatey command (choco install llvm --version=18.1.8 --allow-downgrade -y --force) and PATH precedence logic.

Copilot finished work on behalf of mikeagun July 17, 2025 16:54
@mikeagun mikeagun marked this pull request as ready for review July 17, 2025 17:09
@mikeagun mikeagun changed the title Fix CI/CD failures from GitHub runner update to LLVM 20.1.7 Fix CI/CD failures by installing LLVM 18.1.8 explicitly Jul 17, 2025
@saxena-anurag saxena-anurag merged commit 3058d26 into main Jul 17, 2025
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pipeline failures from Github runner update to LLVM 20.1.7

5 participants