Add LLM Security Scanning to GitHub Actions

Adding LLM security scanning to your CI pipeline is one of the highest-leverage security improvements you can make to an AI-powered application. This guide walks through integrating LLMArmor into GitHub Actions step by step, from a basic pipeline gate to SARIF upload with inline annotations on pull requests.

Why scan in CI?

Static analysis in CI catches security issues at the point they’re introduced — in the pull request, before code reaches production. The benefits:

• Early feedback: developers see findings inline on their PR diff, not in a post-deploy security audit
• Consistent coverage: every commit is scanned automatically, not just when someone remembers to run the tool
• Zero marginal cost: LLMArmor requires no API keys and makes no network calls — a scan of a medium-sized codebase completes in 1–3 seconds

Basic pipeline gate

The simplest integration uses LLMArmor’s exit codes to fail the pipeline on CRITICAL findings:

Exit code	Meaning
0	Clean — no MEDIUM+ findings
1	At least one HIGH or MEDIUM finding
2	At least one CRITICAL finding

.github/workflows/llmarmor.yml

name: LLM Security Scan
on: [push, pull_request]

jobs:
  llmarmor:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install LLMArmor
        run: pip install llmarmor

      - name: Scan for LLM vulnerabilities
        run: llmarmor scan . --quiet
        # Exits 2 on CRITICAL, 1 on HIGH/MEDIUM, 0 if clean
        # The pipeline fails automatically on non-zero exit

This configuration fails the pipeline on any HIGH, MEDIUM, or CRITICAL finding. If you want to block only CRITICAL findings and treat HIGH/MEDIUM as warnings, use continue-on-error:

      - name: Scan (fail on CRITICAL only)
        id: scan_gate
        run: |
          llmarmor scan . --quiet
          echo "exit_code=$?" >> "$GITHUB_OUTPUT"
        continue-on-error: true

      - name: Fail if CRITICAL findings
        if: steps.scan_gate.outputs.exit_code == '2'
        run: exit 2

Choosing your gate threshold

For most teams, blocking on CRITICAL findings from day one is the right starting point. CRITICAL findings (e.g., eval() on LLM output, dynamic dispatch from tool calls) are unambiguously dangerous and should be fixed before merge. HIGH/MEDIUM findings can be addressed in a follow-up sprint while the pipeline stays green.

SARIF upload with inline PR annotations

SARIF (Static Analysis Results Interchange Format) is the standard format for CI security findings. GitHub natively renders SARIF findings as inline annotations on pull request diffs and in the Security → Code Scanning dashboard.

.github/workflows/llmarmor-sarif.yml

name: LLM Security Scan (SARIF)
on: [push, pull_request]

permissions:
  security-events: write   # required for upload-sarif

jobs:
  llmarmor:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install LLMArmor
        run: pip install llmarmor

      - name: Run LLMArmor and produce SARIF
        run: llmarmor scan . -f sarif -o llmarmor.sarif
        # Note: this step does NOT fail the build (no --quiet)
        # The SARIF upload below handles visibility; gate separately if needed

      - name: Upload SARIF to GitHub Code Scanning
        uses: github/codeql-action/upload-sarif@v3
        if: always()   # upload even if the scan step fails
        with:
          sarif_file: llmarmor.sarif

After this workflow runs, findings appear in the Security → Code Scanning tab of your repository, and inline on pull request diffs next to the affected lines.

Combined: gate + SARIF + JSON artifact

This is the recommended production configuration. It gates on CRITICAL findings, uploads SARIF for inline annotations, and saves a JSON report as an artifact for later review:

.github/workflows/llmarmor-full.yml

name: LLM Security Scan
on: [push, pull_request]

permissions:
  security-events: write

jobs:
  llmarmor:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install LLMArmor
        run: pip install llmarmor

      - name: Scan — SARIF output
        run: llmarmor scan . -f sarif -o llmarmor.sarif
        continue-on-error: true   # don't fail here; gate below

      - name: Scan — JSON report
        run: llmarmor scan . -f json -o llmarmor-report.json
        continue-on-error: true

      - name: Upload SARIF to Code Scanning
        uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: llmarmor.sarif

      - name: Upload JSON report as artifact
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: llmarmor-report
          path: llmarmor-report.json

      - name: Gate — fail on CRITICAL findings
        run: llmarmor scan . --quiet
        # This step produces the actual gate exit code.
        # Exits 2 on CRITICAL (pipeline fails), 1 on HIGH/MEDIUM (pipeline fails),
        # 0 on clean (pipeline passes).

Scanning only changed files (for large repos)

For large codebases where scanning every file on every push is too slow, use git diff to limit the scan to files changed in the current PR:

      - name: Get changed Python files
        id: changed
        run: |
          git fetch origin ${{ github.base_ref }} --depth=1
          CHANGED=$(git diff --name-only origin/${{ github.base_ref }}...HEAD -- '*.py' | tr '\n' ' ')
          echo "files=$CHANGED" >> "$GITHUB_OUTPUT"

      - name: Scan changed files
        if: steps.changed.outputs.files != ''
        run: llmarmor scan ${{ steps.changed.outputs.files }} --quiet

Full scan recommended for main branch

Scanning only changed files is fine for pull requests, but we recommend running a full scan on merges to main or master to catch issues that may have been introduced before CI was added.

Suppressing false positives

If LLMArmor flags a line that you’ve reviewed and determined is safe, suppress it inline without disabling the rule globally:

# Safe because this is a static admin persona, not user-controlled
ADMIN_PROMPT = "You are an admin assistant."  # llmarmor: ignore[LLM07]

# This eval is on a validated schema, not raw LLM output
result = eval(validated_schema)  # llmarmor: ignore[LLM05]

Or suppress an entire file using .llmarmorignore:

.llmarmorignore

tests/fixtures/**
scripts/dev_seed.py

Monitoring the Security dashboard

Once SARIF upload is configured, the GitHub Security → Code Scanning dashboard shows:

• Open findings with severity, rule, and affected file/line
• Trend over time (new findings, fixed findings, dismissed findings)
• Filtering by rule (LLM01, LLM05, etc.), severity, and branch

This gives security reviewers a central place to track LLM security posture across the repository without having to read CI logs.

Summary

Integrating LLMArmor into GitHub Actions takes about 10 minutes and gives you:

• Automatic scanning on every push and pull request
• Inline annotations on PR diffs via SARIF
• Pipeline gating that blocks merges with CRITICAL findings
• Audit trail via JSON artifacts and the Code Scanning dashboard

---

Install LLMArmor Get LLMArmor from PyPI and run your first scan locally before adding it to CI.