[NPU]RFC: Ascend CI Integration

[noemotiovon]

Summary

This RFC proposes integrating a Continuous Integration (CI) system for Ascend NPU into the Liger-Kernel project to continuously monitor the support status of operators in the ops/ directory on Ascend devices, ensuring code quality and functional correctness.

Background and Motivation

Current Status

Liger-Kernel already supports CI for multiple hardware platforms:

• NVIDIA GPU (CUDA) - via Modal CI
• AMD GPU (ROCm) - via self-hosted runner
• Intel GPU (XPU) - via self-hosted runner

Ascend NPU support has been implemented at the code level (setup.py platform detection, backends/_ascend/ backend architecture), but lacks automated CI coverage.

Completed Work

Currently, 24 operators require adaptation for Ascend devices, of which 18 operators have passed accuracy verification.

Kernel Name	File Path	Accuracy	Description
Cross Entropy	cross_entropy.py	✅	Cross-entropy loss function
DyT	dyt.py	✅	DyT normalization operation
Embedding	experimental/embedding.py	🟡	Embedding layer (experimental)
Fused Add RMS Norm	fused_add_rms_norm.py	✅	Fused Add + RMS Norm
Fused Linear Cross Entropy	fused_linear_cross_entropy.py	🟡	Fused Linear + Cross Entropy
Fused Linear JSD	fused_linear_jsd.py	🟡	Fused Linear + JSD
Fused Neighborhood Attention	fused_neighborhood_attention.py	🟡	Fused neighborhood attention
GEGLU	geglu.py	✅	GELU gated linear unit
Group Norm	group_norm.py	✅	Group normalization
GRPO Loss	grpo_loss.py	✅	GRPO loss function
JSD	jsd.py	🟡	Jensen-Shannon divergence
KL Div	kl_div.py	✅	KL divergence loss
Layer Norm	layer_norm.py	✅	Layer normalization
Llama4 ROPE	llama4_rope.py	🟡	Llama4 rotary position encoding
Multi Token Attention	multi_token_attention.py	🟡	Multi-token attention
Poly Norm	poly_norm.py	✅	Polynomial normalization
Qwen2VL MRope	qwen2vl_mrope.py	✅	Qwen2VL multi-rotary position encoding
RMS Norm	rms_norm.py	✅	RMS normalization
ROPE	rope.py	✅	Rotary position encoding
Softmax	softmax.py	✅	Softmax activation function
Sparsemax	sparsemax.py	✅	Sparsemax activation function
SWIGLU	swiglu.py	✅	SiLU gated linear unit
Tiled MLP	tiled_mlp.py	✅	Tiled MLP
TVD	tvd.py	✅	TVD loss function

Note: ✅ Accuracy verified, 🟡 In progress.

Problem Statement

The correctness of Ascend operators currently relies on manual testing, lacking automated CI coverage. This creates regression risks, and community contributors cannot quickly verify the correctness of Ascend-related modifications.

Proposal

Objectives

Establish an Ascend CI pipeline by adding an Ascend CI workflow to GitHub Actions, continuously verifying operator correctness, automatically triggering tests on code changes, and promptly detecting regression issues.

Technical Solution

1. CI Workflow Design

Following the existing Intel and AMD CI configurations, create .github/workflows/ascend-ci.yml:

name: Ascend NPU

on:
  push:
    branches:
      - main
    paths:
      - "src/**"
      - "test/**"
  pull_request:
    branches:
      - main
    paths:
      - "src/**"
      - "test/**"
  workflow_dispatch:  # Enable manual trigger

concurrency:
  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
  cancel-in-progress: true

jobs:
  checkstyle:
    runs-on: ubuntu-latest
    steps:
    - name: Checkout code
      uses: actions/checkout@v6

    - name: Set up Python
      uses: actions/setup-python@v6
      with:
        python-version: '3.10'

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r dev/fmt-requirements.txt

    - name: Run checkstyle
      run: make checkstyle

  tests:
    runs-on: [self-hosted, ascend-npu]  # Requires Ascend self-hosted runner configuration
    needs: [checkstyle]  # Wait for checkstyle job to complete
    if: success()  # Only run tests when checkstyle passes
    container:
      image: ascend/ascend-toolkit:latest  # Ascend container image with CANN, torch_npu, triton-ascend pre-installed
      options: --privileged -v /dev/davinci_manager:/dev/davinci_manager -v /dev/devmm_svm:/dev/devmm_svm -v /dev/hisi_hdc:/dev/hisi_hdc --ipc=host
    steps:
    - name: Checkout code
      uses: actions/checkout@v6

    - name: Set up Python
      shell: bash
      run: |
        # Python should be pre-installed in the container image
        python --version

    - name: Verify NPU availability
      shell: bash
      run: |
        npu-smi info
        echo "NPU devices available"

    - name: Setup Dependencies
      shell: bash
      run: |
        python -m pip install --upgrade pip
        pip install -e .[dev]
        # torch_npu and triton-ascend are pre-installed in the container image

    - name: List Python Environments
      shell: bash
      run: python -m pip list

    - name: Run Unit Tests
      shell: bash
      run: |
        # Run tests only after checkstyle passes
        # Initial phase: Run only test cases in test/transformers directory
        python -m pytest test/transformers/ --disable-warnings
        # Future phases will expand to full test suite:
        # make test
        # make test-convergence

Note: The actual configuration needs to be adjusted based on the specific Ascend runner environment, particularly:

• Runner label configuration (self-hosted, ascend-npu)
• Container image with CANN toolkit, torch_npu, and triton-ascend pre-installed
• Container options for NPU device access (device mounting, privileged mode, etc.)

2. Runtime Environment Requirements

• Hardware: Ascend NPU device (e.g., Atlas 800I A2)
• Container Image: Pre-built Docker image containing:
  • Python 3.10+
  • PyTorch with torch_npu (2.7.1)
  • triton-ascend
  • Ascend CANN toolkit
• Container Configuration: Privileged mode and device mounting for NPU access

3. Test Scope

Initial Phase: CI will execute test cases in the test/transformers directory to verify the forward and backward propagation correctness of Ascend-adapted operators.

Future Phases: Gradually expand to the full test suite:

• Unit Tests (make test): Verify all Ascend-adapted operators
• Convergence Tests (make test-convergence): Verify operator convergence on complete models

Code Style Check (make checkstyle) runs as an independent job at all times.

Implementation Plan

1. Infrastructure Preparation: Configure Ascend self-hosted runner, install CANN toolkit and PyTorch + torch_npu environment
2. CI Workflow Development: Create .github/workflows/ascend-ci.yml, implement environment setup, dependency installation, and test execution

Technical Details

Runner Configuration

Self-hosted runner configuration is required (GitHub-hosted runners do not provide Ascend hardware):

• Labels: ascend-npu or ascend-910b4
• Operating System: Ubuntu 20.04/22.04 (according to CANN requirements)
• Hardware: At least one Ascend device (e.g., Atlas 800I A2)

Environment Variables

CANN-related environment variables need to be set, such as ASCEND_RT_VISIBLE_DEVICES, etc.

Testing Strategy

• Test only Ascend-adapted operators to avoid CI failures from unadapted operators
• Use pytest.mark.skip or xfail for known issues
• If multiple NPUs are available, tests can be executed in parallel

Future Work

• Integrate performance benchmarks in CI to track operator performance changes
• Extend to more Ascend device models (e.g., Ascend310, Ascend910A, etc.)
• Continuously adapt more operators to improve Ascend support coverage
• Supplement Ascend usage documentation, best practices, and troubleshooting guides

Conclusion

Integrating Ascend CI will significantly improve Liger-Kernel's support quality for the Ascend platform, ensuring that code changes do not break Ascend operator functionality. Although some infrastructure investment is required, it will greatly reduce maintenance costs and improve community collaboration efficiency in the long run.

Related Resources

• Ascend CANN Documentation
• torch_npu
• triton-ascend

Alternatives

No response

Additional context

No response

[noemotiovon]

Hi @xuedinge233, could you help take a look at this CI integration plan? Also, we’ll need your assistance to complete this work going forward.

[noemotiovon]

Hi @Tcc0403, @hipudding, @zheliuyu, I believe that the precision adaptation of NPU ops under transformers/ is nearing completion. This makes it a good time to integrate Ascend NPU CI, so that we can safeguard precision correctness while pursuing performance improvements in the future. Would you mind helping to review this Ascend CI integration?

[xuedinge233]

Hi @xuedinge233, could you help take a look at this CI integration plan? Also, we’ll need your assistance to complete this work going forward.

It's all right with me.

[Tcc0403]

Try uv instead of pip to setup dependencies faster

ref:
https://github.com/astral-sh/uv
https://github.com/astral-sh/setup-uv

[zheliuyu]

@noemotiovon Awesome work. Need me to supply an NPU machine for the CI?

[noemotiovon]

Hi @zheliuyu, we already have access to an NPU machine (Ascend 910B4), but we really appreciate the generous offer—thank you!

[xuedinge233]

Hi @Tcc0403, The following is the initial plan for this CI. Please help check if there are any changes needed:

1: Fork Validation

• Run the complete NPU CI workflow in the forked repository
• Main objective:
  • Verify the correctness of the workflow
  • Observation of stability and execution duration
  • Identify and isolate flaky cases
• Provide a running link to the forked repository in the CI status section of the README, so that the community can view it transparently.

2: Stability Observation

• Continuous operation of NPU CI (approximately 1-2 weeks)
• Stability determination criteria:
  • Running continuously for several days
  • ≥95% success rate
• At this stage, no workflow code is merged upstream.

3: Upstream Integration

• Merge the verified stable NPU CI workflow into the upstream.
• Apply to connect to the upstream self-hosted NPU runner
• Update the CI status link in the README to point to the official CI of the upstream project.

[Tcc0403]

@xuedinge233 lgtm. Regarding the workflow design, could you also split make test (correctness test) and make test-convergence (convergence test) into two seperated jobs so it will be easier to identify issues?

[xuedinge233]

@xuedinge233 lgtm. Regarding the workflow design, could you also split make test (correctness test) and make test-convergence (convergence test) into two seperated jobs so it will be easier to identify issues?

Ok. Then I'll test them separately

[xuedinge233]

Hi, @Tcc0403. Here is an update on the latest progress. First of all, I would like to state that all test/transformers have been successfully completed . Here is the detailed explanation:

• Except for test_fused_neighborhood_attention.py , this module is currently under repair and will be temporarily skipped.
• The PRs related to NPU that have not been merged yet have been manually merged into the forked repository:
  • [NPU] Add Llama4_rope support on NPU #1035
• Later, the CI program will be automatically executed every day. Here is the link to CI Status. It will automatically synchronize based on the test results.

Next, we will fix and test the content of test/chunked_loss to ensure the complete operation of make test.

(Here is a CI status button for convenient and quick viewing)

[xuedinge233]

Hi @Tcc0403 , The CI has completed a full week of testing and all checks have passed so far.
Would it be acceptable to first connect the CI upstream to the transformers test suite , and then extend it to run the full make test once everything is stable?