[feat, npu] Liger-Kernel is now available for NPU

[zheliuyu]

Motivation

As title, this pr enables the use of Liger-Kernel on npu. A Qwen3-30B-AB SFT fine-tuning example is provided to illustrate its application.

Experiment results

Visualization

Script

This example runs on a single NPU.

qwen3_moe.sh

# Avoid invalid OMP/MKL thread envs (some environments set them to 0/false).
export OMP_NUM_THREADS=1
export MKL_NUM_THREADS=1
# Reduce allocator fragmentation on NPU (best-effort; ignored if unsupported).
export PYTORCH_NPU_ALLOC_CONF=${PYTORCH_NPU_ALLOC_CONF:-max_split_size_mb:32}
export TOKENIZERS_PARALLELISM=false

CUDA_VISIBLE_DEVICES=0 \
swift sft \
    --model /dev/shm/Qwen3 \
    --template qwen3 \
    --tuner_type lora \
    --dataset 'swift/Chinese-Qwen3-235B-2507-Distill-data-110k-SFT#2000' \
              'swift/self-cognition#1000' \
    --load_from_cache_file true \
    --torch_dtype bfloat16 \
    --num_train_epochs 1 \
    --per_device_train_batch_size 1 \
    --per_device_eval_batch_size 1 \
    --learning_rate 1e-4 \
    --lora_rank 4 \
    --lora_alpha 16 \
    --router_aux_loss_coef 0 \
    --gradient_accumulation_steps 16 \
    --max_steps 2 \
    --save_total_limit 2 \
    --logging_steps 1 \
    --max_length 64 \
    --output_dir output \
    --system 'You are a helpful assistant.' \
    --warmup_ratio 0.05 \
    --dataloader_num_workers 0 \
    --model_author swift \
    --use_liger_kernel True \
    --attn_impl eager \
    --model_name swift-robot

use_liger_kernel=False

{
    'loss': '7.281',
    'grad_norm': '6.169',
    'learning_rate': '0.0001',
    'token_acc': '0.3935',
    'epoch': '0.0261',
    'global_step/max_steps': '1/2',
    'elapsed_time': '3m 57s',
    'remaining_time': '3m 57s',
    'memory(GiB)': '60.3',
    'train_speed(s/it)': '237'
}
{
    'loss': '6.51',
    'grad_norm': '5.505',
    'learning_rate': '0',
    'token_acc': '0.4681',
    'epoch': '0.0522',
    'global_step/max_steps': '2/2',
    'elapsed_time': '8m 14s',
    'remaining_time': '0s',
    'memory(GiB)': '60.3',
    'train_speed(s/it)': '247'
}
{
    'train_runtime': '494.6',
    'train_samples_per_second': '0.065',
    'train_steps_per_second': '0.004',
    'train_loss': '6.895',
    'epoch': '0.0522',
    'global_step/max_steps': '2/2',
    'elapsed_time': '8m 15s',
    'remaining_time': '0s',
    'memory(GiB)': '60.3',
    'train_speed(s/it)': '247.3'
}

use_liger_kernel=True

{
    'loss': '7.229',
    'grad_norm': '6.363',
    'learning_rate': '0.0001',
    'epoch': '0.0261',
    'global_step/max_steps': '1/2',
    'elapsed_time': '2m 36s',
    'remaining_time': '2m 36s',
    'memory(GiB)': '60.5',
    'train_speed(s/it)': '155.7'
}
{
    'loss': '6.497',
    'grad_norm': '5.669',
    'learning_rate': '0',
    'epoch': '0.0522',
    'global_step/max_steps': '2/2',
    'elapsed_time': '5m 22s',
    'remaining_time': '0s',
    'memory(GiB)': '60.5',
    'train_speed(s/it)': '160.8'
}
{
    'train_runtime': '322.2',
    'train_samples_per_second': '0.099',
    'train_steps_per_second': '0.006',
    'train_loss': '6.863',
    'epoch': '0.0522',
    'global_step/max_steps': '2/2',
    'elapsed_time': '5m 22s',
    'remaining_time': '0s',
    'memory(GiB)': '60.5',
    'train_speed(s/it)': '161.1'
}

PR type

• Bug Fix
• New Feature
• Document Updates
• More Models or Datasets Support

[gemini-code-assist]

Code Review

This pull request updates the NPU support documentation to reflect that Liger-Kernel is now supported. The changes include updating the status in the support matrix and removing Liger-Kernel from the list of unsupported items. A review comment points out a table alignment issue in the support matrix caused by the change in text length and provides a suggestion to restore visual consistency.

[gemini-code-assist]

The alignment of the table column is broken. To maintain consistent alignment with other rows (like line 370), "已支持" should be followed by three spaces instead of one. This accounts for the width difference between "已支持" (3 characters) and "暂不支持" (4 characters) in a fixed-width context.

Suggested change

	| | Liger-Kernel | 已支持 |
	| | Liger-Kernel | 已支持 |