Skip to content

Latest commit

 

History

35 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

QWEN3C - Qwen3 INT8 Inference Engine

Lightweight transformer inference engine for Qwen3 0.6B, designed for both host development and baremetal RISC-V deployment with hardware acceleration via a 16x16 INT8 systolic array.

Features

  • Qwen3 0.6B model with INT8 quantization (604MB)
  • 28 transformer layers, 16 heads, 1024 dimensions, 32K vocabulary
  • Header-only C implementation for maximum portability
  • Arena-based memory allocation (no malloc/free on baremetal)
  • 16x16 INT8 systolic array hardware acceleration (GEMMA on VEGA AT1051)
  • Dual-target: host (x86/ARM) and baremetal (RISC-V)

Quick Start

Model Files (Required)

The following model files are required but not included in the repository:

  • model_q8.bin (604MB) - Quantized INT8 model weights
  • model_q8.bin.tokenizer (403KB) - BPE tokenizer vocabulary

Generate these files using the export scripts in scripts/:

cd scripts
uv sync
uv run export_q8.py ../model_q8.bin Qwen/Qwen3-0.6B --vocab-size 32000

See scripts/README.md for more options.

Host Build (Development/Testing)

just hc    # Build and run on host

Baremetal Build (RISC-V Target)

just b     # Build for baremetal RISC-V (requires Docker)

Project Structure

qwen3c/
├── main.c                  # Entry point, model loading, tokenizer, inference loop
├── include/
│   ├── config.h            # Build configuration, memory layout, feature flags
│   ├── platform.h          # Platform abstraction (host/baremetal detection)
│   ├── memory.h            # Arena-based memory management (3 arenas)
│   ├── types.h             # Core data structures (tensors, model config)
│   ├── nn.h                # Neural network operations, transformer forward pass
│   ├── gemma.h             # Hardware accelerator interface (systolic array)
│   ├── log.h               # Logging macros and performance timing
│   ├── baremetal.h         # UART, printf, libc implementations for baremetal
│   ├── encoding.h          # RISC-V CSR encoding definitions
│   ├── tests.h             # Test framework dispatcher
│   ├── test_diagnostic.h   # Early system validation tests
│   ├── test_accelerator.h  # Hardware accelerator tests
│   ├── test_memory.h       # Memory management tests
│   ├── test_gemma.h        # GEMMA IP performance tests
│   └── test_debug.h        # Debug system tests
├── crt.S                   # C runtime startup for baremetal
├── mbl.lds                 # Linker script for baremetal memory layout
├── justfile                # Build commands (just hc, just b)
├── build.sh                # Baremetal build script (Docker)
├── makefile                # Alternative build system
├── scripts/                # Python scripts for model export
│   ├── export_q8.py        # Q8_0 quantization and vocabulary pruning
│   ├── model.py            # Qwen3 model architecture
│   └── pyproject.toml      # Python dependencies
├── build/                  # Build artifacts (baremetal ELF, bin, dump)
└── logs/                   # Sample execution logs

Memory Layout (Baremetal)

Region Address Size Purpose
Application 0x80000000 4MB Code and static data
General Arena 0x80400000 248MB Runtime allocations, activations
Cache Arena 0x8FC00000 128MB KV cache (persistent)
Accelerator 0x97C00000 8MB Systolic array workspace (non-cacheable)
Model Data 0x98400000 635MB Pre-loaded model weights (read-only)
Tokenizer 0xBFF00000 1MB Vocabulary data (read-only)

Hardware Acceleration

The engine integrates with a 16x16 INT8 systolic array accelerator (GEMMA) for matrix multiplication:

  • Accelerator registers memory-mapped at 0x20060000
  • Systolic array workspace at 0x97C00000 (non-cacheable)
  • Intelligent routing: hardware used when total_ops >= 65536, d >= 16, group_size >= 64
  • Automatic fallback to software on timeout or failure

Example Output

Host Build (Apple M-series, ~50 tok/s)

[INFO] === Q3INF INFERENCE ENGINE v0.13 ===
[INFO] STAGE 4/7: Loading model and initializing runtime...
[ACCL] Hardware accelerator: HOST BUILD (software only)
[ NN ] CONFIG: layers=28 heads=16 dim=1024 vocab=32000
[INFO] User prompt: What is 2 + 2?

[STEP] === STARTING INFERENCE ===
[PERF] Inference started
[ NN ] Forward pass pos=0, token=31975
[ NN ] Progress: [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 ]
[PERF] LAYER AVG: 5.848ms

[STEP] === RESPONSE GENERATION ===
[ TK ] Token 17 -> '2'
[ TK ] Token 488 -> ' +'
[ TK ] Token 220 -> ' '
[ TK ] Token 17 -> '2'
[ TK ] Token 284 -> ' ='
[ TK ] Token 220 -> ' '
[ TK ] Token 19 -> '4'

[STEP] === FINAL RESPONSE ===
[INFO] >>> 2 + 2 = 4

[PERF] Inference timing breakdown:
[PERF]   Pure inference: 28 tokens in 555.098ms (50.4 tok/s)

Baremetal Build (RISC-V + GEMMA Accelerator)

booting
[INFO] === Q3INF MAIN ENTRY ===
[INFO] STAGE 1/7: System initialization...
[INFO] === Q3INF INFERENCE ENGINE v0.13 ===
[INFO] Setting up non-cacheable region: 0x97c00000 - 0xc0000000
[ACCL] Detecting GEMMA accelerator at 0x20060000
[ACCL] GEMMA accelerator detected and configured successfully
[ACCL] Hardware accelerator: ENABLED
[ACCL]   Implementation: GEMMA VEGA AT1051 (16x16 Systolic Array)
[INFO] [BM] Using pre-loaded model at 0x98400000 (635MB)
[ NN ] CONFIG: layers=28 heads=16 dim=1024 vocab=32000
[INFO] Hardware acceleration: ENABLED (GEMMA VEGA AT1051 (16x16 Systolic Array))
[INFO] User prompt: What is 2 + 2?

[STEP] === STARTING INFERENCE ===
[ NN ] Forward pass pos=0, token=31975
[ NN ] Progress: [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 ]

[STEP] === RESPONSE GENERATION ===
[ TK ] Token 17 -> '2'
[ TK ] Token 488 -> ' +'
[ TK ] Token 220 -> ' '
[ TK ] Token 17 -> '2'
[ TK ] Token 284 -> ' ='
[ TK ] Token 220 -> ' '
[ TK ] Token 19 -> '4'

[STEP] === FINAL RESPONSE ===
[INFO] >>> 2 + 2 = 4

[INFO] === Q3INF INFERENCE ENGINE COMPLETE ===

Build Configuration

Key defines in config.h:

Define Description
HOST_BUILD Defined automatically for host compilation
DEBUG_MODE Enable verbose logging
ENABLE_INFERENCE Enable token generation
ENABLE_TESTS Enable test framework
CONTEXT_LENGTH Maximum sequence length (default: 64)
USE_ACCELERATOR Enable hardware acceleration (baremetal only)

Requirements

  • Host build: GCC or Clang with C99 support
  • Baremetal build: Docker (contains RISC-V toolchain)
  • Model export: Python 3.11+, uv package manager

License

MIT

About

qwen3 inference engine

Resources

Stars

2 stars

Watchers

0 watching

Forks

Contributors

Languages