Alia Architecture

Your hardware. Your AI. No cloud. — Local coding agent launcher v0.1.0

Alia, explained by Alia

A local coding agent drew its own architecture: launcher, llama.cpp server, Qwen Code and your GPU, with real benchmark numbers.

Explore the architecture See the benchmarks

Layered Architecture

Alia sits between the user, the GPU, and Qwen Code — detecting hardware, selecting a profile, starting llama-server on loopback, and launching the agent. Everything stays local.

⌘ User Terminal bash / zsh / tmux via SSH or local Entry Point 🌐 Browser Web Shell → 127.0.0.1:4170 HTTP / WebSocket Alternative Frontend 🔧 Alia Launcher • Hardware Detection (GPU / VRAM / RAM) • Profile Selection (auto / pinned) • Start Lock (flock / PID management) • Server Lifecycle (start / stop / health) • GPU Lock coordination • Ollama unload (if running) Orchestrator 💻 Qwen Code Terminal UI / Web Shell v0.25.1 — @qwen-code/qwen-code OpenAI API compatible API Key: "local" (fixed) QWEN_HOME isolated Coding Agent 🦙 llama-server 127.0.0.1:8090 (loopback only) CUDA inference engine llama.cpp a0e6dc7 MTP drafting (speculative decode) --no-webui · --api-key local Inference Server 🎮 GPU / RAM NVIDIA GPU (detected) VRAM → model layers RAM → expert offload NVMe → page cache Hardware

Swipe sideways to see the whole diagram →

Request Flow

Watch a request travel from your prompt through Qwen Code into llama-server, across GPU and RAM, and back as streamed tokens. Alia starts these pieces once; it is not in the token path.

Prompt your terminal Qwen Code agent loop llama-server 127.0.0.1:8090 GPU attention + KV RAM MoE experts MTP 2-token draft stdin POST /v1/chat CUDA expert offload draft + verify → → → → → ← ← ← ← ← Token Generation Cycle Once: Alia detects the GPU → picks a profile → starts llama-server → launches Qwen Code Per token: Qwen Code → /v1/chat/completions → attention on GPU, experts from RAM → MTP drafts 2 tokens → streamed back

Swipe sideways to see the whole diagram →

Measured Benchmarks

Real numbers from profiles.json — speeds measured on RTX 3070 8 GB, Ryzen 7 5800X, 16 GB RAM with llama.cpp commit a0e6dc7 and each profile's launch flags. The 35B values at 60K come from an earlier run without --jinja; quality scores come from separate runs.

Qwen3.6 35B-A3B — Balanced

rtx8gb-16gb-qwen36-35b · IQ3_XXS · 14.07 GB
TG @ 4K
46.5tok/s
TG @ 32K
35.2tok/s
TG @ 60K
38.2tok/s
PP @ 4K
1,008tok/s
PP @ 32K
1,301tok/s
PP @ 60K
1,041tok/s
MTP Acceptance Rate
62%
+14 % TG @4K, +10 % @32K vs. no MTP
Code Quality
9/10
Separate run, server args not recorded
Tool Use
7/8
Separate run, server args not recorded
RAM Pressure (low free RAM)
24.6 tok/s @32K, 9.7 @60K
32 GB RAM recommended for long contexts — model drops from page cache, streams from SSD

Qwen3.5 9B — Fast

rtx8gb-qwen35-9b-fast · Q4_K_M · 5.7 GB
TG @ 4K
65.7tok/s
TG @ 32K
57.5tok/s
PP @ 4K
2,978tok/s
PP @ 32K
2,708tok/s
Context Window
65,536tokens
Code Quality
8/10
Measured via Ollama (not launch config)
Tool Use
8/8
Measured via Ollama (not launch config)
Model Size
5.7GB
Downloads in ~8 min @ 100 Mbit/s
Recommended RAM
12GB

Qwen3.6 35B-A3B — Q4_K_XL

rtx12-16gb-qwen36-35b-q4 · 20.8 GB · unmeasured
Target Hardware
12–16 GB VRAM · 32 GB RAM
Context
32,768tokens
Server Args
-fa on, --jinja, -b 2048, --cache-ram 4096
Note
Never run on target hardware — unmeasured profile. Requires 35B Q4_K_XL GGUF (~20.8 GB).

Qwen3.6 35B-A3B — Q4_K_XL (24 GB)

rtx24gb-qwen36-35b-q4 · 20.8 GB · unmeasured
Target Hardware
20+ GB VRAM · 32 GB RAM
Context
32,768tokens
Server Args
-ngl 99, -fa on, --jinja, -b 2048
Note
Full GPU offload (-ngl 99) with 20+ GB VRAM. Unmeasured — designed for RTX 4090-class GPUs.

Component Details

Each component in the Alia ecosystem — from launcher to inference, from security to data paths.

🔒 Security Model

  • llama-server bound to 127.0.0.1:8090 only
  • Fixed API key "local" — not exposed externally
  • State dir mode 0700, files mode 0600
  • --no-webui disables llama.cpp web UI
  • No pkill — targets only recorded PID by start time
  • Profile can't override host/port

🔧 Launcher Features

  • Hardware detection via nvidia-smi + /proc/meminfo
  • Multi-GPU: picks largest VRAM GPU
  • CUDA_VISIBLE_DEVICES pinning
  • Start lock via flock
  • GPU lock coordination (shared lock files)
  • Ollama model unload before launch
  • Health check before reuse

📊 Profile Selection

  • Automatic: matches VRAM/RAM, picks highest ranked measured
  • Pinned: explicit profile via alia use
  • Warns on unmeasured or out-of-range
  • 4 profiles: 2 measured, 2 unmeasured
  • Server args validated (no network override)

📁 Data Paths

  • ~/.local/state/alia — PID, server state, logs
  • ~/.local/share/alia/models — downloaded GGUF
  • ~/.config/alia/config.json — profile, language
  • ~/.local/state/alia/qwen-home — isolated Qwen home

🦙 llama-server Configuration

  • llama.cpp commit a0e6dc7 (2026-10-14)
  • CUDA enabled (built from source)
  • MTP drafting: --spec-type draft-mtp
  • Flash attention: -fa on
  • Jinja chat template: --jinja
  • Build path: ~/src/llama.cpp/build/bin/llama-server

💻 Qwen Code Integration

  • Version: @qwen-code/qwen-code@0.25.1
  • OpenAI API compatible endpoint
  • Terminal UI + Web Shell (port 4170)
  • Isolated QWEN_HOME per Alia session
  • Approval modes: ask / auto-edit / yolo
  • Thinking toggle: --fast disables