On my AMD Ryzen AI 9 HX 370 laptop, which has a Radeon 890M iGPU — unified memory — I switched to using a local LLM for development, replacing the cloud models I'd normally reach for. The lean Pi coding agent is also essential for this to work. If you've been wondering whether a local model can really do the job of building a big application, here's what I found.

The setup

The project I am working on is a distributed communication system, split into two Rust codebases:

• a Dioxus frontend, running as WASM in the browser — of roughly 35,000 lines of code;
• an Axum backend running on one or several servers — of roughly 11,000 lines of code.

The model that drives the agent is Tiel-Coder-35B-A3B-MTP:UD-Q4_K_XL a quantised GGUF model served locally by llama.cpp. The model's page is on HuggingFace. It runs on a single NixOS laptop — an AMD Ryzen AI 9 HX 370 with its integrated Radeon 890M GPU — and the whole thing, model and all, takes about 28 GiB of unified memory, so inference never leaves the host. The GPU backend is Vulkan.

The start script

The model is launched with llama-server from a small shell script. The script takes the port as its only argument. Here it is in full.

 1#!/bin/sh
 2#
 3# https://huggingface.co/peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF-MTP
 4#
 5
 6export MODEL="/home/mdo/.models/Tiel-Coder-35B-A3B-MTP-UD-Q4_K_XL.gguf"
 7export MODEL_NAME="Tiel-Coder-35B-A3B-MTP"
 8
 9export CONTEXT_WINDOW_SIZE=262144
10
11# https://github.com/ggml-org/llama.cpp/issues/13178
12# This MTP model has a jinja templated bundled.
13# export QWEN3_CODER_TEMPLATE="./Tiel-Coder-35B-A3B-MTP-UD.jinja"
14
15# BATCH TUNING FOR iGPU: 
16# --batch-size: Logical limit for prompt processing (2048 is safe)
17# --ubatch-size: Physical chunk size processed by the GPU at once.
18# For RDNA3.5 (890M), 512 is the "sweet spot" for speed without driver hangs.
19export BATCH_SIZE=512
20export UBATCH_SIZE=512
21
22# Environment: Force Vulkan to see more of your RAM if needed.
23export GGML_VULKAN_DEVICE_MEMORY_LIMIT=32768
24
25# OPTIMIZE FOR 890M SHARED MEMORY PIPELINE
26export GGML_VULKAN_UNIFIED_MEMORY=1
27
28# 1. FORCE RADV TO TREAT ALL RAM EQUALLY This stops the driver from
29# "moving" data from System RAM to VRAM
30# export GGML_VK_PREFER_HOST_MEMORY=1
31export GGML_VK_PREFER_HOST_MEMORY=1
32
33# Avoid suballocation pool fragmentation
34export GGML_VK_SUBALLOCATION_BLOCK_SIZE=4294967296
35
36# Port as parameter
37if [ $# -ne 1 ]
38then
39  echo "Usage: $0 <port>";
40  exit 1
41fi
42
43export PORT=$1
44
45GGML_BLAS_DEVICE=Vulkan0 GGML_BACKEND=Vulkan llama-server \
46  -m "${MODEL}" \
47  --alias "${MODEL_NAME}" \
48  --spec-type draft-mtp \
49  --spec-draft-n-max 3 \
50  --spec-draft-p-min 0.8 \
51  --host 0.0.0.0 \
52  --port ${PORT} \
53  --load-mode mmap+mlock \
54  -n -1 \
55  -ngl 100 \
56  -np 1 \
57  --cache-prompt \
58  --flash-attn on \
59  -c ${CONTEXT_WINDOW_SIZE} \
60  --cache-type-k q8_0 \
61  --cache-type-v q8_0 \
62  --batch-size ${BATCH_SIZE} \
63  --ubatch-size ${UBATCH_SIZE} \
64  --jinja \
65  -t 24 \
66  -tb 24 \
67  --temp 0.6 \
68  --top-p 0.95 \
69  --min-p 0.01 \
70  --repeat-penalty 1.0 \
71  --presence-penalty 0.0 \
72  --frequency-penalty 0.0 \
73  --no-mmproj

A few things are worth pointing out. The mixture-of-experts (MoE) model uses multi-token prediction (MTP), a flavour of speculative decoding. Rather than training two separate models, the model has extra 'assistant' heads trained on some of its intermediate layers, so those layers can act as a small draft model entirely on their own. That draft model proposes a few tokens at once, and the full model verifies them in a single parallel pass: accepted tokens are kept, and the first rejected token and everything after it are discarded. In the start script, this is what --spec-type draft-mtp selects; --spec-draft-n-max 3 caps the draft at three tokens ahead; --spec-draft-p-min 0.8 sets the minimum probability a draft token must have to be accepted. The context window is a generous 262,144 tokens, which is what lets the agent hold a large slice of the codebase in one prompt.

Pi configuration

The Pi coding agent runs in its dedicated Docker container — see photonsphere / ai-containerized — and is configured to use the locally running model (which I run on port 8092 — remember to open up the port in your firewall).

~/.pi/agent/models.json:

 1{
 2  "providers": {
 3    "llama-server-8092": {
 4      "baseUrl": "http://host.docker.internal:8092/v1",
 5      "api": "openai-completions",
 6      "apiKey": "none",
 7      "compat": {
 8        "supportsDeveloperRole": true,
 9        "supportsReasoningEffort": true,
10        "maxTokensField": "max_tokens"
11      },
12      "models": [
13        {
14          "id": "0.0.0.0:8092",
15          "name": "8092",
16          "reasoning": true,
17          "input": ["text"],
18          "contextWindow": 262144,
19          "maxTokens": 131072,
20          "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 }
21        }
22      ]
23    }
24  }
25}

Note that contextWindow and maxTokens must be set high enough to accommodate the agent's long reasoning turns.

How fast is it, really?

Generation throughput per task — tg (tokens/second or t/s) as reported by llama-server, over the two log captures:

¹ The "first ~60 samples" cover the opening minutes of a session, before contexts (applies to both client and server) grew.

Prompt processing (the agent re-reading its context) is the flip side. The worst cases:

It pays to start a fresh session per task, because the agent runs much faster with a smaller context.

Guiding instructions

Some guiding instructions shaped every session. The 'restraint' rules were explicit: write idiomatic Rust, prefer a test-driven approach where tooling permits, never commit to version control, stop and report a problem rather than get stuck in analysis paralysis or paper over it, avoid spaghetti code in favour of a scalable architecture. The 'role' instructions cast the model as an experienced Rust developer and software architect, well versed in Dioxus, WASM FFI (JavaScript), and Axum, among other things.

Is it feasible?

Yes — and on nothing more than a single laptop, at that. A large and complex project can now be developed with a local model as the sole peer programmer. I have already made the full switch.

The model used here is small enough to run entirely on one workstation and fast enough to stay in the developer's flow. The model is fine-tuned for coding only and its 'coding-fu' genuinely feels on par with some cloud models such as MiMo-V2.5. Running a model locally, however, drains your laptop's battery quickly, so keep it plugged in when you can; also, clean the fans regularly, since the high fan load causes dust to accumulate faster.

Quoting the model itself

Ugh, I'm going in circles. Let me just try the restructure and see if it compiles. Empirical iteration is faster than theorizing.