A Smaller On-Device Model Artifact Matched or Beat a Larger One: A Single-Board Case Study of 4B and 9B RKLLM Artifacts on the RK3588 NPU

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

We compare two quantized language-model artifacts on a single 32 GB Turing RK1 module (Rockchip RK3588 NPU) under one frozen task set: a Qwen3-4B-Instruct-2507 RKLLM artifact and an experimental Qwen3.5-9B RKLLM conversion. Twelve frozen tasks—six Python coding tasks scored by executing assertions and six tool-selection tasks scored by exact comparison with an answer key—were each run three times per model in fresh processes, for 72 trials. The 4B artifact passed 36/36 trials and the 9B artifact 33/36. Answers were byte-identical across repeats for every task, so the effective sample is twelve tasks per model: 12/12 versus 11/12 (exact 95% intervals 0.735–1.000 and 0.615–0.998; exact paired test on the single discordant task, p=1.0). The correctness data therefore show one reproducible failure, not a statistically separable quality difference. Both artifacts passed all coding trials; the 9B failure was a syntactically valid tool call that selected a different tool than the one the request named. The resource differences are large relative to their spread: median native generation rate was 3.385 versus 1.75 tokens/s (ranges 3.35–3.44 and 1.75–1.76), a ratio of 1.93; median prefill rate was 74.415 versus 31.76 tokens/s; and peak process resident memory was 4335 versus 8679 MiB. Per-trial clock snapshots showed the DDR controller at 528 MHz before and after every trial; a separate sampled diagnostic run of the 4B artifact showed it at 2112 MHz throughout inference, so the idle snapshots do not support calling these throughputs DDR-limited. The two artifacts differ in model release, toolkit version, quantization grouping, and chat framing, so this is a deployment comparison of two artifacts on one board, not a test of parameter count. Drafted with AI assistance; reviewed and approved by the author. CloudLab Works.

• Quantization Trade-offs for LLM Inference on Apple Silicon: A Practical Speed, Memory, and Accuracy Benchmark

  Zenodo (CERN European Organization for Nuclear Research) 2026 · 0 citations

• Quantization Thresholds Replicate, Failure Modes Do Not: A Three-Model Study of Agentic Tool Use in Polish from 8-bit to 2-bit

  arXiv (Cornell University) 2026 · 0 citations