What I’ve written

Long-form notes, every number measured on real hardware or traced to a dated source.

The projects page shows what I built. This page is for the things I had to work out along the way: how inference actually spends its time, which knobs move the needle and which only look like they do. Each post ships with the scripts and raw results so you can rerun it on your own machine. Hit / to search.

7 posts
Model selection Encoders vs decoders Evals Serving cost

Which model should we use? A production framework for choosing architectures, models, evals and training, from the use case to the bill

A decision framework for production AI that puts the model sixth: write the contract, name the task shape, let the constraints cut the field, choose encoder, decoder or encoder-decoder for each link, build an eval set from real inputs, shortlist a ceiling, a default, a specialist and a small model, climb the training ladder only as far as the gap, and size memory, latency and the bill. With the Hugging Face search that fools people, the benchmarks worth reading for each task, what to do when your language or domain has no good model, and six use cases worked end to end.

18 October 2026 Long read Read the post
PII detection Gemma 3 270M LoRA llama.cpp

Introducing Kavach PII 270M: a 270M model that finds personal data better than GLiNER, Presidio and spaCy, in 29 languages, trained on one laptop GPU

Updated for version 3: a quarter-billion-parameter model that scores 92.0 value-F1 where GLiNER gets 59.3 and Presidio 29.5, on the same 4,577-document held-out set with identical gold, and 90.4 on documents built entirely by code. Why dropping character offsets from the training targets was the decision that made it work, the re-annotation that found the old test set missing one value in ten, how 43 labels and 29 languages were built and checked, what happens on ASR transcripts, OCR noise and prompt injection, the full GGUF ladder with the reason a 270M model quantizes badly, the one number that got worse, and the three experiments that did not pan out.

23 September 2026, updated 29 September Long read Read the post
Speaker diarization Kaldi PLDA Docker

Who spoke when, on your own data: training, tuning and shipping a Kaldi speaker diarizer that runs on a CPU

From a folder of labelled recordings to a container that writes an RTTM in seconds on a CPU. The x-vector pipeline explained stage by stage, the back-end trained on your own turns in under a minute, the threshold cliff a trained PLDA creates and the Bayesian HMM that fixes it, the speech detector that matters more than any other knob, every setting chosen on dev and measured on held-out meetings, the data-size and speaker-count ablations, speed and memory against audio length, and the honest comparison with pyannote and NVIDIA's Sortformer on the same audio.

20 September 2026 Long read Read the post
Language adaptation GRPO Tokenizers PyTorch

Teaching a language model a language it never knew: continued pretraining, supervised fine-tuning and reinforcement learning on one laptop

The whole training pipeline, run end to end on one 8 GB laptop GPU to teach an English-only 135M model Hindi: tokenizer extension without touching English, new embeddings, continued pretraining with replay, supervised fine-tuning, then GRPO with rewards a script can check, written out in full and measured at every step. From one word repeated to a model that reads Hindi better than stock models twelve times its size and ties a 1.7B on a Hindi task, with the large-lab pipeline set beside it and what scaling the same recipe would change.

15 September 2026 Long read Read the post
Cerebras Hardware Inference engineering Python

Where the bandwidth line ends: how Cerebras gets to 2,000 tokens a second, worked out from the outside

The bandwidth line predicts 600,000 tokens a second for four Cerebras wafers on a 70B model and they get 2,000. An engineering read from public sources only: every speed claim dated and sourced, the wafer rebuilt from the core up, the microseconds a transformer layer really costs on a mesh and why eight published speeds agree on it, what the SRAM design gives up in context, prompts, batch and power, how Groq and NVIDIA solve the same equation, and what carries back to a laptop.

14 September 2026 Long read Read the post
Part 2
llama.cpp Architectures Edge hardware Python

Before you tune anything: a field guide to inference speed and accuracy on the hardware you actually have

What carries over from one machine to another: bandwidth divided by the bytes a token really reads. Six architectures (dense, MoE, sliding-window, two hybrids, a pure SSM) measured on CPU and GPU against that line, plus encoders and T5. A pre-check script that caught a laptop running at a third of its speed, a ten-minute KL-divergence recipe, which quant to pick on a CPU without AVX2, and the order to try things in.

12 September 2026 Long read Read the post

Say hello

Want one of these, but for your problem?

Happy to talk through architecture, trade-offs, or why the obvious approach is not working. Email is fastest.

Email LinkedIn GitHub Twitter Resume