Wren v1: a small language model on four CPU cores

Wren v1 is our own small language model: a dense transformer of about 36.7 million parameters, trained from scratch rather than fine-tuned from someone else's weights.

It is pretrained on 900 million tokens drawn from a 1.24-billion-token corpus, weighted 80% English and 20% Korean, with the Muon optimizer and a final annealing phase on cleaner data. There is no GPU involved: the run uses four CPU threads and moves at roughly 340 tokens a second, so a full pass takes weeks, and it resumes from its last checkpoint after any stop.

The trade-off is plain. A model this small will not match a large one, and it is not on the site yet. We are building it to learn what a small, cheap, fully understood model can do for the tools here, and we will publish results once there are some worth reading.