About
What it is for
A dense language model for Korean and English, trained from scratch on a CPU-only server. In pretraining; not yet available to use.
Wren v1 is our first model. It is pretrained on 900 million tokens, weighted 80% English and 20% Korean, with the Muon optimizer and a final anneal on cleaner data.
Training runs on four CPU threads at roughly 340 tokens a second and resumes from its last checkpoint after any stop. A model this small will not match a large one; we are building it to learn what a small, cheap, fully understood model can do for the tools here.
In short
Wren v1, briefly
01
From scratch
Weights, tokenizer and training loop are ours, not inherited.
02
CPU only
Trained where it will run: four cores, no GPU.
03
Korean + English
A 20 / 80 mix drawn from a 1.24B-token corpus.
Products
Also from Avenpa
Questions about Wren v1?
Write to the studio. A person who works on it will answer.
Contact the studio