All research
01 / Model
Wren 40: a small language model trained from scratch
A 36.7M-parameter model pre-trained on 2 billion tokens and tuned on 400 million tokens of conversation data. It runs in your browser.
Wren 40 is our own small language model: a dense transformer of about 36.7 million parameters, trained from scratch rather than fine-tuned from someone else's weights.
It is pre-trained on 2 billion tokens of mostly English text, then tuned on 400 million tokens of conversation data. A later round of multi-turn tuning raised its score on our held-out 79-case conversation test from 31% to 73%.
It is small enough to run in your browser at ai.avenpa.com. It answers short English questions and simple code. Korean, arithmetic and fact checks are still weak, and it can be wrong.