Voice against general ability
Every model is scored on the same held-out text. Up and down is the owner’s messages (personal CE); left and right is web, dialogue and code (general-weighted CE). Lower is better on both, so the bottom-left is best. The line joins the strategies no other beats on both.
Group means large, single seeds small. 512-token windows.
- Flagship + C, the final model (49M)
- Flagship (49M)
- Probes (17M), mean of 4 seeds
- Frontier
Table view
| Model | Size | Runs | Personal CE | General CE | Frontier |
|---|
The probes are the controlled study: one change at a time, four seeds each. The flagship and its adaptation are single runs of another size, budget and context, so where they sit describes and does not attribute. P1 never saw the owner’s messages; its personal CE is off the scale.
How the final model was made
-
Pretrain the flagship
parameters, tokens of web, dialogue and code at , with the owner’s chat mixed in from the first step: % of the tokens. That is P4’s way in, which at 17M bought the voice at no detectable general cost (below).
-
Adapt it, by a rule fixed in advance
candidate recipes were frozen before the first attempt, each continuing the flagship for tokens. A candidate passes only if personal CE falls by at least and general CE rises by at most on validation. Only passed: % chat at learning rate , personal , general .
-
Read the samples, then choose
The owner read the flagship’s and ’s voice samples and kept ().
-
Open the test split once
Only then was the sealed test split read. ’s gain holds: personal , general . Nothing was chosen from it.
| Native 1024-token context | Flagship, test | Final, test | Change, test | Change, val |
|---|
The exploratory measures are reported beside their real-message references in the repository and never judged. Impersonation AUC is how well a classifier tells the model’s messages from the owner’s real ones; attribution and Delta ask whether a stylometric test names the owner as the author.
What the controlled study found
Each verdict compares runs of the 17M probe that differ in one thing. A difference counts only when its interval excludes zero.
Rotary positions beat learned ones
Learned position embeddings end higher in validation loss than RoPE, against a seed-noise floor of ± ( seeds).
Muon beats AdamW, narrowly
Muon ends lower in validation loss, each optimiser at its own swept learning rate (floor ±).
Adaptation buys the voice, at a general cost that grows with the chat share
| Chat share | Personal CE | General CE |
|---|---|---|
| 25% | ||
| 50% | ||
| 100% |
Against a control adapted on general text alone, over four pretrained bases. All six verdicts resolved.
Mixing the chat in from the start costs no detectable general ability
Against P1: personal , general , unresolved: no cost this design can detect.
Against E1’s 50% adaptation: personal , general . A trade, not a winner.
Differences in nats per token. E3 and E4 are judged against a 95% prediction interval from P1’s seeds; E1 and P4 use 95% intervals, Bonferroni-corrected over each experiment’s verdicts.
Samples
What the models wrote, drawn at random rather than picked. At the owner’s request a script chose them without anyone reading them: from the test split, it kept the samples that pass the gate below and a word filter for profanity, slurs and insults (English and Vietnamese), with prose in English and of moderate length, then drew three voice prompts, one general text and one function with a fixed seed. Each prompt appears twice: the final model’s sample, then the unadapted flagship’s. A word filter is not a reading, so a sample can still say something odd. A voice sample is the model’s next message in a held-out conversation, written after the owner’s preceding messages. Those prompts are the owner’s real messages and are not shown.
Each sample passed a gate before it was published: no run of more than tokens shared with the training chat, and nothing the corpus’s scrubber finds (names, handles, phone numbers, emails, links).
The samples are not published yet.
Tokens such as <NAME> are the corpus’s redactions, which the model learned to write.
What stays private
The owner’s messages, the tokenizer trained on them, the list of names redacted from them and every model checkpoint stay off the internet. MiniMe runs only on the owner’s laptop. There is no endpoint, public or private, and no live model behind this page.
Memorisation: checked, not ruled out
A small model can reproduce its training text verbatim, other people’s words included. The memorisation check gave each model 700 of the owner’s training messages and 700 held-out ones, each after the messages before it, and counted exact reproductions. It could detect memorisation only if the run that overfit the chat (E1 100/0, about seven passes) separated from a model that never saw it (P1). It did not, so by its own rule the check is blind at this scale and no claim of low memorisation is made from it.
| Model | Training messages written exactly | Held-out messages written exactly | Run shared with the training chat, mean | Its 95th percentile |
|---|
A run shared with the training chat is the longest token sequence a voice sample has in common with the owner’s training messages. The owner’s own held-out messages share as much with their past, so overlap alone cannot tell recall from style.
Limits
- At this size a model learns voice, not facts: words, rhythm, punctuation, register. It knows nothing reliable about the owner’s life and does not hold a conversation.
- One person’s messages: every finding about the persona is a case study of one.
- The flagship is a single run. Its distance from the probes mixes size, budget and context, so none of it is attributed to any one of them.
- The personal corpus is the owner’s English and code-switched Instagram messages; mainly-Vietnamese messages were left out by design.