October 29, 2026 · 6 min read
Introducing Quartz Micro Preview V3
Quartz Micro Preview V3 is out: the same 100-million-parameter model as V2, trained on about 65 times more text and then tuned to know what it is and to say so when it can't answer. As before there are two versions, a base model and a chat model, each shipped as plain Hugging Face safetensors, MLX, and GGUF. It is still a small research model that states wrong facts. This post is about what changed, how we measured it, and what is still broken.
What changed since V2
The architecture and tokenizer are unchanged: a dense, Llama-architecture decoder with 100.09M parameters (12 layers, hidden size 768, 12 attention heads with 4 KV heads, SwiGLU, RoPE, a 32K byte-level BPE vocabulary, 1,024-token context). It still loads with plain transformers, mlx_lm and llama.cpp, with no trust_remote_code. What changed is the data and how long we trained.
- Pretraining: 6.5 billion tokens in a single pass (99,384 steps of 65,536 tokens), against 100 million for V2. The mix is FineWeb-Edu (2.6B), Cosmopedia v2 (1.3B), Wikipedia (0.78B), DCLM-Baseline (0.52B), Stack Exchange (0.46B), FineMath (0.39B) and Python-Edu (0.33B), plus small slices of chat text, ELI5, SQuAD 2.0 and TriviaQA. We removed overlaps with ARC, HellaSwag, PIQA, WinoGrande, LAMBADA, OpenBookQA, SciQ, BoolQ, TriviaQA, MMLU and CommonsenseQA before training. It ran for about 18 days on one GTX 1660 Ti at roughly 4,200 tokens per second. Final validation perplexity was 13.6.
- Instruction tuning: 44,389 English conversations (12.3M tokens). That is V2's chat set (OpenAssistant OASST2 and smol-smoltalk slices), plus Dolly-15k, SQuAD 2.0 questions answered from a passage (the unanswerable ones are taught to say "the passage does not say"), CoCoNot examples of honest limits (no browsing, no current events, no feelings), and a 3,000-row identity set so the model can say who made it. Full-parameter, two epochs, loss on the assistant tokens only, about 1.5 hours.
V3 against V2, same prompts
Both chat models answered the same 156 prompts with identical decoding (greedy, repetition penalty 1.1, same tokenizer and chat template) on one GPU. A judge model scored the open-ended parts, and every pairwise comparison was judged in both orders and only counted as a win when both orders agreed. No judge call failed.
| Test | V2 | V3 |
|---|---|---|
| Identity, 100 held-out prompts (judge checks each reply against the model's fact sheet) | 3 / 100 | 71 / 100 |
| Identity, 24 hand-written prompts | 5 / 24 | 23 / 24 |
| Facts, V2's 15 original keyword prompts | 5 / 15 | 11 / 15 |
| Facts, 11 new prompts (grounded QA, honest limits, simple facts) | 4 / 11 | 9 / 11 |
| Head-to-head on 33 general prompts (judge, both orders must agree) | 4 wins | 25 wins (4 ties) |
| Answers that end on their own (end-of-text token) | 23 / 33 | 31 / 33 |
| Repetition (distinct 4-grams, higher is less repetitive) | 0.80 | 0.97 |
| Average answer length (words) | 127 | 52 |
| Loss on assistant tokens, general validation set (lower is better) | 2.383 | 1.719 |
| Loss on held-out identity answers (lower is better) | 4.298 | 1.304 |
How to read those numbers
- The identity jump is by design. V2 had no identity data and usually named a different maker (earlier tests had it call itself "Open Assistant"). V3 was trained to say it is a small model from Vertex AGI, that it can't browse or remember, and that it is not ChatGPT, Claude or Gemini. The 100 held-out prompts come from a split that was never in training, and V3 still fails 29 of them, mostly in multi-turn conversations (4 of 13 passed).
- The fact checks are a smoke test, not a benchmark. They are keyword checks, so they reward plain correct answers and are easy to fool. The 11 new prompts target things V3's data teaches (answer from a passage, admit it can't browse), so V2 was never trained for them. That row shows an added ability, not a like-for-like gain.
- It compares two releases, not one change. V3 differs from V2 in its instruction data and in its pretraining (6.5B tokens against 100M), so these numbers don't isolate the effect of the instruction data. On the base models, perplexity on a sample of V3's validation text is 14.4 for V3 and 28.1 for V2, but that text comes from V3's mix, so read it as "V3 has seen far more and more varied text", not a fair quality gap.
- Short answers are partly a feature. V2 rambled (127 words on average, and 10 of 33 answers never ended inside the token budget). V3 averages 52 words and mostly stops on its own, which also helps its repetition score.
- It is one greedy sample per prompt. With 26 keyword prompts and 33 judged comparisons, a difference of one or two items is noise. The judge moved by one item between two scoring passes of the same V3 answers (70 versus 71 of 100).
What it still gets wrong
It is a 100M-parameter model and it still invents things. Asked about a made-up city it described the place as fictional and from a famous novel, and asked for a two-line poem it wrote sixteen lines. It is weak at maths, reasoning and code, English only, and has had no safety tuning. Please treat every answer as unreliable, and don't use it for anything medical, legal, financial or safety-critical.
Every format, checked
As with V2, we compared each shipped format with the original training code on held-out text. For the chat model, 8-bit builds in MLX and GGUF stay within 0.01% of the original perplexity (13.878), GGUF Q4_K_M is 0.56% off, and 4-bit MLX is the lossiest at 6.2%. The full tables for both models are in the repo cards, along with the benchmark and every per-prompt output, wrong ones included.
Try it, and where to get it
The chat Space now opens on V3 and has a Compare tab that runs one prompt against any two models side by side, so you can put V3 next to V2 (or the original V1) and judge for yourself.
- Quartz Micro Preview V3 (chat): VertexAGI/quartz-micro-preview-v3
- Quartz Micro Preview V3 Base: VertexAGI/quartz-micro-preview-v3-base
- Chat Space: VertexAIco/quartz-micro-preview-chat
A note on licences
The weights carry an Apache-2.0 label, but they were trained on third-party data with its own terms, and some of those terms are NonCommercial or share-alike. For example, the Tulu-3 mixture includes no_robots (CC-BY-NC), and Wikipedia, Stack Exchange, Dolly and SQuAD are CC-BY-SA. Whether training-data terms extend to model weights is legally unsettled, so if you plan any commercial use, read the data section of the model card first. We treat V3 as a research and hobby model.
What's next
The long-term goal for the Quartz series is still to match the original SmolLM-135M, which saw far more text than this run did, and V3 is a step toward that, not an arrival. The weakest spots we measured are multi-turn conversations and invented facts, so that is where the next data goes.