Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.
Official3
ai.google.devofficialauthorityabout the gemma 2 family2,266 words
Google released Gemma 2, a family of lightweight, open-weight text-to-text decoder-only models available in English. The 27B variant was trained with 13 trillion tokens using TPUv5p hardware and JAX and ML Pathways software. These models are designed for tasks like question answering and reasoning while remaining small enough to run on laptops or personal cloud infrastructure.written, reviewed, cited
- The 2B model variant was trained using 2 trillion tokens from diverse web documents, code, and mathematics sources.
- Automated techniques filter out personal information and sensitive data to ensure model safety and reliability.
- Gemma 2 IT 27B achieved a 97.22 score on the Winobias 2_2 benchmark for representational harms evaluation.
developers.googleblog.com2024-08-22officialauthorityabout the gemma 2 familyHN 14 pts · 0 comments891 words
Google released Gemma 2, an open model suite available in 2B, 9B, and 27B parameter sizes that introduces architectural innovations like Alternating Local and Global Attention and Logit Soft-Capping. The 27B variant features 46 layers and a 256000 vocabulary, utilizing Grouped-Query Attention to improve efficiency alongside RMSNorm for Pre and Post-Normalization for training stability. The 2B model demonstrates edge-device capability by outperforming all GPT-3.5 models on the Chatbot Arena.written, reviewed, cited
- The 2B model uses 4 heads for k_proj and v_proj, 8 for q_proj and o_proj with a head size of 256.
- Gemma 2 2B outperforms all GPT-3.5 models on the Chatbot Arena at a size runnable on edge devices.
- Replacing MHA with GQA results in comparable performance while offering parameter efficiency and faster inference times.
mediapipe-studio.webapps.google.com2024-03-07officialabout the gemma 2 familyHN 30 pts · 15 comments23 words
Analysis5
seqpu.com2026-04-15HN front pageabout the gemma 2 familyHN 100 pts · 52 comments
storage.googleapis.com2024-06-27HN front pageabout the gemma 2 familyHN 328 pts · 172 comments9,878 words
Google DeepMind releases Gemma 2, a family of open language models ranging from 2 billion to 27 billion parameters. The 27B model trains on 13 trillion tokens of primarily-English data, while the 2B and 9B models use knowledge distillation to simulate training beyond 50× the compute-optimal quantity. All models inherit a vocabulary with 256k entries designed for multilingual use.written, reviewed, cited
- The 2B and 9B models are trained with knowledge distillation instead of next token prediction.
- Gemma 2 models use interleaved local sliding window and global attention layers in every other layer.
- The 27B model requires 6144 TPUv5p chips for training with 768-way data replication and 8-way model sharding.
arxiv.orgauthorityabout the gemma 2 family10,477 words
Gemma 2 releases open models ranging from 2 billion to 27 billion parameters that apply interleaving local-global attentions and Grouped-Query Attention. The 2B and 9B variants utilize knowledge distillation, training on 2 trillion and 8 trillion tokens respectively to simulate 50x the compute-optimal quantity. The 27B model trains on 13 trillion tokens of primarily-English data across 6144 TPUv5p chips.written, reviewed, cited
- The 2B and 9B models are trained with knowledge distillation instead of next token prediction.
- The 27B model trains on 6144 TPUv5p chips with 768-way data replication and 8-way model sharding.
arxiv.orgauthorityabout the gemma 2 family10,479 words
Gemma 2 releases open models ranging from 2B to 27B parameters with competitive performance relative to larger counterparts. The 27B model trains on 13 trillion tokens while the 2B and 9B variants utilize knowledge distillation to simulate training beyond 50x the compute-optimal quantity. Architectural updates include interleaved local-global attention with an 8192 token context length and group-query attention mechanisms. Pre-training emissions are estimated at 1247.61 tCO2eq across the model family.written, reviewed, cited
- The 2B and 9B models are trained with knowledge distillation instead of next token prediction.
- Gemma 2 models use a sliding window size of 4096 tokens for local attention layers.
- The 27B model trains on 6144 TPUv5p chips with 768-way data replication and 8-way model sharding.
github.comauthorityabout the gemma 2 family2,358 words
Community2
news.ycombinator.com2024-07-01about the gemma 2 familyHN 33 pts · 6 comments301 words
RAGAgent is a new FOSS all-local AI project where chat history is vector searched for context assembly and Fabric is used for system and user messages. Unlike OpenWebUI, it is a real agent rather than just a frontend for default REPLs. The Gemma 2 27B model was broken, but a fix is already committed and underway to the pip repo.written, reviewed, cited
news.ycombinator.com2024-07-31HN 16 pts · 11 comments427 words
