qwenlm.github.io2025-01-26officialauthorityHN front pageHN 307 pts · 108 comments1,608 words
QwenLM releases open-source Qwen2.5-1M models supporting up to one million token contexts. The 7B and 14B checkpoints significantly outperform their 128K counterparts in long-context tasks while maintaining short-text capabilities. A new vLLM-based inference framework accelerates processing by 3x to 7x faster using Dual Chunk Attention. Deploying the 7B model requires at least 120GB VRAM, while the 14B model needs at least 320GB VRAM.written, reviewed, cited
- Training begins with a pre-trained checkpoint that originally had a 4K token context length before expansion.
- The 7B model exhibits minor errors when retrieving information from documents containing up to 1M tokens.
- Processing 1M-token sequences on the 7B model consumes 71GB of VRAM for MLP layer activations.
qwenlm.github.io2024-09-18officialauthorityHN front pageabout the Qwen2.5 familyHN 168 pts · 38 comments1,744 words
Qwen2.5 introduces dense decoder-only models pretrained on up to 18 trillion tokens, available in sizes from 0.5B to 72B parameters. Most variants are licensed under Apache 2.0, excluding the 3B and 72B models, and support a 128K context window with 8K generation limits. Specialized versions for coding and mathematics offer enhanced performance on benchmarks like HumanEval and MATH.written, reviewed, cited
- Qwen2.5-Coder-7B-Instruct outperforms many larger language models across a range of programming languages and tasks.
- Qwen2.5-Math-72B-Instruct general performance surpasses both Qwen2-Math-72B-Instruct and GPT4-o.