Models › Llama-3.1 family

meta-llama Llama 3.1 Community License out 2 years ago

70.6B parameters, text in, text out. 1 provider serves it.

Verified 2026-10-25 Rank #369 in LLMs · score 80.6 method v1 Sources 3 Downloads 30d 40.1k · #398 of 619 tracked

Can I run it?42GB at INT4Estimatefits a 64 GB Mac (unified memory)methodCommunity: Llama 3.1 70B — specs, VRAM, benchmarks (70B) | BestLLMfor →
Can I use it?Llama 3.1 Community License'Built with Llama' attribution; >700M MAU clauselicence file2026-10-24
Do people use it?40.1kdownloads, 30 days#398 of 619 we track · below the median+0 likes, downloads −7% this week444 likesHugging Face2026-10-24

Worth reading

Picked by rule from 42 tracked pieces: the lab's own post, the most shared, a field report, the latest.

Most discussed · 395 HN pts

Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPU

ntransformer is a C++/CUDA inference engine that runs Llama 70B on a single RTX 3090 using a 3-tier adaptive caching system with VRAM-resident layers, pinned RAM, and NVMe/mmap fallback.

  • Disabling IOMMU removes hardware DMA isolation between all PCIe devices, so do not run on multi-tenant or server systems.
  • The NVMe direct backend writes the GGUF model file to raw NVMe blocks via dd before inference begins.
github.com 2026-02-21 395 HN pts · 101 comments 1.6k words
Field report

g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

g1 is an experimental prototype using prompting strategies to improve Llama-3.1-70b reasoning capabilities through o1-like chains.

  • The system requires the LLM to respond in JSON format with 'title', 'content', and 'next_action' keys.
  • The prompt explicitly instructs the model to use at least 3 methods to derive the answer.
github.com 2024-09-15 334 HN pts · 148 comments 1.1k words

25 best of 42 tracked

The 3 picks above, plus these 22. Top is the radar's score: discussion, source authority and the lab's own posts, fading with age.

  1. ArticleLlama 3.1llama.meta.com · 2024-07-23 · 437 HN pts · official
  2. ArticleLlama 3.1 405B now runs at 969 tokens/s on Cerebras Inferencecerebras.ai · 2024-11-19
  3. CodeLlama 3.1 Omni Modelgithub.com · 2024-09-18
  4. CodeShow HN: Tune LLaMa3.1 on Google Cloud TPUsgithub.com · 2024-09-11 · 189 HN pts
  5. CodeLlama 3.1 in Cgithub.com · 2024-07-24 · 212 HN pts
  6. CodeLongwriter – Increase llama3.1 output to 10k wordsgithub.com · 2024-10-07 · 154 HN pts
  7. ArticleLlama 3.1: Our most capable models to dateai.meta.com · 2024-07-23 · 27 HN pts · official
  8. ArticleCerebras Inference now 3x faster: Llama3.1-70B breaks 2,100 tokens/scerebras.ai · 2024-10-24 · 147 HN pts
  9. ArticleHermes 3: The First Fine-Tuned Llama 3.1 405B Modellambdalabs.com · 2024-08-15
  10. DiscussionCerebras launches inference for Llama 3.1; benchmarked at 1846 tokens/s on 8Btwitter.com · 2024-08-27 · 95 HN pts
  11. ArticleLlama 3.1 Instruct 70B Intelligence, Performance & Price ...artificialanalysis.ai · undated
  12. ArticleWhy Llama 3.1 is Importanttimkellogg.me · 2024-07-23
  13. ArticleNvidia presents Llama-Mesh: Generating 3D Mesh with Llama 3.1 8Bresearch.nvidia.com · 2024-11-17
  14. ArticleMeta Llama 3.1 405Bopenrouter.ai · 2024-07-23
  15. ArticleCoding with Llama 3.1, New DeepSeek Coder and Mistral Largeaider.chat · 2024-07-25
  16. ArticleLlama-3.1 supports tool calls via promptingbraintrust.dev · 2024-07-26
  17. ArticleMeta releases an open-weights GPT-4-level AI model, Llama 3.1 405Barstechnica.com · 2024-07-23
  18. ArticleShow HN: Run Llama 3.1 8B in the browserapp.wiz.chat · 2024-07-28
  19. VideoRunning the latest Llama 3.1 8B on Raspberry Pi [video]youtube.com · 2024-07-30
  20. ArticleShow HN: Finetune Llama-3.1 2x faster in a Colabcolab.research.google.com · 2024-07-23
  21. ArticleBenchmarking Llama 3.1 405B on 8x AMD MI300X GPUsdstack.ai · 2024-10-09
  22. DiscussionReflection-Llama-3.1-70B is Llama-3 with LoRAold.reddit.com · 2024-09-08

Where to run it

Live hosted endpoints, cheapest first, USD per million tokens. Ties break on output price, then provider name.

#Cheapest live rowIn /1MOut /1MContextQuantSeen
1 Featherless — — — — Oct 25

All 1 provider →

Where it's used

Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.

Served by 1 API provider; heavily run locally (82 quantized builds); a big base for builders (90 fine-tunes, adapters and merges).

  • As a service1Featherless
  • On own hardware82quantized builds
  • Built on9067 fine-tunes · 8 adapters · 15 merges
  • In apps100+Hugging Face Spaces

Top Spaces running it: pliny-the-prompter/obliteratus ♥ 621 · OpenEvals/evaluation-guidebook ♥ 367 · HuggingFaceFW/finephrase ♥ 285 · lerobot/robot-folding ♥ 158

Will it fit?

Weights only, an estimate: leave room for the KV cache. Method

Fits a 64 GB Mac (unified memory) at INT4

About 42 GB at INT4, 85 GB at FP8, 169 GB at BF16: weights only, so leave room for the KV cache.

16 GB card64 GB Mac2×80 GB8×80 GBINT4: about 42 GB of weights (estimate; KV cache and activations extra)INT442 GBFP8: about 85 GB of weights (estimate; KV cache and activations extra)FP885 GBBF16: about 169 GB of weights (estimate; KV cache and activations extra)BF16169 GB
16 GB card64 GB Mac2×80 GB8×80 GBINT4: about 42 GB of weights (estimate; KV cache and activations extra)INT442 GBFP8: about 85 GB of weights (estimate; KV cache and activations extra)FP885 GBBF16: about 169 GB of weights (estimate; KV cache and activations extra)BF16169 GB

Estimate params × bytes per param × 1.2 · how we estimate

The story so far

Weights, every dated article, and repo commits on one rail.

10 dated articles and 0 repo commits so far

Teal is the weights (2024-07-14), amber triangles are 10 dated articles (each links to it).

What people built on it

172 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.

172 derivatives on the Hub, led by 82 quantized

From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.

82 quantized (Hugging Face model tree)67 fine-tunes (Hugging Face model tree)15 merges (Hugging Face model tree)8 adapters (Hugging Face model tree)82 quantized67 fine-tunes15 merges8 adapters
82 quantized (Hugging Face model tree)67 fine-tunes (Hugging Face model tree)15 merges (Hugging Face model tree)8 adapters (Hugging Face model tree)82 quantized67 fine-tunes15 merges8 adapters

All variants →

Reference

Everything else, folded: the full tables, every source.

Facts

17 fields

Listed in LLMs

Facts

Pipeline tagtext-generation
Librarytransformers
ArchitecturesLlamaForCausalLM
Model typellama
Params by dtypeBF16: 70,553,706,496
Size bandXL
MoEno
Gatedmanual
PapersarXiv:2204.05149
LicenseLlama 3.1 Community License (llama3.1)
Hub snapshot2026-10-25

Servability

vLLM · SGLang

Servability

vLLM: supported · LlamaForCausalLM · 7050ef881ff0835f6890a47db1859a24c65481d6SGLang: supported · LlamaForCausalLM · 773cb938fe9484a8f7e7e46fb1b1fe96412c59ed

vLLM registry (7050ef881ff0835f6890a47db1859a24c65481d6) · SGLang registry (773cb938fe9484a8f7e7e46fb1b1fe96412c59ed) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.

Hardware fit

3 estimates

Hardware fit ESTIMATE

Weights-only memory for Llama-3.1-70B: params × bytes/param × 1.2 overhead. KV cache and activations are extra. How this is computed.

BF16 (2 bytes/param)169 GB · Multi-GPU node (<640 GB)
FP8 (1 byte/param)85 GB · Multi-GPU node (<640 GB)
INT4 (0.5 bytes/param)42 GB · Single high-end GPU (<80 GB)

Weights only (params × bytes/param × 1.2 overhead); KV cache and activations extra.

All variants

40 variants

Lineage

Original model Family: meta-llama/Llama-3.1 172 derivatives on the Hub

quantized: 82finetune: 67merge: 15adapter: 8

Top variants (40 shown, ranked by downloads)

Relation
Quant
Size
40 rows
Variant Relation Quant Params Band License Author Likes Downloads
casperhansen/llama-3.3-70b-instruct-awq quantized awq 70.6B XL llama3.3 casperhansen 53 5.3M
Infermatic/Llama-3.3-70B-Instruct-FP8-Dynamic quantized -FP8 70.6B XL llama3.3 Infermatic 2 302.6k
latam-gpt/Llama-3.1-70B-LatamGPT-SFT-1.0 finetune — 70.6B XL llama3.1 latam-gpt 56 247.4k
NousResearch/Hermes-3-Llama-3.1-70B-FP8 finetune -FP8 70.6B XL llama3 NousResearch 28 239.4k
unsloth/Meta-Llama-3.1-70B-bnb-4bit quantized -bnb 72.7B XL llama3.1 unsloth 32 234.5k
NousResearch/Hermes-3-Llama-3.1-70B-GGUF quantized GGUF — — llama3 NousResearch 54 97.2k
NousResearch/Hermes-4-70B-FP8 quantized -FP8 70.6B XL llama3 NousResearch 43 78.4k
NousResearch/Hermes-4-70B finetune — 70.6B XL llama3 NousResearch 224 60.8k
Mungert/Llama-3.3-70B-Instruct-GGUF quantized GGUF — — llama3.3 Mungert 8 58.9k
unsloth/Meta-Llama-3.1-70B quantized — 70.6B XL llama3.1 unsloth 2 56.6k
ThomasBaruzier/Llama-3.3-70B-Instruct-GGUF quantized GGUF — — llama3.3 ThomasBaruzier 3 34.3k
Mungert/Llama-3.1-70B-Instruct-GGUF quantized GGUF — — llama3.1 Mungert 4 28.6k
deepcogito/cogito-v1-preview-llama-70B finetune — 70.6B XL llama3.1 deepcogito 78 15.9k
mradermacher/Meta-Llama-3.1-70B-i1-GGUF quantized GGUF — — llama3.1 mradermacher 2 18.1k
allenai/Llama-3.1-Tulu-3-70B-SFT finetune — 70.6B XL llama3.1 allenai 7 15.4k
nvidia/OpenMath2-Llama3.1-70B finetune — 70.6B XL llama3.1 nvidia 22 13.7k
RedHatAI/Meta-Llama-3.1-70B-FP8 quantized -FP8 70.6B XL llama3.1 RedHatAI 3 14.3k
WhiteRabbitNeo/Llama-3.1-WhiteRabbitNeo-2-70B finetune — — — llama3.1 WhiteRabbitNeo 52 10.7k
EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1 merge — 70.6B XL other EVA-UNIT-01 21 11.7k
ionos/Llama-3.3-70B-Instruct-FP8 quantized -FP8 70.6B XL llama3.3 ionos 3 12.2k
deepcogito/cogito-v2-preview-llama-70B finetune — 70.6B XL llama3.1 deepcogito 24 11k
mradermacher/Meta-Llama-3.1-70B-GGUF quantized GGUF — — llama3.1 mradermacher 0 9.8k
unsloth/Llama-3.1-70B finetune — 70.6B XL llama3.1 unsloth 0 9.3k
unsloth/Hermes-3-Llama-3.1-70B-bnb-4bit quantized -bnb 72.7B XL llama3.1 unsloth 1 8.2k
tjake/Llama-3.1-70B-Instruct-Jlama-Q4 finetune — 72.7B XL llama3.1 tjake 0 8k
roleplaiapp/Llama-3.3-70B-Instruct-Q4_K_M-GGUF quantized GGUF — — — roleplaiapp 2 7.9k
OpenPipe/Llama-3.1-70B-Instruct finetune — 70.6B XL llama3.1 OpenPipe 0 7.8k
Mungert/cogito-v1-preview-llama-8B-GGUF quantized GGUF — — llama3.1 Mungert 1 7k
sophosympatheia/Nova-Tempus-70B-v0.1 merge — 70.6B XL llama3.3 sophosympatheia 19 6k
Snowflake/Llama-3.1-Arctic-ExCoT-70B finetune — 70.6B XL cc-by-nc-4.0 Snowflake 10 4.9k
hugging-quants/Meta-Llama-3.1-70B-BNB-NF4-BF16 quantized -BNB 21.4B L llama3.1 hugging-quants 2 4.5k
kgrabko/JiRackTernary_70b quantized — — — other kgrabko 6 4.3k
roleplaiapp/Llama-3.3-70B-Instruct-Q4_0-GGUF quantized Q4_0 — — — roleplaiapp 2 4.4k
HPAI-BSC/Llama3.1-Aloe-Beta-70B finetune — 70.6B XL cc-by-nc-4.0 HPAI-BSC 9 2.6k
professorf/Llama-3.3-70B-Instruct-gguf quantized gguf — — llama3.3 professorf 1 2.8k
osllmai-community/Llama-3.3-70B-Instruct finetune — 70.6B XL llama3.3 osllmai-community 2 2.7k
pentagoniac/llamarine finetune — 70.6B XL llama2 pentagoniac 3 2.1k
ISTA-DASLab/Meta-Llama-3.1-70B-AQLM-PV-2Bit-1x16 quantized — 11.0B L — ISTA-DASLab 18 1.1k
tartuNLP/Llama-3.1-EstLLM-70B-0826 finetune — 70.6B XL llama3.1 tartuNLP 0 2k
allenai/llama-3.1-tulu-2-70b finetune — 70.6B XL apache-2.0 allenai 0 2k

All providers

1 provider · 1 endpoint

Where to run 1 providers · 1 endpoints

Hosted endpoints observed for Llama-3.1-70B, sorted by input price (USD per million tokens). Each row names its observer and date; prices change daily, so check the source before quoting.

ProviderModel refContextQuant$/M in$/M outtok/sTTFTToolsStatusObserved
Featherless
hf
meta-llama/Llama-3.1-70B — — — — — — — live 2026-10-25

Articles and coverage

42 pieces

Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.

Official2

llama.meta.com2024-07-23officialHN front pageabout the Llama 3.1 familyHN 437 pts · 269 comments42 words
ai.meta.com2024-07-23officialauthorityabout the Llama 3.1 familyHN 27 pts · 3 comments2,136 words

Analysis7

github.com2026-02-21authorityHN front pageHN 395 pts · 101 comments1,595 words
ntransformer is a C++/CUDA inference engine that runs Llama 70B on a single RTX 3090 using a 3-tier adaptive caching system with VRAM-resident layers, pinned RAM, and NVMe/mmap fallback. The engine achieves an 83x speedup over mmap baseline for 70B on consumer hardware and requires zero external dependencies beyond the CUDA Toolkit. It supports GGUF quantization formats including Q4_K_M and Q6_K under a BSD-2-Clause license.written, reviewed, cited
  • Disabling IOMMU removes hardware DMA isolation between all PCIe devices, so do not run on multi-tenant or server systems.
  • The NVMe direct backend writes the GGUF model file to raw NVMe blocks via dd before inference begins.
  • Layer skip calibration uses cosine similarity to eliminate 20 out of 80 layers per token with minimal quality loss.
github.com2024-09-15authorityHN front pageHN 334 pts · 148 comments1,148 words
g1 is an experimental prototype using prompting strategies to improve Llama-3.1-70b reasoning capabilities through o1-like chains. Without prompting, Llama-3.1-70b had 0% accuracy on the Strawberry problem, but g1 achieves ~70% accuracy on this task. Initial testing shows the system accurately solves simple logic problems 60-80% of the time that usually stump LLMs.written, reviewed, cited
  • The system requires the LLM to respond in JSON format with 'title', 'content', and 'next_action' keys.
  • The prompt explicitly instructs the model to use at least 3 methods to derive the answer.
understandingai.org2025-06-15HN front pageabout the Llama 3.1 familyHN 191 pts · 305 comments
github.com2024-09-11authorityHN front pageabout the Llama 3.1 familyHN 189 pts · 52 comments869 words
github.com2024-07-24authorityHN front pageabout the Llama 3.1 familyHN 212 pts · 36 comments5,858 words
github.com2024-10-07authorityHN front pageabout the Llama 3.1 familyHN 154 pts · 29 comments6,696 words
cerebras.ai2024-10-24HN front pageHN 147 pts · 84 comments

Community1

twitter.com2024-08-27about the Llama 3.1 familyHN 95 pts · 42 comments
All coverage tracked for this model (42) — append-only, nothing is ever removed
TitleSourcePublishedFirst seenSignals
Llama 3.1 Instruct 70B Intelligence, Performance & Price ...artificialanalysis.ai—2026-10-07exa
Evaluation: Llama 3.1 70B vs. Comparable Closed-Source ...vellum.ai—2026-10-07exa
LLM Deep-dive: Llama 3.1predli.com—2026-10-07exa
Meta Llama 3.1 (70B)docs.oracle.com—2026-10-07exa
Llama 3.1 70B — specs, VRAM, benchmarks (70B) | BestLLMforbestllmfor.com—2026-10-07exa
Deep Dive: How Llama 3.1 70B's Transformer Architecture Handles 128K Context Windows — johal.injohal.in2026-04-292026-10-07exa
Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPUgithub.com2026-02-212026-10-07hn
Meta's Llama 3.1 can recall 42 percent of the first Harry Potter bookunderstandingai.org2025-06-152026-10-07hn
Nvidia's new Llama-3.1 Nemotron Ultra outperforms DeepSeek R1 at half the sizeventurebeat.com2025-04-112026-10-07hn
Llama 3.1 405B now runs at 969 tokens/s on Cerebras Inferencecerebras.ai2024-11-192026-10-07hn
Nvidia presents Llama-Mesh: Generating 3D Mesh with Llama 3.1 8Bresearch.nvidia.com2024-11-172026-10-07hn
Cerebras runs Llama 3.1 70B at 2,100 tokens per second, live demo availabletwitter.com2024-10-242026-10-07hn
Cerebras Inference now 3x faster: Llama3.1-70B breaks 2,100 tokens/scerebras.ai2024-10-242026-10-07hn
Explore Llama 3.1 Paper: An In-Depth Manual - Novitablogs.novita.ai2024-10-182026-10-07exa
Nvidia Llama-3.1-Nemotron-70B-Instructbuild.nvidia.com2024-10-162026-10-07hn
Benchmarking Llama 3.1 405B on 8x AMD MI300X GPUsdstack.ai2024-10-092026-10-07hn
Longwriter – Increase llama3.1 output to 10k wordsgithub.com2024-10-072026-10-07hn
Insights from 84,000 comments on "Ask HN: Who Is Hiring" using Llama3.1-70B-FP16exxa.notion.site2024-09-232026-10-07hn
Llama 3.1 Omni Modelgithub.com2024-09-182026-10-07hn
g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chainsgithub.com2024-09-152026-10-07hn
Show HN: Tune LLaMa3.1 on Google Cloud TPUsgithub.com2024-09-112026-10-07hn
Reflection Llama-3.1 70B: Testing and Summary of What We Knowdatacamp.com2024-09-092026-10-07hn
Reflection Llama 3.1 70Bartificialanalysis.ai2024-09-092026-10-07hn
We've now partially replicated Reflection Llama 3.1 70B's eval claimstwitter.com2024-09-082026-10-07hn
Reflection-Llama-3.1-70B is Llama-3 with LoRAold.reddit.com2024-09-082026-10-07hn
Show HN: I built a website where you can easily fine-tune Llama 3.1 modelstunellama.com2024-09-042026-10-07hn
Cerebras launches inference for Llama 3.1; benchmarked at 1846 tokens/s on 8Btwitter.com2024-08-272026-10-07hn
cerebras: 450 tokens/sec llama 3.1 70Btheregister.com2024-08-272026-10-07hn
Hermes 3: The First Fine-Tuned Llama 3.1 405B Modellambdalabs.com2024-08-152026-10-07hn
Running the latest Llama 3.1 8B on Raspberry Pi [video]youtube.com2024-07-302026-10-07hn
Show HN: Run Llama 3.1 8B in the browserapp.wiz.chat2024-07-282026-10-07hn
Llama-3.1 supports tool calls via promptingbraintrust.dev2024-07-262026-10-07hn
Coding with Llama 3.1, New DeepSeek Coder and Mistral Largeaider.chat2024-07-252026-10-07hn
Llama 3.1 in Cgithub.com2024-07-242026-10-07hn
Why Llama 3.1 is Importanttimkellogg.me2024-07-232026-10-07hn
Show HN: Finetune Llama-3.1 2x faster in a Colabcolab.research.google.com2024-07-232026-10-07hn
Meta releases an open-weights GPT-4-level AI model, Llama 3.1 405Barstechnica.com2024-07-232026-10-07hn
Meta Llama 3.1 405Bopenrouter.ai2024-07-232026-10-07hn
Chat with Meta Llama 3.1 405Bllama3.replicate.dev2024-07-232026-10-07hn
Llama 3.1: Our most capable models to dateai.meta.com official2024-07-232026-10-07hn, exa
Llama 3.1llama.meta.com official2024-07-232026-10-07hn
Llama 3.1 70B Review — Benchmarks, Pricing & AI Panel Verdict | TopReviewed.aitopreviewed.ai2024-07-222026-10-07exa

Papers

1 paper

Papers 1

arXiv links found in the model card and its tags. "Peer-reviewed" is shown only with evidence: a published venue in OpenAlex (measured) or the authors' own journal reference on arXiv (reported).

Similar models

3 models

Similar models

Sources

3 sources

Sources

Hub snapshot: 2026-10-25 · HF API: 2026-10-24 21:37 UTC · Providers: 2026-10-24 21:44 UTC · Articles: 2026-10-24 21:46 UTC · Summary: 2026-10-07 15:05 UTC · Last verified 2026-10-25

Follow Llama-3.1-70B

Watch this model

Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.