Models › Llama-3.1 family

Llama-3.1-70B-Instruct

meta-llama Llama 3.1 Community License out 2 years ago

70.6B parameters, text in, text out. 1 provider serves it.

Verified 2026-10-25 Rank #124 in LLMs · score 95.1 method v1 Sources 3 Downloads 30d 89.5k · #325 of 619 tracked

Can I run it?42GB at INT4Estimatefits a 64 GB Mac (unified memory)method
Can I use it?Llama 3.1 Community License'Built with Llama' attribution; >700M MAU clauselicence file2026-10-24
Do people use it?89.5kdownloads, 30 days#325 of 619 we track · below the median+3 likes, downloads +8% this week1,012 likesHugging Face2026-10-24

Worth reading

Picked by rule from 35 tracked pieces: the lab's own post, the most shared, a field report, the latest.

Most discussed · 395 HN pts

Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPU

ntransformer is a C++/CUDA inference engine that runs Llama 70B on a single RTX 3090 using a 3-tier adaptive caching system with VRAM-resident layers, pinned RAM, and NVMe/mmap fallback.

  • Disabling IOMMU removes hardware DMA isolation between all PCIe devices, so do not run on multi-tenant or server systems.
  • The NVMe direct backend writes the GGUF model file to raw NVMe blocks via dd before inference begins.
github.com 2026-02-21 395 HN pts · 101 comments 1.6k words
Field report

g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

g1 is an experimental prototype using prompting strategies to improve Llama-3.1-70b reasoning capabilities through o1-like chains.

  • The system requires the LLM to respond in JSON format with 'title', 'content', and 'next_action' keys.
  • The prompt explicitly instructs the model to use at least 3 methods to derive the answer.
github.com 2024-09-15 334 HN pts · 148 comments 1.1k words

25 best of 35 tracked

The 3 picks above, plus these 22. Top is the radar's score: discussion, source authority and the lab's own posts, fading with age.

  1. ArticleLlama 3.1llama.meta.com · 2024-07-23 · 437 HN pts · official
  2. ArticleIntroducing Llama 3.1: Our most capable models to dateai.meta.com · undated · official
  3. ArticleLlama 3.1 405B now runs at 969 tokens/s on Cerebras Inferencecerebras.ai · 2024-11-19
  4. CodeLlama 3.1 Omni Modelgithub.com · 2024-09-18
  5. CodeShow HN: Tune LLaMa3.1 on Google Cloud TPUsgithub.com · 2024-09-11 · 189 HN pts
  6. CodeLlama 3.1 in Cgithub.com · 2024-07-24 · 212 HN pts
  7. CodeLongwriter – Increase llama3.1 output to 10k wordsgithub.com · 2024-10-07 · 154 HN pts
  8. ArticleHermes 3: The First Fine-Tuned Llama 3.1 405B Modellambdalabs.com · 2024-08-15
  9. ArticleLlama 3.1: Our most capable models to dateai.meta.com · 2024-07-23 · 27 HN pts · official
  10. DiscussionCerebras launches inference for Llama 3.1; benchmarked at 1846 tokens/s on 8Btwitter.com · 2024-08-27 · 95 HN pts
  11. ArticleLlama 3.1 Instruct 70B Intelligence, Performance & Price ...artificialanalysis.ai · undated
  12. ArticleWhy Llama 3.1 is Importanttimkellogg.me · 2024-07-23
  13. ArticleNvidia presents Llama-Mesh: Generating 3D Mesh with Llama 3.1 8Bresearch.nvidia.com · 2024-11-17
  14. ArticleMeta Llama 3.1 405Bopenrouter.ai · 2024-07-23
  15. ArticleCoding with Llama 3.1, New DeepSeek Coder and Mistral Largeaider.chat · 2024-07-25
  16. ArticleLlama-3.1 supports tool calls via promptingbraintrust.dev · 2024-07-26
  17. ArticleMeta releases an open-weights GPT-4-level AI model, Llama 3.1 405Barstechnica.com · 2024-07-23
  18. ArticleShow HN: Run Llama 3.1 8B in the browserapp.wiz.chat · 2024-07-28
  19. VideoRunning the latest Llama 3.1 8B on Raspberry Pi [video]youtube.com · 2024-07-30
  20. ArticleShow HN: Finetune Llama-3.1 2x faster in a Colabcolab.research.google.com · 2024-07-23
  21. ArticleBenchmarking Llama 3.1 405B on 8x AMD MI300X GPUsdstack.ai · 2024-10-09
  22. DiscussionReflection-Llama-3.1-70B is Llama-3 with LoRAold.reddit.com · 2024-09-08

Where to run it

Live hosted endpoints, cheapest first, USD per million tokens. Ties break on output price, then provider name.

#Cheapest live rowIn /1MOut /1MContextQuantSeen
1 Featherless — — — — Oct 25

All 1 provider →

Where it's used

Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.

Served by 1 API provider; heavily run locally (127 quantized builds); a big base for builders (236 fine-tunes, adapters and merges).

  • As a service1Featherless
  • On own hardware127quantized builds
  • Built on236109 fine-tunes · 105 adapters · 22 merges
  • In apps100+Hugging Face Spaces

Top Spaces running it: KingNish/OpenGPT-4o ♥ 4,302 · gokaygokay/FLUX-Prompt-Generator ♥ 1,400 · pliny-the-prompter/obliteratus ♥ 621 · allenai/reward-bench ♥ 438

Will it fit?

Weights only, an estimate: leave room for the KV cache. Method

Fits a 64 GB Mac (unified memory) at INT4

About 42 GB at INT4, 85 GB at FP8, 169 GB at BF16: weights only, so leave room for the KV cache.

16 GB card64 GB Mac2×80 GB8×80 GBINT4: about 42 GB of weights (estimate; KV cache and activations extra)INT442 GBFP8: about 85 GB of weights (estimate; KV cache and activations extra)FP885 GBBF16: about 169 GB of weights (estimate; KV cache and activations extra)BF16169 GB
16 GB card64 GB Mac2×80 GB8×80 GBINT4: about 42 GB of weights (estimate; KV cache and activations extra)INT442 GBFP8: about 85 GB of weights (estimate; KV cache and activations extra)FP885 GBBF16: about 169 GB of weights (estimate; KV cache and activations extra)BF16169 GB

Estimate params × bytes per param × 1.2 · how we estimate

The story so far

Weights, every dated article, and repo commits on one rail.

9 dated articles and 0 repo commits so far

Teal is the weights (2024-07-16), amber triangles are 9 dated articles (each links to it).

What people built on it

363 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.

363 derivatives on the Hub, led by 127 quantized

From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.

127 quantized (Hugging Face model tree)109 fine-tunes (Hugging Face model tree)22 merges (Hugging Face model tree)105 adapters (Hugging Face model tree)127 quantized109 fine-tunes22 merges105 adapters
127 quantized (Hugging Face model tree)109 fine-tunes (Hugging Face model tree)22 merges (Hugging Face model tree)105 adapters (Hugging Face model tree)127 quantized109 fine-tunes22 merges105 adapters

All variants →

Reference

Everything else, folded: the full tables, every source.

Use-case evidence

1 use case

What it is good at 1 use case

Labels are evidence, not verdicts: measured a third-party leaderboard row for this model; author-reported a number the model author published in the card or eval files; structural a deterministic signal from the name, config or chat template, or a decision-model estimate. Each chip links to its source. All use cases.

Facts

17 fields

Listed in LLMs

Facts

Pipeline tagtext-generation
Librarytransformers
ArchitecturesLlamaForCausalLM
Model typellama
Params by dtypeBF16: 70,553,706,496
Size bandXL
MoEno
Gatedmanual
PapersarXiv:2204.05149
LicenseLlama 3.1 Community License (llama3.1)
Hub snapshot2026-10-25

Servability

vLLM · SGLang

Servability

vLLM: supported · LlamaForCausalLM · 7050ef881ff0835f6890a47db1859a24c65481d6SGLang: supported · LlamaForCausalLM · 773cb938fe9484a8f7e7e46fb1b1fe96412c59ed

vLLM registry (7050ef881ff0835f6890a47db1859a24c65481d6) · SGLang registry (773cb938fe9484a8f7e7e46fb1b1fe96412c59ed) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.

Hardware fit

3 estimates

Hardware fit ESTIMATE

Weights-only memory for Llama-3.1-70B-Instruct: params × bytes/param × 1.2 overhead. KV cache and activations are extra. How this is computed.

BF16 (2 bytes/param)169 GB · Multi-GPU node (<640 GB)
FP8 (1 byte/param)85 GB · Multi-GPU node (<640 GB)
INT4 (0.5 bytes/param)42 GB · Single high-end GPU (<80 GB)

Weights only (params × bytes/param × 1.2 overhead); KV cache and activations extra.

All variants

40 variants

Lineage

Fine-tuned from meta-llama/Llama-3.1-70B (same organization) Family: meta-llama/Llama-3.1 363 derivatives on the Hub

Base model: meta-llama/Llama-3.1-70B

quantized: 127finetune: 109adapter: 105merge: 22

Top variants (40 shown, ranked by downloads)

Relation
Quant
Size
40 rows
Variant Relation Quant Params Band License Author Likes Downloads
MaziyarPanahi/Meta-Llama-3.1-70B-Instruct-GGUF quantized GGUF — — — MaziyarPanahi 41 12.1M
RedHatAI/Meta-Llama-3.1-70B-Instruct-FP8 quantized -FP8 70.6B XL llama3.1 RedHatAI 52 2M
RedHatAI/Meta-Llama-3.1-70B-Instruct-quantized.w4a16 quantized -quant 70.6B XL llama3.1 RedHatAI 33 1.9M
mlx-community/Meta-Llama-3.1-70B-Instruct-bf16-CORRECTED finetune -bf16 70.6B XL llama3.1 mlx-community 0 550k
bartowski/Meta-Llama-3.1-70B-Instruct-GGUF quantized GGUF — — llama3.1 bartowski 77 538.7k
reissbaker/llama-3.1-70b-abliterated-lora adapter — — — — reissbaker 4 339.2k
nvidia/Llama-3.1-70B-Instruct-FP8 finetune -FP8 70.6B XL llama3.1 nvidia 21 263k
unsloth/Meta-Llama-3.1-70B-Instruct-bnb-4bit quantized -bnb 70.6B XL llama3.1 unsloth 32 261.3k
unsloth/Meta-Llama-3.1-70B-Instruct quantized — 70.6B XL llama3.1 unsloth 4 243.6k
RedHatAI/Meta-Llama-3.1-70B-Instruct-quantized.w8a8 quantized -quant 70.6B XL llama3.1 RedHatAI 21 234.2k
amd/Llama-3.1-70B-Instruct-FP8-KV quantized -FP8 70.6B XL other amd 5 177.6k
mattshumer/Reflection-Llama-3.1-70B finetune — 70.6B XL llama3.1 mattshumer 1.7k 55.3k
lmstudio-community/Meta-Llama-3.1-70B-Instruct-GGUF quantized GGUF — — llama3.1 lmstudio-community 39 132.2k
RedHatAI/Meta-Llama-3.1-70B-Instruct-FP8-dynamic quantized -FP8 70.6B XL llama3.1 RedHatAI 7 102.8k
nvidia/Llama-3.1-Nemotron-70B-Instruct finetune — — — llama3.1 nvidia 570 43.9k
nvidia/Llama-3.1-Nemotron-70B-Reward-HF finetune — 70.6B XL llama3.1 nvidia 93 59.8k
shenzhi-wang/Llama3.1-70B-Chinese-Chat quantized GGUF 70.6B XL llama3.1 shenzhi-wang 45 56k
legraphista/Meta-Llama-3.1-70B-Instruct-IMat-GGUF quantized GGUF — — llama3.1 legraphista 11 55.1k
mlabonne/Llama-3.1-70B-Instruct-lorablated finetune — 70.6B XL llama3.1 mlabonne 80 50.7k
infly/INF-ORM-Llama3.1-70B finetune — 69.6B XL — infly 27 43.7k
mlabonne/Llama-3.1-70B-Instruct-lorablated-GGUF quantized GGUF — — llama3.1 mlabonne 48 31.7k
mradermacher/Meta-Llama-3.1-70B-Instruct-GGUF quantized GGUF — — llama3.1 mradermacher 3 33.4k
mradermacher/Meta-Llama-3.1-70B-Instruct-i1-GGUF quantized GGUF — — llama3.1 mradermacher 2 27k
EPFLiGHT/Meditron3-70B finetune — 70.6B XL llama3.1 EPFLiGHT 25 25k
qwp4w3hyb/Meta-Llama-3.1-70B-Instruct-iMat-GGUF quantized GGUF — — llama3.1 qwp4w3hyb 8 24.2k
unsloth/Reflection-Llama-3.1-70B finetune — 70.6B XL llama3.1 unsloth 8 20.8k
MaziyarPanahi/calme-2.2-llama3.1-70b finetune — 70.6B XL — MaziyarPanahi 2 18.5k
second-state/Meta-Llama-3.1-70B-Instruct-GGUF quantized GGUF — — llama3.1 second-state 1 17.5k
RedHatAI/Meta-Llama-3.1-70B-Instruct-quantized.w8a16 quantized -quant 70.6B XL llama3.1 RedHatAI 5 15.1k
MaziyarPanahi/calme-2.3-llama3.1-70b finetune — 70.6B XL — MaziyarPanahi 3 13.4k
Skywork/Skywork-Critic-Llama-3.1-70B finetune — — — other Skywork 12 12.8k
HiTZ/Latxa-Llama-3.1-70B-Instruct finetune — 70.6B XL other HiTZ 7 12.3k
c4tdr0ut/grok-oss-Revenant-70B quantized GGUF 70.6B XL other c4tdr0ut 5 11.8k
backyardai/Meta-Llama-3.1-70B-Instruct-GGUF quantized GGUF — — llama3.1 backyardai 3 11.3k
ISTA-DASLab/Meta-Llama-3.1-70B-Instruct-AQLM-PV-2Bit-1x16 quantized — 11.0B L — ISTA-DASLab 45 8.1k
nvidia/Llama-3.1-Nemotron-70B-Reward finetune — — — llama3.1 nvidia 83 4.8k
RedHatAI/Llama-3.1-70B-Instruct-NVFP4 quantized NVFP4 40.6B XL llama3.1 RedHatAI 1 8.3k
SicariusSicariiStuff/Assistant_Pepe_70B finetune — 70.6B XL llama3.1 SicariusSicariiStuff 80 3.9k
gaianet/Meta-Llama-3.1-70B-Instruct-GGUF quantized GGUF — — llama3.1 gaianet 0 7.4k
reissbaker/r1-llama-70b-distill-lora adapter — — — apache-2.0 reissbaker 0 7.1k

All providers

1 provider · 1 endpoint

Where to run 1 providers · 1 endpoints

Hosted endpoints observed for Llama-3.1-70B-Instruct, sorted by input price (USD per million tokens). Each row names its observer and date; prices change daily, so check the source before quoting.

ProviderModel refContextQuant$/M in$/M outtok/sTTFTToolsStatusObserved
Featherless
hf
meta-llama/Llama-3.1-70B-Instruct — — — — — — — live 2026-10-25

Articles and coverage

35 pieces

Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.

Official3

llama.meta.com2024-07-23officialHN front pageabout the Llama 3.1 familyHN 437 pts · 269 comments42 words
ai.meta.comofficialauthorityabout the Llama 3.1 family454 words
Meta released Muse Wonder on OCT 12, 2026, as a frontier model leading on personal and professional agents. Muse Spark 1.3 delivers improved performance across agentic and coding tasks, while Muse Image is the most advanced image generation model yet. The company also announced a partnership with EssilorLuxottica to launch Meta Glasses on June 23, 2026.written, reviewed, cited
ai.meta.com2024-07-23officialauthorityabout the Llama 3.1 familyHN 27 pts · 3 comments2,136 words

Analysis6

github.com2026-02-21authorityHN front pageHN 395 pts · 101 comments1,595 words
ntransformer is a C++/CUDA inference engine that runs Llama 70B on a single RTX 3090 using a 3-tier adaptive caching system with VRAM-resident layers, pinned RAM, and NVMe/mmap fallback. The engine achieves an 83x speedup over mmap baseline for 70B on consumer hardware and requires zero external dependencies beyond the CUDA Toolkit. It supports GGUF quantization formats including Q4_K_M and Q6_K under a BSD-2-Clause license.written, reviewed, cited
  • Disabling IOMMU removes hardware DMA isolation between all PCIe devices, so do not run on multi-tenant or server systems.
  • The NVMe direct backend writes the GGUF model file to raw NVMe blocks via dd before inference begins.
  • Layer skip calibration uses cosine similarity to eliminate 20 out of 80 layers per token with minimal quality loss.
github.com2024-09-15authorityHN front pageHN 334 pts · 148 comments1,148 words
g1 is an experimental prototype using prompting strategies to improve Llama-3.1-70b reasoning capabilities through o1-like chains. Without prompting, Llama-3.1-70b had 0% accuracy on the Strawberry problem, but g1 achieves ~70% accuracy on this task. Initial testing shows the system accurately solves simple logic problems 60-80% of the time that usually stump LLMs.written, reviewed, cited
  • The system requires the LLM to respond in JSON format with 'title', 'content', and 'next_action' keys.
  • The prompt explicitly instructs the model to use at least 3 methods to derive the answer.
understandingai.org2025-06-15HN front pageabout the Llama 3.1 familyHN 191 pts · 305 comments
github.com2024-09-11authorityHN front pageabout the Llama 3.1 familyHN 189 pts · 52 comments869 words
github.com2024-07-24authorityHN front pageabout the Llama 3.1 familyHN 212 pts · 36 comments5,858 words
github.com2024-10-07authorityHN front pageabout the Llama 3.1 familyHN 154 pts · 29 comments6,696 words

Community1

twitter.com2024-08-27about the Llama 3.1 familyHN 95 pts · 42 comments
All coverage tracked for this model (35) — append-only, nothing is ever removed
TitleSourcePublishedFirst seenSignals
Introducing Llama 3.1: Our most capable models to dateai.meta.com official—2026-10-07exa
Llama 3.1 Instruct 70B Intelligence, Performance & Price ...artificialanalysis.ai—2026-10-07exa
LLM Deep-dive: Llama 3.1predli.com—2026-10-07exa
Llama 3.1 70B Instruct Benchmarks, Pricing & Context Windowllm-stats.com—2026-10-07exa
Llama 3.1 70B Instructcatalog.ngc.nvidia.com—2026-10-07exa
Evaluation: Llama 3.1 70B vs. Comparable Closed-Source ...vellum.ai—2026-10-07exa
Meta: Llama 3.1 70B Instruct - AI Model Details & Benchmarksbenchable.ai—2026-10-07exa
Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPUgithub.com2026-02-212026-10-07hn
Meta's Llama 3.1 can recall 42 percent of the first Harry Potter bookunderstandingai.org2025-06-152026-10-07hn
Nvidia's new Llama-3.1 Nemotron Ultra outperforms DeepSeek R1 at half the sizeventurebeat.com2025-04-112026-10-07hn
Llama 3.1 405B now runs at 969 tokens/s on Cerebras Inferencecerebras.ai2024-11-192026-10-07hn
Nvidia presents Llama-Mesh: Generating 3D Mesh with Llama 3.1 8Bresearch.nvidia.com2024-11-172026-10-07hn
Nvidia Llama-3.1-Nemotron-70B-Instructbuild.nvidia.com2024-10-162026-10-07hn
Benchmarking Llama 3.1 405B on 8x AMD MI300X GPUsdstack.ai2024-10-092026-10-07hn
Longwriter – Increase llama3.1 output to 10k wordsgithub.com2024-10-072026-10-07hn
Llama 3.1 Omni Modelgithub.com2024-09-182026-10-07hn
g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chainsgithub.com2024-09-152026-10-07hn
Show HN: Tune LLaMa3.1 on Google Cloud TPUsgithub.com2024-09-112026-10-07hn
Reflection-Llama-3.1-70B is Llama-3 with LoRAold.reddit.com2024-09-082026-10-07hn
Cerebras launches inference for Llama 3.1; benchmarked at 1846 tokens/s on 8Btwitter.com2024-08-272026-10-07hn
Hermes 3: The First Fine-Tuned Llama 3.1 405B Modellambdalabs.com2024-08-152026-10-07hn
Running the latest Llama 3.1 8B on Raspberry Pi [video]youtube.com2024-07-302026-10-07hn
Show HN: Run Llama 3.1 8B in the browserapp.wiz.chat2024-07-282026-10-07hn
Llama-3.1 supports tool calls via promptingbraintrust.dev2024-07-262026-10-07hn
Coding with Llama 3.1, New DeepSeek Coder and Mistral Largeaider.chat2024-07-252026-10-07hn
Llama 3.1 in Cgithub.com2024-07-242026-10-07hn
Why Llama 3.1 is Importanttimkellogg.me2024-07-232026-10-07hn
Show HN: Finetune Llama-3.1 2x faster in a Colabcolab.research.google.com2024-07-232026-10-07hn
Meta releases an open-weights GPT-4-level AI model, Llama 3.1 405Barstechnica.com2024-07-232026-10-07hn
Meta Llama 3.1 405Bopenrouter.ai2024-07-232026-10-07hn
Chat with Meta Llama 3.1 405Bllama3.replicate.dev2024-07-232026-10-07hn
Llama 3.1: Our most capable models to dateai.meta.com official2024-07-232026-10-07hn
Llama 3.1llama.meta.com official2024-07-232026-10-07hn
Llama 3.1 70B Instruct – 128k context, open weights | LLM Referencellmreference.com2024-07-232026-10-07exa
Llama 3.1 70B Review — Benchmarks, Pricing & AI Panel Verdict | TopReviewed.aitopreviewed.ai2024-07-222026-10-07exa

Papers

1 paper

Papers 1

arXiv links found in the model card and its tags. "Peer-reviewed" is shown only with evidence: a published venue in OpenAlex (measured) or the authors' own journal reference on arXiv (reported).

Similar models

3 models

Similar models

Sources

3 sources

Sources

Hub snapshot: 2026-10-25 · HF API: 2026-10-24 21:34 UTC · Providers: 2026-10-24 21:44 UTC · Articles: 2026-10-24 21:45 UTC · Summary: 2026-10-07 15:01 UTC · Last verified 2026-10-25

Follow Llama-3.1-70B-Instruct

Watch this model

Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.