Model Follower.

Up to date

Rebuilt
2026-10-25 06:54 UTC
Release radar
117 on watch
Hub snapshot
2026-10-25
Re-checked
daily · every number sourced and dated

Models › codebert family

codebert-base

Hugging Face ↗ Follow changes

microsoft Not declared out 5 years ago

Text in, embeddings out, 514 tokens of context. 1 provider serves it.

Verified 2026-10-25 Rank #36 in Embeddings · score 92.5 method v1 Sources 3 Downloads 30d 206.5k · #249 of 619 tracked

Can I use it?Not declaredNo license declared on the model cardlicence file2026-10-24
Do people use it?206.5kdownloads, 30 days#249 of 619 we track · top 40%+2 likes, downloads −8% this week299 likesHugging Face2026-10-24

Worth reading

Picked by rule from 7 tracked pieces: the lab's own post, the most shared, a field report, the latest.

Latest · 6 months ago

Benchmark: 4 Code Embedding Models – CodeBERT vs. GraphCodeBERT vs. CodeT5+ for RAG Pipelines — johal.in

This benchmark evaluates four open-source code embedding models—CodeBERT-base, GraphCodeBERT-base, and two CodeT5+ variants—for RAG pipelines.

  • The benchmark script explicitly reduces batch size by half when an out-of-memory error occurs.
  • CodeT5+ models utilize the pooler_output for embeddings while CodeBERT uses the last hidden state.
johal.in 2026-05-02 3.6k words about the codebert family family

7 best of 7 tracked

The 1 pick above, plus these 6. Top is the radar's score: discussion, source authority and the lab's own posts, fading with age.

  1. Paper[2002.08155] CodeBERT: A Pre-Trained Model for Programming and Natural Languagesar5iv.labs.arxiv.org · undated
  2. Codemicrosoft/CodeBERTgithub.com · undated
  3. ArticleShow HN: CodeBERT a RoBERTa model pre-trained on Python Code (Colab)colab.research.google.com · 2020-06-13 · 2 HN pts
  4. ArticleMicrosoft’s CodeBERT ingests public GitHub repositories to help you find codeventurebeat.com · 2020-02-24 · 2 HN pts
  5. ArticleCode Search | microsoft/CodeBERT | DeepWikideepwiki.com · 2025-04-23
  6. ArticleA Deep Dive into CodeBERT: Microsoft’s Pre-trained Language Model for Source Codediversedaily.com · 2024-08-04

Where to run it

Live hosted endpoints, cheapest first, USD per million tokens. Ties break on output price, then provider name.

#Cheapest live rowIn /1MOut /1MContextQuantSeen
1 HF Inference — — — — Oct 25

All 1 provider →

Where it's used

Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.

Served by 1 API provider; 7 quantized builds for local use; a big base for builders (160 fine-tunes, adapters and merges).

  • As a service1HF Inference
  • On own hardware7quantized builds
  • Built on160152 fine-tunes · 7 adapters · 1 merges
  • In apps100+Hugging Face Spaces

Top Spaces running it: Molbap/transformers-modular-refactor ♥ 17 · awacke1/Feature-Extraction-microsoft-codebert-base ♥ 3 · alexpaul/microsoft-codebert-base ♥ 2 · Not-Grim-Refer/Code-to-Detailed-English-Description ♥ 2

The story so far

Weights, every dated article, and repo commits on one rail.

5 dated articles and 16 repo commits so far

Teal is the weights (2022-03-02), amber triangles are 5 dated articles (each links to it), grey ticks are repo commits.

MarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOct2022-02-11: 1 repo commit2021-05-20: 2 repo commits2020-12-11: 1 repo commit2020-07-10: 12 repo commitsBenchmark: 4 Code Embedding Models – CodeBERT vs. GraphCodeBERT vs. CodeT5+ for RAG Pipelines — johal.in · 2026-05-02Show HN: CodeBERT a RoBERTa model pre-trained on Python Code (Colab) · 2020-06-13Microsoft’s CodeBERT ingests public GitHub repositories to help you find code · 2020-02-24A Deep Dive into CodeBERT: Microsoft’s Pre-trained Language Model for Source Code · 2024-08-04Code Search | microsoft/CodeBERT | DeepWiki · 2025-04-23Weights 2022-03-02 · 2022-03-02Weights 2022-03-02now · 1 provider
MarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOct2022-02-11: 1 repo commit2021-05-20: 2 repo commits2020-12-11: 1 repo commit2020-07-10: 12 repo commitsBenchmark: 4 Code Embedding Models – CodeBERT vs. GraphCodeBERT vs. CodeT5+ for RAG Pipelines — johal.in · 2026-05-02Show HN: CodeBERT a RoBERTa model pre-trained on Python Code (Colab) · 2020-06-13Microsoft’s CodeBERT ingests public GitHub repositories to help you find code · 2020-02-24A Deep Dive into CodeBERT: Microsoft’s Pre-trained Language Model for Source Code · 2024-08-04Code Search | microsoft/CodeBERT | DeepWiki · 2025-04-23Weights 2022-03-02 · 2022-03-02Weights 2022-03-02now · 1 provider

What people built on it

167 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.

167 derivatives on the Hub, most of them fine-tunes

From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.

7 quantized (Hugging Face model tree)152 fine-tunes (Hugging Face model tree)1 merge (Hugging Face model tree)7 adapters (Hugging Face model tree)7 quantized152 fine-tunes1 merge7 adapters
7 quantized (Hugging Face model tree)152 fine-tunes (Hugging Face model tree)1 merge (Hugging Face model tree)7 adapters (Hugging Face model tree)7 quantized152 fine-tunes1 merge7 adapters

All variants →

Reference

Everything else, folded: the full tables, every source.

What the card says

3 cited sentences

Summary generated, cited

CodeBERT-base is a RobertaModel architecture designed for programming and natural languages.[1][2] The model is trained on bi-modal data from CodeSearchNet using MLM+RTD objectives.[3] It supports tasks such as code search and code-to-document generation.[4]

  1. architectures
  2. “CodeBERT: A Pre-Trained Model for Programming and Natural…”
  3. “trained on bi-modal data (documents & code) of CodeSearchNet”
  4. “scripts that support "code search" and "code-to-document …”

Generated 2026-10-07 15:03 UTC by parasail-gemma-4-26b-a4b-it; each numeral points at the fact it rests on. Not a verdict.

Use-case evidence

2 use cases

What it is good at 2 use cases

Labels are evidence, not verdicts: measured a third-party leaderboard row for this model; author-reported a number the model author published in the card or eval files; structural a deterministic signal from the name, config or chat template, or a decision-model estimate. Each chip links to its source. All use cases.

Facts

16 fields

Listed in Embeddings

Facts

Pipeline tagfeature-extraction
Librarytransformers
ArchitecturesRobertaModel
Model typeroberta
MoEno
Context length514 (config.json max_position_embeddings)
PapersarXiv:2002.08155
LicenseNot declared
Hub snapshot2026-10-25

Servability

vLLM · SGLang

Servability

vLLM: supported · RobertaModel · 7050ef881ff0835f6890a47db1859a24c65481d6SGLang: supported · RobertaModel · 773cb938fe9484a8f7e7e46fb1b1fe96412c59ed

vLLM registry (7050ef881ff0835f6890a47db1859a24c65481d6) · SGLang registry (773cb938fe9484a8f7e7e46fb1b1fe96412c59ed) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.

All variants

40 variants

Lineage

Original model Family: microsoft/codebert 167 derivatives on the Hub

finetune: 152quantized: 7adapter: 7merge: 1

Top variants (40 shown, ranked by downloads)

Relation
40 rows
Variant Relation Quant Params Band License Author Likes Downloads
angusleung100/CodeBERT-Base-Solidity-Vulnerability finetune — 125M S — angusleung100 1 6.4k
mahdin70/codebert-devign-code-vulnerability-detector finetune — 125M S mit mahdin70 1 5k
4luc/codebert-code-clone-detector finetune — 125M S mit 4luc 0 1.8k
mahdin70/CodeBERT-PrimeVul-BigVul finetune — 125M S mit mahdin70 2 1.4k
sohailtsm/semeval_taskB_model finetune — 125M S — sohailtsm 0 1.2k
mcanoglu/microsoft-codebert-base-finetuned-defect-cwe-group-detection finetune — 125M S — mcanoglu 0 968
Hachirou18/NyerAndroidMalware finetune — 125M S apache-2.0 Hachirou18 0 884
mahdin70/CodeBERT-VulnCWE finetune — 126M S mit mahdin70 1 814
mcanoglu/microsoft-codebert-base-finetuned-defect-detection finetune — 125M S — mcanoglu 0 797
salmane11/SQLQueryShield finetune — 125M S — salmane11 1 634
buelfhood/SOCO-Java-CodeBERT-ST finetune — 125M S — buelfhood 1 583
ahmedosama2003/techforum-duplicate-detector finetune — 125M S — ahmedosama2003 1 571
sohailtsm/semeval_taskA_model finetune — 125M S — sohailtsm 0 613
shubharuidas/codebert-embed-base-dense-retriever finetune — 125M S apache-2.0 shubharuidas 2 473
AdnanRiaz107/CodeBert-finetuned-the-stack-bash finetune — 125M S — AdnanRiaz107 0 476
JetBrains-Research/rocq-language-theorem-embeddings finetune — 125M S apache-2.0 JetBrains-Research 0 474
elinaparajuli/Codebert-Code-finetuned finetune — 125M S — elinaparajuli 0 450
sohailtsm/semeval_taskC_model finetune — 125M S — sohailtsm 0 440
jayansh21/codesheriff-bug-classifier finetune — 125M S — jayansh21 1 359
NamCyan/codebert-base-technical-debt-code-tesoro finetune — 125M S — NamCyan 0 408
RayenLLM/Vulnerability_Detection_Using_CodeBERT finetune — 125M S — RayenLLM 1 348
eyeonyou/tag_clf finetune — 125M S — eyeonyou 0 386
itsanan/codebert-finetuned-crewai-base finetune — 125M S apache-2.0 itsanan 0 376
buelfhood/SOCO-Java-codebert-mnrl-triplets-ep1-bs16-lr2e-05-split0.0 finetune — 125M S — buelfhood 0 374
SashaSheykina/codeBert-finetuned-cXg-nl-to-code finetune — 125M S — SashaSheykina 0 369
emekaphilians/trustchainai-codebert finetune — 125M S mit emekaphilians 0 368
Sab286/ai-code-detector-finetuned finetune — 125M S — Sab286 0 358
itsanan/codebert-embed-crewai-base finetune — 125M S apache-2.0 itsanan 0 356
buelfhood/SOCO-C-CodeBERT-ST finetune — 125M S — buelfhood 0 354
aidencary/codepulse-codebert finetune — 125M S mit aidencary 0 349
csc-unipd/lilybert quantized — 125M S apache-2.0 csc-unipd 1 291
onnx-community/codebert-base-ONNX quantized — — — — onnx-community 0 310
AdnanRiaz107/CBertbase-APPS10k finetune — 125M S — AdnanRiaz107 0 289
azherali/CodeGenDetect-CodeBert_Lora adapter — — — — azherali 0 265
AbijithwearsHUGGIES/codebert-smart-contract-vuln quantized — 125M S mit AbijithwearsHUGGIES 1 140
sng1789/results finetune — 125M S — sng1789 0 189
aieng-lab/codebert-base_sentiment finetune — 125M S mit aieng-lab 0 183
mmock/NLBSE2025_java finetune — 125M S mit mmock 0 166
phamtungthuy/codebert-defect-detection finetune — 125M S mit phamtungthuy 0 154
mmock/NLBSE2025_python finetune — 125M S mit mmock 0 151

Repository changelog

16 commits

Repository changelog 16 commits

DateCommitSHA
2022-02-11 Addition of Rust model 3b0952f
2021-05-20 upload flax model 3a68f56
2021-05-20 allow flax 3ff5798
2020-12-11 Migrate model card from transformers-repo aa14b80
2020-07-10 Update tf_model.h5 0fef132
2020-07-10 Update vocab.json b5e9225
2020-07-10 Update pytorch_model.bin 9bc0ec8
2020-07-10 Update merges.txt 8b0f31a
2020-07-10 Update config.json 5319141
2020-07-10 Update special_tokens_map.json 1a3831e
2020-07-10 Update tokenizer_config.json 0047fa0
2020-07-10 Update merges.txt ffc990b
2020-07-10 Update config.json 09a9eff
2020-07-10 Update special_tokens_map.json 8d2ad35
2020-07-10 Update tokenizer_config.json dedb8f2
2020-07-10 initial commit 93d55bc

All providers

1 provider · 1 endpoint

Where to run 1 providers · 1 endpoints

Hosted endpoints observed for codebert-base, sorted by input price (USD per million tokens). Each row names its observer and date; prices change daily, so check the source before quoting.

ProviderModel refContextQuant$/M in$/M outtok/sTTFTToolsStatusObserved
HF Inference
hf
 microsoft/codebert-base — — — — — — — live 2026-10-25

Articles and coverage

7 pieces

Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.

Official1

Show HN: CodeBERT a RoBERTa model pre-trained on Python Code (Colab)
colab.research.google.com2020-06-13about the codebert familyHN 2 pts · 1 comments1,783 words
A Colab notebook demonstrates setting up and testing CodeBERT, a model for code understanding. The tutorial uses the codistai/codeBERT-small-v2 model, which has a maximum positional embedding limit of 1024. The code parsing mechanism relies on special tokens like 'indent' and 'dedent' to specify logical structure.written, reviewed, cited
  • The code-bert library lacks a pip wheel and must be installed via git clone and editable pip install.
  • The process_code function outputs a list where each item represents one logical line of code.
  • The tokenizer issues a FutureWarning that the max_len parameter is deprecated in favor of model_max_length.

Analysis6

[2002.08155] CodeBERT: A Pre-Trained Model for Programming and Natural Languages
ar5iv.labs.arxiv.orgauthorityabout the codebert family6,571 words
CodeBERT is a bimodal pre-trained model for programming and natural languages that uses a Transformer-based architecture. The model contains 125M parameters and is trained on data from six programming languages, including 2.1M bimodal datapoints and 6.4M unimodal codes. It employs a hybrid objective function combining masked language modeling and replaced token detection to support tasks like code search and documentation generation.written, reviewed, cited
  • CodeBERT is trained without explicit markers to denote the input programming language across six languages.
  • The training data filters out functions shorter than three lines and documentations shorter than three tokens.
  • CodeBERT consistently outperforms RoBERTa in a zero-shot scenario where pre-trained model parameters are fixed.
microsoft/CodeBERT
github.comauthorityabout the codebert family750 words
Microsoft released the CodeBERT series, a collection of six pre-trained models for programming and natural languages. The series includes CodeBERT, GraphCodeBERT, UniXcoder, CodeReviewer, CodeExecutor, and LongCoder, all available in this repository as of June 2023. These models are multi-programming-lingual models pre-trained on NL-PL pairs in 6 programming languages including Python, Java, JavaScript, PHP, Ruby, and Go. LongCoder is specifically designed as a sparse and efficient pre-trained Transformer model for long code modeling.written, reviewed, cited
  • The base CodeBERT model is explicitly stated as not suitable for mask prediction tasks.
  • CodeExecutor learns to predict execution traces using a specific code execution pre-training task.
Benchmark: 4 Code Embedding Models – CodeBERT vs. GraphCodeBERT vs. CodeT5+ for RAG Pipelines — johal.in
johal.in2026-05-02about the codebert family3,575 words
This benchmark evaluates four open-source code embedding models—CodeBERT-base, GraphCodeBERT-base, and two CodeT5+ variants—for RAG pipelines. CodeT5+ 220m achieves 0.89 recall@10, 22% higher than CodeBERT base, while GraphCodeBERT base reduces p99 embedding latency by 18% compared to CodeBERT base on NVIDIA T4 GPUs. This trade-off between accuracy and latency guides selection, with CodeT5+ costing $12.70 per million embeddings.written, reviewed, cited
  • The benchmark script explicitly reduces batch size by half when an out-of-memory error occurs.
  • CodeT5+ models utilize the pooler_output for embeddings while CodeBERT uses the last hidden state.
Microsoft’s CodeBERT ingests public GitHub repositories to help you find code
venturebeat.com2020-02-24about the codebert familyHN 2 pts · 0 comments
A Deep Dive into CodeBERT: Microsoft’s Pre-trained Language Model for Source Code
diversedaily.com2024-08-04about the codebert family
Code Search | microsoft/CodeBERT | DeepWiki
deepwiki.com2025-04-23about the codebert family

Papers

1 paper

Papers 1

arXiv links found in the model card and its tags. "Peer-reviewed" is shown only with evidence: a published venue in OpenAlex (measured) or the authors' own journal reference on arXiv (reported).

Similar models

3 models

Similar models

Sources

3 sources

Sources

Hub snapshot: 2026-10-25 · HF API: 2026-10-24 21:35 UTC · Providers: 2026-10-24 21:44 UTC · Articles: 2026-10-24 21:45 UTC · Summary: 2026-10-07 15:03 UTC · Last verified 2026-10-25

Follow codebert-base

Watch this model

Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.