Florence-2-base

232M parameters, images and text in, text out, 1K tokens of context.

Verified 2026-10-26 07:27 UTC Rank #197 in LLMs · score 89.2 method v1 Sources 3 Downloads 30d 3.1M · #47 of 619 tracked

Changed this week: 1 new quantized build on the Hub (6 in all).2026-10-24

Can I run it?0.1GB at INT4Estimatefits a 16 GB cardmethod

Can I use it?MITPermissivelicence file2026-10-26

Do people use it?3.1Mdownloads, 30 days#47 of 619 we track · top 8%+4 likes, downloads +6% this week431 likesHugging Face2026-10-26

Worth reading

Picked by rule from 12 tracked pieces: the lab's own post, the most shared, a field report, the latest.

Latest · 2 years ago

Playing with Microsoft Florence 2

Microsoft released Florence 2, a model capable of image captioning, segmentation, and OCR that runs on CPU and GPU.

objectgraph.com 2024-08-06 5 HN pts 214 words about the Florence 2 family family

12 best of 12 tracked

The 1 pick above, plus these 11. Top is the radar's score: discussion, source authority and the lab's own posts, fading with age.

ArticleFlorence-2: Advancing a Unified Representation for ...microsoft.com · undated · official
Paper[2311.06242] Florence-2: Advancing a Unified Representation for a Variety of Vision Tasksar5iv.labs.arxiv.org · undated
PaperFlorence-2: Advancing a Unified Representation for a Variety of Vision Tasksarxiv.org · 2023-11-13 · 1 HN pts
CodeShow HN: Florence2-Sharp – Advanced Image Understanding and OCR in C#github.com · 2024-07-25 · 1 HN pts
ArticleFlorence-2: Advancing a Unified Representation for a Variety of Vision Tasksopenaccess.thecvf.com · undated
ArticleFlorence-2: MIT Open Source Vision Foundation Model by Microsoftblog.roboflow.com · 2024-06-20 · 4 HN pts
ArticleHow Microsoft's New Large Vision Model "Florence-2" Worksassemblyai.com · 2024-07-15 · 1 HN pts
ArticleExploring Florence-2 - m0nadsmonads.substack.com · 2024-08-12
ArticleFlorence-2: Multimodal Vision Model | MaximoFNmaximofn.com · 2024-06-23
ArticleFlorence-2: A Unified Model for Many Vision Tasks | Abhik Sarkarabhik.ai · 2024-01-10
ArticleFlorence-2: Advancing a Unified Representation for a Variety of Vision Tasks [Quick Review]liner.com · 2023-11-10

Show all 11

Where it's used

Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.

No API provider serves it, so its downloads are people running it themselves: 6 quantized builds and 100 Spaces.

As a service0no API provider listed
On own hardware6quantized builds
Built on2821 fine-tunes · 7 adapters · 0 merges
In apps100+Hugging Face Spaces

Top Spaces running it: gokaygokay/Florence-2 ♥ 848 · ovi054/image-to-prompt ♥ 836 · SkalskiP/florence-sam ♥ 519 · microsoft/OmniParser-v2 ♥ 463

Will it fit?

Weights only, an estimate: leave room for the KV cache. Method

Fits a 16 GB card at INT4

About 0.1 GB at INT4, 0.3 GB at FP8, 0.6 GB at BF16: weights only, so leave room for the KV cache.

Estimate params × bytes per param × 1.2 · how we estimate

The story so far

Weights, every dated article, and repo commits on one rail.

Most coverage came in the first two weeks

Teal is the weights (2024-06-15), amber triangles are 7 dated articles (each links to it), grey ticks are repo commits.

What people built on it

34 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.

34 derivatives on the Hub, most of them fine-tunes

From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.

All variants →

Reference

Everything else, folded: the full tables, every source.

Benchmarks

20 rows

Benchmarks 20 rows

measured third-party leaderboard rowauthor-reported number from the model card or eval files (20)No confidence intervals unless the source gave them; settings differ between rows, so compare within a benchmark, not across. Date = the benchmark's own date when it gives one, else when we read it.

Benchmark	Value	Setting / notes	Reported by	Date
author-reported COCO Cap. test CIDEr	133	Florence-2-base	model author	2026-10-07
author-reported COCO Caption Karpathy test CIDEr	140	Florence-2-base-ft	model author	2026-10-07
author-reported COCO Det. val2017 mAP	34.7	Florence-2-base	model author	2026-10-07
author-reported Flickr30k test R@1	83.6	Florence-2-base	model author	2026-10-07
author-reported NoCaps val CIDEr	118.7	Florence-2-base	model author	2026-10-07
author-reported NoCaps val CIDEr	116.7	Florence-2-base-ft	model author	2026-10-07
author-reported RefCOCO · Understand images and video	53.9 percent	Florence-2-base	model author	2026-10-07
author-reported RefCOCO · Understand images and video	58.4 percent	Florence-2-base	model author	2026-10-07
author-reported RefCOCO · Understand images and video	49.7 percent	Florence-2-base	model author	2026-10-07
author-reported RefCOCO · Understand images and video	51.5 percent	Florence-2-base	model author	2026-10-07
author-reported RefCOCO · Understand images and video	56.4 percent	Florence-2-base	model author	2026-10-07
author-reported RefCOCO · Understand images and video	47.9 percent	Florence-2-base	model author	2026-10-07
author-reported RefCOCO · Understand images and video	66.3 percent	Florence-2-base	model author	2026-10-07
author-reported RefCOCO · Understand images and video	65.1 percent	Florence-2-base	model author	2026-10-07
author-reported RefCOCO · Understand images and video	34.6	Florence-2-base	model author	2026-10-07
author-reported TextCaps val CIDEr	70.1	Florence-2-base	model author	2026-10-07
author-reported TextCaps val CIDEr	143.9	Florence-2-base-ft	model author	2026-10-07
author-reported TextVQA · Understand images and video	63.6 percent	Florence-2-base-ft	model author	2026-10-07
author-reported VQAv2 · Understand images and video	79.7 percent	Florence-2-base-ft	model author	2026-10-07
author-reported VizWiz VQA test-dev Acc	63.6 percent	Florence-2-base-ft	model author	2026-10-07

What the card says

1 cited sentence

Summary generated, cited

The model uses a sequence-to-sequence architecture to perform vision tasks like captioning, object detection, and segmentation.^[1]

“sequence-to-sequence architecture enables it to excel”

Generated 2026-10-07 15:03 UTC by parasail-gemma-4-26b-a4b-it; each numeral points at the fact it rests on. Not a verdict.

Use-case evidence

2 use cases

What it is good at 2 use cases

Labels are evidence, not verdicts: measured a third-party leaderboard row for this model; author-reported a number the model author published in the card or eval files; structural a deterministic signal from the name, config or chat template, or a decision-model estimate. Each chip links to its source. All use cases.

Facts

21 fields

Listed in LLMs

Facts

Florence-2-base has 232M parameters. (safetensors metadata, observed 2026-10-26)
Context length is 1,024 tokens (1K). (config.json max_position_embeddings, observed 2026-10-26)
License: MIT (friction: none). Permissive. (license file, observed 2026-10-26)
Created 2024-06-15, last modified 2025-08-04. (HF API, observed 2026-10-26)

Pipeline tag	image-text-to-text
Library	transformers
Architectures	Florence2ForConditionalGeneration
Model type	florence2
Dtype	float16
Params by dtype	F16: 231,567,705
Size band	S
MoE	no
Context length	1,024 (config.json max_position_embeddings)
Papers	arXiv:2311.06242
License	MIT (mit)
Hub snapshot	2026-10-26

Servability

vLLM · SGLang

Servability

vLLM: not in registry · Florence2ForConditionalGeneration · 4b1c04e76fe9ac3ca8b7e541dd423e7d5f38dc60SGLang: not in registry · 60d941293e3af27a12c6eb64e2c3e2e1a17f100f

vLLM registry (4b1c04e76fe9ac3ca8b7e541dd423e7d5f38dc60) · SGLang registry (60d941293e3af27a12c6eb64e2c3e2e1a17f100f) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.

Hardware fit

3 estimates

Hardware fit ESTIMATE

Weights-only memory for Florence-2-base: params × bytes/param × 1.2 overhead. KV cache and activations are extra. How this is computed.

BF16 (2 bytes/param)	0.6 GB · Edge (<4 GB)
FP8 (1 byte/param)	0.3 GB · Edge (<4 GB)
INT4 (0.5 bytes/param)	0.1 GB · Edge (<4 GB)

Weights only (params × bytes/param × 1.2 overhead); KV cache and activations extra.

All variants

34 variants

Lineage

Original model Family: microsoft/Florence-2 34 derivatives on the Hub

finetune: 21quantized: 6adapter: 7

Top variants (34 shown, ranked by downloads)

Variant	Relation	Quant	Params	Band	License	Author	Likes	Downloads
gokaygokay/Florence-2-Flux	finetune	—	271M	S	apache-2.0	gokaygokay	15	94.4k
AskUI/PTA-1	finetune	—	271M	S	mit	AskUI	101	59.8k
sahilnishad/Florence-2-FT-DocVQA	finetune	—	271M	S	mit	sahilnishad	1	54.7k
onnx-community/Florence-2-base	quantized	—	—	—	mit	onnx-community	19	14.7k
PJMixers-Images/Florence-2-base-Castollux-v0.5	finetune	—	232M	S	—	PJMixers-Images	6	6.6k
PJMixers-Images/Florence-2-base-Castollux-v0.2	finetune	—	271M	S	—	PJMixers-Images	2	782
maxiw/Florence-2-ScreenQA-base	finetune	—	271M	S	—	maxiw	2	661
PJMixers-Images/Florence-2-base-Castollux-v0.4	finetune	—	271M	S	—	PJMixers-Images	1	489
PJMixers-Images/Florence-2-base-Castollux-v0.1	finetune	—	271M	S	—	PJMixers-Images	0	504
mlboydaisuke/Florence-2-base-ExecuTorch	quantized	—	—	—	mit	mlboydaisuke	0	480
PJMixers-Images/Florence-2-base-danbooru2022-316k	finetune	—	271M	S	—	PJMixers-Images	2	281
josley/florence-2-icon-caption	finetune	—	350M	S	mit	josley	1	328
kzawistowsk/TinyClick	finetune	—	—	—	mit	kzawistowsk	4	0
alakxender/florence-base-dv01-01-dv-e10	finetune	—	271M	S	—	alakxender	0	178
phonsobon/Images_captioning_fine_tune_Florence-2-base	adapter	—	232M	S	mit	phonsobon	1	104
mlboydaisuke/Florence-2-base-CoreML	quantized	—	—	—	mit	mlboydaisuke	0	140
francipaolo/florence-2-pal-comp-v1	finetune	—	271M	S	—	francipaolo	0	109
Spravil/caption-via-translation-0_4B	finetune	—	429M	S	—	Spravil	0	105
BusyCoder/results	finetune	—	271M	S	mit	BusyCoder	0	90
minhvn4/florence2-icon	finetune	—	232M	S	mit	minhvn4	0	89
francipaolo/florence-2-pal-comp-v2	finetune	—	271M	S	—	francipaolo	0	82
thaidinhz1/kie-receipt-florence2-lora	adapter	—	—	—	—	thaidinhz1	0	74
mmrech/florence2-pitvqa-finetuned	adapter	—	—	—	mit	mmrech	0	71
alakxender/florence-base-dv01-01	finetune	—	271M	S	—	alakxender	0	57
MrEngineer/florence-2-vqa-lora	adapter	—	—	—	—	MrEngineer	0	55
webnn/Florence-2-base	quantized	—	—	—	mit	webnn	1	0
birder-project/davit_fl_base_florence-v2	finetune	—	—	—	mit	birder-project	0	50
1of2/omniparser-v2-coreml	quantized	—	—	—	mit	1of2	0	42
mmrech/florence2_output	adapter	—	—	—	mit	mmrech	0	36
WC-ECL/florence-2-base-onnx	quantized	—	—	—	mit	WC-ECL	0	24
peeache/Florence2-LoRA-r32-a64-KvasirVQAx1	adapter	—	—	—	—	peeache	0	1
lamkser/OmniParserv2.0	finetune	—	—	—	—	lamkser	0	0
Ardelyo/dara-v1	finetune	—	—	—	apache-2.0	Ardelyo	0	0
Papizo123/florence2-nssf-ocr-final	adapter	—	770M	S	mit	Papizo123	0	0

Repository changelog

16 commits

changed in the last 7 daysOther: 2026-10-24all changes

Repository changelog 16 commits

Date	Commit	SHA
2025-08-04	add confidence scores for `od` and `description_with_bboxes` tasks (#25)	5ca5edf
2025-08-04	Fix invalid escape sequences (#20)	a495aed
2025-08-04	Fix the error "only DaViT is supported for now" appears whenloading the fine-tuned model (#31)	fe9219b
2025-08-04	Fix: resize_token_embeddings's Interface for transformers v4.49.0 (#26)	2f6bee8
2025-08-04	Fix weight init, deprecation warnings, safetensors (#27)	1896e7c
2024-11-04	Inherit from GenerationMixin (#22)	ceaf371
2024-07-20	update_model_init_fp16 (#12)	ee1f1f1
2024-07-19	update_ocr_postprocess (#11)	09eb282
2024-07-01	Update pipeline tag	53f3fb4
2024-06-30	Fix modeling code (typos/bugs) (#5)	4fab3c0
2024-06-26	Fix incorrect bos_token, eos_token, and pad_token ids in config.json (#6)	78e1dd4
2024-06-18	Update README.md	a2f0be7
2024-06-18	update_readme_example (#3)	0f6358c
2024-06-18	add_grounding_example (#2)	37070d5
2024-06-15	init_model (#1)	a15cd1f
2024-06-15	initial commit	c3302ad

Articles and coverage

12 pieces

Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.

Official1

Florence-2: Advancing a Unified Representation for ...

microsoft.comofficialauthorityabout the Florence 2 family201 words

Microsoft introduces Florence-2, a vision foundation model using a unified, prompt-based representation for diverse computer vision and vision-language tasks. The model was trained on FLD-5B, which consists of 5.4 billion comprehensive visual annotations on 126 million images using an iterative strategy of automated image annotation and model refinement. A sequence-to-sequence structure enables the model to perform versatile and comprehensive vision tasks with unprecedented zero-shot and fine-tuning capabilities.written, reviewed, cited

Analysis9

[2311.06242] Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks

ar5iv.labs.arxiv.orgauthorityabout the Florence 2 family15,794 words

Florence-2 is a novel vision foundation model utilizing a unified, prompt-based representation for diverse computer vision and vision-language tasks. The model was trained on FLD-5B, which consists of 5.4 billion comprehensive visual annotations on 126 million images using an iterative strategy of automated image annotation and model refinement. It employs a sequence-to-sequence structure to achieve unprecedented zero-shot and fine-tuning capabilities across tasks like captioning, object detection, grounding, and segmentation.written, reviewed, cited

The model improves training efficiency by 4 times compared to pre-trained models on ImageNet.
The pre-trained backbone surpasses supervised and self-supervised models on ADE20K semantic segmentation.
Task activation is achieved through textual prompts, reflecting the approach used by Large Language Models.

Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks

arxiv.org2023-11-13authorityabout the Florence 2 familyHN 1 pts · 0 comments345 words

Florence-2 is a novel vision foundation model featuring a unified, prompt-based representation for diverse computer vision and vision-language tasks. The model utilizes a sequence-to-sequence structure trained on FLD-5B, which consists of 5.4 billion comprehensive visual annotations on 126 million images. Extensive evaluations demonstrate the model's strong zero-shot and fine-tuning capabilities across numerous tasks.written, reviewed, cited

Playing with Microsoft Florence 2

objectgraph.com2024-08-06about the Florence 2 familyHN 5 pts · 0 comments214 words

Microsoft released Florence 2, a model capable of image captioning, segmentation, and OCR that runs on CPU and GPU. The author created a playground space on Hugging Face for testing the microsoft/Florence-2-large-ft model. A patch is required because the latest transformers version tries to import flash attention on Mac M machines and fails.written, reviewed, cited

Show HN: Florence2-Sharp – Advanced Image Understanding and OCR in C#

github.com2024-07-25authorityabout the Florence 2 familyHN 1 pts · 0 comments529 words

A new C# library provides .NET developers with a clean API to run Microsoft's Florence-2-base models locally without requiring Python. It supports multiple image understanding tasks including captioning, OCR, and object detection through automatic downloads of Florence-2-base ONNX models. The project is available as a NuGet package under an MIT license.written, reviewed, cited

Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks

openaccess.thecvf.comabout the Florence 2 family8,268 words

Microsoft introduces Florence-2, a vision foundation model using a unified, prompt-based representation for various computer vision and vision-language tasks. The model utilizes a sequence-to-sequence structure trained on the FLD-5B dataset, which consists of 5.4 billion comprehensive visual annotations on 126 million images. Extensive evaluations demonstrate Florence-2 to be a strong contender with unprecedented zero-shot and fine-tuning capabilities.written, reviewed, cited

The FLD-5B dataset generation engine uses multiple models to reach a consensus on image annotations.
All annotations in the FLD-5B dataset are uniformly standardized into textual outputs for consistent optimization.
Task activation in Florence-2 is achieved through textual prompts reflecting the approach used by Large Language Models.

Florence-2: MIT Open Source Vision Foundation Model by Microsoft

blog.roboflow.com2024-06-20about the Florence 2 familyHN 4 pts · 0 comments724 words

Microsoft released Florence-2, a lightweight vision-language model under the MIT license that unifies over ten tasks like captioning and segmentation. The model relies on the FLD-5B dataset, which contains 126 million images and 5.4 billion annotations, to achieve performance comparable to much larger models. It comes in two variants with 0.23 billion and 0.77 billion parameters, enabling deployment on mobile devices or an NVIDIA T4 GPU.written, reviewed, cited

The FLD-5B dataset is not yet publicly available and was announced for upcoming release during CVPR 2024.
Running Florence-2 on an NVIDIA T4 GPU yields inferences taking approximately 1 second per image.

How Microsoft's New Large Vision Model "Florence-2" Works

assemblyai.com2024-07-15about the Florence 2 familyHN 1 pts · 0 comments2,037 words

Microsoft released Florence-2, a compact Large Vision Model that unifies image-language tasks like captioning and object detection. The model trains on FLD-5B, an open-source dataset containing 5.4 billion annotations on 126 million images, and encodes images using a DaViT encoder into visual tokens. It processes location information via special tokens <loc_x><loc_y> where coordinates have a 0.1% resolution, enabling unified training without task-specific heads.written, reviewed, cited

Florence-2 uses a multi-step engine where the model generates pseudolabels, filters them for quality, and iteratively mixes them back into training data.
The model achieves consensus on original pseudolabels by having many different specialized models work together to generate the final labels.

Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks [Quick Review]

liner.com2023-11-10about the Florence 2 family783 words

Florence-2: A Unified Model for Many Vision Tasks | Abhik Sarkar

abhik.ai2024-01-10about the Florence 2 family1,696 words

All coverage tracked for this model (12) — append-only, nothing is ever removed

Title	Source	Published	First seen	Signals
Florence-2: Advancing a Unified Representation for ...	microsoft.com official	—	2026-10-07	exa
[2311.06242] Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks	ar5iv.labs.arxiv.org	—	2026-10-07	exa
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks	openaccess.thecvf.com	—	2026-10-07	exa
Exploring Florence-2 - m0nads	monads.substack.com	2024-08-12	2026-10-07	exa
Playing with Microsoft Florence 2	objectgraph.com	2024-08-06	2026-10-07	hn
Show HN: Florence2-Sharp – Advanced Image Understanding and OCR in C#	github.com	2024-07-25	2026-10-07	hn
How Microsoft's New Large Vision Model "Florence-2" Works	assemblyai.com	2024-07-15	2026-10-07	hn
Florence-2: Multimodal Vision Model \| MaximoFN	maximofn.com	2024-06-23	2026-10-07	exa
Florence-2: MIT Open Source Vision Foundation Model by Microsoft	blog.roboflow.com	2024-06-20	2026-10-07	hn
Florence-2: A Unified Model for Many Vision Tasks \| Abhik Sarkar	abhik.ai	2024-01-10	2026-10-07	exa
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks	arxiv.org	2023-11-13	2026-10-07	hn
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks [Quick Review]	liner.com	2023-11-10	2026-10-07	exa

Papers

1 paper

Papers 1

arXiv links found in the model card and its tags. "Peer-reviewed" is shown only with evidence: a published venue in OpenAlex (measured) or the authors' own journal reference on arXiv (reported).

Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks this model

Bin Xiao, Haiping Wu, Weijian Xu +6 · 2023-11-10 · PDF · HF paper page (98 upvotes)

Similar models

3 models

Similar models

granite-docling-258M · 258M
same size band (S), same task (image-text-to-text), permissive license
LFM2.5-VL-450M · 449M
same size band (S), same task (image-text-to-text)
GOT-OCR-2.0-hf · 561M
same size band (S), same task (image-text-to-text), permissive license

Sources

3 sources

Sources

Hub snapshot: 2026-10-26 · HF API: 2026-10-26 07:17 UTC · Providers: 2026-10-26 07:26 UTC · Articles: 2026-10-26 07:27 UTC · Summary: 2026-10-07 15:03 UTC · Last verified 2026-10-26 07:27 UTC

Follow Florence-2-base

Watch this model

/feeds/models/microsoft--Florence-2-base.xml(RSS 2.0) · every change to Florence-2-base
/feeds/families/microsoft--Florence-2.xml(RSS 2.0) · every change across the Florence-2 family

Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.