Florence-2-large

777M parameters, images and text in, text out, 4K tokens of context.

Verified 2026-10-25 Rank #142 in LLMs · score 93.6 method v1 Sources 3 Downloads 30d 374.8k · #190 of 619 tracked

Can I run it?0.5GB at INT4Estimatefits a 16 GB cardmethod

Can I use it?MITPermissivelicence file2026-10-24

Do people use it?374.8kdownloads, 30 days#190 of 619 we track · top 31%+4 likes, downloads +4% this week1,880 likesHugging Face2026-10-24

Worth reading

Picked by rule from 14 tracked pieces: the lab's own post, the most shared, a field report, the latest.

Latest · 2 years ago

Playing with Microsoft Florence 2

Microsoft released Florence 2, a model capable of image captioning, segmentation, and OCR that runs on CPU and GPU.

objectgraph.com 2024-08-06 5 HN pts 214 words about the Florence 2 family family

14 best of 14 tracked

The 1 pick above, plus these 13. Top is the radar's score: discussion, source authority and the lab's own posts, fading with age.

Paper[2311.06242] Florence-2: Advancing a Unified Representation for a Variety of Vision Tasksar5iv.labs.arxiv.org · undated
PaperFlorence-2: Advancing a Unified Representation for a Variety of Vision Tasksarxiv.org · 2023-11-13 · 1 HN pts
CodeShow HN: Florence2-Sharp – Advanced Image Understanding and OCR in C#github.com · 2024-07-25 · 1 HN pts
ArticleFlorence-2: Advancing a Unified Representation for a Variety of Vision Tasksopenaccess.thecvf.com · undated
ArticleFlorence-2: Advancing a Unified Representation for a Variety of Vision Tasksieeexplore.ieee.org · undated
ArticleFlorence-2 — Edgepediaedgechat.ai · undated
ArticleFlorence-2: Advancing a Unified Representation for a Variety of Vision Tasks — Paper Graphpaper-graph-ui.vercel.app · undated
ArticleFlorence-2: MIT Open Source Vision Foundation Model by Microsoftblog.roboflow.com · 2024-06-20 · 4 HN pts
ArticleHow Microsoft's New Large Vision Model "Florence-2" Worksassemblyai.com · 2024-07-15 · 1 HN pts
ArticlePapers Explained 214: Florence-2. While existing large vision models… | by Ritvik Rastogi | Mediumritvik19.medium.com · 2024-09-19
ArticleFlorence-2: Multimodal Vision Model | MaximoFNmaximofn.com · 2024-06-23
ArticleFlorence-2: A Unified Model for Many Vision Tasks | Abhik Sarkarabhik.ai · 2024-01-10
ArticleFlorence-2: Advancing a Unified Representation for a Variety of Vision Tasks [Quick Review]liner.com · 2023-11-10

Show all 13

Where it's used

Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.

No API provider serves it, so its downloads are people running it themselves: 3 quantized builds and 100 Spaces.

As a service0no API provider listed
On own hardware3quantized builds
Built on5443 fine-tunes · 10 adapters · 1 merges
In apps100+Hugging Face Spaces

Top Spaces running it: gokaygokay/FLUX-Prompt-Generator ♥ 1,400 · gokaygokay/Florence-2 ♥ 848 · John6666/danbooru-tags-transformer-v2-with-wd-tagger ♥ 322 · SkalskiP/better-florence-2 ♥ 198

Will it fit?

Weights only, an estimate: leave room for the KV cache. Method

Fits a 16 GB card at INT4

About 0.5 GB at INT4, 0.9 GB at FP8, 1.9 GB at BF16: weights only, so leave room for the KV cache.

Estimate params × bytes per param × 1.2 · how we estimate

The story so far

Weights, every dated article, and repo commits on one rail.

5 dated articles and 20 repo commits so far

Teal is the weights (2024-06-15), amber triangles are 5 dated articles (each links to it), grey ticks are repo commits.

What people built on it

57 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.

57 derivatives on the Hub, most of them fine-tunes

From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.

All variants →

Reference

Everything else, folded: the full tables, every source.

Benchmarks

28 rows

Benchmarks 28 rows

measured third-party leaderboard rowauthor-reported number from the model card or eval files (28)No confidence intervals unless the source gave them; settings differ between rows, so compare within a benchmark, not across. Date = the benchmark's own date when it gives one, else when we read it.

Benchmark	Value	Setting / notes	Reported by	Date
author-reported COCO Cap. test CIDEr	133	Florence-2-base	model author	2026-10-07
author-reported COCO Cap. test CIDEr	135.6	Florence-2-large	model author	2026-10-07
author-reported COCO Det. val2017 mAP	34.7	Florence-2-base	model author	2026-10-07
author-reported COCO Det. val2017 mAP	37.5	Florence-2-large	model author	2026-10-07
author-reported Flickr30k test R@1	83.6	Florence-2-base	model author	2026-10-07
author-reported Flickr30k test R@1	84.4	Florence-2-large	model author	2026-10-07
author-reported NoCaps val CIDEr	118.7	Florence-2-base	model author	2026-10-07
author-reported NoCaps val CIDEr	120.8	Florence-2-large	model author	2026-10-07
author-reported RefCOCO · Understand images and video	53.9 percent	Florence-2-base	model author	2026-10-07
author-reported RefCOCO · Understand images and video	56.3 percent	Florence-2-large	model author	2026-10-07
author-reported RefCOCO · Understand images and video	58.4 percent	Florence-2-base	model author	2026-10-07
author-reported RefCOCO · Understand images and video	61.6 percent	Florence-2-large	model author	2026-10-07
author-reported RefCOCO · Understand images and video	49.7 percent	Florence-2-base	model author	2026-10-07
author-reported RefCOCO · Understand images and video	51.4 percent	Florence-2-large	model author	2026-10-07
author-reported RefCOCO · Understand images and video	51.5 percent	Florence-2-base	model author	2026-10-07
author-reported RefCOCO · Understand images and video	53.6 percent	Florence-2-large	model author	2026-10-07
author-reported RefCOCO · Understand images and video	56.4 percent	Florence-2-base	model author	2026-10-07
author-reported RefCOCO · Understand images and video	57.9 percent	Florence-2-large	model author	2026-10-07
author-reported RefCOCO · Understand images and video	47.9 percent	Florence-2-base	model author	2026-10-07
author-reported RefCOCO · Understand images and video	49.9 percent	Florence-2-large	model author	2026-10-07
author-reported RefCOCO · Understand images and video	66.3 percent	Florence-2-base	model author	2026-10-07
author-reported RefCOCO · Understand images and video	68 percent	Florence-2-large	model author	2026-10-07
author-reported RefCOCO · Understand images and video	65.1 percent	Florence-2-base	model author	2026-10-07
author-reported RefCOCO · Understand images and video	67 percent	Florence-2-large	model author	2026-10-07
author-reported RefCOCO · Understand images and video	34.6	Florence-2-base	model author	2026-10-07

Show all 28 rows (3 more)

Benchmark	Value	Setting / notes	Reported by	Date
author-reported RefCOCO · Understand images and video	35.8	Florence-2-large	model author	2026-10-07
author-reported TextCaps val CIDEr	70.1	Florence-2-base	model author	2026-10-07
author-reported TextCaps val CIDEr	72.8	Florence-2-large	model author	2026-10-07

What the card says

1 cited sentence

Summary generated, cited

The model uses a prompt-based approach to perform vision tasks such as captioning, object detection, and segmentation.^[1]

“perform tasks like captioning, object detection, and segm…”

Generated 2026-10-07 15:01 UTC by parasail-gemma-4-26b-a4b-it; each numeral points at the fact it rests on. Not a verdict.

Use-case evidence

2 use cases

What it is good at 2 use cases

Labels are evidence, not verdicts: measured a third-party leaderboard row for this model; author-reported a number the model author published in the card or eval files; structural a deterministic signal from the name, config or chat template, or a decision-model estimate. Each chip links to its source. All use cases.

Facts

21 fields

Listed in LLMs

Facts

Florence-2-large has 777M parameters. (safetensors metadata, observed 2026-10-24)
Context length is 4,096 tokens (4K). (config.json max_position_embeddings, observed 2026-10-24)
License: MIT (friction: none). Permissive. (license file, observed 2026-10-24)
Created 2024-06-15, last modified 2025-08-04. (HF API, observed 2026-10-24)

Pipeline tag	image-text-to-text
Library	transformers
Architectures	Florence2ForConditionalGeneration
Model type	florence2
Dtype	float16
Params by dtype	F16: 776,721,497
Size band	S
MoE	no
Context length	4,096 (config.json max_position_embeddings)
Papers	arXiv:2311.06242
License	MIT (mit)
Hub snapshot	2026-10-25

Servability

vLLM · SGLang

Servability

vLLM: not in registry · Florence2ForConditionalGeneration · 7050ef881ff0835f6890a47db1859a24c65481d6SGLang: not in registry · 773cb938fe9484a8f7e7e46fb1b1fe96412c59ed

vLLM registry (7050ef881ff0835f6890a47db1859a24c65481d6) · SGLang registry (773cb938fe9484a8f7e7e46fb1b1fe96412c59ed) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.

Hardware fit

3 estimates

Hardware fit ESTIMATE

Weights-only memory for Florence-2-large: params × bytes/param × 1.2 overhead. KV cache and activations are extra. How this is computed.

BF16 (2 bytes/param)	1.9 GB · Edge (<4 GB)
FP8 (1 byte/param)	0.9 GB · Edge (<4 GB)
INT4 (0.5 bytes/param)	0.5 GB · Edge (<4 GB)

Weights only (params × bytes/param × 1.2 overhead); KV cache and activations extra.

All variants

40 variants

Lineage

Original model Family: microsoft/Florence-2 57 derivatives on the Hub

finetune: 43quantized: 3adapter: 10merge: 1

Top variants (40 shown, ranked by downloads)

Variant	Relation	Quant	Params	Band	License	Author	Likes	Downloads
gokaygokay/Florence-2-Flux-Large	finetune	—	823M	S	apache-2.0	gokaygokay	62	206.4k
thwri/CogFlorence-2.2-Large	finetune	—	823M	S	mit	thwri	45	148.4k
2toINF/X-VLA-Pt	finetune	—	880M	S	apache-2.0	2toINF	14	130.9k
thwri/CogFlorence-2.1-Large	finetune	—	823M	S	mit	thwri	28	43.8k
2toINF/X-VLA-Libero	finetune	—	880M	S	apache-2.0	2toINF	2	15.1k
thwri/CogFlorence-2-Large-Freeze	finetune	—	823M	S	mit	thwri	14	13.8k
2toINF/X-VLA-SoftFold	finetune	—	880M	S	apache-2.0	2toINF	4	10.9k
2toINF/X-VLA-WidowX	finetune	—	880M	S	apache-2.0	2toINF	3	7.7k
2toINF/X-VLA-RoboTwin2	finetune	—	880M	S	apache-2.0	2toINF	4	5.6k
onnx-community/Florence-2-large	quantized	—	—	—	mit	onnx-community	18	2.1k
2toINF/X-VLA-Calvin-ABC_D	finetune	—	880M	S	apache-2.0	2toINF	0	3k
HongxinLi/GoClick-Large	finetune	—	829M	S	mit	HongxinLi	4	2k
HongxinLi/GoClick-Base	finetune	—	271M	S	mit	HongxinLi	1	1.9k
anushreeberlia/fashion-florence	adapter	—	777M	S	mit	anushreeberlia	3	1.7k
2toINF/X-VLA-Google-Robot	finetune	—	880M	S	apache-2.0	2toINF	1	1k
silveroxides/furrence2-large	finetune	—	823M	S	mit	silveroxides	4	773
2toINF/X-VLA-VLABench	finetune	—	880M	S	apache-2.0	2toINF	1	921
diffusers/shot-categorizer-v0	finetune	—	829M	S	mit	diffusers	2	664
Qingyun/Florence-2-large-DOTA-v1.0-lmmrotate	finetune	—	930M	S	mit	Qingyun	3	496
mbreuss/flower_libero_10	finetune	—	1.00B	S	mit	mbreuss	4	340
2toINF/X-VLA-AgiWorld-Challenge	finetune	—	880M	S	apache-2.0	2toINF	0	522
mbreuss/flower_calvin_d	finetune	—	1.00B	S	mit	mbreuss	4	312
mbreuss/flower_libero_goal	finetune	—	1.00B	S	mit	mbreuss	4	259
mbreuss/flower_libero_90	finetune	—	1.00B	S	mit	mbreuss	4	253
mbreuss/flower_libero_spatial	finetune	—	1.00B	S	mit	mbreuss	3	269
Mitchins/Florence-Chan-2.1	adapter	—	777M	S	mit	Mitchins	1	326
Mit1208/Florence-2-large-DocLayNet	finetune	—	823M	S	mit	Mit1208	3	225
bkbj/Atest	merge	—	—	—	bigscience-openrail-m	bkbj	0	263
Nikhil-aka-Nick/FlorenceDropout2	finetune	—	829M	S	mit	Nikhil-aka-Nick	1	161
narmvasu99/florence2-large-loan-doc-extractor-v1	finetune	—	829M	S	mit	narmvasu99	0	193
StarVLA/Florence-GR00T-Bridge-RT-1	finetune	—	—	—	—	StarVLA	0	186
ultra2207/florence_2_ocr_peft	adapter	—	—	—	—	ultra2207	1	126
thorscribe/thorscribe-model-3	finetune	—	829M	S	mit	thorscribe	0	169
d3p4rt/newtype-cognition	finetune	—	777M	S	mit	d3p4rt	0	156
Qingyun/Florence-2-models-lmmrotate	finetune	—	—	—	mit	Qingyun	3	0
EricChan1122/florence2-parasitic-egg-detection-version2	adapter	—	—	—	mit	EricChan1122	0	143
Nikhil-aka-Nick/florence2-finalV11	finetune	—	829M	S	mit	Nikhil-aka-Nick	0	140
EricChan1122/florence2-parasitic-egg-detection-version3	adapter	—	—	—	mit	EricChan1122	0	138
mlboydaisuke/Florence-2-large-ExecuTorch	quantized	—	—	—	mit	mlboydaisuke	0	122
2toINF/X-VLA-Pt-Auto-action-space	finetune	—	880M	S	apache-2.0	2toINF	0	120

Repository changelog

20 commits

Repository changelog 20 commits

Date	Commit	SHA
2025-08-04	Update configuration_florence2.py (#62)	21a599d
2025-08-04	Fix: resize_token_embeddings's Interface for transformers v4.49.0 (#97)	6e29045
2025-08-04	Fix weight init, deprecation warnings, safetensors (#103)	a935c70
2024-12-08	initial tuned 4k context length model (#84)	00d2f15
2024-11-15	add_confidence_score (#56)	f0acedb
2024-11-04	Inherit from GenerationMixin (#80)	f928440
2024-08-21	Fix invalid escape sequences (#65)	39ddb41
2024-07-20	update_model_init_fp16 (#53)	6bf1792
2024-07-19	update_ocr_postprocess (#52)	df93020
2024-07-01	Update pipeline tag	15aa04e
2024-06-30	Fix modeling code (typos/bugs) (#12)	484465f
2024-06-26	Fix runon lines (#16)	aec3bc5
2024-06-26	Fix incorrect bos_token, eos_token, and pad_token ids in config.json (#17)	8651614
2024-06-25	add_caption_grounding (#23)	ef29c9b
2024-06-18	Update README.md	6e0aa88
2024-06-18	update_readme_example (#5)	0f7c938
2024-06-18	add_grounding_example (#3)	9509157
2024-06-18	add_grounding_example (#2)	0c7be59
2024-06-15	add_model (#1)	33ee8e1
2024-06-15	initial commit	77dfafb

Articles and coverage

14 pieces

Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.

Analysis10

[2311.06242] Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks

ar5iv.labs.arxiv.orgauthorityabout the Florence 2 family15,794 words

Florence-2 is a novel vision foundation model utilizing a unified, prompt-based representation for diverse computer vision and vision-language tasks. The model was trained on FLD-5B, which consists of 5.4 billion comprehensive visual annotations on 126 million images using an iterative strategy of automated image annotation and model refinement. It employs a sequence-to-sequence structure to achieve unprecedented zero-shot and fine-tuning capabilities across tasks like captioning, object detection, grounding, and segmentation.written, reviewed, cited

The model improves training efficiency by 4 times compared to pre-trained models on ImageNet.
The pre-trained backbone surpasses supervised and self-supervised models on ADE20K semantic segmentation.
Task activation is achieved through textual prompts, reflecting the approach used by Large Language Models.

Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks

arxiv.org2023-11-13authorityabout the Florence 2 familyHN 1 pts · 0 comments345 words

Florence-2 is a novel vision foundation model featuring a unified, prompt-based representation for diverse computer vision and vision-language tasks. The model utilizes a sequence-to-sequence structure trained on FLD-5B, which consists of 5.4 billion comprehensive visual annotations on 126 million images. Extensive evaluations demonstrate the model's strong zero-shot and fine-tuning capabilities across numerous tasks.written, reviewed, cited

Playing with Microsoft Florence 2

objectgraph.com2024-08-06about the Florence 2 familyHN 5 pts · 0 comments214 words

Microsoft released Florence 2, a model capable of image captioning, segmentation, and OCR that runs on CPU and GPU. The author created a playground space on Hugging Face for testing the microsoft/Florence-2-large-ft model. A patch is required because the latest transformers version tries to import flash attention on Mac M machines and fails.written, reviewed, cited

Show HN: Florence2-Sharp – Advanced Image Understanding and OCR in C#

github.com2024-07-25authorityabout the Florence 2 familyHN 1 pts · 0 comments529 words

A new C# library provides .NET developers with a clean API to run Microsoft's Florence-2-base models locally without requiring Python. It supports multiple image understanding tasks including captioning, OCR, and object detection through automatic downloads of Florence-2-base ONNX models. The project is available as a NuGet package under an MIT license.written, reviewed, cited

Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks

openaccess.thecvf.comabout the Florence 2 family8,268 words

Microsoft introduces Florence-2, a vision foundation model using a unified, prompt-based representation for various computer vision and vision-language tasks. The model utilizes a sequence-to-sequence structure trained on the FLD-5B dataset, which consists of 5.4 billion comprehensive visual annotations on 126 million images. Extensive evaluations demonstrate Florence-2 to be a strong contender with unprecedented zero-shot and fine-tuning capabilities.written, reviewed, cited

The FLD-5B dataset generation engine uses multiple models to reach a consensus on image annotations.
All annotations in the FLD-5B dataset are uniformly standardized into textual outputs for consistent optimization.
Task activation in Florence-2 is achieved through textual prompts reflecting the approach used by Large Language Models.

Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks

ieeexplore.ieee.orgabout the Florence 2 family18 words

Florence-2 — Edgepedia

edgechat.aiabout the Florence 2 family913 words

Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks — Paper Graph

paper-graph-ui.vercel.appabout the Florence 2 family

Florence-2: MIT Open Source Vision Foundation Model by Microsoft

blog.roboflow.com2024-06-20about the Florence 2 familyHN 4 pts · 0 comments724 words

Microsoft released Florence-2, a lightweight vision-language model under the MIT license that unifies over ten tasks like captioning and segmentation. The model relies on the FLD-5B dataset, which contains 126 million images and 5.4 billion annotations, to achieve performance comparable to much larger models. It comes in two variants with 0.23 billion and 0.77 billion parameters, enabling deployment on mobile devices or an NVIDIA T4 GPU.written, reviewed, cited

The FLD-5B dataset is not yet publicly available and was announced for upcoming release during CVPR 2024.
Running Florence-2 on an NVIDIA T4 GPU yields inferences taking approximately 1 second per image.

How Microsoft's New Large Vision Model "Florence-2" Works

assemblyai.com2024-07-15about the Florence 2 familyHN 1 pts · 0 comments2,037 words

Microsoft released Florence-2, a compact Large Vision Model that unifies image-language tasks like captioning and object detection. The model trains on FLD-5B, an open-source dataset containing 5.4 billion annotations on 126 million images, and encodes images using a DaViT encoder into visual tokens. It processes location information via special tokens <loc_x><loc_y> where coordinates have a 0.1% resolution, enabling unified training without task-specific heads.written, reviewed, cited

Florence-2 uses a multi-step engine where the model generates pseudolabels, filters them for quality, and iteratively mixes them back into training data.
The model achieves consensus on original pseudolabels by having many different specialized models work together to generate the final labels.

All coverage tracked for this model (14) — append-only, nothing is ever removed

Title	Source	Published	First seen	Signals
[2311.06242] Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks	ar5iv.labs.arxiv.org	—	2026-10-07	exa
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks	openaccess.thecvf.com	—	2026-10-07	exa
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks	ieeexplore.ieee.org	—	2026-10-07	exa
Florence-2 — Edgepedia	edgechat.ai	—	2026-10-07	exa
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks — Paper Graph	paper-graph-ui.vercel.app	—	2026-10-07	exa
Papers Explained 214: Florence-2. While existing large vision models… \| by Ritvik Rastogi \| Medium	ritvik19.medium.com	2024-09-19	2026-10-07	exa
Playing with Microsoft Florence 2	objectgraph.com	2024-08-06	2026-10-07	hn
Show HN: Florence2-Sharp – Advanced Image Understanding and OCR in C#	github.com	2024-07-25	2026-10-07	hn
How Microsoft's New Large Vision Model "Florence-2" Works	assemblyai.com	2024-07-15	2026-10-07	hn
Florence-2: Multimodal Vision Model \| MaximoFN	maximofn.com	2024-06-23	2026-10-07	exa
Florence-2: MIT Open Source Vision Foundation Model by Microsoft	blog.roboflow.com	2024-06-20	2026-10-07	hn
Florence-2: A Unified Model for Many Vision Tasks \| Abhik Sarkar	abhik.ai	2024-01-10	2026-10-07	exa
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks	arxiv.org	2023-11-13	2026-10-07	hn
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks [Quick Review]	liner.com	2023-11-10	2026-10-07	exa

Papers

1 paper

Papers 1

arXiv links found in the model card and its tags. "Peer-reviewed" is shown only with evidence: a published venue in OpenAlex (measured) or the authors' own journal reference on arXiv (reported).

Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks this model

Bin Xiao, Haiping Wu, Weijian Xu +6 · 2023-11-10 · PDF · HF paper page (98 upvotes)

Similar models

3 models

Similar models

Qwen3.5-0.8B · 873M
same size band (S), same task (image-text-to-text), more providers, permissive license
GOT-OCR2_0 · 716M
same size band (S), same task (image-text-to-text), permissive license
t5gemma-2-270m-270m · 786M
same size band (S), same task (image-text-to-text)

Sources

3 sources

Sources

Hub snapshot: 2026-10-25 · HF API: 2026-10-24 21:35 UTC · Providers: 2026-10-24 21:44 UTC · Articles: 2026-10-24 21:45 UTC · Summary: 2026-10-07 15:01 UTC · Last verified 2026-10-25

Follow Florence-2-large

Watch this model

/feeds/models/microsoft--Florence-2-large.xml(RSS 2.0) · every change to Florence-2-large
/feeds/families/microsoft--Florence-2.xml(RSS 2.0) · every change across the Florence-2 family

Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.