Florence-2-large-ft

770M parameters, images and text in, text out, 1K tokens of context.

Verified 2026-10-25 Rank #514 in LLMs · score 76 method v1 Sources 3 Downloads 30d 46.8k · #389 of 619 tracked

Can I run it?0.5GB at INT4Estimatefits a 16 GB cardmethod

Can I use it?MITPermissivelicence file2026-10-24

Do people use it?46.8kdownloads, 30 days#389 of 619 we track · below the median+2 likes, downloads −5% this week397 likesHugging Face2026-10-24

Worth reading

Picked by rule from 5 tracked pieces: the lab's own post, the most shared, a field report, the latest.

Latest · 2 years ago

Playing with Microsoft Florence 2

Microsoft released Florence 2, a model capable of image captioning, segmentation, and OCR that runs on CPU and GPU.

objectgraph.com 2024-08-06 5 HN pts 214 words about the Florence 2 family family

5 best of 5 tracked

The 1 pick above, plus these 4. Top is the radar's score: discussion, source authority and the lab's own posts, fading with age.

PaperFlorence-2: Advancing a Unified Representation for a Variety of Vision Tasksarxiv.org · 2023-11-13 · 1 HN pts
CodeShow HN: Florence2-Sharp – Advanced Image Understanding and OCR in C#github.com · 2024-07-25 · 1 HN pts
ArticleFlorence-2: MIT Open Source Vision Foundation Model by Microsoftblog.roboflow.com · 2024-06-20 · 4 HN pts
ArticleHow Microsoft's New Large Vision Model "Florence-2" Worksassemblyai.com · 2024-07-15 · 1 HN pts

Where it's used

Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.

No API provider serves it, so its downloads are people running it themselves: 1 quantized builds and 64 Spaces.

As a service0no API provider listed
On own hardware1quantized builds
Built on119 fine-tunes · 2 adapters · 0 merges
In apps64Hugging Face Spaces

Top Spaces running it: gokaygokay/Florence-2 ♥ 848 · SkalskiP/better-florence-2 ♥ 198 · SixOpen/Florence-2-large-ft ♥ 44 · DamarJati/Remove-watermark ♥ 39

Will it fit?

Weights only, an estimate: leave room for the KV cache. Method

Fits a 16 GB card at INT4

About 0.5 GB at INT4, 0.9 GB at FP8, 1.8 GB at BF16: weights only, so leave room for the KV cache.

Estimate params × bytes per param × 1.2 · how we estimate

The story so far

Weights, every dated article, and repo commits on one rail.

5 dated articles and 11 repo commits so far

Teal is the weights (2024-06-15), amber triangles are 5 dated articles (each links to it), grey ticks are repo commits.

What people built on it

12 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.

12 derivatives on the Hub, most of them fine-tunes

From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.

All variants →

Reference

Everything else, folded: the full tables, every source.

Benchmarks

6 rows

Benchmarks 6 rows

measured third-party leaderboard rowauthor-reported number from the model card or eval files (6)No confidence intervals unless the source gave them; settings differ between rows, so compare within a benchmark, not across. Date = the benchmark's own date when it gives one, else when we read it.

Benchmark	Value	Setting / notes	Reported by	Date
author-reported COCO Caption Karpathy test CIDEr	143.3	Florence-2-large-ft	model author	2026-10-07
author-reported NoCaps val CIDEr	124.9	Florence-2-large-ft	model author	2026-10-07
author-reported TextCaps val CIDEr	151.1	Florence-2-large-ft	model author	2026-10-07
author-reported TextVQA · Understand images and video	73.5	Florence-2-large-ft	model author	2026-10-07
author-reported VQAv2 · Understand images and video	81.7	Florence-2-large-ft	model author	2026-10-07
author-reported VizWiz VQA test-dev Acc	72.6	Florence-2-large-ft	model author	2026-10-07

What the card says

1 cited sentence

Summary generated, cited

This finetuned model uses a sequence-to-sequence architecture to perform vision tasks like captioning and object detection.^[1]^[2]

“Finetuned model on a colletion of downstream tasks”
“sequence-to-sequence architecture”

Generated 2026-10-07 15:13 UTC by parasail-gemma-4-26b-a4b-it; each numeral points at the fact it rests on. Not a verdict.

Use-case evidence

2 use cases

What it is good at 2 use cases

Labels are evidence, not verdicts: measured a third-party leaderboard row for this model; author-reported a number the model author published in the card or eval files; structural a deterministic signal from the name, config or chat template, or a decision-model estimate. Each chip links to its source. All use cases.

Facts

21 fields

Listed in LLMs

Facts

Florence-2-large-ft has 770M parameters. (safetensors metadata, observed 2026-10-24)
Context length is 1,024 tokens (1K). (config.json max_position_embeddings, observed 2026-10-24)
License: MIT (friction: none). Permissive. (license file, observed 2026-10-24)
Created 2024-06-15, last modified 2025-08-04. (HF API, observed 2026-10-24)

Pipeline tag	image-text-to-text
Library	transformers
Architectures	Florence2ForConditionalGeneration
Model type	florence2
Dtype	float16
Params by dtype	F16: 770,430,041
Size band	S
MoE	no
Context length	1,024 (config.json max_position_embeddings)
Papers	arXiv:2311.06242
License	MIT (mit)
Hub snapshot	2026-10-25

Servability

vLLM · SGLang

Servability

vLLM: not in registry · Florence2ForConditionalGeneration · 7050ef881ff0835f6890a47db1859a24c65481d6SGLang: not in registry · 773cb938fe9484a8f7e7e46fb1b1fe96412c59ed

vLLM registry (7050ef881ff0835f6890a47db1859a24c65481d6) · SGLang registry (773cb938fe9484a8f7e7e46fb1b1fe96412c59ed) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.

Hardware fit

3 estimates

Hardware fit ESTIMATE

Weights-only memory for Florence-2-large-ft: params × bytes/param × 1.2 overhead. KV cache and activations are extra. How this is computed.

BF16 (2 bytes/param)	1.8 GB · Edge (<4 GB)
FP8 (1 byte/param)	0.9 GB · Edge (<4 GB)
INT4 (0.5 bytes/param)	0.5 GB · Edge (<4 GB)

Weights only (params × bytes/param × 1.2 overhead); KV cache and activations extra.

All variants

12 variants

Lineage

Original model Family: microsoft/Florence-2 12 derivatives on the Hub

finetune: 9quantized: 1adapter: 2

Top variants (12 shown, ranked by downloads)

Variant	Relation	Quant	Params	Band	License	Author	Likes	Downloads
congruency/florence-2-large-ft-moredetailed	finetune	—	823M	S	mit	congruency	16	14.4k
yifeihu/Florence-2-DocLayNet-Fixed	finetune	—	823M	S	apache-2.0	yifeihu	11	14.3k
onnx-community/Florence-2-large-ft	quantized	—	—	—	mit	onnx-community	8	12.5k
ucsahin/Florence-2-large-TableDetection	finetune	—	823M	S	mit	ucsahin	19	11k
Musa07/Florence-2-large-FormClassification-ft	finetune	—	823M	S	mit	Musa07	1	1.2k
YxBxRyXJx/Florence-2-OD-COCO	finetune	—	823M	S	mit	YxBxRyXJx	0	306
zhaohungtan1209/created_model	finetune	—	823M	S	mit	zhaohungtan1209	0	234
byh711/table_detection_outputs	finetune	—	823M	S	mit	byh711	0	177
vikas351/florence_final	finetune	—	823M	S	mit	vikas351	0	101
Khabner/florence-large-lora-v1	adapter	—	—	—	mit	Khabner	0	77
MWirelabs/assamese-ocr	finetune	—	—	—	apache-2.0	MWirelabs	1	0
Khabner/florence-large-lora-v22	adapter	—	—	—	mit	Khabner	0	47

Repository changelog

11 commits

Repository changelog 11 commits

Date	Commit	SHA
2025-08-04	Fix weight init, deprecation warnings, safetensors (#38)	4a12a2b
2024-07-20	update_model_init_fp16 (#20)	bb44b80
2024-07-19	update_ocr_postprocess (#19)	13c0e86
2024-07-01	Update pipeline tag	3112cd2
2024-06-30	Fix modeling code (typos/bugs) (#10)	45e7f61
2024-06-26	Fix incorrect bos_token, eos_token, and pad_token ids in config.json (#11)	c669c6b
2024-06-18	Update README.md	f3c2bbf
2024-06-18	add_grounding_example (#3)	9896a44
2024-06-18	add_grounding_example (#2)	8743885
2024-06-15	init_model (#1)	d5b20a4
2024-06-15	initial commit	880fa77

Articles and coverage

5 pieces

Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.

Analysis5

Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks

arxiv.org2023-11-13authorityabout the Florence 2 familyHN 1 pts · 0 comments345 words

Florence-2 is a novel vision foundation model featuring a unified, prompt-based representation for diverse computer vision and vision-language tasks. The model utilizes a sequence-to-sequence structure trained on FLD-5B, which consists of 5.4 billion comprehensive visual annotations on 126 million images. Extensive evaluations demonstrate the model's strong zero-shot and fine-tuning capabilities across numerous tasks.written, reviewed, cited

Playing with Microsoft Florence 2

objectgraph.com2024-08-06about the Florence 2 familyHN 5 pts · 0 comments214 words

Microsoft released Florence 2, a model capable of image captioning, segmentation, and OCR that runs on CPU and GPU. The author created a playground space on Hugging Face for testing the microsoft/Florence-2-large-ft model. A patch is required because the latest transformers version tries to import flash attention on Mac M machines and fails.written, reviewed, cited

Show HN: Florence2-Sharp – Advanced Image Understanding and OCR in C#

github.com2024-07-25authorityabout the Florence 2 familyHN 1 pts · 0 comments529 words

A new C# library provides .NET developers with a clean API to run Microsoft's Florence-2-base models locally without requiring Python. It supports multiple image understanding tasks including captioning, OCR, and object detection through automatic downloads of Florence-2-base ONNX models. The project is available as a NuGet package under an MIT license.written, reviewed, cited

Florence-2: MIT Open Source Vision Foundation Model by Microsoft

blog.roboflow.com2024-06-20about the Florence 2 familyHN 4 pts · 0 comments724 words

Microsoft released Florence-2, a lightweight vision-language model under the MIT license that unifies over ten tasks like captioning and segmentation. The model relies on the FLD-5B dataset, which contains 126 million images and 5.4 billion annotations, to achieve performance comparable to much larger models. It comes in two variants with 0.23 billion and 0.77 billion parameters, enabling deployment on mobile devices or an NVIDIA T4 GPU.written, reviewed, cited

The FLD-5B dataset is not yet publicly available and was announced for upcoming release during CVPR 2024.
Running Florence-2 on an NVIDIA T4 GPU yields inferences taking approximately 1 second per image.

How Microsoft's New Large Vision Model "Florence-2" Works

assemblyai.com2024-07-15about the Florence 2 familyHN 1 pts · 0 comments2,037 words

Microsoft released Florence-2, a compact Large Vision Model that unifies image-language tasks like captioning and object detection. The model trains on FLD-5B, an open-source dataset containing 5.4 billion annotations on 126 million images, and encodes images using a DaViT encoder into visual tokens. It processes location information via special tokens <loc_x><loc_y> where coordinates have a 0.1% resolution, enabling unified training without task-specific heads.written, reviewed, cited

Florence-2 uses a multi-step engine where the model generates pseudolabels, filters them for quality, and iteratively mixes them back into training data.
The model achieves consensus on original pseudolabels by having many different specialized models work together to generate the final labels.

Papers

1 paper

Papers 1

arXiv links found in the model card and its tags. "Peer-reviewed" is shown only with evidence: a published venue in OpenAlex (measured) or the authors' own journal reference on arXiv (reported).

Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks this model

Bin Xiao, Haiping Wu, Weijian Xu +6 · 2023-11-10 · PDF · HF paper page (98 upvotes)

Similar models

3 models

Similar models

GOT-OCR2_0 · 716M
same size band (S), same task (image-text-to-text), permissive license
Qwen3.5-0.8B · 873M
same size band (S), same task (image-text-to-text), more providers, permissive license
t5gemma-2-270m-270m · 786M
same size band (S), same task (image-text-to-text)

Sources

3 sources

Sources

Hub snapshot: 2026-10-25 · HF API: 2026-10-24 21:41 UTC · Providers: 2026-10-24 21:44 UTC · Articles: 2026-10-24 21:46 UTC · Summary: 2026-10-07 15:13 UTC · Last verified 2026-10-25

Follow Florence-2-large-ft

Watch this model

/feeds/models/microsoft--Florence-2-large-ft.xml(RSS 2.0) · every change to Florence-2-large-ft
/feeds/families/microsoft--Florence-2.xml(RSS 2.0) · every change across the Florence-2 family

Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.