Model Follower: Understand images and video

Inkling-Small: Author reports mmmu_pro_standard_10_options 74

Sat, 24 Oct 2026 21:39:08 GMT

thinkingmachines/Inkling-Small has author-reported evidence for "Understand images and video": Author reports mmmu_pro_standard_10_options 74. Source: https://huggingface.co/thinkingmachines/Inkling-Small/raw/main/.eval_results/mmmu_pro.yaml. Observed at 2026-10-24T21:39:08Z.

Kimi-K2.6: Author reports mmmu_pro_vision 79.4

Sat, 24 Oct 2026 21:32:01 GMT

moonshotai/Kimi-K2.6 has author-reported evidence for "Understand images and video": Author reports mmmu_pro_vision 79.4. Source: https://huggingface.co/moonshotai/Kimi-K2.6/raw/main/.eval_results/mmmu_pro.yaml. Observed at 2026-10-24T21:32:01Z.

MiniMax-M3: Author reports mmmu_pro_standard_10_options 78.1

Sat, 24 Oct 2026 21:31:59 GMT

MiniMaxAI/MiniMax-M3 has author-reported evidence for "Understand images and video": Author reports mmmu_pro_standard_10_options 78.1 (MMMU-Pro score extracted from the model card benchmark graph. The exact task variant (vision, standard 4-options, or standard 10-options) is not explicitly stated.). Source: https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/.eval_results/minimax-m3.yaml. Observed at 2026-10-24T21:31:59Z.

Eikos-27B-FP8: Author reports MMStar 70.7%

Wed, 07 Oct 2026 20:27:02 GMT

caiovicentino1/Eikos-27B-FP8 has author-reported evidence for "Understand images and video": Author reports MMStar 70.7% (Zero-shot). Source: https://huggingface.co/caiovicentino1/Eikos-27B-FP8/raw/main/README.md. Observed at 2026-10-07T20:27:02.451825+00:00.

MiniCPM-V-4_5: Author reports Video-MME 73.5

Wed, 07 Oct 2026 20:24:24 GMT

openbmb/MiniCPM-V-4_5 has author-reported evidence for "Understand images and video": Author reports Video-MME 73.5 (8.7B). Source: https://huggingface.co/openbmb/MiniCPM-V-4_5/raw/main/README.md. Observed at 2026-10-07T20:24:24.708922+00:00.

EXAONE-4.5-33B: Author reports MMMU 78.7

Wed, 07 Oct 2026 20:23:59 GMT

LGAI-EXAONE/EXAONE-4.5-33B has author-reported evidence for "Understand images and video": Author reports MMMU 78.7 (EXAONE 4.5 33B (Reasoning)). Source: https://huggingface.co/LGAI-EXAONE/EXAONE-4.5-33B/raw/main/README.md. Observed at 2026-10-07T20:23:59.146267+00:00.

Mage-VL: Author reports AI2D w/ Mask 83.16

Wed, 07 Oct 2026 20:23:55 GMT

microsoft/Mage-VL has author-reported evidence for "Understand images and video": Author reports AI2D w/ Mask 83.16 (Mage-VL-4B). Source: https://huggingface.co/microsoft/Mage-VL/raw/main/README.md. Observed at 2026-10-07T20:23:55.322190+00:00.

Kimi-VL-A3B-Thinking-2506: Author reports MMBench-EN-v1.1 (Acc) 84.4

Wed, 07 Oct 2026 20:23:39 GMT

moonshotai/Kimi-VL-A3B-Thinking-2506 has author-reported evidence for "Understand images and video": Author reports MMBench-EN-v1.1 (Acc) 84.4. Source: https://huggingface.co/moonshotai/Kimi-VL-A3B-Thinking-2506/raw/main/README.md. Observed at 2026-10-07T20:23:39.433875+00:00.

Florence-2-large-ft: Author reports VQAv2 test-dev Acc 81.7

Wed, 07 Oct 2026 20:23:37 GMT

microsoft/Florence-2-large-ft has author-reported evidence for "Understand images and video": Author reports VQAv2 test-dev Acc 81.7 (Florence-2-large-ft). Source: https://huggingface.co/microsoft/Florence-2-large-ft/raw/main/README.md. Observed at 2026-10-07T20:23:37.765825+00:00.

Step3-VL-10B: Author reports MMMU 80.11

Wed, 07 Oct 2026 20:23:35 GMT

stepfun-ai/Step3-VL-10B has author-reported evidence for "Understand images and video": Author reports MMMU 80.11 (PaCoRe). Source: https://huggingface.co/stepfun-ai/Step3-VL-10B/raw/main/README.md. Observed at 2026-10-07T20:23:35.103901+00:00.

Florence-2-base-ft: Author reports VQAv2 test-dev Acc 79.7

Wed, 07 Oct 2026 20:23:20 GMT

microsoft/Florence-2-base-ft has author-reported evidence for "Understand images and video": Author reports VQAv2 test-dev Acc 79.7 (Florence-2-base-ft). Source: https://huggingface.co/microsoft/Florence-2-base-ft/raw/main/README.md. Observed at 2026-10-07T20:23:20.963985+00:00.

LFM2.5-VL-1.6B: Author reports MMStar 50.67

Wed, 07 Oct 2026 20:23:16 GMT

LiquidAI/LFM2.5-VL-1.6B has author-reported evidence for "Understand images and video": Author reports MMStar 50.67. Source: https://huggingface.co/LiquidAI/LFM2.5-VL-1.6B/raw/main/README.md. Observed at 2026-10-07T20:23:16.936967+00:00.

LFM2.5-VL-450M: Author reports MMStar 43.00

Wed, 07 Oct 2026 20:23:15 GMT

LiquidAI/LFM2.5-VL-450M has author-reported evidence for "Understand images and video": Author reports MMStar 43.00. Source: https://huggingface.co/LiquidAI/LFM2.5-VL-450M/raw/main/README.md. Observed at 2026-10-07T20:23:15.249332+00:00.

paligemma2-3b-pt-224: Author reports [AI2D][ai2d] 84.6

Wed, 07 Oct 2026 20:22:52 GMT

google/paligemma2-3b-pt-224 has author-reported evidence for "Understand images and video": Author reports [AI2D][ai2d] 84.6 (448-28B). Source: https://huggingface.co/google/paligemma2-3b-pt-224/raw/main/README.md. Observed at 2026-10-07T20:22:52.873751+00:00.

QVQ-72B-Preview: Author reports MMMU(val) 70.3

Wed, 07 Oct 2026 20:22:49 GMT

Qwen/QVQ-72B-Preview has author-reported evidence for "Understand images and video": Author reports MMMU(val) 70.3. Source: https://huggingface.co/Qwen/QVQ-72B-Preview/raw/main/README.md. Observed at 2026-10-07T20:22:49.192734+00:00.

Qwen2-VL-72B-Instruct: Author reports RealWorldQA 77.8

Wed, 07 Oct 2026 20:22:39 GMT

Qwen/Qwen2-VL-72B-Instruct has author-reported evidence for "Understand images and video": Author reports RealWorldQA 77.8. Source: https://huggingface.co/Qwen/Qwen2-VL-72B-Instruct/raw/main/README.md. Observed at 2026-10-07T20:22:39.572000+00:00.

gemma-4-26B-A4B-it-assistant: Author reports MMMU Pro 73.8%

Wed, 07 Oct 2026 20:22:38 GMT

google/gemma-4-26B-A4B-it-assistant has author-reported evidence for "Understand images and video": Author reports MMMU Pro 73.8% (Gemma 4 26B A4B). Source: https://huggingface.co/google/gemma-4-26B-A4B-it-assistant/raw/main/README.md. Observed at 2026-10-07T20:22:38.751487+00:00.

Idefics3-8B-Llama3: Author reports MMMU
(val) 46.6

Wed, 07 Oct 2026 20:22:37 GMT

HuggingFaceM4/Idefics3-8B-Llama3 has author-reported evidence for "Understand images and video": Author reports MMMU
(val) 46.6 (Idefics3-8B). Source: https://huggingface.co/HuggingFaceM4/Idefics3-8B-Llama3/raw/main/README.md. Observed at 2026-10-07T20:22:37.724508+00:00.

NVLM-D-72B: Author reports MMMU (val / test) 58.7 / 54.9

Wed, 07 Oct 2026 20:22:35 GMT

nvidia/NVLM-D-72B has author-reported evidence for "Understand images and video": Author reports MMMU (val / test) 58.7 / 54.9 (NVLM-D 1.0 72B (Huggingface)). Source: https://huggingface.co/nvidia/NVLM-D-72B/raw/main/README.md. Observed at 2026-10-07T20:22:35.886125+00:00.

Llama-3.1-Nemotron-Nano-VL-8B-V1: Author reports MMMU Val with chatGPT as a judge 48.2%

Wed, 07 Oct 2026 20:22:34 GMT

nvidia/Llama-3.1-Nemotron-Nano-VL-8B-V1 has author-reported evidence for "Understand images and video": Author reports MMMU Val with chatGPT as a judge 48.2%. Source: https://huggingface.co/nvidia/Llama-3.1-Nemotron-Nano-VL-8B-V1/raw/main/README.md. Observed at 2026-10-07T20:22:34.092116+00:00.

Kimi-VL-A3B-Thinking: Author reports MathVision (full) (Pass@1) 36.8

Wed, 07 Oct 2026 20:22:29 GMT

moonshotai/Kimi-VL-A3B-Thinking has author-reported evidence for "Understand images and video": Author reports MathVision (full) (Pass@1) 36.8 (Kimi-VL-Thinking-A3B). Source: https://huggingface.co/moonshotai/Kimi-VL-A3B-Thinking/raw/main/README.md. Observed at 2026-10-07T20:22:29.984710+00:00.

SmolVLM2-2.2B-Instruct: Author reports Mathvista 51.5

Wed, 07 Oct 2026 20:22:27 GMT

HuggingFaceTB/SmolVLM2-2.2B-Instruct has author-reported evidence for "Understand images and video": Author reports Mathvista 51.5 (SmolVLM2 2.2B). Source: https://huggingface.co/HuggingFaceTB/SmolVLM2-2.2B-Instruct/raw/main/README.md. Observed at 2026-10-07T20:22:27.316359+00:00.

Kimi-VL-A3B-Instruct: Author reports MMMU-Val (Pass@1) 61.7

Wed, 07 Oct 2026 20:21:24 GMT

moonshotai/Kimi-VL-A3B-Instruct has author-reported evidence for "Understand images and video": Author reports MMMU-Val (Pass@1) 61.7 (Kimi-VL-A3B-Thinking). Source: https://huggingface.co/moonshotai/Kimi-VL-A3B-Instruct/raw/main/README.md. Observed at 2026-10-07T20:21:24.726912+00:00.

gemma-4-31B-it-assistant: Author reports MMMU Pro 76.9%

Wed, 07 Oct 2026 20:21:17 GMT

google/gemma-4-31B-it-assistant has author-reported evidence for "Understand images and video": Author reports MMMU Pro 76.9% (Gemma 4 31B). Source: https://huggingface.co/google/gemma-4-31B-it-assistant/raw/main/README.md. Observed at 2026-10-07T20:21:17.344339+00:00.

gemma-4-12B: Author reports MMMU Pro 69.1%

Wed, 07 Oct 2026 20:21:05 GMT

google/gemma-4-12B has author-reported evidence for "Understand images and video": Author reports MMMU Pro 69.1% (Gemma 4 12B Unified). Source: https://huggingface.co/google/gemma-4-12B/raw/main/README.md. Observed at 2026-10-07T20:21:05.712743+00:00.

Qianfan-OCR: Author reports ChartQA 88.1

Wed, 07 Oct 2026 20:20:57 GMT

baidu/Qianfan-OCR has author-reported evidence for "Understand images and video": Author reports ChartQA 88.1. Source: https://huggingface.co/baidu/Qianfan-OCR/raw/main/README.md. Observed at 2026-10-07T20:20:57.718951+00:00.

gemma-4-E2B: Author reports MMMU Pro 44.2%

Wed, 07 Oct 2026 20:20:29 GMT

google/gemma-4-E2B has author-reported evidence for "Understand images and video": Author reports MMMU Pro 44.2% (Gemma 4 E2B). Source: https://huggingface.co/google/gemma-4-E2B/raw/main/README.md. Observed at 2026-10-07T20:20:29.538589+00:00.

gemma-4-26B-A4B: Author reports MMMU Pro 73.8%

Wed, 07 Oct 2026 20:20:11 GMT

google/gemma-4-26B-A4B has author-reported evidence for "Understand images and video": Author reports MMMU Pro 73.8% (Gemma 4 26B A4B). Source: https://huggingface.co/google/gemma-4-26B-A4B/raw/main/README.md. Observed at 2026-10-07T20:20:11.847069+00:00.

Florence-2-base: Author reports Refcocog val Accuracy 66.3

Wed, 07 Oct 2026 20:19:58 GMT

microsoft/Florence-2-base has author-reported evidence for "Understand images and video": Author reports Refcocog val Accuracy 66.3 (Florence-2-base). Source: https://huggingface.co/microsoft/Florence-2-base/raw/main/README.md. Observed at 2026-10-07T20:19:58.335540+00:00.

gemma-4-E4B: Author reports MMMU Pro 52.6%

Wed, 07 Oct 2026 20:19:55 GMT

google/gemma-4-E4B has author-reported evidence for "Understand images and video": Author reports MMMU Pro 52.6% (Gemma 4 E4B). Source: https://huggingface.co/google/gemma-4-E4B/raw/main/README.md. Observed at 2026-10-07T20:19:55.830091+00:00.

diffusiongemma-26B-A4B-it: Author reports MMMU Pro 54.3%

Wed, 07 Oct 2026 20:19:48 GMT

google/diffusiongemma-26B-A4B-it has author-reported evidence for "Understand images and video": Author reports MMMU Pro 54.3% (DiffusionGemma 26B A4B). Source: https://huggingface.co/google/diffusiongemma-26B-A4B-it/raw/main/README.md. Observed at 2026-10-07T20:19:48.067859+00:00.

Qwen2-VL-2B-Instruct: Author reports RealWorldQA 62.9

Wed, 07 Oct 2026 20:19:40 GMT

Qwen/Qwen2-VL-2B-Instruct has author-reported evidence for "Understand images and video": Author reports RealWorldQA 62.9. Source: https://huggingface.co/Qwen/Qwen2-VL-2B-Instruct/raw/main/README.md. Observed at 2026-10-07T20:19:40.274493+00:00.

Florence-2-large: Author reports Refcocog val Accuracy 68.0

Wed, 07 Oct 2026 20:19:31 GMT

microsoft/Florence-2-large has author-reported evidence for "Understand images and video": Author reports Refcocog val Accuracy 68.0 (Florence-2-large). Source: https://huggingface.co/microsoft/Florence-2-large/raw/main/README.md. Observed at 2026-10-07T20:19:31.598627+00:00.

gemma-4-31B: Author reports MMMU Pro 76.9%

Wed, 07 Oct 2026 20:19:22 GMT

google/gemma-4-31B has author-reported evidence for "Understand images and video": Author reports MMMU Pro 76.9% (Gemma 4 31B). Source: https://huggingface.co/google/gemma-4-31B/raw/main/README.md. Observed at 2026-10-07T20:19:22.456254+00:00.

Qwen2-VL-7B-Instruct: Author reports RealWorldQA 70.1

Wed, 07 Oct 2026 20:19:22 GMT

Qwen/Qwen2-VL-7B-Instruct has author-reported evidence for "Understand images and video": Author reports RealWorldQA 70.1. Source: https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct/raw/main/README.md. Observed at 2026-10-07T20:19:22.003834+00:00.

Muse-Glimmer-30B: Author reports MMMU Pro 74

Wed, 07 Oct 2026 20:19:12 GMT

meta-models/Muse-Glimmer-30B has author-reported evidence for "Understand images and video": Author reports MMMU Pro 74 (High Reasoning). Source: https://huggingface.co/meta-models/Muse-Glimmer-30B/raw/main/README.md. Observed at 2026-10-07T20:19:12.040914+00:00.

Inkling: Author reports MMMU Pro (Standard 10) 73.5%

Wed, 07 Oct 2026 20:19:05 GMT

thinkingmachines/Inkling has author-reported evidence for "Understand images and video": Author reports MMMU Pro (Standard 10) 73.5%. Source: https://huggingface.co/thinkingmachines/Inkling/raw/main/README.md. Observed at 2026-10-07T20:19:05.702369+00:00.

Qwen2.5-VL-32B-Instruct: Author reports MMMU 70

Wed, 07 Oct 2026 20:18:28 GMT

Qwen/Qwen2.5-VL-32B-Instruct has author-reported evidence for "Understand images and video": Author reports MMMU 70. Source: https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct/raw/main/README.md. Observed at 2026-10-07T20:18:28.397309+00:00.

gemma-4-E2B-it: Author reports MMMU Pro 44.2%

Wed, 07 Oct 2026 20:18:23 GMT

google/gemma-4-E2B-it has author-reported evidence for "Understand images and video": Author reports MMMU Pro 44.2% (Gemma 4 E2B). Source: https://huggingface.co/google/gemma-4-E2B-it/raw/main/README.md. Observed at 2026-10-07T20:18:23.721758+00:00.

gemma-4-12B-it: Author reports MMMU Pro 69.1%

Wed, 07 Oct 2026 20:18:18 GMT

google/gemma-4-12B-it has author-reported evidence for "Understand images and video": Author reports MMMU Pro 69.1% (Gemma 4 12B Unified). Source: https://huggingface.co/google/gemma-4-12B-it/raw/main/README.md. Observed at 2026-10-07T20:18:18.754019+00:00.

Qwen2.5-VL-72B-Instruct: Author reports MMMU_Pro 51.1

Wed, 07 Oct 2026 20:17:56 GMT

Qwen/Qwen2.5-VL-72B-Instruct has author-reported evidence for "Understand images and video": Author reports MMMU_Pro 51.1. Source: https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct/raw/main/README.md. Observed at 2026-10-07T20:17:56.062619+00:00.

Qwen2.5-VL-7B-Instruct: Author reports MMStar 63.9

Wed, 07 Oct 2026 20:17:17 GMT

Qwen/Qwen2.5-VL-7B-Instruct has author-reported evidence for "Understand images and video": Author reports MMStar 63.9. Source: https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct/raw/main/README.md. Observed at 2026-10-07T20:17:17.042337+00:00.

gemma-4-E4B-it: Author reports MMMU Pro 52.6%

Wed, 07 Oct 2026 20:17:16 GMT

google/gemma-4-E4B-it has author-reported evidence for "Understand images and video": Author reports MMMU Pro 52.6% (Gemma 4 E4B). Source: https://huggingface.co/google/gemma-4-E4B-it/raw/main/README.md. Observed at 2026-10-07T20:17:16.087842+00:00.

Kimi-K2.5: Author reports MMMU-Pro 78.5

Wed, 07 Oct 2026 20:16:46 GMT

moonshotai/Kimi-K2.5 has author-reported evidence for "Understand images and video": Author reports MMMU-Pro 78.5 (Thinking). Source: https://huggingface.co/moonshotai/Kimi-K2.5/raw/main/README.md. Observed at 2026-10-07T20:16:46.965549+00:00.

DeepSeek-V4.1-Flash: Author reports MMMU-Pro (EM) 56.5

Wed, 07 Oct 2026 20:15:43 GMT

deepseek-ai/DeepSeek-V4.1-Flash has author-reported evidence for "Understand images and video": Author reports MMMU-Pro (EM) 56.5 (DeepSeek-V4.1-Flash-Base). Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/raw/main/README.md. Observed at 2026-10-07T20:15:43.203420+00:00.

gemma-4-31B-it: Author reports MMMU Pro 76.9%

Wed, 07 Oct 2026 20:15:39 GMT

google/gemma-4-31B-it has author-reported evidence for "Understand images and video": Author reports MMMU Pro 76.9% (Gemma 4 31B). Source: https://huggingface.co/google/gemma-4-31B-it/raw/main/README.md. Observed at 2026-10-07T20:15:39.011299+00:00.

gemma-4-26B-A4B-it: Author reports MMMU Pro 73.8%

Wed, 07 Oct 2026 20:14:26 GMT

google/gemma-4-26B-A4B-it has author-reported evidence for "Understand images and video": Author reports MMMU Pro 73.8% (Gemma 4 26B A4B). Source: https://huggingface.co/google/gemma-4-26B-A4B-it/raw/main/README.md. Observed at 2026-10-07T20:14:26.358568+00:00.