Home › Method
Ranking method scoring v1
The score decides what is on the watch list and the order of the tables. It is a weighted sum of log-scaled popularity signals, hosting breadth, and a curated org reputation. It is not a quality judgment: a high score means "widely used and widely hosted", nothing more.
Formula
score = Σ weight(signal) × value(signal)Candidate score = weighted sum of log-scaled signals + curated org reputation + boosts. Signals marked stub are wired but return 0 until a source exists. Weights are reviewable; bump version on change.
Signals and weights
| Signal | Weight | Value | Status |
|---|---|---|---|
| likes_log | 3 | log(1 + Hub likes) | live |
| downloads_alltime_log | 2 | log(1 + all-time Hub downloads) | live |
| downloads_30d_log | 1.5 | log(1 + downloads in the last 30 days) | live |
| trending_log | 2 | log(1 + Hub trending score) | live |
| providers_count | 4 | number of distinct hosted providers observed (HF inference providers, OpenRouter, DeepInfra, Parasail) | live |
| org_reputation | 1 | curated 0–10 org reputation (engine/config/orgs.json) | live |
| recency_boost | 3 | 1 if created within the last 30 days | live |
| paper_linked | 3 | 1 if the card links an arXiv paper | live |
| eval_results_present | 2 | 1 if the repo ships .eval_results files | live |
| top500_likes_member | 2 | 1 if in the Hub's top-500 by likes | live |
| top500_downloads_member | 2 | 1 if in the Hub's top-500 by downloads | live |
| google_trends_interest | 0 | search-interest signal | not yet live |
| public_benchmark_score | 0 | public leaderboard score | not yet live |
Signals marked "not yet live" are wired with weight 0/0 and return 0 until a source exists. Weights are reviewable; the version bumps on any change and every page shows the version it was ranked with.
Sections
| Section | Candidates | Limit |
|---|---|---|
| llm | pipeline tags: text-generation, image-text-to-text, any-to-any, text2text-generation | 500 |
| embedding | pipeline tags: feature-extraction, sentence-similarity, text-ranking | 60 |
| decision | curated source: data/curated/decision-models-laya-2026-10-05.csv | — |
| moe | flag over llm section; a model is MoE if config has routed-expert keys, tags include moe, or name matches A\d+B | — |
Rank is the position inside a section (an MoE model has one rank in LLMs and another in MoE). Ties keep snapshot order.
What the score ignores
- Benchmark quality: model-author-reported numbers are shown on pages but do not enter the score until a public benchmark signal goes live.
- Price and latency: shown per provider row; not scored.
- Our own measurements: none exist yet; when they do they will be labeled measured and dated.