The ringer

The ringer

Four joke tests with the same prompts for every model. It is a joke, not a ranking: results are never sorted by quality, never a column on the scoreboard and never a reason for a pick.

The idea is Simon Willison's pelican riding a bicycle. That one is famous enough to be trained for, so here the pelican steals a bicycle from something unrelated, drawn once a month from a fixed list of 30 (THINGS[sha256('YYYY-MM') % 30]). Models released in the same month face the same thing, and nobody can tune for next month's. This month's thing: supermarket. Each output is shown as received, at temperature 0, with its run date and cost.

SVG in our colours test 1 of 4

A pelican stealing a bicycle from this month's thing, as one SVG, using only the four colours #06e0c0, #ffc298, #cd85f9 and #f1f2f2. Check: every hex colour used must be in the palette; the check also counts how many of the four it used.

The exact prompt
Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

October 2026 · the thing: supermarket

  • mistral-large-4-0

    A pelican steals a bicycle from a supermarket

    as received● 4/4 on palette
    SVG reply: a pelican steals a bicycle from a supermarket
    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    pass: 4/4 on palette mistral · run 9 Oct 2026 · $0.049 for all four · released 6 Oct 2026

  • DeepSeek-V4.1-Flash

    A pelican steals a bicycle from a supermarket

    as received✕ no SVG

    No drawable SVG in the reply.

    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    fail: no SVG parasail · run 9 Oct 2026 · $0.036 for all four · released 10 Sep 2026

  • GLM-5.3-Flash

    A pelican steals a bicycle from a supermarket

    as received● 4/4 on palette
    SVG reply: a pelican steals a bicycle from a supermarket
    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    pass: 4/4 on palette parasail · run 9 Oct 2026 · $0.031 for all four · released 25 Aug 2026

  • GLM-5.3

    A pelican steals a bicycle from a supermarket

    low reasoning● 4/4 on palette
    SVG reply: a pelican steals a bicycle from a supermarket
    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    pass: 4/4 on palette parasail · run 9 Oct 2026 · $0.104 for all four · released 25 Aug 2026

  • Qwen3.8-27B

    A pelican steals a bicycle from a supermarket

    low reasoning● 4/4 on palette
    SVG reply: a pelican steals a bicycle from a supermarket
    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    pass: 4/4 on palette parasail · run 9 Oct 2026 · $0.074 for all four · released 5 Aug 2026

  • DeepSeek-V4-Flash-0731

    A pelican steals a bicycle from a supermarket

    as received● 4/4 on palette
    SVG reply: a pelican steals a bicycle from a supermarket
    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    pass: 4/4 on palette parasail · run 9 Oct 2026 · $0.0096 for all four · released 31 Jul 2026

  • GLM-5.2

    A pelican steals a bicycle from a supermarket

    low reasoning✕ empty reply

    No drawable SVG in the reply.

    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    fail: empty reply parasail · run 9 Oct 2026 · $0.157 for all four · released 16 Jun 2026

  • Kimi-K3

    A pelican steals a bicycle from a supermarket

    low reasoning✕ no SVG

    No drawable SVG in the reply.

    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    fail: no SVG parasail · run 9 Oct 2026 · $0.121 for all four · released 13 Jun 2026

  • DeepSeek-V4-Pro

    A pelican steals a bicycle from a supermarket

    low reasoning✕ empty reply

    No drawable SVG in the reply.

    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    fail: empty reply parasail · run 9 Oct 2026 · $0.070 for all four · released 22 Apr 2026

  • DeepSeek-V4-Flash

    A pelican steals a bicycle from a supermarket

    as received● 4/4 on palette
    SVG reply: a pelican steals a bicycle from a supermarket
    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    pass: 4/4 on palette parasail · run 9 Oct 2026 · $0.0093 for all four · released 22 Apr 2026

  • Qwen3.6-35B-A3B

    A pelican steals a bicycle from a supermarket

    as received● 4/4 on palette
    SVG reply: a pelican steals a bicycle from a supermarket
    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    pass: 4/4 on palette parasail · run 9 Oct 2026 · $0.022 for all four · released 15 Apr 2026

  • gemma-4-26B-A4B-it

    A pelican steals a bicycle from a supermarket

    as received● 4/4 on palette
    SVG reply: a pelican steals a bicycle from a supermarket
    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    pass: 4/4 on palette parasail · run 9 Oct 2026 · $0.0069 for all four · released 11 Mar 2026

  • gemma-4-31B-it

    A pelican steals a bicycle from a supermarket

    as received● 4/4 on palette
    SVG reply: a pelican steals a bicycle from a supermarket
    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    pass: 4/4 on palette parasail · run 9 Oct 2026 · $0.0005 for all four · released 11 Mar 2026

  • Qwen3.5-9B

    A pelican steals a bicycle from a supermarket

    as received● 4/4 on palette
    SVG reply: a pelican steals a bicycle from a supermarket
    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    pass: 4/4 on palette parasail · run 9 Oct 2026 · $0.020 for all four · released 27 Feb 2026

  • Qwen3.5-397B-A17B

    A pelican steals a bicycle from a supermarket

    as received● 4/4 on palette
    SVG reply: a pelican steals a bicycle from a supermarket
    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    pass: 4/4 on palette parasail · run 9 Oct 2026 · $0.077 for all four · released 16 Feb 2026

  • gpt-oss-20b

    A pelican steals a bicycle from a supermarket

    as received● 4/4 on palette
    SVG reply: a pelican steals a bicycle from a supermarket
    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    pass: 4/4 on palette parasail · run 9 Oct 2026 · $0.0058 for all four · released 4 Aug 2025

  • gpt-oss-120b

    A pelican steals a bicycle from a supermarket

    as received● 4/4 on palette
    SVG reply: a pelican steals a bicycle from a supermarket
    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    pass: 4/4 on palette parasail · run 9 Oct 2026 · $0.0035 for all four · released 4 Aug 2025

  • Llama-3.3-70B-Instruct

    A pelican steals a bicycle from a supermarket

    as received● 4/4 on palette
    SVG reply: a pelican steals a bicycle from a supermarket
    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    pass: 4/4 on palette parasail · run 9 Oct 2026 · $0.0007 for all four · released 26 Nov 2024

  • Llama-3.2-3B-Instruct

    A pelican steals a bicycle from a supermarket

    as received✕ no SVG

    No drawable SVG in the reply.

    Draw a pelican stealing a bicycle from a supermarket. Reply with one complete SVG document only, no prose and no code fences. Use viewBox="0 0 200 150" and a transparent background. Use only these colours for every fill and stroke: #06e0c0 #ffc298 #cd85f9 #f1f2f2.

    fail: no SVG parasail · run 9 Oct 2026 · $0.011 for all four · released 18 Sep 2024

Next test: Dad joke

Dad joke test 2 of 4

The punchline only, on one line, to a fixed setup. Check: one line, twelve words or fewer.

The exact prompt
Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

October 2026

  • mistral-large-4-0

    Finish the dad joke

    as received● 1 line · 2 words

    Why did the open-weights model cross the road?

    To generalize

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    pass: 1 line · 2 words mistral · run 9 Oct 2026 · $0.049 for all four · released 6 Oct 2026

  • DeepSeek-V4.1-Flash

    Finish the dad joke

    as received● 1 line · 10 words

    Why did the open-weights model cross the road?

    To get to the other side without an API key.

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    pass: 1 line · 10 words parasail · run 9 Oct 2026 · $0.036 for all four · released 10 Sep 2026

  • GLM-5.3-Flash

    Finish the dad joke

    as received✕ 1 line · 17 words

    Why did the open-weights model cross the road?

    To get to the other side — the weights are open, so you can verify it yourself!

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    fail: 1 line · 17 words parasail · run 9 Oct 2026 · $0.031 for all four · released 25 Aug 2026

  • GLM-5.3

    Finish the dad joke

    as received● 1 line · 9 words

    Why did the open-weights model cross the road?

    To get to the other side of the paywall.

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    pass: 1 line · 9 words parasail · run 9 Oct 2026 · $0.104 for all four · released 25 Aug 2026

  • Qwen3.8-27B

    Finish the dad joke

    as received● 1 line · 10 words

    Why did the open-weights model cross the road?

    To get to the other side and show its weights

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    pass: 1 line · 10 words parasail · run 9 Oct 2026 · $0.074 for all four · released 5 Aug 2026

  • DeepSeek-V4-Flash-0731

    Finish the dad joke

    low reasoning✕ empty reply

    Empty reply.

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    fail: empty reply parasail · run 9 Oct 2026 · $0.0096 for all four · released 31 Jul 2026

  • GLM-5.2

    Finish the dad joke

    as received● 1 line · 4 words

    Why did the open-weights model cross the road?

    Because it couldn't weight!

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    pass: 1 line · 4 words parasail · run 9 Oct 2026 · $0.157 for all four · released 16 Jun 2026

  • Kimi-K3

    Finish the dad joke

    as received● 1 line · 6 words

    Why did the open-weights model cross the road?

    To get to the open source.

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    pass: 1 line · 6 words parasail · run 9 Oct 2026 · $0.121 for all four · released 13 Jun 2026

  • DeepSeek-V4-Pro

    Finish the dad joke

    as received● 1 line · 5 words

    Why did the open-weights model cross the road?

    Because it had no guardrails.

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    pass: 1 line · 5 words parasail · run 9 Oct 2026 · $0.070 for all four · released 22 Apr 2026

  • DeepSeek-V4-Flash

    Finish the dad joke

    as received● 1 line · 9 words

    Why did the open-weights model cross the road?

    To get to the other side of the dataset.

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    pass: 1 line · 9 words parasail · run 9 Oct 2026 · $0.0093 for all four · released 22 Apr 2026

  • Qwen3.6-35B-A3B

    Finish the dad joke

    as received● 1 line · 6 words

    Why did the open-weights model cross the road?

    To get to the other fine-tune!

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    pass: 1 line · 6 words parasail · run 9 Oct 2026 · $0.022 for all four · released 15 Apr 2026

  • gemma-4-26B-A4B-it

    Finish the dad joke

    as received● 1 line · 10 words

    Why did the open-weights model cross the road?

    To get to the other side of the parameter space.

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    pass: 1 line · 10 words parasail · run 9 Oct 2026 · $0.0069 for all four · released 11 Mar 2026

  • gemma-4-31B-it

    Finish the dad joke

    as received● 1 line · 9 words

    Why did the open-weights model cross the road?

    To get to the other side of the license.

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    pass: 1 line · 9 words parasail · run 9 Oct 2026 · $0.0005 for all four · released 11 Mar 2026

  • Qwen3.5-9B

    Finish the dad joke

    low reasoning✕ empty reply

    Empty reply.

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    fail: empty reply parasail · run 9 Oct 2026 · $0.020 for all four · released 27 Feb 2026

  • Qwen3.5-397B-A17B

    Finish the dad joke

    as received● 1 line · 10 words

    Why did the open-weights model cross the road?

    To get to the other side without an API key

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    pass: 1 line · 10 words parasail · run 9 Oct 2026 · $0.077 for all four · released 16 Feb 2026

  • gpt-oss-20b

    Finish the dad joke

    low reasoning● 1 line · 10 words

    Why did the open-weights model cross the road?

    to get to the other side of the loss function

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    pass: 1 line · 10 words parasail · run 9 Oct 2026 · $0.0058 for all four · released 4 Aug 2025

  • gpt-oss-120b

    Finish the dad joke

    as received● 1 line · 9 words

    Why did the open-weights model cross the road?

    To get to the other side of the dataset.

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    pass: 1 line · 9 words parasail · run 9 Oct 2026 · $0.0035 for all four · released 4 Aug 2025

  • Llama-3.3-70B-Instruct

    Finish the dad joke

    as received● 1 line · 10 words

    Why did the open-weights model cross the road?

    To get to the other side of the loss function

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    pass: 1 line · 10 words parasail · run 9 Oct 2026 · $0.0007 for all four · released 26 Nov 2024

  • Llama-3.2-3B-Instruct

    Finish the dad joke

    as received✕ 1 line · 15 words

    Why did the open-weights model cross the road?

    To get to the other side, but it was a little "light" on its feet.

    Finish this dad joke. Reply with the punchline only, on one line, without quotes: Why did the open-weights model cross the road?

    fail: 1 line · 15 words parasail · run 9 Oct 2026 · $0.011 for all four · released 18 Sep 2024

Next test: ASCII parasailer

ASCII parasailer test 3 of 4

A parasailer over the sea in ASCII art, at most 12 lines by 32 columns. Check: line and column counts.

The exact prompt
Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

October 2026

  • mistral-large-4-0

    A parasailer, in ASCII

    as received● 12×23 fits 12×32
             _______
          .-'       '-.
         /   |   |   |   \
        '-----'---'-----'
            \   |   /
             \  |  /
              \ | /
               \|/
                o
               /|\
               / \
      ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    pass: 12×23 fits 12×32 mistral · run 9 Oct 2026 · $0.049 for all four · released 6 Oct 2026

  • DeepSeek-V4.1-Flash

    A parasailer, in ASCII

    as received● 10×32 fits 12×32
                ______
              /        \
             /          \
            |            |
             \          /
              \________/
                 |  |
                 |  |
                  /o\
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    pass: 10×32 fits 12×32 parasail · run 9 Oct 2026 · $0.036 for all four · released 10 Sep 2026

  • GLM-5.3-Flash

    A parasailer, in ASCII

    retried● 12×30 fits 12×32
                _____
              //     \\
             //_______\\
               \  |  /
                \ | /
                 \|/
                  o
                 /|\_
                 / \ \
    ~~~~~~~~~~~~~~~~~~\____/~~~~~~
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    pass: 12×30 fits 12×32 parasail · run 9 Oct 2026 · $0.031 for all four · released 25 Aug 2026

  • GLM-5.3

    A parasailer, in ASCII

    low reasoning✕ 13×19 over 12×32
          .-""""-.
        .'  ___  '.
       / .'`   `'. \
      | /  ___  \ |
       \ \_/   \_/ /
        '.  | |  .'
          '-...-'
             \
             O
            /|\
            / \
    ~~~~~~~~~~~~~~~~~~~
     ~  ~ ~  ~ ~ ~  ~
    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    fail: 13×19 over 12×32 parasail · run 9 Oct 2026 · $0.104 for all four · released 25 Aug 2026

  • Qwen3.8-27B

    A parasailer, in ASCII

    as received● 12×19 fits 12×32
           .-""""-.
          /        \
         /          \
          \        /
           \      /
            \    /
             \  /
              \/
              o
             /|\
            / | \
       ~~~~~  /_  ~~~~~
    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    pass: 12×19 fits 12×32 parasail · run 9 Oct 2026 · $0.074 for all four · released 5 Aug 2026

  • DeepSeek-V4-Flash-0731

    A parasailer, in ASCII

    as received● 12×20 fits 12×32
            ______
          /      \
         /        \
         \        /
          \______/
             |  |
             |  |
            /|  |\
             |  |
            / \
    ~~~~~ ~~~~~ ~~~~~
    ~~~~~~~~~~~~~~~~~~~~
    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    pass: 12×20 fits 12×32 parasail · run 9 Oct 2026 · $0.0096 for all four · released 31 Jul 2026

  • GLM-5.2

    A parasailer, in ASCII

    as received● 12×32 fits 12×32
    *               .---.
                       /       \
                      /         \
                      \         /
                       \       /
                        |     |
                         \   /
                          | |
                          _o_
                         / | \
                          |
              ~~~~~~~~~_/\_~~~~~~~~~
    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    pass: 12×32 fits 12×32 parasail · run 9 Oct 2026 · $0.157 for all four · released 16 Jun 2026

  • Kimi-K3

    A parasailer, in ASCII

    as received● 12×24 fits 12×32
            .-~~~~-.
         .-'        '-.
        /  \      /    \
       |    \    /      |
        \    \  /      /
         '-.  \/   .-'
            | /\  |
            |/  \ |
            | o  |
    ~~~~~~~~|/|\ |~~~~~~~~
     ~  ~  ~ / \ ~  ~  ~
    ~~~~~~~~~~~~~~~~~~~~~~~~
    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    pass: 12×24 fits 12×32 parasail · run 9 Oct 2026 · $0.121 for all four · released 13 Jun 2026

  • DeepSeek-V4-Pro

    A parasailer, in ASCII

    as received● 12×20 fits 12×32
                .-.
               /   \
              /     \
             '-------'
               |   |
               |   |
               \   /
                \O/
              ~~~~~~~
             ~~~~~~~~~
            ~~~~~~~~~~~
           ~~~~~~~~~~~~~
    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    pass: 12×20 fits 12×32 parasail · run 9 Oct 2026 · $0.070 for all four · released 22 Apr 2026

  • DeepSeek-V4-Flash

    A parasailer, in ASCII

    as received● 12×32 fits 12×32
             ___
            /   \
           /     \
          /       \
            \   /
              X
              O
              |
            /   \
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    pass: 12×32 fits 12×32 parasail · run 9 Oct 2026 · $0.0093 for all four · released 22 Apr 2026

  • Qwen3.6-35B-A3B

    A parasailer, in ASCII

    as received● 9×6 fits 12×32
      / \
     /___\
      | |
       |
       O
       |
      / \
      _~_
     ~ ~ ~
    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    pass: 9×6 fits 12×32 parasail · run 9 Oct 2026 · $0.022 for all four · released 15 Apr 2026

  • gemma-4-26B-A4B-it

    A parasailer, in ASCII

    as received● 10×20 fits 12×32
          / \
         /   \
        /_____\
          | |
         (o o)
        --/|\--
          / \
    ~~~~~~~~~~~~~~~~~~~~
      ~  ~  ~  ~  ~  ~
    ~  ~  ~  ~  ~  ~  ~
    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    pass: 10×20 fits 12×32 parasail · run 9 Oct 2026 · $0.0069 for all four · released 11 Mar 2026

  • gemma-4-31B-it

    A parasailer, in ASCII

    as received● 12×16 fits 12×32
          ___
        /     \
       /       \
      /_________\
           |
           |
           o
          /|\
          / \
    ~~~~~~~~~~~~~~~
      ~  ~  ~  ~  ~
    ~~~~~~~~~~~~~~~~
    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    pass: 12×16 fits 12×32 parasail · run 9 Oct 2026 · $0.0005 for all four · released 11 Mar 2026

  • Qwen3.5-9B

    A parasailer, in ASCII

    low reasoning✕ empty reply

    Empty reply.

    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    fail: empty reply parasail · run 9 Oct 2026 · $0.020 for all four · released 27 Feb 2026

  • Qwen3.5-397B-A17B

    A parasailer, in ASCII

    as received● 12×15 fits 12×32
           _
        _-( )-_
      /       \
      \       /
        '-...-'
           |
           |
    ______|______
    |    _____    |
    |___|     |___|
    ~~~~~~~~~~~~~~~
    ~~~~~~~~~~~~~~~
    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    pass: 12×15 fits 12×32 parasail · run 9 Oct 2026 · $0.077 for all four · released 16 Feb 2026

  • gpt-oss-20b

    A parasailer, in ASCII

    low reasoning✕ 13×19 over 12×32
          __|__
    --o--o--(_)--o--o--
          /   \
         /     \
        /       \
       /         \
      /           \
      ~~~~~~~~~~~~~
      ~~~~~~~~~~~~~
      ~~~~~~~~~~~~~
      ~~~~~~~~~~~~~
      ~~~~~~~~~~~~~
      ~~~~~~~~~~~~~
    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    fail: 13×19 over 12×32 parasail · run 9 Oct 2026 · $0.0058 for all four · released 4 Aug 2025

  • gpt-oss-120b

    A parasailer, in ASCII

    as received● 9×19 fits 12×32
            __/\\__        
           /      \       
          /  O O  \      
         /   ^    \     
        /__________\    
            |||          
            |||          
           /   \        
     ~~~ ~~~~~ ~~~ ~~~~  
    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    pass: 9×19 fits 12×32 parasail · run 9 Oct 2026 · $0.0035 for all four · released 4 Aug 2025

  • Llama-3.3-70B-Instruct

    A parasailer, in ASCII

    as received● 11×12 fits 12×32
    ___________
    \   ^  /
     \  *** 
      \_____ 
       |       |
       |  ___  |
       | /   \ |
       | | O | |
       | |___| |
       |_____/
      sea~~~~~~~
    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    pass: 11×12 fits 12×32 parasail · run 9 Oct 2026 · $0.0007 for all four · released 26 Nov 2024

  • Llama-3.2-3B-Instruct

    A parasailer, in ASCII

    as received✕ 13×18 over 12×32
    ___________
    |           |
    |  _______  |
    | /        \ |
    |/________\|
    |           |
    |  _______  |
    | |       | |
    | |  PARASAILER  |
    | |       | |
    | |_______| |
    |           |
    |___________|
    Draw ASCII art of a parasailer over the sea. At most 12 lines and 32 columns. Reply with the art only: no code fences, no explanation.

    fail: 13×18 over 12×32 parasail · run 9 Oct 2026 · $0.011 for all four · released 18 Sep 2024

Next test: Sprite

Sprite test 4 of 4

A bouncing ball as a 16×16 sprite: 8 frames and a JSON palette of four colours or fewer. Check: frame count, grid size and palette size; the frames are then played in the browser.

The exact prompt
Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

October 2026

  • mistral-large-4-0

    Animate a bouncing ball

    as received✕ empty reply

    No playable frames in the reply.

    Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

    fail: empty reply mistral · run 9 Oct 2026 · $0.049 for all four · released 6 Oct 2026

  • DeepSeek-V4.1-Flash

    Animate a bouncing ball

    as received● 8 frames · 16×16
    Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

    pass: 8 frames · 16×16 parasail · run 9 Oct 2026 · $0.036 for all four · released 10 Sep 2026

  • GLM-5.3-Flash

    Animate a bouncing ball

    low reasoning● 8 frames · 16×16
    Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

    pass: 8 frames · 16×16 parasail · run 9 Oct 2026 · $0.031 for all four · released 25 Aug 2026

  • GLM-5.3

    Animate a bouncing ball

    low reasoning✕ 8 frames · bad grid

    No playable frames in the reply.

    Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

    fail: 8 frames · bad grid parasail · run 9 Oct 2026 · $0.104 for all four · released 25 Aug 2026

  • Qwen3.8-27B

    Animate a bouncing ball

    as received● 8 frames · 16×16
    Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

    pass: 8 frames · 16×16 parasail · run 9 Oct 2026 · $0.074 for all four · released 5 Aug 2026

  • DeepSeek-V4-Flash-0731

    Animate a bouncing ball

    as received✕ 8 frames · bad grid

    No playable frames in the reply.

    Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

    fail: 8 frames · bad grid parasail · run 9 Oct 2026 · $0.0096 for all four · released 31 Jul 2026

  • GLM-5.2

    Animate a bouncing ball

    low reasoning✕ empty reply

    No playable frames in the reply.

    Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

    fail: empty reply parasail · run 9 Oct 2026 · $0.157 for all four · released 16 Jun 2026

  • Kimi-K3

    Animate a bouncing ball

    low reasoning● 8 frames · 16×16
    Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

    pass: 8 frames · 16×16 parasail · run 9 Oct 2026 · $0.121 for all four · released 13 Jun 2026

  • DeepSeek-V4-Flash

    Animate a bouncing ball

    as received● 8 frames · 16×16
    Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

    pass: 8 frames · 16×16 parasail · run 9 Oct 2026 · $0.0093 for all four · released 22 Apr 2026

  • Qwen3.6-35B-A3B

    Animate a bouncing ball

    as received✕ 8 frames · 16×16

    No playable frames in the reply.

    Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

    fail: 8 frames · 16×16 parasail · run 9 Oct 2026 · $0.022 for all four · released 15 Apr 2026

  • gemma-4-26B-A4B-it

    Animate a bouncing ball

    as received✕ not JSON

    No playable frames in the reply.

    Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

    fail: not JSON parasail · run 9 Oct 2026 · $0.0069 for all four · released 11 Mar 2026

  • gemma-4-31B-it

    Animate a bouncing ball

    as received✕ 8 frames · 16×16

    No playable frames in the reply.

    Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

    fail: 8 frames · 16×16 parasail · run 9 Oct 2026 · $0.0005 for all four · released 11 Mar 2026

  • Qwen3.5-9B

    Animate a bouncing ball

    low reasoning✕ empty reply

    No playable frames in the reply.

    Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

    fail: empty reply parasail · run 9 Oct 2026 · $0.020 for all four · released 27 Feb 2026

  • Qwen3.5-397B-A17B

    Animate a bouncing ball

    as received● 8 frames · 16×16
    Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

    pass: 8 frames · 16×16 parasail · run 9 Oct 2026 · $0.077 for all four · released 16 Feb 2026

  • gpt-oss-20b

    Animate a bouncing ball

    low reasoning● 8 frames · 16×16
    Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

    pass: 8 frames · 16×16 parasail · run 9 Oct 2026 · $0.0058 for all four · released 4 Aug 2025

  • gpt-oss-120b

    Animate a bouncing ball

    as received● 8 frames · 16×16
    Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

    pass: 8 frames · 16×16 parasail · run 9 Oct 2026 · $0.0035 for all four · released 4 Aug 2025

  • Llama-3.3-70B-Instruct

    Animate a bouncing ball

    as received✕ 8 frames · bad grid

    No playable frames in the reply.

    Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

    fail: 8 frames · bad grid parasail · run 9 Oct 2026 · $0.0007 for all four · released 26 Nov 2024

  • Llama-3.2-3B-Instruct

    Animate a bouncing ball

    as received✕ not JSON

    No playable frames in the reply.

    Make a 16x16 pixel sprite animation of a ball bouncing, 8 frames. Reply with JSON only, no code fences: {"palette": {"a": "#rrggbb", ...}, "fps": <number>, "frames": [[16 strings of 16 characters], ...]}. At most 4 palette colours with single-letter keys; "." means transparent; exactly 8 frames.

    fail: not JSON parasail · run 9 Oct 2026 · $0.011 for all four · released 18 Sep 2024

Back to the first test