Track New AI Models on the Arena Leaderboard With a 70-Line Scraper

Every week I write the Model Buzz Report, and every week part of the job is staring at the Arena leaderboard trying to remember what it looked like last week. Which of these is new? Was that one there before? When did GLM 5.3 show up in the top 20, because it sure felt like it came out of nowhere?

That is a question a scraper cannot answer. A scraper hands you the page as it is right now, and “what’s new” is a comparison between now and then. You need a then.

So this post is about the then. It is the smallest useful version of the thing most scraping recipes in this series is going to need: save each run with a timestamp, run it on a schedule, and compare this run with the last one. The example is deliberately easy, one page and one question.

• Why “Now” Is Almost Never the Question
• The Page: Arena’s Text Leaderboard
• The Harness, All 70 Lines
  • Storage: SQLite, Not a JSON File
  • The Diff: Two Kinds of “New”
  • The Schedule: One Cron Line
• Day One Has No Then
• Two Things It Got Wrong
  • That “NEW #1” Is a Rename
  • It Missed the One That Started This
• Why Not Just Use Firecrawl’s Change Tracking?
• What This Is For

Why “Now” Is Almost Never the Question

Think about what people actually want out of a scraped page. Is this price a deal? Did the competitor change their pricing? Which models are new near the top? None of those can be answered from one snapshot. “Is this a deal” needs the price history. “Did they change” needs the old page. “What’s new” needs the old list.

Firecrawl solves the ugly part of scraping, which is getting a clean page out of a site that renders everything with JavaScript and would rather you went away. It does not solve the then. Nothing that fetches a page can, because the then is data you had to collect yourself, back when it was the now.

That takes three boring things:

1. Storage. Save each crawl with a timestamp instead of printing it and forgetting it.
2. A schedule. One crawl tells you nothing. A cron line fixes that.
3. A diff. Compare this crawl with the last one and only say something when something changed.

That is the whole harness. I keep wanting to call it a framework and it keeps being 70 lines.

The Page: Arena’s Text Leaderboard

The target is arena.ai/leaderboard/text, the overall text leaderboard. About 400 models, ranked by head-to-head human votes, with score, vote count, price, and context window per row.

It is also a page that doesn’t want to be read by a simple fetch. The table is rendered client-side, and when I built the Model Buzz skill I learned that some fetch tools hand back the wrong leaderboard on category URLs without any error at all. That is the worst kind of failure: data that looks plausible.

Firecrawl got it clean on the first try:

firecrawl scrape https://arena.ai/leaderboard/text --only-main-content --wait-for 3000

What comes back is markdown, and the leaderboard is a plain markdown table:

| Rank | Rank Spread | Model | Score | Votes | Price $/M | Context |
| --- | --- | --- | --- | --- | --- | --- |
| 1 | 17 | Anthropic<br>[claude-fable-5-high](https://www.anthropic.com/news/claude-fable-5-mythos-5)<br>Anthropic · Proprietary | 1506±5 | 30,057 | $10 / $50 | 1M |
...
| 19 | 737 | [glm-5.3-max](https://z.ai/blog/glm-5.3)<br>Z.ai · MIT | 1483±6 | 10,960 | $1.40 / $4.40 | 1M |

I gave it three seconds to let the page render. I didn’t test whether it needs that, so treat the number as superstition you are free to delete.

No JSON schema, no LLM extraction, and no selectors. A regex reads that table fine. A plain scrape is one credit a page, and Firecrawl’s JSON extraction adds four more on top, so asking an LLM to parse a table this regular would be paying five times over for something a regex already does. If you haven’t installed the CLI yet, the setup post covers it and the installer you should skip.

The Harness, All 70 Lines

Python, standard library only, calling the Firecrawl CLI through subprocess. No SDK, so the only install is the CLI you already have.

#!/usr/bin/env python3
"""Watch the Arena text leaderboard and report models that just showed up near the top.

    python3 arena_watch.py                      # scrape now, save, compare with last run
    python3 arena_watch.py --url <wayback url> --at 2026-09-02   # backfill an old snapshot
"""
import argparse, re, sqlite3, subprocess, sys
from datetime import datetime, timezone

URL = "https://arena.ai/leaderboard/text"
DB = "arena.db"
TOP = 20

ROW = re.compile(r"^\| (\d+) \| \d+ \| (.+?) \| (\d+)±")
NAME = re.compile(r"\[([^\]]+)\]")


def scrape(url):
    out = subprocess.run(
        ["firecrawl", "scrape", url, "--only-main-content", "--wait-for", "3000"],
        capture_output=True, text=True, check=True).stdout
    rows = []
    for line in out.splitlines():
        m = ROW.match(line)
        name = m and NAME.search(m.group(2))
        if not name:
            continue  # skips the calendar widget and anything else that isn't a model row
        org = m.group(2).split("<br>")[-1].split(" · ")[0]
        rows.append((int(m.group(1)), name.group(1), org, int(m.group(3))))
    return rows


def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--url", default=URL)
    ap.add_argument("--at", help="timestamp to record, for backfilling old snapshots")
    args = ap.parse_args()

    rows = scrape(args.url)
    if len(rows) < TOP:
        sys.exit(f"only parsed {len(rows)} rows, the page layout probably changed")

    taken = args.at or datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M")
    db = sqlite3.connect(DB)
    db.execute("""CREATE TABLE IF NOT EXISTS ranks
                  (taken TEXT, rank INT, model TEXT, org TEXT, score INT)""")
    db.executemany("INSERT INTO ranks VALUES (?,?,?,?,?)", [(taken, *r) for r in rows])
    db.commit()

    prev = db.execute("SELECT MAX(taken) FROM ranks WHERE taken < ?", (taken,)).fetchone()[0]
    if not prev:
        print(f"{taken}: first run, saved {len(rows)} models. Nothing to compare yet.")
        return


<img src="/images/2026/track-new-ai-models-on-the-arena-leaderboard-body-2.jpg" alt="The Harness, All 70 Lines" srcset="            /assets/resized/480/track-new-ai-models-on-the-arena-leaderboard-body-2.jpg 480w,            /assets/resized/800/track-new-ai-models-on-the-arena-leaderboard-body-2.jpg 800w,            /assets/resized/1400/track-new-ai-models-on-the-arena-leaderboard-body-2.jpg 1400w,    " loading="lazy">


    was_top = {m for (m,) in db.execute(
        "SELECT model FROM ranks WHERE taken = ? AND rank <= ?", (prev, TOP))}
    seen = {m for (m,) in db.execute("SELECT DISTINCT model FROM ranks WHERE taken < ?", (taken,))}

    print(f"{taken} vs {prev}")
    for rank, model, org, score in rows:
        if rank > TOP:
            break
        if model not in seen:
            print(f"  NEW      #{rank:<3} {model} ({org}, {score})")
        elif model not in was_top:
            print(f"  MOVED UP #{rank:<3} {model} ({org}, {score})")


if __name__ == "__main__":
    main()

Here is what each piece is doing, mapped to the three boring things.

Storage: SQLite, Not a JSON File

Every run inserts every row, all 400 or so, stamped with the time it was taken. One table, five columns.

I went back and forth on JSONL here. A line of JSON per run is simpler to look at and fine for a diff against the last run. But the questions I actually care about later are history questions: how long has this model been in the top 20, when did it first appear anywhere on the board, is it climbing or sliding. With JSONL you write a loop for each of those. With SQLite you write a query. And SQLite ships with Python, so it costs nothing to install.

Saving the whole board instead of just the top 20 is on purpose. Storage is cheap and you can’t go back and scrape last Tuesday. A model that debuts at #24 is not news today, but the day it cracks the top 20 you want to know it has been lurking for a week.

The Diff: Two Kinds of “New”

The script reports two different things, and the difference matters.

• NEW means the model has never appeared anywhere on the board in any previous run. This is the one I actually wanted. A brand new model landing in the top 20 on its first appearance is the “where did that come from” moment.
• MOVED UP means the model was on the board before but was not in the top 20 last run. A climber. Less exciting, still worth a look.

Everything else, a model shuffling from #7 to #9, stays quiet. The whole point of the diff is that most runs should print almost nothing.

The Schedule: One Cron Line

0 8 * * * cd /path/to/arena-watch && python3 arena_watch.py >> watch.log 2>&1

Daily at 8am, appended to a log. That lives on my mini PC, which is on all the time. On a Mac you can use cron too, or launchd if you enjoy writing XML. Once a day is plenty for a leaderboard that needs thousands of votes to move a model. Each run is one scrape, which on Firecrawl is one credit.

Day One Has No Then

Here is the catch with every history-based tool: the first run is useless. It saves 400 rows and prints “Nothing to compare yet.” You have to wait a day for the second run before the thing does anything at all, and a week before it does anything interesting.

I didn’t want to wait a week to write this post, so I cheated with the Wayback Machine. The Internet Archive snapshots the Arena leaderboard most days, and Firecrawl will scrape an archived copy just like the live page. That is what the --url and --at flags are for: point the script at an old snapshot and tell it what date to record.

python3 arena_watch.py --url "https://web.archive.org/web/20260902190730/https://arena.ai/leaderboard/text" --at "2026-09-02 19:07"
python3 arena_watch.py --url "https://web.archive.org/web/20260909040114/https://arena.ai/leaderboard/text" --at "2026-09-09 04:01"
python3 arena_watch.py --url "https://web.archive.org/web/20260917153328/https://arena.ai/leaderboard/text" --at "2026-09-17 15:33"
python3 arena_watch.py

The archived pages parse the same as the live one, since the table structure is identical and only the link URLs change. Three weeks of history in four commands:

2026-09-02 19:07: first run, saved 399 models. Nothing to compare yet.
2026-09-09 04:01 vs 2026-09-02 19:07
  NEW      #3   claude-fable-5.1-max (Anthropic, 1504)
2026-09-17 15:33 vs 2026-09-09 04:01
  NEW      #8   muse-spark-1.3-max (Meta, 1493)
2026-09-23 01:27 vs 2026-09-17 15:33
  NEW      #1   claude-fable-5-high (Anthropic, 1506)

Fable 5.1 Max debuting at #3 and Meta’s Muse Spark 1.3 Max at #8 are exactly the kind of thing I wanted flagged. Midway through doing this the Internet Archive went down with a “Temporarily Offline” page, which is a nice reminder that the backfill trick is a trick and not infrastructure.

Two Things It Got Wrong

That “NEW #1” Is a Rename

Look at the last line again. claude-fable-5-high at #1 as a brand new model. It is not. The database says so:

sqlite3 arena.db "SELECT model, taken, rank, score FROM ranks WHERE model LIKE 'claude-fable-5%' ORDER BY model, taken;"

claude-fable-5|2026-09-02 19:07|1|1508
claude-fable-5|2026-09-09 04:01|1|1507
claude-fable-5|2026-09-17 15:33|1|1506
claude-fable-5-high|2026-09-23 01:27|1|1506

Same rank, same score, new name. This is the most common way change detection lies to you, and it is not a Firecrawl problem or a SQLite problem. It is an identity problem: the thing you are tracking needs a stable key, and the page does not give you one.

The cheap fix is a sanity check before calling anything NEW: if an unseen name sits at the exact rank and score of a name that just disappeared, call it a rename. I left it out of the script because 70 lines that lie once a month teach more than 90 lines that hide it. Your mileage may vary once it wakes you up at 8am about a model that is three months old.

It Missed the One That Started This

GLM 5.3 Max, the model that made me want this in the first place, never shows up as NEW. It was already sitting at #18 on September 2, the oldest snapshot I loaded, so as far as the database knows it has always been there:

2026-09-02 19:07|18|1482
2026-09-09 04:01|20|1482
2026-09-17 15:33|19|1483
2026-09-23 01:27|19|1483

The harness can only see changes that happen after it starts watching. You can’t get the then after the fact. Start the cron before you need it.

Also notice GLM wobbling from #18 to #20 to #19 while its score barely moves. Arena ranks a lot of models within a few points of each other near the top, and a model right on the #20 line will flicker in and out. Had it dropped to #21 for one run, the next run would have called it MOVED UP. If that gets noisy, the fix is to compare against the top 20 from any of the last few runs instead of just the last one.

Why Not Just Use Firecrawl’s Change Tracking?

Fair question, since Firecrawl has a changeTracking format and a whole Monitor product built around scheduled checks. I didn’t use either here, and the reason is the leaderboard itself: the vote counts on every row change constantly, so “did this page change” is always yes. What I want to know is not whether the page changed but what changed in one column of one table, compared against a history I can query.

That is not a knock on those features. For a page that should be static, a pricing page or a terms of service, “tell me when this changes” is exactly the right tool. They are worth their own post. This is the case where you want your own history, because the question you ask of it next month is not one you know yet.

What This Is For

As a standalone tool this is a small convenience. I will run it next to the Model Buzz Report and it will save me the “wait, was that there last week” squint.

The reason it is the first real recipe in this series is the shape. Scrape, save with a timestamp, compare with the last run, speak only when something changed. Every recipe coming after this one is that shape pointed at a different page with a different question: price history instead of rank history, a job board instead of a leaderboard. Firecrawl handles getting the page. These 70 lines handle remembering it.

And the database is already there, filling up once a day, waiting for whatever question I think of next.