Firecrawl Universal Scrape Can Read Podcasts. The Terms Say You Can't Keep Them.

On October 8, Firecrawl launched Universal Scrape. The scrape endpoint I have been feeding URLs to for this whole series now answers questions about podcasts, people profiles, company financials and website traffic too. Ask it about podcasts and it gets you transcripts.

The 70-line harness from the Arena post already does the hard part: run on a schedule, save every result to SQLite, compare this week to last week, speak up when something changes. Point it at podcast mentions instead of a leaderboard and you have a brand monitor. Which podcasts talked about your product this week? Is that number going up? Who mentioned it for the first time?

Then I read the terms and had to accept them to use the feature. And the terms forbid storing the data, reusing it for a later query, keeping it past the session, and showing it to anyone who isn’t one of your own users. That rules out the harness, the comparison, my AI agents, and other things I haven’t thought of yet.

So this post is not what I originally planned. It’s about the fine print for every paid provider behind Universal Scrape, plus the version of the podcast tracker that survives it.

• What Universal Scrape Adds
• The Recipe I Was Going to Write
• What Particle’s Terms Actually Say
• I Read the Terms for Every Paid Provider
• The Tracker That Survives the Terms
  • The second gotcha: don’t pass a domain
  • The script
  • What I can show you
• Read the Schedule Before You Write the Recipe

What Universal Scrape Adds

Two things, really.

Files by URL. PDFs, images, Word documents, spreadsheets and presentations, if you can link to them. You pass the URL like any web page and get markdown back. The scrape docs put PDF parsing at 1 credit per PDF page. None of that has provider terms attached.

Provider data. This is Alexandria, the data catalog Firecrawl launched a couple of weeks ago, folded into the scrape endpoint. The launch post names the headliners: Apollo, FullEnrich and Data Legion for people and company profiles, Fiscal.ai and Benzinga for financial data, and Particle for podcasts.

In the CLI, a provider call is a scrape where the URL is a provider/capability address:

firecrawl scrape particle/podcasts/mentions/timeseries \
  --options '{"company_id": "anthropic", "interval": "month"}' --json

That syntax needs a recent CLI. Mine was 1.23.3, so I updated to 1.26.3.

npm install -g firecrawl-cli@latest
firecrawl --version   # 1.26.3

Same plain npm install as the setup post, so nothing touched my editor configs, and the key still comes from FIRECRAWL_API_KEY.

The Recipe I Was Going to Write

Every useful scraping job in this series turned out to be the same job. A scraper tells you what a page says right now. Change needs history, history needs storage, and storage is why the harness exists.

For podcasts, the plan was one weekly call, a row per episode in SQLite, and three alerts:

• mentions this week versus the last four
• shows mentioning the term for the first time
• the transcript line where each mention happens, so you can read it without listening to a 90-minute episode

Particle has the tools for all of it. Episode search, transcripts, mention lookups, speaker lists, even which sponsors read ads on which shows. 105 tools, at 5, 15, 25 or 50 credits a call.

What Particle’s Terms Actually Say

Before the first call, Particle makes you accept its terms from the CLI:

firecrawl alexandria terms show particle --json
firecrawl alexandria terms accept particle \
  --terms-version B-1.0.0 --digest <sha256 from the show output> --confirm

Credit to Firecrawl here. You accept an exact version, pinned by a hash of the document, and the JSON that terms show returns tells an AI agent to present the terms to a human and wait for an explicit yes. My agent did exactly that. It read them, came back, and told me the recipe was dead before it accepted anything.

The document is Schedule B, which pulls in Particle’s Data Use Terms. The parts that matter, quoted:

(g) Download, save, or store Clips, Full Episode Audio, or Transcripts, other than (i) transient caching necessary for real-time display and the operation of your application, (ii) retaining a response you received, unaltered, so that your users can view that prior response…

(h) Reuse stored Podcast Data to answer a later or different query. Each query is served by a new request through the Services.

(i) Where an Automated Agent handles Podcast Data, allow it to retain that data beyond the session context or short-term memory reasonably necessary to process the immediate request. No persistent storage by agents.

(a) Redistribute, resell, sublicense, or make Podcast Data available to anyone other than your own authorized users…

Go back through the plan with those in hand.

The SQLite table is out. Saving transcripts is (g). Saving episode rows and comparing them next week is (h), almost word for word: stored data answering a later query.

The agent workflow is out. Every post in this series was built with a coding agent driving the CLI. Under (i), the agent can hold the data for the request and that’s it.

This blog post is out, mostly. You are not one of my authorized users. A table of which shows mentioned which company, in a public post, is (a).

There is also (j), no training models on it, and section B.7, which says Particle “may not hold licenses from the podcast creators and publishers” for the content. Particle’s position is fair use, and the schedule says that the podcast owners “may not share that assessment.”

I Read the Terms for Every Paid Provider

If the podcast provider looks like that, what about the rest? Alexandria has a terms catalog, and terms show works on any provider ID, so I walked the categories and asked about every provider I could find.

Most of them have no terms in the catalog at all. Government data, public registries, app stores, package registries, Yahoo Finance. terms show returns “Provider not found in the accessible terms catalog.” That doesn’t mean no rules. It means Firecrawl has no extra contract for that source, and the source’s own terms of service apply the way they would to any scrape.

The commercial partners each carry a lettered schedule. Seven of them:

“Accept first?” is the required flag. When it says No, the call goes through without an acceptance step. The schedules still describe how you are allowed to use the data, and nothing in them suggests skipping the acceptance step lets you skip the rules.

So it sorts into three piles. The contact providers are built for your sales team and forbid publishing. Similarweb and Particle forbid keeping anything. The financial providers leave you room to build a product.

None of this is Firecrawl being sneaky. Licensed data has always come with rules like these. What changed is that it now sits behind the same firecrawl scrape you use on a public web page, where the only rules are robots.txt and the site’s terms. One command, two completely different legal situations, and nothing in the syntax tells you which one you are in.

The Tracker That Survives the Terms

The thing (h) forbids, keeping your own history to answer later questions, is a thing Particle sells you instead. podcasts/mentions/timeseries costs 15 credits and returns the whole history, bucketed by day, week or month, in one call. You don’t need a database if every run asks for everything.

I pointed it at Anthropic, monthly, since January. The episode counts climbed steeply all year. It looked like a story. Then I ran the same call on Coca-Cola as a control (a company that didn’t get more famous in 2026), and it climbed steeply too.

That is not Anthropic or Coca-Cola. That is Particle’s index growing. More shows transcribed, more episodes covered, more mentions of everybody. A raw mention count from a source whose coverage changes every month will tell you your brand is exploding when the source is the thing exploding.

The fix is old: divide by a control. Same source, same window, a company whose real share of the conversation should hold steady. Whatever index growth inflates in one, it inflates in the other.

The second gotcha: don’t pass a domain

company_id says it takes a slug, a domain or an ID. anthropic.com worked. coca-cola.com came back as Simply Orange, one juice brand out of Coca-Cola’s two hundred, with a handful of mentions. No error, no warning. If I hadn’t looked at the company block in the response, I’d have used a juice brand as my control and believed the result.

So the script resolves names first with entities/search (5 credits) and takes the exact match.

The script

No SQLite, no saved files. It prints to the terminal and forgets:

#!/usr/bin/env python3
import json, subprocess, sys


def call(tool, options):
    out = subprocess.run(
        ["firecrawl", "scrape", f"particle/{tool}",
         "--options", json.dumps(options), "--json"],
        capture_output=True, text=True, check=True).stdout
    return json.loads(out[out.index("{"):])["data"]["alexandria"][0]["data"]


def company_slug(name):
    # Never pass a bare domain: coca-cola.com resolved to Simply Orange.
    hits = call("entities/search", {"q": name, "type": "company", "limit": 5})["data"]
    exact = [h for h in hits if h.get("match_quality") == "exact"] or hits
    if not exact:
        sys.exit(f"no company found for {name!r}")
    company = exact[0]["company"]
    print(f"{name!r} -> {company['name']} ({company['slug']})")
    return company["slug"]


def monthly(slug, since):
    data = call("podcasts/mentions/timeseries",
                {"company_id": slug, "interval": "month", "published_after": since})
    return {b["start"][:7]: b["episode_count"] for b in data["buckets"]}


def main():
    target, control = sys.argv[1], sys.argv[2]
    since = sys.argv[3] if len(sys.argv) > 3 else "2026-01-01"
    t = monthly(company_slug(target), since)
    c = monthly(company_slug(control), since)
    print(f"\nmonth    {target} episodes per {control} episode")
    for month in sorted(t):
        ratio = t[month] / c[month] if c.get(month) else float("nan")
        print(f"{month}  {ratio:6.2f}")


if __name__ == "__main__":
    main()

python3 podcast_share.py "Anthropic" "The Coca-Cola Company"

Each run is two searches and two timeseries calls: 40 credits. Monthly, that’s 4% of the free 1,000. Weekly, about 17%. Daily is 1,200, which is more than the free plan.

What I can show you

And (a) bites the post itself: Particle’s counts are Podcast Data, so I’m not printing them. The ratio is a number I computed from them, which I think puts it on my side of the line. I am not a lawyer, and you should make that call for your own use, not borrow mine.

month    Anthropic episodes per The Coca-Cola Company episode
2026-01    1.57
2026-02    2.94
2026-03    3.87
2026-04    3.27
2026-05    3.15
2026-06    3.57
2026-07    2.78
2026-08    3.14
2026-09    4.89
2026-10    3.89

Read that against the raw curve, which climbed all year. Corrected for index growth, Anthropic’s share roughly doubled from January to February and then sat around three for six months. September is the only real spike. October is a partial month as I write this. The steep line was mostly Particle getting bigger.

Read the Schedule Before You Write the Recipe

The rule from the Alexandria post was check the catalog before you write a scraper. Universal Scrape adds a step after that: check the terms before you write anything that stores, compares or publishes.

The check is one free command per provider:

firecrawl alexandria terms show <provider> --json

If it says the provider is not in the terms catalog, you’re in normal scraping territory, with the site’s own terms and everything the series has said so far. If it returns a schedule, search it for “store”, “retain”, “available to” and “agent” before you design anything. Those four words decide whether your idea is a cron job, a dashboard for your own team, or nothing.

My podcast tracker ended up as a 40-line script that forgets everything it learns the moment it prints. That’s less than I wanted. It’s also the only version I can actually run, and it gets right what the version I planned would have gotten wrong: the growth curve was mostly the index.

The financial providers are the ones whose terms let you build something real, which makes Fiscal.ai the next thing I want to look at.