Your Ebook Library Beats the Web for Research, and Your Coding Agent Can Search It Without RAG

Ask a coding agent to research “how does X actually work” and it will use web search and find some blog posts about X. Those blog posts are summaries of a book somebody wrote about X. Sometimes they’re summaries of other blog posts that summarized the book. For a settled, conceptual question, that summary-of-a-summary chain is all there is.

I own about 30k ebooks. Somewhere in that collection, a writer spent 300 pages on the topic a blog post only has 800 words on. So I built a skill, ebook-research, that lets my agent find the book, find the chapter, get the text, and cite it. This week I moved it into my public agent-skills repo.

You might assume it’s a RAG project. Instead, it’s a full-text index that has been around for over 14 years, plus a Python script.

• Recoll
• The Rule: Never Let a Book Into Context
• The Four Steps It Takes
  • Level 1: Which books cover this
  • Level 2: Where inside the book
  • Level 3: Read the prose
  • Level 4: Cite it
• Books for the Foundation, the Web for This Week
• The One Thing That Isn’t Standalone
• Where It Stands

Recoll

Recoll is a desktop full-text search tool. You point it at a folder, it indexes everything inside (PDF, EPUB, MOBI, DOC, whatever it has a filter for) into a Xapian database, and then you can query that database from the command line with recollq. recollq reads the index off disk. There is no running service.

A coding agent is good at running shell commands and reading their output. It doesn’t need a semantic search layer to find the Stoicism books. A semantic layer would also surface chunks on adjacent topics you didn’t have words for, and this won’t. The trade is worth it: what comes back here is whole books, not the tiny chunks an embedding index deals in. It needs a query language that supports phrases and proximity, and here’s Recoll’s:

The agent writes the query. What it can’t do is handle what comes back.

The Rule: Never Let a Book Into Context

A large PDF extracts to 600KB of text, most of a 200K-token context window. Let an agent cat that, or run pdftotext on it with no output file, and one book just burned most of the session.

So a core rule of the skill:

Never cat, pdftotext, or recollq -d a book straight into context.

Everything goes through scripts/shelf.py, a stdlib-only Python script (562 lines). Each subcommand caps what it prints. Full extractions go to ~/.cache/ebook-research/ on disk, and the agent gets back a slice plus the path to the rest. If it needs more, it reads the cache file with an offset and a limit, or re-runs with --grep.

The Four Steps It Takes

The skill walks the agent down four levels and tells it never to stop at the first one.

Level 1: Which books cover this

python3 scripts/shelf.py search '"negative visualization"' --books 5 --snips 1

This ranks books, not fragments, and handles duplicates. Here’s a real run against my library while I was writing this:

query: "negative visualization"
13 books matched (showing 5)

[1] A-guide-to-the-good-life-the-ancient-art-of-Stoic-joy
    epub, 14 hit(s), rel 100%
    · Section0004.xhtml | F O U R Negative Visualization What's the

[2] 101_Essays_Interior-107
    epub, 2 hit(s), rel 71%
    · id34 | in your head. Practice negative visualization. Create tangible solutions for

[3] Stoicism and the Art of Happiness
    epub, 2 hit(s), rel 71%
    (1 duplicate cop(ies) elsewhere in the library)
    · id11 | He coined the term 'negative visualization' to refer to Stoic

[4] Become an Effective Software Engineering Manager
    epub, 1 hit(s), rel 63%
    · f_0083 | example, regular practicing of negative visualization, where we imagine that

[5] Handbook_for_New_Stoics_7P-65
    epub, 1 hit(s), rel 47%
    · MovingForward.xhtml | how to engage in negative visualization, practicing self-denial

Fourteen hits and a whole chapter in William Irvine’s A Guide to the Good Life versus one passing mention in a software engineering management book. That spread is the signal. (Of course a management book covers negative visualization. Imagining the worst is the job.)

This is what happens without the quotes:

query: negative visualization
38 books matched (showing 5)

[1] BeautifulVisualization - Unknown
    pdf, 2 hit(s), rel 28%

Thirty-eight books, led by an O’Reilly data-viz book. Stemming is on by default, and an unquoted query is an AND, so anything with “negative” somewhere and “visualization” somewhere wins. The skill tells the agent to quote phrases for this reason. Level 1 snippets are 40 characters. They’re good for ranking books and worthless as evidence.

Level 2: Where inside the book

python3 scripts/shelf.py hits "/path/to/book.epub" '"negative visualization"' --max 12

Lists every matching section with a ready-to-paste locator: --page N for a PDF, --ipath "chapter.xhtml" for an EPUB. EPUBs have no page numbers, so the skill locates them by chapter file.

Level 3: Read the prose

python3 scripts/shelf.py read "/path/A-guide-to-the-good-life.epub" \
    --ipath "Section0004.xhtml" --grep "negative visualization" --matches 1 --chars 800

What comes back:

A-guide-to-the-good-life-the-ancient-art-of-Stoic-joy.epub — chapter Section0004.xhtml
5869 words, 33963 chars → full text saved to:
~/.cache/ebook-research/A-guide-to-the-good-life-...-Section0004-xhtml.txt
---
F O U R
Negative Visualization
What's the Worst That Can Happen?
...
Doing this, the Stoics thought, will make us value our wife, our car,
and our job more than we otherwise would. This technique — let us refer
to it as negative visualization — was employed by the Stoics at least as
far back as Chrysippus. It is, I think, the single most valuable
technique in the Stoics' psychological tool kit.
--- 1 passage(s) shown; read [cache file] for the rest

A 34,000-character chapter, and 800 characters of it entered the context. The rest is on disk if the agent wants it.

Nice of the chapter to ask. “What’s the worst that can happen?” is the whole question behind the Rule, and the answer is 600KB of PDF in the context window.

Level 4: Cite it

Every claim the agent pulls from the library carries a title, an author, and a locator. When several books agree, it says so. When they disagree, it says that too.

The skill also tells the agent to distinguish “this book asserts” from “this is established.” A book is one author’s argument, not a fact-check.

Books for the Foundation, the Web for This Week

The skill says it’s the wrong tool for anything current. Model releases, framework versions, news: web search. Books are for what’s settled, deep, or conceptual. Philosophy, craft, history, psychology, business principles, the theory underneath a thing.

Many questions want both. The skill does a web pass for what changed this month, then a library pass for the principle under it, presented separately. A claim from a 2015 book never blends into a paragraph about this week’s news.

And if the library has nothing good, the agent says so and goes back to the web. Even a library this size has holes.

The One Thing That Isn’t Standalone

Install Recoll, point its indexing config at your ebook folder (the indexing preferences in the GUI on first launch, or a topdirs line in ~/.recoll/recoll.conf), run recollindex, and wait. Mine took about 15 minutes.

There’s no live monitor, so after you add books re-run recollindex. The skill tells the agent to suggest you run it yourself in case your forget.

Where It Stands

skills/ebook-research in the agent-skills repo, a SKILL.md and one Python script, next to fetch-anything and living-plan. If you use agent skills, you know where the folder goes. If you have a folder full of PDFs and EPUBs you bought and never finished, you already have a better research corpus than whatever your agent gets from a search engine.