Stop Asking AI to Fix the Whole Problem: The Zwicky Box Method

I have a Python script called ai_report.py that builds me an AI news report every day. It pulls about twenty RSS feeds (Anthropic, OpenAI, Simon Willison, Latent.Space, a pile of Medium tags), a dozen subreddits, and a few Hacker News searches. It scores everything, dedupes it against a persistent index, sorts it into categories, writes one markdown file, and emails it to me every morning. Nobody reads it but me.

And I mostly don’t read it either. It returns so much that opening it feels like a chore, and the only filter that actually works is me, skimming, on the days I bother.

I have thought about fixing it a few times, but just getting it to work in the first place was a pain in the ass. But that was probably the same month that OpenClaw came out and switched names three or four times. So models were dumber then.

All the scoring, deduping, and other stuff I had it do was simply because the list was too damn long. So this time I asked a model the question the way I’d been asking it in my head. Claude Sonnet 5, through OpenRouter, a paragraph describing the script, and then:

How do I fix it?

The answer was good. It told me I’d built “a collection pipeline, not a filtering pipeline,” then told me to rank everything, take a hard top N (“5, maybe 8”), and use an LLM as the judge against a written description of my taste. Tier the list. Archive the rest. Tune the prompt weekly.

I’d probably have built that. And I’d have ended up with a shorter file I don’t read.

This post is taking that problem and using a specific creativity method on it: take the thing apart before you try to solve it. It has four steps, each one a different tool for breaking things down. I also hand the tedious part to a model and keep the part it can’t do for myself.

• Step One: Ask Why Until the Problem Changes Shape
• Step Two: List What the Thing Is Made Of
• Step Three: Change All of Them at Once
• Step Four: Hand the Model the Tedious Part
  • What came back
  • Where it went wrong
• The Part You Can’t Hand Over
• The Cell I Circled

Step One: Ask Why Until the Problem Changes Shape

The first tool came out of a loom factory. Sakichi Toyoda, whose company became Toyota, is usually credited with the habit of asking “why” five times when something breaks, and Taiichi Ohno made it famous as part of the Toyota Production System. The first answer to “why” is almost always just the symptom.

First, on paper, with no AI model:

1. Why don’t I read the report? Because it’s too long.
2. Why is it too long? Because it returns everything that survives dedup, and dedup only removes repeating items. It does nothing with what I don’t care about.
3. Why doesn’t anything remove what I don’t care about? Because the scoring only sorts. It ranks each item by how much it looks like articles I’ve clipped, and then keeps all of them.
4. Why would a longer list be a problem at all? I could just skim. More items means I read fewer of them, not more. Digests are normally tuned on the assumption that more is better, and for a reader of one, that assumption runs backwards.
5. Why is it built as a daily digest? Because that’s what news reports look like. I never decided it. It was the default.

Look at where that landed. I started with “it’s too long” and ended with “I inherited a format I never chose, and the format sucks.” The model’s answer to the whole question was a fix for answer number one. A shorter daily digest is still a daily digest.

And five is not a magic number. You stop when the answer stops being about the symptom. Sometimes that’s three whys and sometimes it’s eight.

Step Two: List What the Thing Is Made Of

Now that I know the format is the problem, I need to see the format in parts.

Robert P. Crawford laid out the tool for this in his 1954 book The Techniques of Creative Thinking, and it’s called attribute listing. You write down every attribute of the thing you’re trying to improve, the literal ones, and then you change them one at a time and see what each change gives you.

Here’s my report, listed out:

Crawford’s move is to change one at a time. So:

• Change the volume: cap it at the top 8. That’s the model’s answer, almost word for word.
• Change the cadence: make it weekly. Fewer files, same problem, just once a week.
• Change the output: skip the email and drop it straight into my vault. Now it’s a long note I don’t read.

Every single-attribute change is a fix I’ve already thought of. Changing one thing at a time is how you improve a design you’re happy with. I’m not happy with this one. I need to change several attributes at once, and use a different tool.

Step Three: Change All of Them at Once

Fritz Zwicky was a Swiss astronomer at Caltech, the one who proposed dark matter in 1933 and spent the rest of his career being right a decade before everyone else. During the 1940s he did engineering work on jet and rocket propulsion at Aerojet, and that’s where he built the tool he spent his later years evangelizing. He called it the morphological approach. Most people now call the grid itself a Zwicky box.

The idea is simple, but the work is not:

1. Break the problem into independent parameters. If changing one forces a change in another, they’re the same parameter with two names.
2. For each parameter, list every value you can think of.
3. Walk all the combinations. The interesting answer is usually a combination nobody would have proposed as a whole.

My attribute list from step two is already most of the way to step one, which is why I did Crawford first. Turning attributes into parameters is mostly a matter of asking “what else could this be?” for each row:

That (c) under “Who filters” is not random. I clip articles into my Obsidian vault by hand, deliberately. Those clips are a record of what I found worth keeping, and the report already uses them, but only to decide what order to show me everything in.

The (c) under “Output” is the far end of that row: the version where the report stops being for me at all. I’d never pick it, but a Zwicky box is supposed to have values you’d never pick. That’s how you find out where the edges are.

3 X 4 X 3 X 3 X 3 is 324 combinations.

Now put the model’s first answer on this grid. It’s one cell: single item, model scoring, top N, daily, email. My current report is single item, nobody filters (the sort cuts nothing), no cap, daily, email. The answer to “how do I fix it?” changed two of the five values. It’s the cell next door.

That’s not a bad answer. It’s the average answer, the one that’s the closest to what I already have. Sure, a better question would get a better answer, but nobody opens with a Zwicky box. The default question buys the cell next door. There are 323 other cells and I hadn’t looked at any of them.

You can walk 324 combinations by hand. But you’d need an afternoon and a lot of coffee, and the first forty are going to make you want to stop. This is where most people quit.

Step Four: Hand the Model the Tedious Part

This is the part of the method a model was born for. Walking a grid is the kind of work nobody finishes by hand, and a model will grind through it without getting bored or skipping cells out of laziness.

There’s one step to add before the walk, though. Later practitioners of Zwicky’s method added a pass called cross-consistency assessment: go through every pair of values and cross out the ones that can’t coexist. If nobody filters, there’s no score, so “fixed top N” is meaningless. Any combination containing that pair is dead.

Here’s the prompt, after the same paragraph describing the script:

I'm running Zwicky's morphological analysis on this. I've already picked the
parameters and values. Your job is the tedious part.

[the five parameters and their values, as in the table above]

That's 3 x 4 x 3 x 3 x 3 = 324 combinations.

Do this:
1. Cross-consistency first. Go through every PAIR of values across parameters
   and flag the pairs that can't coexist or make no sense together. Explain
   each flag in a few words.
2. Count how many of the 324 combinations survive after removing every
   combination containing a flagged pair.
3. Walk every surviving combination. Give each a one-line description of what
   that system would actually be. Number them.
4. Mark the one combination that describes what I have now with [CURRENT].
5. Pick the five surviving combinations that are furthest from [CURRENT] and
   still buildable by one person in a weekend. For each, say what would change
   and what I'd lose.

Notice that the prompt doesn’t ask how to fix my report. It hands over a process I’d already started and tells the model which step I want it to do.

What came back

The cross-consistency pass took seconds and found ten conflicting pairs. Most were right. “Model scoring” plus “no cap” got flagged because “you’d be computing a score and then never using it to cut anything.” Fair. “Nobody filters” plus “only when something crosses a bar” got flagged because the bar needs something automated watching it. Also fair. It even flagged one of its own calls as shaky: “A couple of these, like F1, are judgment calls about what ‘keyword rules’ means.”

It counted 138 survivors out of 324: the ten flagged pairs killed 186 combinations between them.

Then it was supposed to walk all 138, one line each. It didn’t. It grouped them into 50 “shapes” and collapsed the unit parameter into a bracket, and its excuse was:

I’m collapsing Unit into a bracket rather than writing three near-identical lines each time; that’s the Zwicky box itself demonstrating your actual problem.

Which is funny. It got handed a list too long to be worth reading, and it clustered it while walking a grid where “cluster of items about the same story” was one of the options for my report.

Where it went wrong

Mostly in the pick, and that one’s on me.

“Furthest from current” was a dumb thing to ask for. I asked for the five combinations furthest from what I have, and it gave me exactly that: five combinations that flip all five parameters. Three of the five are keyword filters, which is the weakest filter on the grid. Two of them auto-post to my blog with no review, which is the far end of a row I put there knowing I’d never pick it. Distance from the current cell isn’t the same thing as interesting. I gave it a measurable criterion because measurable felt safe, and it measured.

The walk was fine. What it missed was meaning: what I actually wanted from the pick, which I never told it.

The Part You Can’t Hand Over

So I tried the obvious next experiment. Same paragraph about the script, but this time the model picks the parameters too:

I want to run Zwicky's morphological analysis on this. Step one only: list the
independent parameters of the problem, and for each, the possible values. Do
not solve anything yet.

It came back with twelve parameters. Purpose, source scope, relevance scoring basis, dedup granularity, output unit, volume control, target size, summarization depth, ordering, human feedback loop, novelty window, cadence. Between four and eight values each.

That’s roughly 102 million combinations.

It’s a thorough list, and most of the parameters are real. But it isn’t a Zwicky box. Three things go wrong with it:

• The parameters aren’t independent. “Volume control mechanism” includes “fixed top-N cutoff.” “Target size/length” includes “fixed count (e.g., top 10).” That’s the same thing listed twice. Step one of the method is independence, and the list fails it.
• One value can’t exist. Under the feedback loop it offers “implicit signal capture (what you actually open/click).” The script writes a markdown file and emails it to me. There’s no tracking, no server, nothing that sees a click. That idea came up once already in my planning notes and it dead-ends there.
• Nothing got cut. A list of everything that could vary is easy. Deciding which five things matter for this report isn’t.

But I’m not going to pretend it was useless, because its first parameter was better than anything on my list. Purpose: what would “working” even mean? Ambient awareness, triage for research, raw material to write from, a searchable archive, an early warning for a few specific things. I never asked that.

So the division of labor looks like this. The model is good at walking the grid and decent at suggesting things to put on it. Picking the parameters, cutting them down to the few that are independent and yours, and knowing what each value means is the judgment half, and it doesn’t transfer. The model doesn’t know what I’d do with a better report, and the parameters are where all of that lives.

I went looking for research on whether decomposition methods like this one work better when you run them with a model, and I couldn’t find anything I’d point you to. There are papers on combining ideas and papers on models converging on the same answers. Nobody seems to have tested the Zwicky box. So this is one run on one problem, and it’s worth exactly that much.

The Cell I Circled

With the purpose question added, the walk reads differently. This report has one job: keep me up to date on all this AI stuff, and now and then hand me something I haven’t run into, which sometimes turns into a post idea. It’s for me. Every cell that ends in “auto-post to blog” is dead on arrival, and so is every cell that makes me do the filtering, because that’s the one I’m already ignoring.

The one I circled: clusters of items about the same story, filtered by similarity to what I’ve clipped into my notes, capped at a top N, daily, in my inbox. That’s shape #32 in the model’s walk, with clusters as the unit. And the first answer is still in there: the cap, and a filter that actually cuts. The average answer is a special case of the walk.

The most useful sentence in the whole model run is about #44, a neighbor of that cell (same filter, a threshold instead of a top N), in the “what you’d lose” section:

discovery of genuinely new topics (similarity filters reinforce past interests, they don’t surface unknown unknowns)

That’s true. Half the point of this report is surfacing things I haven’t run into, and a filter built on what I’ve already clipped is going to just create an echo chamber. My report already sorts by exactly that similarity. So the cell needs a small slot for things that don’t look like my clips. That’s a sixth parameter I didn’t have, and I only found it because the model walked the cells carefully enough.

I haven’t built it yet. If I don’t start opening that email in the morning, the cell was wrong.

None of that is the answer I got when I asked how to fix my report. Taking it apart got me a format I never chose, five parts I’d been treating as one, 324 combinations, a purpose I’d never stated, and one cell I’d never have proposed as a whole, because as a whole it sounds like a weird way to read the news.

Next time you catch yourself asking a model to fix a problem, write down its parts first. Five whys, a list of attributes, a grid of values. You’ll pick the rows yourself, then hand the walk to the machine. It’s the one part of the job it will do all afternoon without complaining.