Skip to content
Ben StewartField manual

All chaptersChapter 4 · Working with AI

Entry 4.8

What Level Does This Task Deserve?

Two questions I ask before I hand anything to AI

7 min readFiled 3 October 20261,510 words

I'm a massive fan of AI Twitter. I know, I know, but it's the only Twitter vice I actually have, and it's how I keep up. The other day a post from Jaatster turned up and I spent some time thinking about it. It's where the idea for this post came from, so credit first; here's the actual tweet.

In much smaller type at the bottom of the card, it says "Workflow foundation: Level 5". Jaatster printed both numbers, which is more honest than most of the frameworks behind these cards, because a single number on a card has to be the best thing you've ever done with the tools.

The cards come from Every's Eight Levels of AI Adoption, a prompt that asks you five questions and hands you a level. I took it with one change, which was to skip the questions and tell the AI to go and read a month of my own session transcripts and grade me on what I'd actually done.

That month I spent a lot of time working with AI on some deep projects, so seven of those sessions were substantial, and the biggest had 22 prompts from me. It came back with a Level 5 (no shame in that), with medium to high confidence and a range of 3 to 7. Of course I'd have liked to score myself higher, but you answer five questions thinking about your best day and call it a level.

sessions in a month
24
tool calls in one session
388
subagents in one go
37
my level (3 to 7)
5

When I looked at how it graded me and worked out where the 3s and the 7s came from, they followed the pattern of work I'd been doing over the past two weeks.

Why everyone wants a level

It doesn't matter who I talk to, they're worried they're behind on AI. I spoke to someone this week who really meant it and was genuinely worried about it. They'd been watching what their colleagues were doing, and what some other teams were doing, and they'd decided they were miles off the pace. Once we talked through how they actually work, I didn't reckon they were that far behind at all, and that's a pattern I keep seeing. There's real existential dread out there, yet when people lift their heads above the parapet they find that, overall, they're not as far behind as they think they are.

That's where the appeal of a level comes from. Hardly anyone can show the ROI on any of this yet, so there's no real way to show how far ahead you are. A number is the next best thing. It turns "am I behind?" into "I'm a 4" and you can wear it like a badge. A number also fits on a slide, and once it's on a slide it becomes a target.

The idea of levels comes from self-driving cars, which I had to look up. In 2014 SAE International, which writes many of the car industry's engineering standards, set out six levels of driving automation, running from Level 0, where you do all the driving, to Level 5, where the car drives itself anywhere and in any conditions. AI frameworks have borrowed that shape too (Google DeepMind said so outright when it proposed levels of AGI in 2023). It works for cars because each level describes how much of the driving the machine has taken over. Point the same idea at a person and it sounds rigorous and isn't. All the frameworks I've read say higher isn't better, then list the signs you're ready for the next level, which tells you which way they expect you to go.

I don't want to be grumpy about this, because the levels gave me something. I can now say "that was a job for a 3, whereas that one needed a little bit more of a 5" and mean something specific, and I couldn't do that a month ago. I'd keep that, and drop the idea that the number describes me.

Two questions

What the levels did make me do, for the first time in a while, was stop and ask how I'm actually using AI, and how I can deliberately look at my workflow, my thinking and my systems thinking to make sure I'm getting the best out of it. The models are getting so good (Fable and Opus 5.5 from Anthropic, and the latest ChatGPT) that I can do things in September and October 2026 that I couldn't do in March. These are the two questions I keep asking myself:

Checking doesn't have to mean reading every line myself. It can mean getting another model to verify it, or being able to trace each claim back to where it came from. If I can't do any of those, I can't check it.

Put those on two axes and you get a grid.

Expensive if wrong · Easy to check

Level5

Let it work, then check everything

Anything senior leadership will read

Expensive if wrong · Hard to check

Level3

Stay close

Thinking work

Cheap if wrong · Easy to check

Level4 / 7

Let go, check a sample

Finishing a pattern, mechanical fan-out

Cheap if wrong · Hard to check

Level6

Ask whether it should run at all

The weekly research sweep, switched off in July

Thinking work sits top right. Working out what I actually believe about something is expensive to get wrong, because I end up holding a weaker idea and presenting it as mine, and there's no answer key. My instruction there is nearly always some version of "just tell me what you would do, don't do it". That's the 3 on my card, on purpose. Some of it shouldn't go near the tool at all yet.

Anything going to senior leadership, peers or stakeholders is expensive too, though you can check it, because numbers trace back to sources. So the AI does a lot of the work and I say "I need every single number and claim checked" before any of it goes in front of senior people. That's where the 5 came from. The AI does more here than on the thinking work and I trust it less, which still feels slightly backwards to me.

Bottom left is where I let go. Once I've done the first few of something and know what good looks like, I say "do everything that's left" and stop watching. That's a 4. The 37 subagents live here too, a manager model handing a pile of repetitive linking work out to cheaper models in one session. I didn't read 37 outputs. The highest number on my card, the 7, came from the most boring work I did all month.

Bottom right is cheap and hard to check. I used to run a weekly automated research sweep, and nothing it found was worth reading unless I'd gone looking for it. I switched it off in July and run the research when I need it. There are no scheduled AI jobs on my machine now, so Level 6 on my card is a zero.

In a month of reading about levels, I didn't find anyone arguing for stepping down a level. The closest was u/Fyren-1131 on r/ExperiencedDevs.

I am trying to find the level of AI use that makes sense for me.
u/Fyren-1131, r/ExperiencedDevs

Somebody with a Level 8 card and somebody switching things off can both be doing that well.

If I could change one thing about level frameworks, Every's included, I'd score the task and leave the person out of it. A person's level is just the mix of tasks on their desk this week. Mine moved between 3 and 7 inside a month and I didn't get any better or worse at this in between.

We're all ahead or behind depending on what day it is, and on what's just been released over the past week. My current mindset is that as long as AI is making big or deep tasks easier for me, I must be doing something right, and when that stops being the case it's time to re-level myself.

A single trading card reading Level 5, range 3 to 7, beside a hand of five task cards: thinking work at Level 3, finishing a pattern at 4, anything leaders read at 5, mechanical fan-out at 7, and a face-down Level 6 card marked switched off.

In short

I asked AI to grade my AI level from a month of my own session transcripts. It came back as Level 5 with a range of 3 to 7, and where I landed each time depended on the task in front of me. So I've stopped asking what level I am and ask two questions of every task instead. What does it cost if the output is wrong, and can I check what comes back? Score the task and leave the person out of it.


How this was written: Claude read the month of transcripts, three different models drafted versions of this post, two blind judges scored them, and I rewrote the winner by hand.

This is where Two questions in the toolkit comes from.