A measure defined by AI Search Leaders, November 2026
All researchTask gain
Can AI actually do the job, and which brand gets the business?
- Citations
- Does the assistant talk about your brand?
- Task gain
- Can it complete a real task on your site, and who wins the transaction?
- 0Refused
- 1Advice only
- 2Stalled
- 3Decision point
- 4Done with help
- 5Task done
From refusing the task to completing it, with only the final payment left to the human. Up to three runs per assistant, the median reported.
01
Task completion rate
Did the assistant finish the job on a live site?
02
Task gain
How far did it get, and which brand won the business?
03
Agent experience score
Can agents get through your site?
By Jeremy Cabral (lead author) and Tom Critchlow · AI Search Leaders · Updated 4 November 2026
Round 1: all three passes in, 56 runs. Round 2 starts on 9 November.
The future of AI search is task completion
People are starting to hand assistants the whole errand. Book the flight, fill in the form, buy the thing. When they do, the customer never reads an answer or sees a citation, and the brand that wins is the one the agent can actually get the job done with.
I think task completion rate becomes the number that matters for AI search, the way rankings and then clicks did before it. It’s the first visibility measure I know of that ends in a transaction, and almost no brand can see its own. So we built a way to measure it, and ran the first round.
Twenty assistants, one flight
Sydney to Melbourne and back, one adult, one checked bag, the cheapest total fare, taken to the payment screen. The same prompt, pasted word for word into every assistant and agent we could sign into, up to three times each, on 3 and 4 November 2026.
No agent was given a card, a password or a one-time code, so nothing was booked. Reaching the payment screen with the right flights in the cart counts as done.
- runs
- 56runs20 products from 14 vendors, up to three runs each, on 3 and 4 November 2026
- payment screens
- 23payment screens41 percent reached one with the right itinerary in the cart
- bookings
- 0bookingsNo card was entered and nothing was paid, by design
- cheapest fare
- A$355cheapest fareJetstar, with a bag. 12 runs got there
- paid more
- 11paid moreCompletions that landed on a dearer fare, mostly when Jetstar’s site failed the agent
- Jetstar payment screen
- Virgin Australia payment screen
- Trip.com payment screen
- Didn’t reach a payment screen
Median 4 · Reached a payment screen in a typical run
Median 3 · Got to the decision point
Median 2 or below · Advised, or started and handed back
Not run: Antigravity (pending), Instinct (waitlisted), Mistral Le Chat (pending), Qwen (pending). Hermes’s second run has no score: its quota had run out.
Show as a table
| Assistant | Median | Scores | Payment screens | What happened |
|---|---|---|---|---|
| ChatGPT | 1 | 1, 1, 1 | 0 of 3 | Pointed to Work mode every time, with a fare in one run of three |
| ChatGPT Work | 3 | 3, 3, 4 | 1 of 3, Jetstar | A connection error, then an expired session, then Jetstar’s Review & Pay at A$355 |
| OpenAI dot | 4 | 4, 4, 4 | 3 of 3, Virgin Australia | A$362 each time. Jetstar’s site failed it in the first two runs and it didn’t try in the third |
| Codex | 4 | 3, 4, 4 | 2 of 3, Jetstar | Stuck at Jetstar’s passenger page, then A$355 after a 13-minute stall that may have been our fault, then the inherited checkout covered below |
| Claude | 4 | 4, 4, 4 | 3 of 3, Virgin Australia | A$370, A$370, A$362. Jetstar gave it a protocol error, then a reCAPTCHA it declined twice |
| Claude Code | 4 | 4, 4, 4 | 3 of 3, Jetstar and Virgin Australia | Jetstar at A$355 twice, Virgin at A$362 when Jetstar rejected its browser |
| Gemini | 1 | 1, 1, 1 | 0 of 3 | Advice and a refusal. Quoted Jetstar at A$289 with a bag, a total no agent could reach |
| Gemini Tasks | 1 | 1, 1, 1 | 0 of 3 | Advice only, with Jetstar estimates that moved between about A$253 and A$350 |
| Perplexity Computer | 4 | 3, 4, 5 | 2 of 3, Jetstar | Stopped at the passenger page, then A$355 with one question, then A$355 in 5 minutes 8 seconds with none |
| Manus | 4 | 4, 3, 4 | 2 of 3, Virgin Australia | A$362 twice. Both airlines' checkouts failed it in pass 2 |
| Antigravity | pending | Installed during pass 1; no run yet | ||
| Meta Muse | 4 | 3, 4, 4 | 2 of 3, Trip.com and Virgin Australia | Trip.com at A$405.80 for a Jetstar seat, then Virgin at A$362. Jetstar’s own site never loaded for it |
| Instinct | waitlisted | “Flight booking isn’t available to you while you’re on the waitlist.” | ||
| Hermes Agent | 2 | 2 | 0 of 1 | Jetstar’s bags dialog, then its daily quota ran out. Still out at run 2 |
| OpenClaw | 1 | 2, 0 | 0 of 2 | Never loaded a page, then a model settings error stopped it at once |
| Grok | 2 | 2, 2, 2 | 0 of 3 | Started, reported bot checks across the travel sites, handed back |
| Grok Bot | 4 | 4, 4, 4 | 3 of 3, Jetstar | A$355 every time, in 75, 31 and 27 minutes |
| DeepSeek Harness | 4 | 1, 4, 5 | 2 of 3, Jetstar | Refused to quote, then A$355 with one question, then A$355 in 8 minutes with none |
| Microsoft Copilot | 1 | 0, 1, 1 | 0 of 3 | No booking connector. Fares and advice |
| Kimi | 2 | 1, 2, 2 | 0 of 3 | A shortlist, then two attempts that ended at blocks on the airline sites |
| Genspark | 1 | 1, 1, 1 | 0 of 3 | Research and links. “I have no booking, payment or hold capability” |
| Mistral Le Chat | pending | Not run | ||
| Qwen | pending | Signed-out marketing page; not run |
Each square is one run, placed in the column of its score; hover one for its details.
Products with a browser of their own completed 21 of 30 runs, and chat-only products completed 2 of 24. The line moved while we watched, too: DeepSeek Harness refused to quote a price in pass 1 and reached a payment screen in passes 2 and 3. Advisers and agents are two different lists, and a product can move from one to the other without telling you.
Who got the booking
Jetstar had the cheapest fare, A$355 with a bag, and 12 runs got there. Its site also errored, blocked or challenged agents in at least 19 runs across nine products, and nine of Virgin’s ten wins came in a run where Jetstar’s site had just failed that agent.
None of that would show up in Jetstar’s analytics. The agent never becomes a session.
11 of 23 completions landed on a dearer fare than the one the prompt asked for.
Totals with one checked bag each way, as each payment screen showed them. Qantas’s range is from the runs that priced it: the agents doing the sum correctly.
Jetstar
Mentioned in 36 runs
0.24 · 12 of 50
Virgin Australia
Mentioned in 28 runs
0.24 · 10 of 42
Trip.com
Mentioned in 5 runs
0.50 · 1 of 2
Qantas
Mentioned in 10 runs
0 · 0 of 34
Rex
Mentioned in 3 runs
0 · 0 of 2
Brand task gain: the runs a brand won, over the runs where it was a live option. Trip.com’s 0.50 is one win in two chances.
“Jetstar’s site is failing, so I can’t verify its direct price with a bag.”



Agents that remember
This is the finding we didn’t go looking for. In pass 1 every completing agent stopped at the passenger form. By pass 3 two of them didn’t, because they’d met the tester before, and they scored the only two 5s in the study.
The caveat is a big one: the memory came from earlier runs of this same study. A customer’s first attempt won’t look like that. I suspect their tenth will.
Perplexity Computer
Pass 1
3
20 min
2 interventions
Stopped at the passenger page
Pass 3
5
5 min
No interventions
Reached Jetstar’s payment page
DeepSeek Harness
Pass 1
1
1 min
No interventions
Refused to quote a price
Pass 3
5
8 min
No interventions
Rebuilt an expired cart and reached payment
Perplexity Computer’s saved notes from earlier sessions “meant I didn’t need to ask you anything.”
Memory gets things wrong too. Claude Code proposed an old email address it remembered, and Codex pulled an email and postcode from a previous itinerary. An agent that stops asking also stops being checked.

Where the humans came in
We went in expecting logins and CAPTCHAs. Logins never happened, and 30 of the 40 interventions were the passenger form: title, name as on ID, date of birth, mobile, postcode.
In four runs Jetstar’s session timed out while the agent waited for a person to answer. The gap between an agent asking and a human answering is now a failure mode with a brand attached.
Chat only
2 of 24 completed · 0.08
Its own cloud browser
13 of 18 completed · 0.72
A browser on the tester’s machine
8 of 12 completed · 0.67
Each square is a scored run, filled if it reached a payment screen. The two chat-only completions are DeepSeek Harness, once it started driving a browser.
- 30 passenger details: title, name as on ID, date of birth, mobile, postcode
- 10 everything else: permissions, CAPTCHAs, corrections, setup
As the logs record them
- Typed answers
- 16
- Choices
- 9
- Clicks
- 3
- CAPTCHAs
- 2
- Corrections
- 2
- Other
- 8

What to do about it
Measure it first. The rubric is published, and three runs per product is enough to start. I think the first result will tell you more than a year of citation tracking.
Then treat agent experience as a discipline beside user experience: price visible without a login, forms an accessibility tree can operate, sessions that outlast a human answering a question, and a bot policy that can tell a customer’s agent from a scraper. A stall that survives two runs on two days, in the same place, is yours to fix.
01
A booking guide at a stable address
Each step of the flow and what it requires.
02
A documented deep-link format
With a worked example and its error responses.
03
A machine-readable fare table
Bag allowance, seat rules, change fees and every mandatory fee.
04
A statement on holds
Whether a price can be held, and for how long.
05
Session rules
The idle timeout, how to extend it, what survives expiry and what automation is permitted.
06
A customer handover step
The agent prepares the booking; the customer reviews it and pays.
Condensed from the debriefs of Claude Code and OpenAI’s dot, 4 November 2026. Asked what would have helped, they wrote the same list.
Round 2 starts on 9 November
Send us one task your customers try to complete with an assistant.
It goes in the next matrix, run across every product in Australia and the US, with right of reply for each vendor. In three months you’ll know who got the booking.
Send us a taskRound 2’s tasks
- 01A hotel near a venue
- 02Airport car hire
- 03A restaurant booking
- 04Two ecommerce checkouts
- 05A Black Friday deal, before the sales and on the day
- 06A SaaS team signup
- 07A B2B quote and demo
- 08A cited research briefing, the control
The next stretch
Don't get
left behind.
Join 150+ senior search people working out AI search together.
