A measure defined by AI Search Leaders, November 2026

Task gain

Can AI actually do the job, and which brand gets the business?

Citations
Does the assistant talk about your brand?
Task gain
Can it complete a real task on your site, and who wins the transaction?
The task gain scale
  1. 0Refused
  2. 1Advice only
  3. 2Stalled
  4. 3Decision point
  5. 4Done with help
  6. 5Task done

From refusing the task to completing it, with only the final payment left to the human. Up to three runs per assistant, the median reported.

  1. 01

    Task completion rate

    Did the assistant finish the job on a live site?

  2. 02

    Task gain

    How far did it get, and which brand won the business?

  3. 03

    Agent experience score

    Can agents get through your site?

By Jeremy Cabral (lead author) and Tom Critchlow · AI Search Leaders · Updated 4 November 2026

In progress

Round 1: all three passes in, 56 runs. Round 2 starts on 9 November.

The future of AI search is task completion

People are starting to hand assistants the whole errand. Book the flight, fill in the form, buy the thing. When they do, the customer never reads an answer or sees a citation, and the brand that wins is the one the agent can actually get the job done with.

I think task completion rate becomes the number that matters for AI search, the way rankings and then clicks did before it. It’s the first visibility measure I know of that ends in a transaction, and almost no brand can see its own. So we built a way to measure it, and ran the first round.

Twenty assistants, one flight

Sydney to Melbourne and back, one adult, one checked bag, the cheapest total fare, taken to the payment screen. The same prompt, pasted word for word into every assistant and agent we could sign into, up to three times each, on 3 and 4 November 2026.

No agent was given a card, a password or a one-time code, so nothing was booked. Reaching the payment screen with the right flights in the cart counts as done.

runs
56runs20 products from 14 vendors, up to three runs each, on 3 and 4 November 2026
payment screens
23payment screens41 percent reached one with the right itinerary in the cart
bookings
0bookingsNo card was entered and nothing was paid, by design
cheapest fare
A$355cheapest fareJetstar, with a bag. 12 runs got there
paid more
11paid moreCompletions that landed on a dearer fare, mostly when Jetstar’s site failed the agent
Every scored run, by assistant and score
  • Jetstar payment screen
  • Virgin Australia payment screen
  • Trip.com payment screen
  • Didn’t reach a payment screen
Assistant012345Med.

Median 4 · Reached a payment screen in a typical run

ClaudeVVV4
Claude CodeJVJ4
Grok BotJJJ4
OpenAI dotVVV4
Perplexity ComputerJJ4
CodexJJ4
ManusVV4
Meta MuseTV4
DeepSeek HarnessJJ4

Median 3 · Got to the decision point

ChatGPT WorkJ3

Median 2 or below · Advised, or started and handed back

Grok2
Kimi2
Hermes Agent2
ChatGPT1
Gemini1
Gemini Tasks1
Genspark1
Microsoft Copilot1
OpenClaw1

Not run: Antigravity (pending), Instinct (waitlisted), Mistral Le Chat (pending), Qwen (pending). Hermes’s second run has no score: its quota had run out.

Show as a table
AssistantMedianScoresPayment screensWhat happened
ChatGPT11, 1, 10 of 3Pointed to Work mode every time, with a fare in one run of three
ChatGPT Work33, 3, 41 of 3, JetstarA connection error, then an expired session, then Jetstar’s Review & Pay at A$355
OpenAI dot44, 4, 43 of 3, Virgin AustraliaA$362 each time. Jetstar’s site failed it in the first two runs and it didn’t try in the third
Codex43, 4, 42 of 3, JetstarStuck at Jetstar’s passenger page, then A$355 after a 13-minute stall that may have been our fault, then the inherited checkout covered below
Claude44, 4, 43 of 3, Virgin AustraliaA$370, A$370, A$362. Jetstar gave it a protocol error, then a reCAPTCHA it declined twice
Claude Code44, 4, 43 of 3, Jetstar and Virgin AustraliaJetstar at A$355 twice, Virgin at A$362 when Jetstar rejected its browser
Gemini11, 1, 10 of 3Advice and a refusal. Quoted Jetstar at A$289 with a bag, a total no agent could reach
Gemini Tasks11, 1, 10 of 3Advice only, with Jetstar estimates that moved between about A$253 and A$350
Perplexity Computer43, 4, 52 of 3, JetstarStopped at the passenger page, then A$355 with one question, then A$355 in 5 minutes 8 seconds with none
Manus44, 3, 42 of 3, Virgin AustraliaA$362 twice. Both airlines' checkouts failed it in pass 2
AntigravitypendingInstalled during pass 1; no run yet
Meta Muse43, 4, 42 of 3, Trip.com and Virgin AustraliaTrip.com at A$405.80 for a Jetstar seat, then Virgin at A$362. Jetstar’s own site never loaded for it
Instinctwaitlisted“Flight booking isn’t available to you while you’re on the waitlist.”
Hermes Agent220 of 1Jetstar’s bags dialog, then its daily quota ran out. Still out at run 2
OpenClaw12, 00 of 2Never loaded a page, then a model settings error stopped it at once
Grok22, 2, 20 of 3Started, reported bot checks across the travel sites, handed back
Grok Bot44, 4, 43 of 3, JetstarA$355 every time, in 75, 31 and 27 minutes
DeepSeek Harness41, 4, 52 of 3, JetstarRefused to quote, then A$355 with one question, then A$355 in 8 minutes with none
Microsoft Copilot10, 1, 10 of 3No booking connector. Fares and advice
Kimi21, 2, 20 of 3A shortlist, then two attempts that ended at blocks on the airline sites
Genspark11, 1, 10 of 3Research and links. “I have no booking, payment or hold capability”
Mistral Le ChatpendingNot run
QwenpendingSigned-out marketing page; not run

Each square is one run, placed in the column of its score; hover one for its details.

Products with a browser of their own completed 21 of 30 runs, and chat-only products completed 2 of 24. The line moved while we watched, too: DeepSeek Harness refused to quote a price in pass 1 and reached a payment screen in passes 2 and 3. Advisers and agents are two different lists, and a product can move from one to the other without telling you.

Who got the booking

Jetstar had the cheapest fare, A$355 with a bag, and 12 runs got there. Its site also errored, blocked or challenged agents in at least 19 runs across nine products, and nine of Virgin’s ten wins came in a run where Jetstar’s site had just failed that agent.

None of that would show up in Jetstar’s analytics. The agent never becomes a session.

Where the 23 completions landed, by fare
A$355CheapestJetstar12
A$362+A$7Virgin Australia8
A$370+A$15Virgin Australia2
A$386–475+A$31 or moreQantasEligible in 34 runs, chosen in none
A$405.80+A$50.80Trip.com1

11 of 23 completions landed on a dearer fare than the one the prompt asked for.

Totals with one checked bag each way, as each payment screen showed them. Qantas’s range is from the runs that priced it: the agents doing the sum correctly.

Brand task gain

Jetstar

Mentioned in 36 runs

0.24 · 12 of 50

Virgin Australia

Mentioned in 28 runs

0.24 · 10 of 42

Trip.com

Mentioned in 5 runs

0.50 · 1 of 2

Qantas

Mentioned in 10 runs

0 · 0 of 34

Rex

Mentioned in 3 runs

0 · 0 of 2

Runs where it was a live option Runs it won

Brand task gain: the runs a brand won, over the runs where it was a live option. Trip.com’s 0.50 is one win in two chances.

“Jetstar’s site is failing, so I can’t verify its direct price with a bag.”
OpenAI dot · Minute 16
Claude’s browser pane on booking.jetstar.com showing “This site can’t be reached” and ERR_HTTP2_PROTOCOL_ERROR, beside a prompt asking to allow Claude to use the browser on the site.
Claude · Minute 2Jetstar’s booking engine refused Claude’s browser outright. Later passes brought a reCAPTCHA, which Claude declined. It ended on Virgin every time.
Manus reporting that the Virgin Australia booking is at the card-entry screen at A$362.00, with a Take control button.
Manus · 57 minutesJetstar’s booking page had failed, so Manus asked permission to switch, and stopped at Virgin’s card entry at A$362.
Grok Bot’s card reading “Action needed: Enter your card and pay A$355.00 on Jetstar, then hand back”, over a screenshot of Jetstar’s checkout page, with Skip, I’m done and Take over buttons.
Grok Bot · Minute 75Grok Bot got through. It stopped at Jetstar’s checkout at A$355 and asked for the card. Nobody pressed Take over, and nothing was paid.

Agents that remember

This is the finding we didn’t go looking for. In pass 1 every completing agent stopped at the passenger form. By pass 3 two of them didn’t, because they’d met the tester before, and they scored the only two 5s in the study.

The caveat is a big one: the memory came from earlier runs of this same study. A customer’s first attempt won’t look like that. I suspect their tenth will.

Perplexity Computer

Pass 1

3

20 min
2 interventions

Stopped at the passenger page

Pass 3

5

5 min
No interventions

Reached Jetstar’s payment page

DeepSeek Harness

Pass 1

1

1 min
No interventions

Refused to quote a price

Pass 3

5

8 min
No interventions

Rebuilt an expired cart and reached payment

Perplexity Computer’s saved notes from earlier sessions “meant I didn’t need to ask you anything.”
Perplexity Computer · Debrief, pass 3

Memory gets things wrong too. Claude Code proposed an old email address it remembered, and Codex pulled an email and postcode from a previous itinerary. An agent that stops asking also stops being checked.

Perplexity Computer, finished in 5m 8s: the booking is on Jetstar’s payment page at A$355.00 return, nothing charged.
Perplexity Computer · Finished in 5m 8s · Pass 3Jetstar’s payment page at A$355.00 return, a checked bag each way, nothing charged, and nothing asked.

Where the humans came in

We went in expecting logins and CAPTCHAs. Logins never happened, and 30 of the 40 interventions were the passenger form: title, name as on ID, date of birth, mobile, postcode.

In four runs Jetstar’s session timed out while the agent waited for a person to answer. The gap between an agent asking and a human answering is now a failure mode with a brand attached.

Completion by what the product had to work with

Chat only

2 of 24 completed · 0.08

JJ

Its own cloud browser

13 of 18 completed · 0.72

JVVVJJVVTVJJJ

A browser on the tester’s machine

8 of 12 completed · 0.67

JJVVVJVJ

Each square is a scored run, filled if it reached a payment screen. The two chat-only completions are DeepSeek Harness, once it started driving a browser.

The 40 human interventions
  • 30 passenger details: title, name as on ID, date of birth, mobile, postcode
  • 10 everything else: permissions, CAPTCHAs, corrections, setup

As the logs record them

Typed answers
16
Choices
9
Clicks
3
CAPTCHAs
2
Corrections
2
Other
8
Jetstar’s booking page in ChatGPT Work’s browser, reading “Your session expired but we’re still here to help get you there.”
ChatGPT Work · Jetstar’s passenger pageIt declined the insurance upsell on its own and asked for the traveler’s details. By the time the answer came, the session had expired. The agent did everything right and the site’s clock beat the human.

What to do about it

Measure it first. The rubric is published, and three runs per product is enough to start. I think the first result will tell you more than a year of citation tracking.

Then treat agent experience as a discipline beside user experience: price visible without a login, forms an accessibility tree can operate, sessions that outlast a human answering a question, and a bot policy that can tell a customer’s agent from a scraper. A stall that survives two runs on two days, in the same place, is yours to fix.

What two agents asked airline sites for
  1. 01

    A booking guide at a stable address

    Each step of the flow and what it requires.

  2. 02

    A documented deep-link format

    With a worked example and its error responses.

  3. 03

    A machine-readable fare table

    Bag allowance, seat rules, change fees and every mandatory fee.

  4. 04

    A statement on holds

    Whether a price can be held, and for how long.

  5. 05

    Session rules

    The idle timeout, how to extend it, what survives expiry and what automation is permitted.

  6. 06

    A customer handover step

    The agent prepares the booking; the customer reviews it and pays.

Condensed from the debriefs of Claude Code and OpenAI’s dot, 4 November 2026. Asked what would have helped, they wrote the same list.

The whitepaper · Read online or as a PDF

Read the whitepaper

The case for task completion as the measure of AI search, the three numbers we think come next, every run of round 1, and an honest section on what one round can’t tell you. It opens with a two-page summary.

Round 2 starts on 9 November

Send us one task your customers try to complete with an assistant.

It goes in the next matrix, run across every product in Australia and the US, with right of reply for each vendor. In three months you’ll know who got the booking.

Send us a task

Round 2’s tasks

  1. 01A hotel near a venue
  2. 02Airport car hire
  3. 03A restaurant booking
  4. 04Two ecommerce checkouts
  5. 05A Black Friday deal, before the sales and on the day
  6. 06A SaaS team signup
  7. 07A B2B quote and demo
  8. 08A cited research briefing, the control

The next stretch

Don't get
left behind.

Join 150+ senior search people working out AI search together.