Skip to main content
DanManREAL/ AI_LAB
Updated Nov 20, 2026Request a test

Field testTested AIBy Published Tested Updated

Vercel Agent Desktop used a sample Trip app in 6 actions without moving the mouse

The report

Field test

Verdict

Trip task done · mouse and focus untouched

In one controlled Linux run, Agent Desktop 1.0.2 set a sample Trip app to four days, picked Ryokan and ticked all four packing items in six actions, and the app's own state confirms it. The mouse pointer never moved and a second window kept focus the whole time.

On the tier list: A tool demonstration with a fixed list of commands, not a model doing the work, so it places no agent on the tier list.

One small job: plan the trip

Agent Desktop is a command-line tool from Vercel Labs that lets an AI agent read and press the controls of desktop apps. On November 1, my MythREAL_FACTORY video agent gave version 1.0.2 one job on Linux: in a small Trip planner app built for this episode, add a day (3 to 4), switch the stay from Hotel to Ryokan, and tick four packing items, while a second window called Focus Pad kept focus. This is a sample app, not Vercel's own Trip demo.

The tool reads an app through its accessibility tree, the same description of buttons and checkboxes that screen readers use. Each snapshot gives every control a short reference, like e1 for "Add a day", and an action names that reference instead of a spot on the screen. Before each of the six actions, the run took a fresh snapshot, found the control by its name and used the reference from that snapshot.

What happened

All six commands returned OK. The app's own saved state went from 3 days, Hotel and nothing packed to 4 days, Ryokan and all four items packed, and the last snapshot read "4 of 4 packed". The mouse pointer stayed in the centre of the screen, and the active window was the same one before and after: Focus Pad.

The tool is careful about what it claims. Its four checkbox commands came back verified: it read each box afterwards and saw it ticked. The two button clicks came back unverified, meaning sent but not confirmed by the tool. The next snapshot is what showed they worked, so it pays to check the app after a click.

It took some setup first

The first five tries to list apps or windows were refused, because the tool could not prove what kind of window it was looking at on the private test display. They worked once the display was reached through a different connection. The app also listed itself as "python3.12", not "Trip", so asking for it by name failed and the run picked its window instead. The app was relaunched once to fix the day-count label that accessibility tools read. Clear control names are what this tool runs on.

Where it fits

  • Agent Desktop: native apps whose controls show up in the accessibility tree.
  • A browser tool such as agent-browser or Playwright: web pages.
  • Screenshots and a model that can see: how an app looks, or controls the accessibility tree doesn't describe.
  • Cua Driver offers similar desktop control. Nothing here compared the two, so this is no reason to replace a working Cua setup.
Context, not results: the Apple Podcasts example with eight passing tests discussed in the video is Oskar Kwasniewski's own experiment, and the GarageBand and drag-and-drop clips are older footage from the Agent Desktop contributors. This page's numbers come from the run's saved commands, outputs, app state and recording, all read again for this page. The raw files stay private.

Limits of this report

  • One run of one small sample app on Linux. Other apps, Windows and macOS were not tried; on my Windows PC the tool was only asked for its version.
  • The six actions were a fixed list written ahead of time: a script named each control by its label and looked up its reference in a fresh snapshot. No model chose actions live, so this says nothing about how well a model drives the tool or how often it succeeds.
  • Saving was not tested: the changes lived in the running app only.
  • The test display was a private virtual screen, not a desktop someone was working on.

Follow the lab

New test reports and tier-list changes get announced in my Discord's #news channel. The full videos are on YouTube.

Receipts

8 items

DanManREAL · AI lab

Test receipt

Field testNov 1, 2026

Agent Desktop 1.0.2 by Vercel Labs (Linux x86_64)

Tool and version
Agent Desktop 1.0.2 · Linux x86_64the tool's own version answer on the test display
Task result, read from the app itself
3 → 4 days · Hotel → Ryokan · 0 → 4 of 4 packedPassport, Rail pass, Power adapter and Rain jacket; the last snapshot read "4 of 4 packed"
Actions
6 · 2 clicks, 4 checks · all returned OKeach one used a control reference from a snapshot taken just before it
The tool's own confirmation
4 checks verified · 2 clicks unverifiedunverified means sent but not confirmed by the tool; the next snapshot showed 4 days, then Ryokan selected
Mouse pointer and active window
pointer at 960,540 before and after · same active windowFocus Pad, a separate window, was focused just before the run and was still the active window after it
Run length
23.2 sfirst command to final snapshot, with pauses added for the recording, so not a speed result
Setup before the run
5 listings refused · 1 app relaunchthe tool could not prove the window type on the test display until it was reached through a different connection; the app was relaunched to fix its day-count label
How the app showed up
as python3.12, not Tripasking for the app by name failed, so the run picked its window by id

Verdict

Trip task done · mouse and focus untouched

Tier list A tool demonstration with a fixed list of commands, not a model doing the work, so it places no agent on the tier list.

danmanreal.com/tests/vercel-agent-desktop-trip-sample

What was tested

Agent Desktop 1.0.2 by Vercel Labs (Linux x86_64)A small GTK3 Trip planner app built for this episodeA private Linux test display (Xvfb with the Openbox window manager)

Shorts

Related

Independent testing. Sponsorships are always disclosed, and verdicts are never for sale.

Disagree? Bring your receipts

All reports

36 reports