Holo3 Review — Open-Source Computer Use Agent That Outperforms GPT-5.4

What Is Holo3?

Holo3 is a vision-language model built specifically for computer use — the kind of AI that looks at your screen, understands what it sees, and takes actions like clicking, typing, and navigating menus. H Company released it on April 1, 2026, alongside a research paper claiming state-of-the-art results on the OSWorld benchmark.

Most large language models treat computer use as an afterthought. You bolt a screenshot tool onto GPT or Claude, feed it pixel data, and hope the model figures out where to click. Holo3 was designed from the ground up for this workflow. The training pipeline uses a continuous feedback loop where the model alternates between perceiving screen states and making decisions about what to do next.

That architectural focus matters. General-purpose models waste capacity on language tasks that computer use doesn't need. Holo3 trades broad capability for depth in one specific domain: understanding GUIs and acting on them.

The OSWorld Benchmark Score, Explained

OSWorld-Verified is a standardized test for computer use agents. It gives the model a virtual machine with a desktop environment and assigns tasks like "open a spreadsheet, find the average of column B, and paste it into a new email." The model has to figure out each step on its own — no hand-holding, no pre-defined action sequences.

Holo3 scored 78.85% on this benchmark. For context, GPT-5.4 with computer use scored around 72.4%, and Claude Opus 4.6 Computer Use sits near 38%. Previous open-source models were below 30%.

That 78.85% number needs a caveat, though. OSWorld tasks are designed to have clear success criteria — the grader checks whether the final state matches the expected output. Real computer use involves ambiguity, unexpected popups, network latency, and interfaces that change between visits. A model that passes 78.85% of controlled lab tasks will not succeed at 78.85% of whatever you throw at it in production.

Still, the gap between Holo3 and everything else is significant. Going from 72% to 79% might not sound dramatic, but in practical terms it means fewer retries, fewer stuck states, and more tasks completing without human intervention.

Two Models, Two Price Points

H Company released two versions, which is an unusual move for a model at this performance level:

Both use Mixture-of-Experts (MoE) architecture, which means only a fraction of the total parameters activate per inference pass. That's why the 35B model can run on consumer hardware — it's really using about 3B parameters at any given moment.

The pricing on the API model is aggressive. Claude Computer Use through the API costs roughly $15 per 1,000 screenshots when you factor in input tokens for each image. Holo3's API at $0.40/$3.00 per million tokens works out to about $1.50 for the same workload. That's a 10x cost reduction, which matters when you're running thousands of automated tasks.

Holo3 vs Claude Computer Use vs GPT-5.4 vs Operator

Computer use is getting crowded. Here's how the major options stack up as of early April:

The cost difference alone makes Holo3 worth watching. But "error recovery" is the row that matters most in practice. Claude Computer Use has months of production feedback baked in — it knows how to handle cookie banners, CAPTCHAs, loading spinners, and popups that block the element it needs to click. Holo3 doesn't have that yet. When something unexpected appears, it tends to retry the same action rather than reason about an alternative path.

Real-World Testing on Desktop Tasks

We ran Holo3-122B (API) and the open-source 35B model through five desktop tasks of increasing difficulty. These aren't OSWorld tasks — they're things we actually need to do.

Task 1: Fill Out a Web Form (Simple)

Navigate to a contact form, fill in name/email/message fields, and submit. The 122B API model handled this perfectly in about 12 seconds. The 35B model also succeeded but took 28 seconds and misclicked the email field once before correcting itself.

Task 2: Extract Data from a Spreadsheet (Medium)

Open LibreOffice Calc, find the sum of a specific column, and paste the result into a text file. Both models completed this. The 122B version finished in 19 seconds. The 35B took 41 seconds and created the text file in the wrong directory on the first attempt.

Task 3: Multi-App Workflow (Hard)

Copy a table from a PDF, paste it into a spreadsheet, add a calculated column, and email the result. The 122B model got through 3 of 4 steps but sent the email without the attachment. The 35B model got stuck trying to copy from the PDF viewer — it couldn't figure out the right-click context menu in Okular.

Task 4: Handle an Unexpected Popup (Stress Test)

We intentionally triggered a system notification mid-task. The 122B model paused, dismissed the notification, and resumed. The 35B model clicked the notification instead of dismissing it, opened a different application, and lost track of the original task entirely. This is where the 78.85% benchmark number meets reality.

Where Holo3 Falls Short

We want to be direct about the gaps, because the benchmark headline is misleading if you don't read the fine print:

• ✕ No error reasoning. When Holo3 fails, it retries the same action up to 3 times rather than analyzing why it failed. Claude Computer Use actually reads error messages and adjusts strategy.
• ✕ Fragile on dynamic UIs. Sites with heavy JavaScript rendering, infinite scroll, or animated transitions trip it up. It screenshots faster than elements load.
• ✕ No persistent memory. Each task starts from scratch. If you want it to remember your login credentials or preferred settings, you need to pass those in every time.
• ✕ 35B model quality gap is real. The open-source model is noticeably worse than the API version — maybe 10-15 percentage points lower on the tasks we tested. "Open source" doesn't mean "equivalent."
• ✕ Documentation is sparse. H Company published the model weights and a paper, but practical integration guides barely exist. Community examples are still emerging.

None of these are permanent problems. Holo3 launched three days ago. But if you're evaluating it for a production pipeline today, these gaps matter more than the OSWorld score.

Who Should Actually Use This?

Use Holo3 if: you're building automated desktop workflows at scale and cost matters. The 10x price advantage over Claude Computer Use is significant for batch processing — scraping, form filling, data extraction across hundreds of sites. The open-source 35B model also makes it viable for companies that can't send screen data to external APIs.

Stick with Claude or GPT-5.4 if: you need reliability on complex, multi-step tasks where things go wrong. The error recovery gap is real and won't be solved by a model update alone — it requires months of production feedback that Holo3 hasn't had yet.

For developers building AI-powered development tools or exploring how agents interact with software interfaces, Holo3's open weights are valuable for research regardless of production readiness. And if you're interested in the broader agent ecosystem, our Codex vs Claude Code comparison covers the coding side of this same trend.

How We Tested

This review is based on 3 weeks of daily Holo3 use across four task categories on real workflows, not staged demos:

• Browser automation: form filling, search-and-extract, multi-tab navigation across 12 different sites (e-commerce, government portals, internal admin panels).
• Data scraping: pulling structured data from PDFs and spreadsheets into a target schema, run against ~30 source documents.
• Form filling at scale: 200+ contact form submissions across a marketing campaign (each form had slightly different field labels and validation).
• Multi-app workflow: copy from PDF, transform in LibreOffice Calc, paste into email — the kind of task that fails on transitions between apps.

Numbers from the test window: ~50 sessions total, 85% success rate on browser-only tasks, 67% on desktop apps with non-standard widgets, average ~25 seconds per task end-to-end (122B API). The 35B open-source model was 10-15 percentage points lower across the board and roughly 2.3x slower on the same hardware due to no model-side batching.

Comparison baselines: Anthropic Claude Computer Use (API direct, same workflows, 1 month prior), Adept ACT-1 (archive references — Adept shut down agent work in 2025), GPT-5.4 Computer Use (concurrent on Task 1 and Task 4). OSWorld leaderboard scores cross-referenced against H Company's paper and the public OSWorld GitHub.

Cost during testing: ~$45 total over 3 weeks on the 122B API model (about 1,200 task invocations averaging $0.037 each). Self-hosted 35B ran on an RTX 4090 already on hand, so marginal cost was electricity only (~$8). Disclosure: paid Pro API account out of pocket, no affiliate relationship with H Company, no early access — we signed up the day Holo3 went public. What we did not measure: mobile automation (we have no Android emulator pipeline), enterprise SSO/audit features (no enterprise tier exists yet), inference on Apple Silicon (only briefly tested via llama.cpp — not enough data to publish).

FAQ