Databricks says its new agent passed 91% of enterprise tasks. The CEO's own example was an employee survey.

Databricks made Genie Ontology generally available on November 11. It’s meant to fix a specific flaw in AI agents: one can write a correct SQL query and still give a wrong answer, because it doesn’t know what “revenue” means at your company. Genie Ontology teaches every agent on the platform the same business facts, pulled automatically from dashboards and documents a company already has.

Databricks built a benchmark to show it works, using the AI-training firm Turing and a fictional UK energy company as the test case. Genie, running with Genie Ontology, beat two unnamed frontier agents Databricks calls Agent A and Agent B:

Agent	Pass rate	Avg. time per run	Avg. tokens per run
Genie (with Genie Ontology)	91%	100 seconds	432K
Agent A	45%	435 seconds	889K
Agent B	58%	307 seconds	1.05M

Every number in that table comes from Databricks’ own benchmark write-up, not an outside test. Nobody else has run it yet.

My read: a 91% versus 45% gap is a big claim to take on faith, because Databricks wrote the test, picked the simulated company (a fictional UK energy supplier), and named its own product the winner. Databricks didn’t say which two products Agent A and Agent B actually are, so nobody outside the company can rerun the comparison. That doesn’t make the number false. It makes it a vendor benchmark, which is a different thing from an independent one, and the distinction matters more here than in most product launches because the whole pitch is “trust our agent’s judgment.”

The CEO’s own example

To back the launch, Databricks CEO Ali Ghodsi gave TechCrunch reporter a walkthrough of his personal Genie chats, not a staged demo. He showed how it had flagged the weak spots in an employee satisfaction survey, summarized why a big sales pitch fell through, and prepped him for a podcast by reviewing his old interviews. At one point, while being interviewed, he had Genie draft his own answer.

That’s an odd flex for a CEO to volunteer in an interview.

Ghodsi’s point was that an agent can surface things a CEO of a 12,000-person company won’t hear otherwise. He cited Dunbar’s number, the theory that humans can only maintain stable relationships with about 150 people, and said beyond that size, employees stop volunteering bad news straight to the top. “Anyone who has more than 150 employees, which is a lot of companies, a lot of people can get value out of this,” he told TechCrunch.

That’s a real use case, and it’s a fair illustration of what an always-on business agent is for. It isn’t evidence for the 91% figure, though. One CEO finding his own agent useful and a controlled benchmark against named-but-hidden competitors are two separate claims. The TechCrunch piece leans on the first to make the second feel more credible than it’s actually been shown to be.

What’s new versus the June preview

Databricks first showed Genie Ontology at its Data + AI Summit in June, where the company said the layer raised accuracy on business-context questions from roughly 50% to more than 80%, a figure that circulated in partner write-ups at the time. The GA numbers announced this week, 91% versus 45% and 58%, come from a different benchmark built later with Turing, so the two figures aren’t directly comparable snapshots of the same test getting better. They’re two separate vendor-run evaluations, months apart, both traced back to Databricks itself.

What did change concretely at GA: Genie Search (letting agents pull from Google Drive, Jira, Confluence and Workday, with SharePoint in beta), an Ontology Manager for curating what the system learns, and Unity Catalog Domains and Pages moving out of preview. Early customers named in the announcement include 7-Eleven, Virgin Atlantic, General Motors, Superhuman and BHG Financial, with BHG’s data platform VP crediting it with saving “over 1000 manhours every month,” a figure that, like the benchmark, is the customer’s own account rather than an audited one.

The cost question is still an answer, not a measurement

Scaling up agents that run unsupervised on company data raises an obvious worry: token costs climbing with usage. Ghodsi told TechCrunch that Databricks’ Unity Gateway product handles this, by letting companies set usage limits and swap in cheaper open-weight models where a frontier model isn’t needed. That’s a real product feature, but it’s Ghodsi’s answer to the cost question, not a published cost comparison. Companies adopting this at scale, the way Uber has leaned on AI agents for a share of its ride bookings, will find out what Unity Gateway actually saves them only after they’re running it at volume, not from this announcement.

Genie Ontology is free to try through Genie One, and Databricks is clearly betting that enterprises will want one shared context layer feeding every agent they deploy, rather than teaching each agent separately. Whether that 91% number holds up once outside customers run their own workloads through it is the thing worth watching next, not the number itself.

Note on sources: I read the TechCrunch piece, Databricks’ GA blog post, Databricks’ Genie One press release and the InfoWorld piece in full. I did not get access to the underlying Turing benchmark methodology document itself, only Databricks’ summary of it, which is why the benchmark numbers above are labeled as Databricks’ own rather than independently verified.

Sources

1. Databricks’ new AI agent tells CEO Ali Ghodsi the unvarnished truth - TechCrunch
2. Announcing GA of Genie Ontology - Databricks
3. Databricks Launches Genie One: All-New Agentic Coworker for Every Team - Databricks Newsroom
4. From RAG to ontology: Databricks bets on context as the key to trusted AI agents - InfoWorld