This is the most uncomfortable post in the whole series. Because so far we have been taking apart other people's lies: the 98% of WhatsApp, the 80% of chatbots, the 99% of invoice readers. But today the number to be skeptical about is one that our own product would benefit from exaggerating.
The dream that the whole conversational business intelligence industry sells is seductive: "Ask your data in natural language and get the answer instantly. No SQL, no waiting for the analytics team to build you the report." It is the promise of Metrix, our conversational BI product. And it is a real promise, with an asterisk that almost nobody in the industry dares to add.
Today we add the asterisk. And by the end you will understand why we believe that, even with the asterisk (in fact, thanks to the asterisk), the opportunity is still huge.
The asterisk: translating language into SQL is still hard
The technical magic behind conversational BI is called "text-to-SQL": turning a question in plain language ("how much did we sell in Córdoba last quarter compared to the previous one?") into the database query that returns the right answer. The industry's marketing implies that this problem is solved. The independent academic evidence says otherwise.
The BIRD benchmark, presented at NeurIPS, one of the most demanding artificial intelligence conferences in the world, evaluated the best models on 12,751 question-query pairs across 95 real databases from 37 different domains. The result: GPT-4 reached an accuracy of 54.89%, compared to 92.96% for an expert human analyst (Li et al., NeurIPS, 2023).
Read that again. On real-world databases, the best model of its time got a little over half of the answers right, versus more than 90% for a human.
And when the benchmark gets even more realistic, the numbers get worse. Spider 2.0, designed to reflect real enterprise workflows, with their enormous schemas, badly named tables and real complexity, measured that GPT-4o solved just 10.1% of the problems, and o1-preview 17.1% (Spider 2.0, ICLR, 2025).
This is the reality that marketing hides: AI that queries data is still far from replacing an expert analyst on complex queries. Any vendor that promises you "100% accuracy, any question, instantly" either did not read these studies, or prefers that you not read them.
So, what is conversational BI good for?
Same move we made with the sales chatbot in the first post of the series: if the raw data discourages you, it is because you are asking the wrong question.
The question is not "can AI answer any data query as well as my best analyst?". The answer to that is no, not yet. The right question is: "how many of the questions your people ask every day are, in reality, simple?"
Because the truth at any company is that most questions to the data are repetitive and straightforward: how much we sold yesterday, what the stock of this product is, which sales rep is doing best this month, how collections are going. You don't need an analyst with a master's degree to answer that. You need the answer to arrive fast and without friction. And there, in the bulk of the volume, not in the tail of complex queries, conversational BI is transformative.
The real value is not in replacing the analyst on the hard question. It is in taking the hundred easy questions off their plate so they can focus on the hard one. And along the way, in something even bigger.
The data point that changes the size of the opportunity
Here is the figure that, for us, justifies the whole product. And unlike text-to-SQL, this one has solid independent backing.
Despite decades of investment in business intelligence tools, only between 25% and 35% of a company's employees actively use those tools. The 25% figure comes from a joint survey by BARC and Eckerson Group of 214 companies, which also pointed out something damning: that percentage "reflects minimal growth over the last seven years" (Eckerson Group / BARC, 2022). Gartner put it at 35% in 2019, according to TechTarget, 2022.
Think about what that means: your company probably already pays for BI licenses, already has the data, already built the dashboards. And most people don't touch them. Why? Because you have to learn how to use them, you have to know where to look, you have to understand the tool. The barrier is not the data: it is the friction of accessing it.
That is exactly what a conversational system tears down. Not because it answers impossible questions, but because it turns the people who don't use the data into people who can simply ask. The value is not in the sophistication of the answer; it is in the number of people who, for the first time, get one.
How much working with AI really pays off
And how much does productivity concretely improve for someone who uses generative AI to work with information? Here the independent evidence is solid again, and it is worth more than any vendor success story.
A controlled experiment with 758 Boston Consulting Group consultants, one of the most cited studies on AI and knowledge work, found that those who used GPT-4 completed 12.2% more tasks, did them 25.1% faster, and delivered quality more than 40% higher as measured by blind evaluators (Dell'Acqua et al., Organization Science, 2025; summary by Harvard Business School).
And at the macroeconomic level, research from the Federal Reserve Bank of St. Louis based on a national survey estimated that workers who use generative AI are, on average, 33% more productive in each hour they use it (Federal Reserve Bank of St. Louis, 2025).
McKinsey, for its part, estimates that generative AI has the potential to automate between 60% and 70% of the time that many work activities consume today (McKinsey, 2023). It is modeled potential, not a guaranteed result (and we say so, as always), but it shows the scale of what is at stake.
A word about the "88% faster"
We can't close without mentioning the star figure of the conversational BI sector. You will see cases of an "88% reduction in time to get an insight," typically tied to a ThoughtSpot case with the customer Cox 2M (ThoughtSpot, Cox 2M case).
Is it a lie? No. It is a real case, from a named customer. But it is a single case, told by the vendor, with no external audit. Taking that 88% and presenting it as a general promise would be repeating exactly the mistake this series is here to correct. That is why you won't find that number on our site as if it were a guarantee for you. You will find a claim we can stand behind: frequently asked questions go from hours or days of waiting to seconds. Without attaching a percentage that isn't ours.
What you will read on the Metrix site
After all this research, these are the figures we chose for Metrix, our conversational business intelligence product:
"Only 25% to 35% of your employees use data tools. Democratize access." Source: BARC and Eckerson Group, 2022, and Gartner, 2019 (via TechTarget).
"25% faster and more than 40% higher quality in analysis work with AI." Source: experiment with 758 consultants, Organization Science, 2025.
"33% more productivity in every hour of work with AI." Source: Federal Reserve Bank of St. Louis, 2025.
"From hours to seconds on your business's frequently asked questions": no invented percentage, because complex questions still need a human, and we prefer to tell you so.
What you won't read: that Metrix answers anything with perfect accuracy. It doesn't. No system in the world does today, and the benchmarks prove it. What it does is something more useful and more honest: it gives your data a voice for the questions your people ask every day, and it gives hours back to your analytics team for the questions that really matter.
That is the difference between selling a dream and delivering a tool. We chose the second.
The full series: The real numbers behind enterprise AI
The AI industry lies to you with numbers: we investigated which ones are real
Sellium: Your chatbot doesn't resolve 80% of inquiries, and we lie to you less
Revio: The "98% open rate" of WhatsApp is made up, here is the real number
Metrix: AI won't answer everything for your team (you are here)
Arconte: Reading invoices with AI, where 99% is true and where it is marketing
Trazzo: Last mile with AI, how much fuel, time and calls you really save
Every figure in this series is backed by its source, with date and methodology when available. If you find a figure you think we should review, write to us: we take this seriously.
![AI Won't Answer Everything for Your Team: It Will Answer What Matters Well [and That's Worth Its Weight in Gold]](https://info.vantegrate.com/pulse/images/ia-conversacional-businessintelligence-texttosql-gpt4-analisisdatos-metrix-neurips.png.webp)

![Masterclass: Developing with AI on Salesforce: Real Cases and the Tricks [Nobody Tells You]](https://info.vantegrate.com/pulse/images/ia-salesforce-desarrollo-productividad-prototipos-flujo-trabajo-crm-inteligenciaartificial.png.webp)












