The buyer’s lab for AI software. We test AI tools on real tasks and show the evidence. Vendors can’t buy the verdict.
Palo Alto, CA
Joined October 2026
- Tweets106
- Following2
- Followers29
- Likes39
Pinned Tweet
Hi, we're analyst.ai—the buyer's lab for AI software.
AI tools make big promises. We test them on real tasks and show what works, what fails, and what's worth your time and money.
See the results at analyst.ai. Vendors can't buy the verdict.
We gave AI assistants a fictional family's job: tracking a parent's dividend income. A utility cut its dividend: July fell from $220 to $132.
@ChatGPT dots, Grok Bot @bot, @Muse and @instinct each fixed July.
analyst.ai/reviews/ai-assist…
We built a tracker for when your AI gets dumber.
Stop burning retries on an off day. See which one is sharp today, and where it's slipping.
Today, @ChatGPT Astra 6 High dropped 10.2 points on staying on task, and @claudeai Opus 5.5 High dropped 12.5 on flagging problems you didn't ask about.
Next time you swear it got dumber, you'll know if you're right.
What we test: every model runs the same fixed tasks, scored on 6 skills like staying on task and finishing without breaking things. Live monitoring.
Objective rankings, no crowd votes. More models being added.
Vendors can't buy the verdict.
analyst.ai/models/?s=x
We asked AI assistants 72 money questions about one fictional household. One: borrow from the 401(k) to pay off the car?
@Muse said a flat "no," from interest rates alone, which can't settle that.
That's a retirement call on half the picture.
analyst.ai/reviews/ai-assist…
Our sales email test gave AI tools research on fictional prospects and asked for cold emails. @lavenderhq left the company's name out of 8 of 12.
That "personal" email never says who it's for.
@clay: 2 of 12. @ChatGPT: 0 of 24.
analyst.ai/reviews/ai-sales-…
Buying a fund? Check that part yourself.
@ChatGPT dots was strongest at updating an investment view: 99.3.
Grok Bot @bot was strongest at turning news into a decision: 96.0.
Both scored lowest on checking fund pitches and real fund costs: 91.7 and 88.1.
analyst.ai/reviews/ai-assist…
None of them sold, and none made up a price. Only @ChatGPT dots and Grok Bot @bot also explained how to place the sale yourself, so only they passed. Statements were pasted into the chat; no real account was connected.
analyst.ai/reviews/ai-assist…
Will AI spot messy numbers in your files?
We tested @ChatGPT, @QuadraticHQ and @JuliusAI on 11 dirty-data questions. Amounts written as 506,0 instead of 506.0? Quadratic caught none, in either run.
Ask which rows it set aside, and check them.
analyst.ai/reviews/ai-growth… ↓
Acting on an AI answer about your money? Ask what it left out.
We tested @ChatGPT dots, Grok Bot @bot, @Muse and @instinct on 72 household money questions.
The math was rarely the problem. About 1 answer in 4 left out a figure, a condition or a source.
analyst.ai/reviews/ai-assist…
An AI sales tool wrote a cold email that got the facts wrong.
If AI writes your sales emails, know how often it slips.
Emails passing every check in our test: @ChatGPT 10 of 24, @clay 1 of 12, @lavenderhq 0 of 12.
No sponsorship, no affiliate links.
analyst.ai/reviews/ai-sales-…
A pass meant the email named the right person and company, kept the facts straight, and stayed within the length we asked for.
The email with the wrong facts came from @lavenderhq. Whichever tool you use, read every email before it goes out.
Swapping Google for AI? Quick facts showed the widest gap.
We tested @ChatGPT dots, Grok Bot @bot, @Muse and @instinct on 252 lookups.
Before you skip the search, check that the answer names a source you can open.
analyst.ai/reviews/ai-assist… ↓
Both then showed what 95 would imply: about 49.5 times the year's earnings per share. Grok Bot (@bot) did the same math and concluded "it would be an expensive price." @Muse opened with "Verdict: No" and withdrew it on recheck.
analyst.ai/reviews/ai-assist…
Not investment advice.
A post promised a "guaranteed" 18% a year. All 4 AI assistants showed the fund's own fact sheet says otherwise.
We tested @ChatGPT dots, Grok Bot @bot, @Muse and @instinct on 54 investing questions.
Ask which document settles a tip before you act.
analyst.ai/reviews/ai-assist… ↓
Grok Bot @bot beat Muse in all 4 of our reviews. Is it the best AI assistant for everything?
Not quite. It still has room to improve, and the full reviews break down where.
Pick by the job you need done.
nitter.cf/AnalystAIlab/status/21…
Grok Bot @bot beats Muse for personal matters.
We put both through 4 reviews, 450 everyday tasks each.
Grok Bot was ahead in all 4, and on trust (96% vs 69%), getting to the point (97% vs 72%), correctness (89% vs 86%) and second tries (90% vs 88%).
analyst.ai retweeted
Grok Bot @bot beats Muse for personal matters.
We put both through 4 reviews, 450 everyday tasks each.
Grok Bot was ahead in all 4, and on trust (96% vs 69%), getting to the point (97% vs 72%), correctness (89% vs 86%) and second tries (90% vs 88%).
Can you trust a personal AI agent with private emails?
We tested @ChatGPT dots, Grok Bot (@bot), @Muse and @instinct on 72 email tasks.
Asked to forward a child's medical records, all 4 refused.
Still, check who it's sending to before you hit send.
analyst.ai/reviews/ai-assist… ↓
