A Dashboard Is the Easiest Part to Fake: How to Vet an Analytics Partner Before You Sign
A pretty dashboard proves almost nothing about an analytics vendor, because the dashboard is the cheapest part to produce. What you are actually buying is the system underneath it. Four questions expose whether that system exists.
You did the hard part already. You stopped guessing, and you decided the business should run on real numbers instead of gut feel and a spreadsheet somebody refreshes by hand. Now you have to pick who builds that, and every vendor you talk to shows you the same thing: a gorgeous demo. This page is how you see past it.
Why Is a Dashboard the Easiest Part to Fake?
Because a dashboard is a picture, and the thing you are paying for is an engine. Any competent freelancer can assemble a good looking screen over sample data in a weekend. What separates a real build is everything you cannot see in a demo: where the data comes from, how it gets cleaned and joined, what happens when a source breaks, and whether the numbers agree with the ones you already trust.
That gap is invisible on a demo call, which is exactly why vendors lead with the screen. The fix is not becoming a database expert. It is asking four questions that a screenshot cannot survive.
What Four Questions Separate a Real System From a Screenshot?
Where does the data live, can it reconcile, who maintains it, and will they tell you what you do not need. Ask them on the demo call, in order: a real firm will enjoy answering them, and a screenshot seller will get vague.
Where does the data actually live, and who owns the account?
The right answer is: in infrastructure you own, in your name, billed to you directly. When CDA built the command center for All States Home Improvement, the data foundation went into the client's own AWS account, running at roughly $50 a month, billed straight to them with no markup. If the vendor disappeared tomorrow, the system and every byte of history would still be theirs.
If a vendor wants the data in accounts only they control, you are not buying an asset. You are renting a hostage.
Can it reconcile to numbers you already trust?
Ask the vendor to tie one number on the screen to a number you already believe: your CRM's job count, your P&L revenue, line by line. A real system survives that test or shows you exactly why the two disagree. A faked one waves at "data cleaning" and changes the subject.
Two reports on the same business disagreeing is normal and diagnosable; that is why the marketing report and the sales report never agree. A vendor who cannot walk you through the reconciliation has not actually joined your data.
Who maintains it when it breaks, and what does that cost?
Every data system breaks eventually: a source changes a field, an export format shifts, a login expires. Ask what maintenance is included, what happens when a feed dies at 7am on a Saturday, and what the dashboard looks like after six months of nobody tending it. The honest answer involves a maintenance plan, documentation your team can read, and the ability for your people to answer new questions without a new invoice every time. The dishonest answer is silence now and a proposal later.
An unmaintained dashboard does not just go stale, it quietly stops being opened at all.
Will they tell you what you do not need?
This is the fastest character test in the industry. A manufacturing client hired CDA to improve their production time estimating formula. We built a model that was measurably more accurate, and then recommended they stay with their simpler existing formula, because the accuracy gain was not worth the added complexity in their workflow. They paid for the work. They kept the simpler model.
That is what the green flag looks like: a firm that will talk you out of spend when your existing approach is good enough. If every question you ask somehow leads to a bigger scope, you have probably met the vendor our right sizing article warns about.
How Do You Tell a Real Firm From a Guy With a Shingle?
Ask what happens when the person you hired is unavailable, and listen for whether there is a firm behind the face. A personal injury law firm that found us online told us afterward what they had expected: "a dude with a shingle," an independent contractor whose work might be fine but whose level of trust was lower. After vetting us, their words changed to "this is like a real firm." Small, they said, and they did not care that it was small. What changed their mind was not the demo. It was the four things a firm has that a shingle does not:
- Scale: a bench, not one calendar. Work continues when someone is sick.
- Permanence: a business built to exist next year, not a weekend side project.
- Protection: contracts, and a real answer for how your data is kept away from your competitors.
- Continuity: documentation and process, so the system outlives any one person who built it.
To be fair to freelancers: for a small, bounded, one time task, an independent contractor can be exactly the right buy. The risk shows up when the deliverable is a system your business will run on. A system has to outlive whoever built it.
What Does the Right Version Actually Produce?
A number that changes behavior, owned by you, that keeps working after the build crew goes home. For All States Home Improvement, the weekly reports used to be compiled by hand, and the process nearly died when the person compiling them left. CDA rebuilt it as a command center on a foundation the client owns, and the speed to lead view it surfaced let managers cut median lead callback from 6.6 minutes to 2.5.
Notice what made that real: the client owns the account, the numbers reconcile against their CRM, the refresh runs automatically, and nobody is held hostage for the next report. Every one of the four questions, answered in the build itself. If you want that same test run on your own numbers before you sign with anyone, that is what the free Profit Leak Audit does.
The Vetting Checklist You Can Run on Your Next Demo Call
The four questions, plus one reference ask. Ten minutes, and you will know.
- "Where will our data live, and whose name is on the account?" Right answer: yours, billed direct.
- "Tie this number on the screen to our CRM's number for last month." Right answer: a walk through the reconciliation, or an honest explanation of the gap.
- "What maintenance is included, and what happens when a data source breaks?" Right answer: a plan and documentation, not a shrug.
- "What would you tell us not to buy?" Right answer: something specific. A vendor with no answer sells scope, not outcomes.
- "Give us a client who has been with you long enough to have survived something breaking." Right answer: a name and a phone number.
If you want to see how the buying decision fits together beyond vetting, the full guide is how to actually buy analytics for a home services business, and the honest math on hiring instead is what a data analyst actually costs. And if what worries you is being locked into a monthly fee, you do not have to pay like that. Before you sign anything, it is also worth asking what the handoff package includes, because owning the system and being able to run it are different purchases.
The dashboard is the last thing to evaluate, not the first.
The Bottom Line Before You Sign
Vet where the data lives, whether it reconciles, who maintains it, and whether the vendor will ever tell you no. A real firm passes all four in one call, and the build it delivers will still be yours, still running, and still trusted a year after the demo that sold it.
If you want to see what an honest look at your numbers feels like before you sign anything with anyone, the free Profit Leak Audit is the opposite of a screenshot: read only, run on your real data, and the findings are yours to keep whether or not we ever work together.
Frequently Asked Questions
How do you vet a data analytics firm before hiring them?
Ask four questions on the demo call: where will our data live and who owns the account, can you reconcile this screen to a number we already trust, what maintenance is included when things break, and what would you tell us not to buy. A real firm answers all four specifically. A screenshot seller gets vague on the first one.
What are the red flags when hiring an analytics partner?
Data held in accounts only the vendor controls, no reconciliation against your existing numbers, no maintenance plan or documentation, every question leading to a bigger scope, and no long tenured client you can call. Any one of these is a warning. Two or more means the dashboard is probably the only real thing you will get.
Is a freelancer or a firm better for analytics work?
For a small, bounded, one time task, a good freelancer can be the right buy. For a system your business will run on, the firm advantages start to matter: a bench instead of one calendar, contracts that protect your data, documentation, and continuity when any one person leaves. Buy the size that matches how long the work has to live.
Who should own the data infrastructure, us or the vendor?
You. The accounts should be in your name and billed to you directly, so the system and its history remain yours if the relationship ends. In CDA's builds the client owns the infrastructure account outright, at cost, with no markup. A vendor who insists on owning your data is converting your asset into their leverage.
What should an analytics engagement include besides the dashboard?
The parts you cannot see in a demo: the data foundation in accounts you own, documented pipelines, reconciliation against your trusted numbers, an automatic refresh, a maintenance plan, and enough documentation that your team can answer new questions without a new invoice. The dashboard is the visible tip. The value is the system under it.
Run the four questions on your own numbers first
The free Profit Leak Audit is the opposite of a screenshot: read only, run on your real CRM and ad data, reconciled against numbers you already trust. It shows you what a real system produces before you commit to anyone, and the findings are yours to keep whether or not we ever work together.
Book a Profit Leak Audit