Barely a client conversation goes by right now without someone asking whether they should add an AI feature to their product. It's rarely framed as a specific problem to solve — it's framed as a fear of being left behind. Our job in that conversation is usually to slow things down and find the actual problem before recommending a solution, AI or otherwise.
The question we ask before anything else
The first question is simple: what specific, recurring task is currently slow, expensive, or error-prone enough that solving it would clearly move a business metric? If the honest answer is “nothing specific, we just want to have an AI feature,” that's a strong signal to wait. AI features built to check a marketing box tend to be underused by real customers and expensive to maintain, and they rarely justify themselves once the initial novelty wears off.
Where the case gets real is when there's a genuinely repetitive, pattern-based task currently costing real time or money — sorting and prioritizing support tickets, extracting structured data from unstructured documents, generating a first-draft summary of a long document a human then reviews. Those are tasks where AI's specific strength — finding patterns in messy, high-volume data — maps directly onto a real cost center, and where even an imperfect AI-assisted first pass genuinely saves time downstream.
What we check before building it
Data availability is the next filter, and it's the one that kills more AI feature ideas than anything else. A useful AI feature needs a reasonable volume of relevant, reasonably clean historical data to work from — a support ticket categorization feature needs a real archive of past tickets and how they were actually categorized. A client with a great idea but no historical data to build on is looking at a much longer, more expensive project than they expect, often starting with months of manual data collection before the AI feature can even be trained meaningfully.
We also push clients to think through the failure case upfront, not after launch. What happens when the AI feature gets it wrong — and it will, some percentage of the time, for any real-world feature? A recommendation engine that's occasionally off is low-stakes. An AI feature making decisions about a customer's account status or eligibility needs a human review step built in from day one, not bolted on after the first embarrassing mistake goes public.
Where this usually lands
More often than not, this conversation ends with the client either scoping a much smaller, more specific AI feature than they originally imagined, or deciding a well-designed rules-based system solves their actual problem just as well without the added complexity and ongoing cost of a machine learning model. Both are good outcomes. The bad outcome is building an AI feature because it felt necessary, discovering six months later that a simpler tool would have done the job, and having spent the budget that could have solved the actual problem in the meantime.
Estimating the real cost, not just the build cost
Clients consistently underestimate the ongoing cost of an AI feature relative to a traditional one. A rules-based feature, once built and tested, mostly just runs — maintenance is occasional and predictable. An AI feature needs ongoing monitoring for model drift (performance quietly degrading as real-world data shifts away from what the model was trained on), periodic retraining as that drift accumulates, and per-request API costs that scale with usage in a way a traditional feature's server costs typically don't. We now build a rough ongoing-cost estimate into every AI feature proposal, not just the initial build estimate, specifically because this gap between build cost and running cost is where AI feature budgets most often blow past what a client expected.
That ongoing cost isn't a reason to avoid AI features where they're genuinely warranted — it's a reason to make sure the problem being solved is valuable enough to justify a cost structure that traditional software mostly doesn't have. A feature saving a support team meaningful hours every week clears that bar easily. A feature adding a marginal convenience for a small fraction of users usually doesn't.
A simple test we run before committing to a build
Before committing engineering time, we build a rough manual or semi-automated version of the proposed AI feature first — sometimes literally a human doing the categorization or summarization task the AI would eventually do, timed and tracked for a couple of weeks. It sounds like a strange extra step, but it does two things at once: it confirms whether the underlying task is genuinely valuable enough to be worth automating at all, and it produces exactly the kind of real, task-specific data that makes training or fine-tuning the eventual AI feature far more effective than starting from a generic pretrained model with no domain-specific grounding.
More than once, this manual test has revealed that the “problem” wasn't actually as costly as assumed once someone measured it properly, and the client was better served skipping the AI feature altogether. That's not a failed exercise — catching that before six weeks of development is exactly the outcome the test is designed to surface.
The businesses that get real value from AI features are rarely the ones chasing the technology for its own sake — they're the ones that already had a clearly defined, measurable problem and happened to find that AI was the right tool for it. Starting from the problem instead of the technology is a small shift in framing that changes almost every decision downstream.
