The phrase “AI-powered” has been appended to so many products in the last two years that it’s become almost meaningless. Most of the time it means someone has called an API and passed the result to a text box. That’s not a bad thing - sometimes that’s genuinely useful - but it’s not the same as building AI into a product in a way that changes what the product can do.
We build real AI features for real products, including our own. Here’s what we’ve learned about the difference between AI that works and AI that’s just there for the landing page.
The Question to Start With
Before any code is written, the useful question is: what decision or action does this AI feature change for the user?
Not “how do we add AI?” but “what does the user currently have to do manually, or slowly, or unreliably, that AI could improve?” If you can answer that clearly, you have a foundation for a feature worth building. If you can’t, you’re likely building a feature nobody will use more than once.
This sounds obvious, but it gets skipped constantly. Teams get excited about the technology and start building before they’ve identified the actual user problem. The result is features that technically work but don’t improve anything.
Real Integration vs Wrapper Integration
A wrapper integration takes an input, sends it to a model, and returns the output. That’s sometimes the right approach. A simple summarisation tool, a FAQ chatbot, a form auto-fill - these can be built as wrappers and they deliver value.
Real integration means the AI output affects something in the product’s core logic. It changes state. It triggers downstream actions. It learns from the context of that specific product and that specific user.
For example, in My Member Buddy - our membership management software - we’ve built AI features that do more than surface information. They analyse engagement patterns, flag members who are likely to churn before renewal, and suggest personalised follow-up actions to the association manager. The AI isn’t a chatbot bolted on. It’s woven into the workflow.
Getting to that level of integration takes more than an API call. It requires understanding the data model, the user workflow, and what “good” looks like in that specific domain.
Data Is the Real Foundation
AI features that deliver genuine value are almost always built on top of good data. The model itself - whether that’s GPT, Claude, Gemini, or an open-source alternative - is a commodity. What differentiates AI features in real products is the quality and relevance of the data fed into them and the context provided with each request.
If a product has poor data hygiene - inconsistent formats, missing fields, unreliable historical records - AI features will amplify those problems, not solve them. One of the first things we do when scoping an AI feature is assess the data it needs to operate on. Sometimes the AI work is straightforward and the data work is where most of the effort goes.
Prompt Engineering Is Real Engineering
There’s still a tendency to treat prompt writing as a minor detail - something you figure out in an afternoon. In practice, for complex features in production products, getting prompts right is a significant engineering task.
A prompt that works reliably across diverse real-world inputs, handles edge cases gracefully, produces consistently structured output, and degrades sensibly when the model is uncertain requires iteration and testing. The difference between a prompt that works 70% of the time and one that works 95% of the time matters enormously once you’re shipping to real users.
This is one area where experience working with models daily makes a real difference. We know where models tend to fail, what kinds of instructions they handle poorly, and how to structure requests to get reliable results.
Build for Failure
AI models are probabilistic. They will produce wrong answers, hallucinate, and misunderstand inputs. Any product that doesn’t account for this is building on an unstable foundation.
Good AI product design includes graceful failure modes. What does the product do when the model produces output that doesn’t meet confidence thresholds? How does the user know when AI output should be treated as a suggestion rather than a fact? When should a human be in the loop?
These aren’t edge cases to be handled later. They’re core design decisions that should be made before the first line of code is written.
What Working With a Builder Looks Like
When founders and operators come to us to build AI into their product, the first conversation isn’t about which model to use. It’s about what the product does today, who uses it, what they’re trying to accomplish, and where there’s friction or untapped opportunity.
From there we scope the AI features as part of the broader product architecture, not as an add-on. We prototype quickly to validate that the approach works on real data before committing to full implementation. And we build with the expectation that AI features will need ongoing tuning as usage grows and real-world inputs reveal edge cases the prototype didn’t surface.
The goal is always the same: AI that makes the product meaningfully better, not AI that makes the product description sound impressive.
If you’re building a product and want to talk through where AI fits - and where it doesn’t - we’d be glad to have that conversation. Visit Orana Software to get in touch.