The demo trap
Every AI project starts the same way. Someone shows a demo on clean data. The room gets excited. Budget gets approved. Then the project hits real data and everything stops.
Demos work because they cheat. The data is curated, the edge cases are removed, and the integration layer is a mock. In healthcare, HL7 messages arrive malformed half the time. FHIR resources are missing required fields. EHR exports use undocumented formats that vary between installations of the same vendor. In finance, schemas change between API versions without notice, legacy core banking systems don’t expose APIs at all, and regulatory reporting formats differ by jurisdiction, sometimes by state.
The gap between “works on sample data” and “works on your data” is where budgets go to die. I’ve seen organizations spend six months on a proof of concept, get approval, then spend eighteen months discovering their data isn’t what they thought it was.
Integration is the expensive part
A language model that can’t reach your data is a toy. It doesn’t matter how good the model is if it sits in a sandbox disconnected from the systems people actually use.
Most organizations have five to fifteen systems touching a single patient record or financial transaction. Each boundary is an integration point. Each integration point is a failure mode, a security surface, and a cost center. The patient’s medication history lives in the EHR. The lab results come from a reference lab via HL7v2. The insurance eligibility check hits a clearinghouse API. The clinical decision support tool needs all three, in real time, with the right permissions.
Nobody budgets for this. They budget for “the AI.” Then they discover the model is 10% of the project and integration is 80%. The remaining 10% is arguing about whose budget covers the integration work.
Compliance is architecture, not paperwork
In regulated industries, compliance isn’t something you bolt on after the system works. It’s a set of architectural constraints that shape every design decision from the first commit.
Data residency determines where you can process information. Audit logging determines how you store it. Role-based access determines who sees what. Decision traceability determines whether a regulator can reconstruct why the system did what it did. These aren’t features you add in sprint twelve. They’re load-bearing walls. Move them later and the building falls down.
I’ve watched teams build a working system, then spend twice the original budget retrofitting compliance controls. The audit logging alone required restructuring the data pipeline. They would have saved a year by drawing the compliance boundaries on day one.
The organizational gap
The hardest problem isn’t technical. It’s answering one question: who owns this system after the vendor leaves?
AI changes what the work is, not whether there is work. Someone needs to validate outputs when the model is uncertain. Someone needs to investigate when predictions drift. Someone needs to decide when to retrain and what data to include. These are ongoing operational responsibilities, not a one-time deployment task.
If the organization doesn’t have capacity for that – or won’t create it – the AI slowly degrades. Confidence scores drop. False positives increase. Users start ignoring the system. Eventually someone turns it off and nobody notices for a month. I’ve seen this happen three times in two years. The technology worked. The organization didn’t.
What actually works
Start with the data, not the model. Spend the first weeks mapping what data exists, where it lives, how clean it is, and who owns it. If the data isn’t ready, the model doesn’t matter. No amount of fine-tuning fixes garbage inputs.
Ship the smallest useful thing. One workflow. One integration point. One department. Prove it works in production with real users and real data before expanding scope. And keep one person accountable from research to deployment. No handoffs between a researcher, an architect, a developer, and a DevOps engineer. That’s four chances for context to get lost, and it always does.