Trust and context

Most healthcare leaders I meet are investing heavily in new data systems and AI. The thinking is simple: if we can finally unify our messy records and let intelligent tools process them, we’ll see faster approvals, safer care decisions, and lower costs. Most of these initiatives will underperform or fail. Not because the technology can’t handle it, but because we still haven’t addressed two fundamentals: trust, and clinical context.

It’s not a technology gap. It’s a trust gap.

For years, building intelligent systems meant painstaking work by small teams of specialists. Now, platforms exist that allow hundreds of people inside a health system or payer to create automations and smart workflows. That’s powerful; and dangerous, because as this creation democratizes, the traditional controls are falling away.

In many organizations, there are no agreed standards for what “good enough” looks like. No rigorous ways to test whether a summary preserves all critical facts. No systematic checks for consistency. In short, no universal language for trust.

We’ve seen this play out too many times. A promising new system gets deployed - maybe it’s designed to summarize patient files for call center agents or utilization nurses. On paper, it cuts call times by nearly two minutes, yet in practice, agents keep flipping back to the original records. They don’t trust the summary to be complete or clinically faithful, so the efficiency gains vanish. The same pattern repeats with care teams, claims processors, and even clinical directors. They still verify everything manually, because no one is willing to gamble on incomplete or misleading data when lives and compliance are on the line. It adds to the cognitive burden that the new system had promised to reduce in the first place.

Multiply that hesitation across thousands of transactions, and you understand why so many pilots look good in a demo but never scale. And it’s exactly why trust has to be engineered, not assumed.

Trust is measurable if you build for it

We still see vendor scorecards touting accuracy without explaining what that means in a clinical context. We still see risk measures imported from retail or financial services, where a missed prediction doesn’t jeopardize someone’s health. Meanwhile, the operational truth is that healthcare requires a different bar.

We’ve learned that traditional analytics metrics like precision, recall, or F1 simply don’t capture the risk. Healthcare needs its own standards and so, we’ve learned to quantify trust through hard questions:

• Is the output faithful to the underlying medical record (Clinical match scoring)? Not just generally right, but down to each coded diagnosis, symptom, or treatment? If a patient’s chart lists five critical conditions and a summary only captures three, that’s a 0.6 match. Which means 40% of essential clinical data is missing.
• Does it catch the non-negotiables (Agreement scoring)? For example, if a patient is allergic to penicillin, does the system report that fact or misses it?
• Is it factual or making assumptions (Hallucination scoring)? Does it provide answers strictly adhering to the context provided? Or does it make a few things up? Does it provide the fidelity a workflow needs? Would it support clinical decisions with verbatim accuracy?
• Is it consistent (Consistency scoring)? Will processing the same patient file ten times yield the same core conclusions? Because unlike consumer chat, slight differences aren’t acceptable here.
• Can it cite the source accurately for the response generated (Citation Scoring). Are the responses verbatim to source where the answer is derived from? The thresholds of citation accuracy that need to be pre-determined and agreed upon based on the use case.
• Is the language used for generating responses consistent with a certain style and word precision of a clinical guideline (Language Scoring)? Trust in provided answers erodes when generated responses don’t have the linguistic style of guidelines that clinicians are used to.

None of these are philosophical issues. They’re operational ones. Modern data platforms tolerate more unstructured, unclean data than traditional systems ever could, making these guardrails matter even more. Without them, speed just amplifies risk. When you hardwire these checks into your processes, frontline teams see the difference, and they begin to trust the system enough to stop double-checking every output.

Context is the second barrier

Even with robust trust metrics, you can’t succeed if your AI systems don’t understand your clinical, regulatory, and operational context. Healthcare decisions aren’t simple question-and-answer transactions; they’re multi-step judgments grounded in local pathways, specialty guidelines, payer policies, and embedded clinician knowledge.

Most of the technology now deployed was never designed for this. You can build the best general-purpose engine in the world, but if it doesn’t understand your clinical and regulatory nuances, it’s going to disappoint. We keep seeing the same failures: models that handle text beautifully but miss why a cardiology protocol differs from an oncology pathway. Automations that look clean on a dashboard but stumble when they encounter a subtle reimbursement rule.

Many teams tried bolting on traditional search & retrieval architectures, hoping that would solve the context problem. It hasn’t. Healthcare carries too much implicit context for retrieval alone to close the gap.

The way forward: Codifying your expertise

This is why we talk about knowledge platforms. It’s not just a technical term; it’s how you make your organization’s expertise machine-consumable. That means taking clinical pathways, specialty guidelines, and local compliance rules that today live in PDFs and expert minds, and turning them into structured and governed knowledge assets.

Some health systems are already feeding cardiac care guidelines into systems that build dynamic knowledge graphs, so the next steps surface inside the EHR without a physician having to search. Others are encoding local approval rules, so their summaries align with payer criteria right out of the gate. The smartest are creating modular logic blocks that can be plugged into different workflows, maintaining consistency whether the process is authorization, discharge planning, or population health management. The important distinction is to build and plug them in a way that enables smart contextualization inside of the workflows, without the need to search or jump to another workflow – more of a push rather than a pull.

It’s slower upfront, but it pays off in fewer denials, safer care, and far less wasted time second-guessing systems that seem like black boxes.

The enterprise difference: Embedding trust and context by design

None of this works at scale without a deliberate shift in how we govern these systems. In our experience, organizations mature through clear stages.

At the earliest level, projects are ad hoc. Everyone builds their own, often relying on vendor assurances. Eventually, organizations move to defining internal metrics, creating playbooks for clinicians, engineers, and data scientists, and understanding what each guardrail does and doesn’t do.

Then comes the real turning point: embedding these trust and context measures directly into every new AI project. At this point, clinical match scoring, agreement checks, hallucination controls, and consistency expectations become entry criteria. If a proposed system can’t prove it meets your standards, it doesn’t launch.

At the highest levels, organizations are continuously responsive. They use feedback from users, metrics from production systems, and automated checks to adapt. Because trust isn’t static, it’s something you uphold daily.

The real measure of success

The truth is, any organization can buy the latest data platforms, but few will put in the work to codify their knowledge, build playbooks, and engineer trust at scale.

That means establishing enterprise-wide standards for what trust looks like and measuring it in ways that go beyond generic “accuracy.” It means investing in codifying your own knowledge and expertise, so your systems think like your best people, not like a generic training dataset. And it means pushing past pilots to embed these checks into every project, with guardrails that evolve as your needs and risks change.

If we fail to do this, we’ll end up with a landscape littered with abandoned pilots, skeptical staff, and enormous sunk costs. Worse, we’ll undermine the very confidence we need to move forward, not just in smart systems, but in the decisions that we make every day for patients.

This article is published on Forbes.