Rethinking Experience Evaluation in an AI World
Why Measuring Usability Is No Longer Enough When Experiences Are Powered by AI
You ask AI for advice, and the answer comes back in seconds: clear, well-written, confident. Helpful, even. You follow it only to later discover it wasn’t completely accurate.
Was that a good experience? Judged against the standards of traditional UX, probably yes: The interaction was smooth, the task got done, nothing felt broken.
We spend a lot of time talking about how to design and build human-AI experiences, and comparatively little talking about how to evaluate them, whether the experience someone just had was actually successful, not just frictionless.
For decades, we've measured technology by how easy it is to use. That work still matters, but user experience now includes much more.
The Evolution of User Experience
With a traditional digital product, failure tends to announce itself: a payment doesn't go through, a page won't load, a form flags the field you got wrong. AI breaks differently. A wrong answer can arrive in the same fluent, confident tone as a right one. So, the question isn't just whether people can use the thing. It's whether they can tell when not to trust it. Usability is still the foundation; AI must be user-friendly and recoverable when things go wrong. But usability was built to judge interactions. AI is asking something bigger: to judge a relationship.
Relationships form over time, as do trust and dependency. The more someone uses an AI tool, the more their behavior adapts to it. They get more confident or more reliant. AI isn't static either; its capabilities shift, its behavior varies, and what one person experiences today may not be what someone else gets tomorrow.
To build sustainable value, we need a wider lens on the human–AI experience.
From Evaluating Interactions to Evaluating Relationships: The A.C.T.I.V.E. Framework
That wider lens means adding a layer on top of usability. We introduce the A.C.T.I.V.E. framework to examine six dimensions of the human-AI relationship:
A — Alignment
Is the AI solving the right problem, in the right way?
AI can be technically correct while missing the real objective. For instance, a procurement copilot may recommend the lowest-cost supplier while ignoring resilience, sustainability or risk. A customer service assistant may optimize for speed while reducing quality.
Alignment is about whether the AI understood what actually mattered, not just what was asked. As AI becomes more independent, getting this wrong becomes costlier.
In enterprise settings, this is critical because many AI systems sit inside complex decision environments. The “best” answer is rarely defined by one variable. It often requires trade-offs between cost, quality, risk, compliance and trust. Evaluation should therefore ask:
- Is AI optimizing for the right business outcomes?
- Does it reflect the organization’s policies and decision criteria?
- Does it handle edge cases and competing priorities appropriately?
- Are users being guided toward better decisions, not just faster ones?
C — Comprehension
Do people understand how AI works and when it's likely to fail?
Nobody needs to understand the model under the hood. But they do need a roughly accurate sense of what it can and can't do, whether an answer is verified fact or a generated guess, where its limits sit and when something's worth double-checking.
Disclaimers and labels aren't the same thing as comprehension. Without an accurate mental model, people end up trusting AI blindly in the moments they should be careful and dismissing it in the moments it could actually help. Evaluation should test whether users can answer questions such as:
- What is this AI tool good at?
- Where is it likely to fail?
- When should I verify its output?
- When should I escalate to a human expert?
T — Trust Calibration
Do people trust the AI enough?
"Building trust" has become a major objective for organizations building AI. But more AI trust is not always better.
Over-trust can lead to automation bias, where people accept AI outputs without sufficient scrutiny. Under-trust can lead to low adoption, duplicated work and missed return on investment. And a successful AI experience sits between those extremes.
Evaluation should therefore look at behavior, not just attitudes:
- When do users accept, edit, reject or escalate AI outputs?
- Do they check more carefully when uncertainty is high?
- Do they over-rely on confident-sounding answers?
- Does trust increase or decrease appropriately after errors?
- Are confidence scores, citations or explanations actually changing behavior?
Trust calibration connects experience design directly to governance. It helps organizations understand not only whether people like the AI, but whether they are using it responsibly.
I — Interaction Resilience
When the AI gets it wrong, can people spot it and recover without losing control?
AI will make mistakes. That's not a design flaw to be engineered away. So, the more useful question is: What happens next?
That means evaluation should not only focus on ideal scenarios. It should examine what happens when the AI misunderstands, hallucinates, omits context, takes the wrong action or makes a recommendation that conflicts with policy.
This becomes especially important as AI moves from copilot to agent.
A drafting assistant that produces an imperfect email creates one level of risk. An AI agent that books travel, updates a customer record, triggers an IT workflow or submits a purchase order creates another. In these contexts, recovery is not just a usability concern. It is an operational and governance concern.
A resilient AI experience gives users ways to detect, interrupt, correct, undo and escalate. Evaluation should ask:
- Can users tell when something is wrong?
- Are high-risk actions previewed before execution?
- Are approval thresholds clear?
- Can users undo or roll back actions?
- Can control be transferred smoothly back to a human?
The more autonomy an AI system has, the more important resilience becomes. Good AI experience design does not assume the system will always be right. It ensures people can regain control when it is not.
V — Values & Fairness
Is the experience fair, safe and responsible for everyone it is designed to serve, not just the average user?
An AI can perform well overall while consistently performing poorly for certain people or situations. For example, a voice assistant may work perfectly for most users but struggle with particular accents. A hiring or talent mobility tool may provide less relevant recommendations for certain roles, regions or career paths. A decision-support system may produce different error patterns across customer segments.
If evaluation focuses only on aggregate performance, these issues can remain invisible.
In enterprise environments, fairness is not only an ethical concern. It is also a regulatory, reputational and adoption risk. Employees and customers are less likely to trust systems that feel opaque, inconsistent or unfair. Leaders also need evidence that AI systems are being monitored for differential impact.
Evaluation should assess and identify:
- Who is having a good experience, and who is not?
- Are errors distributed evenly, or concentrated among certain groups?
- Are outputs accessible and inclusive?
- Are users given a way to challenge, appeal or correct AI-driven outcomes?
Responsible AI cannot be assessed only at the model level. It must be evaluated at the experience level too — where people encounter the system and feel its consequences.
E — Evolution
How does the relationship between people and AI change over time?
Most user testing captures a moment: Give someone a task, observe what happens and identify where they struggle. But the effects of AI often develop gradually.
Someone may check every answer when they first start using an AI tool. After weeks of good experiences, they may stop checking altogether. Trust becomes reliance — and sometimes over-reliance.
The opposite can happen, too. One serious failure after months of reliable use can quickly destroy trust.
AI can also change how people work. For instance, a coding copilot may improve speed while weakening review habits. A service agent assistant may reduce handle time while changing how employees learn, escalate or apply judgment. A decision-support tool may improve consistency while narrowing the range of options people consider.
Those changes are difficult to see in a single session. Evaluation therefore needs to continue after deployment, and organizations should monitor:
- Adoption quality, not just usage volume
- Override, edit and escalation rates
- Decision quality and error patterns
- User confidence and reliance over time
- Customer and employee outcomes
- Changes in skill, judgment or workflow behavior
Better Human-AI Experiences Require Better Evaluation
As AI takes on more of the work of finding, recommending, deciding and acting on our behalf, usability won't tell us whether any of that is going well. To achieve value, we must evaluate UX more comprehensively.
That doesn't mean running the full A.C.T.I.V.E. framework on every feature or every release. Not every AI experience carries the same risks.
The starting point is simpler: Where would getting it wrong matter most?
A low-stakes recommendation tool may need the most attention on Alignment and Comprehension. An AI agent that can spend money, cancel a booking or act on someone's behalf raises very different questions around Trust Calibration and Interaction Resilience.
A.C.T.I.V.E. helps identify where evaluation needs to go deeper. But the reason it needs to go deeper at all is simple: A good AI experience was never just about how easily someone could use it. It's about what the system does with their trust, their decisions and their agency once they've handed them over. That's the case for making human-AI Evaluation more A.C.T.I.V.E.
