Watch the illustrated presentation
A 5-minute illustrated explanation with American Michael narration.
Cover: AI-generated conceptual illustration, not a real client event.
By Ahmad Al Shami | Strategy, Enterprise Architecture & AI Transformation
Your agents completed 10,000 tasks. What improved for the business?
A task count can show that a system is active. It cannot establish whether customers received better service, employees spent less time correcting errors or the organization earned a return on its investment.
The leadership question is whether the redesigned workflow produces acceptable outcomes at an acceptable cost, with the controls the business requires. That question connects AI performance to the operating model, rather than treating model usage as a separate success story.
Why the measurement conversation needs to change
McKinsey’s 2026 State of AI research examines the gap between individual productivity and enterprise financial impact. It reinforces the need to distinguish local efficiency from business value. [1]
The FinOps Foundation’s AI guidance addresses cost complexity, allocation, forecasting and unit economics, with spending decisions connected to business value. Its business-value domain also emphasizes objective measures of technology outcomes. [2][3]
The scorecard below is my proposed way to apply these concerns to an agent-enabled workflow. It is not an official benchmark or a prescribed implementation of either source.
Define a successful outcome first
Consider the maintenance-enquiry example from my previous article. An agent classifies a request, retrieves approved service information and helps arrange an appointment, while a coordinator handles exceptions.
A useful outcome is an eligible enquiry correctly routed and, where required, booked into a valid slot, with the customer informed and the required approval recorded.
Agree the acceptance criteria before the pilot. Specify how long a case must remain free from an attributable correction or reopening before it counts as accepted. A message sent, a classification produced or a booking tool called is an intermediate event.
Keep separate measures for routing success and completed maintenance. Correctly routing a request does not establish that the repair was completed or that the customer’s underlying problem was resolved.
Build a scorecard around the workflow
| Dimension | Example measure | What it helps decide |
| Business outcome | Eligible enquiries correctly routed or booked within the agreed service window | Is the workflow improving the intended service? |
| Quality | Incorrect bookings, reopened cases and repeat customer contacts | Is faster work creating downstream correction? |
| Human effort | Review, correction and exception-handling minutes per case | Where is work being removed or transferred? |
| Economics | Full operating cost per accepted outcome | Is the overall delivery model becoming more efficient? |
| Control | Unauthorized actions, missing approvals and successful recovery tests | Can the workflow operate within agreed boundaries? |
| Customer experience | Effort, complaints and relevant satisfaction measures | Does the improvement matter to the service user? |
Report the numerator, denominator, period, scope and data owner for each measure. Include volumes of ineligible, escalated and failed cases so that apparent improvement cannot come from quietly changing the case mix.
Measure cost per accepted outcome
A practical unit measure is:
Cost per accepted outcome = total in-scope workflow operating cost ÷ accepted outcomes in the measurement period.
The numerator should include the costs of unsuccessful attempts as well as successful ones. Depending on scope, it may include model usage, retrieval, orchestration, integrations, infrastructure, monitoring, human review, corrections, exception handling and allocated support costs.
Use a consistent allocation method for shared services. Avoid counting the same human work in both an allocated service charge and a separate labour line. Separate fixed and variable costs so leaders can understand what may change with volume.
Record implementation and transition investment separately, then include it in the investment case over an appropriate time horizon. An operating unit-cost measure is not, by itself, an ROI calculation.
An illustrative comparison
The following numbers are invented solely to demonstrate the calculation. They are not client results, targets or industry benchmarks. Both cohorts contain 1,000 comparable eligible enquiries and use the same acceptance rules and measurement window.
| Measure | Baseline workflow | Agent-enabled workflow |
| Eligible enquiries | 1,000 | 1,000 |
| Accepted outcomes | 900 | 950 |
| Total allocated operating cost | AED 20,000 | AED 18,000 |
| Cost per accepted outcome | AED 22.22 | AED 18.95 |
Using unrounded values, the example shows an approximately 14.7% reduction in allocated operating cost per accepted outcome. It also shows why looking only at the cost of model calls would miss the relevant comparison.
It does not demonstrate cash savings or positive investment ROI. Implementation costs are excluded from this operating comparison, and allocated labour costs may remain payable even when a workflow requires fewer minutes.
Nor does a before-and-after table prove that AI caused the change. Real evaluations should account for differences in demand, case complexity, staffing, seasonality and other process changes.
Distinguish released capacity from cash savings
If coordinators spend fewer hours on intake, the organization may gain capacity. That capacity could reduce a backlog, improve customer service or support additional demand.
Cash savings require a change in actual expenditure, such as reduced overtime or a lower external service bill. Avoided future hiring is a separate benefit category with its own assumptions. Do not add all three categories together when they describe the same hours.
Assign a benefit owner who can explain how released capacity will be used and what evidence will show that it produced value. Apply the same discipline to revenue claims: opportunities influenced, sales won and incremental contribution are different measures.
Establish a fair comparison
Capture a baseline before launch. Where feasible, compare similar cohorts or use a controlled rollout. Keep eligibility, acceptance criteria and observation periods consistent, and record changes that could affect the result.
Combine automated telemetry with business-system records and proportionate quality review. A tool returning “success” may only confirm that an operation was received. The CRM, service platform or other authoritative system should establish the resulting business state.
Check whether rejected cases disappear from reporting, whether retries create duplicates, and whether staff are doing unrecorded correction work. Describe uncertainty when sample sizes are small or outcomes take time to mature.
For costs, reconcile operational usage with billing and allocated service costs. For outcomes, reconcile workflow records with accepted business results. Both reconciliations need stable identifiers without collecting unnecessary personal data.
Make scale, redesign and stop decisions explicit
Agree decision criteria with the business owner before the pilot. Set appropriate outcome, quality, control and cost expectations for the workflow’s risk and purpose.
A lower cost per outcome should not compensate automatically for an unacceptable control failure. Conversely, a service may justify higher unit cost if it demonstrably improves an outcome the business values and the investment case supports that choice.
Use the evidence to decide whether to:
- Scale: outcomes and controls meet agreed requirements, with a credible operating and investment case.
- Redesign: value is promising, but handoffs, quality or costs need improvement.
- Pause or stop: the workflow fails required controls or does not justify continued investment under the agreed assumptions.
These are management decisions, not a universal scoring formula. Document the rationale, owner and next review point.
The board-level question
The strongest AI progress report connects the strategic objective to a specific capability, a changed workflow, observed outcomes and the full economics of delivery.
Keep activity measures for engineering and operational diagnosis. Lead business reviews with what changed for the customer and organization, how confidently it was measured, and which decision the evidence supports.
Strategy → workflow change → accepted outcome → full cost and benefit → investment decision.
Continue the discussion
Which AI metric would your leadership team trust enough to approve the next release or investment?
Read the related article on designing the AI operating model, explore EA repository readiness, or discuss your transformation.
Sources
[1] McKinsey, The state of AI in 2026: On the road to ROI. https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai
[2] FinOps Foundation, FinOps for AI, accessed 16 October 2026. https://www.finops.org/framework/technology-categories/ai/
[3] FinOps Foundation, Quantify Business Value, accessed 16 October 2026. https://www.finops.org/framework/domains/quantify-business-value/