Measuring the Return on a Custom AI Agent

Before any build, measure the workflow as it runs today, over enough weeks to smooth out spikes:

• Time per task and total hours per month.
• Cost per task: the loaded cost of the people doing it, plus tools.
• Volume: how many tasks are done, and how many are left undone or done late.
• Quality: error and rework rates, found by checking a sample.
• Value at stake: revenue or margin the work drives, or losses when it's missed.

Write down how each number was measured, so the after numbers use the same definitions. Gartner's warning that many agentic AI projects will be canceled over unclear business value is, in practice, a warning about missing baselines.^[1]

Pick one primary metric tied to money and two or three supporting ones. Capacity metrics need a value attached: work done faster only pays if the freed time or the extra volume turns into something worth having.

A simple, defensible calculation:

1. Gain per month = (baseline cost per task − agent cost per task) × monthly volume, plus any new value from added capacity.
2. Cost to run per month = model usage + hosting + monitoring and maintenance + human review time.
3. Build cost, one-time.
4. Payback period = build cost ÷ (gain per month − cost to run per month).

Published case studies show the structure. In Eigenomic's litigation case, four matters a month × ten hours saved per matter × the firm's $1,000 hourly rate gives the $40,000 a month in freed billable capacity it reports.^[4] In its dealer case, 25–30 additional good buys a month at $2–3K margin each is the basis for the roughly $50K a month it reports.^[3] Check every input: freed hours only count if they are rebilled or redeployed.

Ask for production evidence before you credit a result. For tasks with checkable answers, prefer metrics code can score over opinions; Eigenomic's BORE benchmark is built on that rule for business document work, ranking models only on exact checks against ground truth and publishing the outputs so anyone can re-score them.^[5] Its own notes say such benchmarks are not fully representative of live enterprise performance, which is why your own measurements on your own work are what count.^[5]

Speed without accuracy moves errors downstream faster. Keep a standing sample:

• Checkable outputs (fields, categories, amounts): score automatically against known answers.
• Judgment outputs (briefs, summaries): have an expert grade a sample against a rubric.
• Overrides and escalations: track the rate; a rise means drift.

NIST's AI Risk Management Framework makes measurement an ongoing function of managing AI risk, alongside governing, mapping and managing it.^[6]

Ranking candidate workflows by the time they consume and the value they drive is a project in itself. Eigenomic's AI Operations Audit is built for exactly this: it observes how work gets done and returns a ranked list of what to automate first, with the ROI math and a 90-day plan.^[7] Doing it in-house works too: list your top ten repetitive workflows, estimate hours and error cost for each, and start with the one that scores highest on the decision framework.