How to measure AI ROI: a one-page worksheet for time actually saved

Here is how to measure AI ROI without guessing: pick one task, time it before the tool arrives, and time it again during a trial. Count the hours spent reviewing, fixing, and maintaining the AI output as part of the new cost. Count saved hours as return only when they turn into capacity you can name, such as more quotes answered the same day, and compare that against the full cost of the change.

That is slower than typing an estimate into a calculator, but it survives a skeptical question. Write a stop threshold before the trial so the result can tell you to quit, not just to continue.

Self-reported time savings can be badly wrong. In one randomized trial published in July 2025 (opens in a new tab), METR had 16 experienced open-source developers work on 246 real issues, with AI tools allowed or disallowed at random. The developers forecast that AI would make them 24% faster and afterward believed it had made them 20% faster. Measured, they took 19% longer with AI. That result covers early-2025 tools and experienced developers on code they knew well; METR's February 2026 update (opens in a new tab) suggests later tools sped developers up, while calling its own data "only very weak evidence" for the size of that change. The lesson that holds is about method: self-reports and the stopwatch disagreed.

Measured gains can be real, and they are often uneven. In a study published in The Quarterly Journal of Economics in 2025 (opens in a new tab), first issued as an NBER working paper in April 2023, Brynjolfsson, Li, and Raymond studied 5,172 customer support agents who received a generative AI assistant on a staggered schedule. Productivity, measured as issues resolved per hour, rose 15% on average, with substantial heterogeneity across workers. Less experienced and lower-skilled workers improved both the speed and quality of their output, while the most experienced and highest-skilled workers saw small gains in speed and small declines in quality. Your average will depend on who uses the tool and for what work.

Both studies measured time directly instead of asking people. Do the same for one task.

If you have several candidate tasks, the scoring sheet in AI use case prioritization helps pick the one to measure first. Define one unit of work so it means the same thing in both periods: "a written reply to a quote request from the web form," not "customer communication." Then record, for a fixed window before the trial:

• Volume. Units completed per week, from the CRM, the sent folder, or the job system.
• Time per unit. Median minutes, timed rather than guessed.
• Rework rate. Share of units needing a correction, follow-up apology, or second attempt, and how long each fix took.
• Customer-facing result. What a customer notices, such as hours from request to reply.

Time at least 20 real instances over at least two weeks of normal work; a task that comes up less often needs a longer window to reach 20. If the business is seasonal, compare like periods, not a quiet month against a busy one.

If a task now takes three fewer hours a week and nobody's week changes, the business has not gained three hours of value. It has gained slack, which may be welcome but is not return on the tool.

Decide before the trial what freed time is for: answering quote requests the same day, calling back open estimates, taking work currently sent to a contractor, or absorbing growth without a new hire. Track whether the hours went there, and count only those with a named, observed use.

Write two rules before the trial: what result stops it and what result expands it. In the worksheet example, the trial stops if net hours freed are under 10% of baseline or rework exceeds 15% after four weeks, and expands to a second task only if counted capacity holds for a full month. Name who can call either decision.

NIST's AI Risk Management Framework (opens in a new tab) frames this as deciding whether a system achieves its stated objectives and whether deployment should proceed (MANAGE 1.1), with someone assigned to disengage systems that perform inconsistently with their intended use (MANAGE 2.4). NIST notes that the framework is being revised (opens in a new tab) as of October 2026. For more on pass and fail conditions, see why AI pilots stall after the demo.

AI output that a person must check is not free output. During the trial, time the review as you timed the original task. Track rework separately, because an AI draft that reads well can carry an error that surfaces only after a customer sees it. The NIST framework's MAP 3.2 makes the same point in general terms: examine and document the potential costs, including non-monetary costs, of expected or realized AI errors.

List each cost with your own figures beside it. The first three go into the worksheet's hours; the rest go on line 13.

• Review time per unit, from the trial measurements.
• Rework time, including customer follow-up caused by errors.
• Upkeep: maintaining prompts, templates, and settings, and re-testing after the vendor changes models.
• Subscriptions or seats, and usage charges for API-based tools.
• Setup and configuration time, including connecting the tool to your systems.
• Training time for staff, and the slower first weeks while people learn.
• Exit cost: what it would take to switch tools or return to the old process.

If you are still choosing between buying a finished tool, configuring one on a platform, and commissioning custom work, build vs buy AI compares how these costs fall under each option.

Copy this into a spreadsheet. The example columns describe a hypothetical small office that writes replies to web quote requests; lines 11 to 14 take your own figures.

Counting only the trial's drafting minutes (2.67 hours against the 10.33-hour baseline), the tool appears to save 7.67 hours a week, about 74%. Counting review, rework, and upkeep brings that to 3.17 hours. Counting only hours with a named use brings it to 2.0. All three numbers are true; only the last one is return.

Hours become a return figure only when you say what an hour of that capacity is worth, which depends on where the hours went:

• They replaced paid time, such as overtime, temporary help, or work sent to a contractor. Value them at what that time would have cost you.
• They produced more work, such as more quotes answered the same day. Value them by the margin on the extra work you can trace to them, not by the wage of the person whose time was freed.
• You cannot say which. Leave line 11 blank. The ROI is unproven, whatever a calculator says.

For line 13, spread one-time costs such as setup and training over the weeks you are evaluating, and record the number of weeks so the next review uses the same period. Value setup and training hours the same way you valued line 11.

Lagging metrics show whether the change mattered but arrive late. Leading metrics show early whether the trial is on track.

A calculator multiplies the numbers you type in, and many ask for estimated hours saved per person, the number the METR trial shows can be far off. Use one as a template if it helps, with lines 3 through 10 of your worksheet in place of the estimate. It still cannot tell you whether the freed hours went anywhere, so treat its output as the most optimistic version of the answer, not the answer. If the calculator comes from a vendor you are evaluating, test the product on your own examples instead; the AI vendor evaluation checklist sets out a bake-off on 30 to 50 of them.

For one task, you can run this worksheet yourself. It gets harder when several AI uses run at once: each needs its own baseline, decision record, and stop call, plus a monthly report leadership can read. If nobody owns that work, it is one of the recurring responsibilities a fractional Chief AI Officer mandate can carry; what shapes fractional CAIO cost and scope explains how that mandate is sized. Cleland & Co. offers the Fractional CAIO engagement when the work justifies a standing seat.