Sign in or create a free account to save this Card to your Deck.
Practice Card · Evidence of progress
Measure the result of a 30-day AI implementation
AN AI PILOT NEEDS A BEFORE AND AFTER.
A thirty-day implementation creates useful evidence when the company defines the baseline before deployment and compares business performance under reasonably similar conditions afterward. Usage, prompts, generated outputs, and employee enthusiasm show activity. They do not establish ROI. The test should examine the result the implementation was designed to change, including quality, cycle time, cost, completed work, customer impact, or decision performance. Where conditions changed during the test, record the confounder instead of assigning every improvement to AI.
Expanded teaching
Expanded teaching
Define the baseline before launch
Capture the current method, volume, time, quality, error, cost, user experience, and any other measure that matters to the use case. A baseline created after the pilot invites selective memory and makes the result difficult to interpret.
Choose one primary result
A pilot can create several benefits, but one primary outcome keeps the test interpretable. Secondary measures can include quality, staff adoption, exceptions, risk incidents, and customer effects. Avoid a scorecard so large that every result can be declared successful.
Measure comparable work
Compare similar tasks, customers, transaction types, or decision situations where possible. Note changes in demand, staffing, seasonality, pricing, policy, or workflow that could affect the result independently of AI.
Treat saved time carefully
Hours saved are capacity, not automatically cash. Record where the recovered time went. Value appears when capacity produces additional useful work, avoids planned spend, reduces purchased services, or supports another measurable priority while quality is maintained.
Where AI can help
AI can structure before-and-after data, summarize exceptions, calculate transparent scenarios, and prepare the review. Humans verify the underlying records, judge output quality, identify confounders, and decide whether the evidence supports continuation.
Close the test with a decision
At day 30, choose among four outcomes: stop, revise and retest, continue at current scope, or expand. A pilot that ends with “people liked it” but no decision or business evidence has not completed the implementation cycle.
Diagnostic inquiry
Questions to ask, not conclusions
At the end of 30 days, can you show what changed against a pre-deployment baseline and separate business improvement from tool activity?
Questions that distinguish possible explanations
- Which single result was the pilot designed primarily to improve?
- What changed during the month that could have influenced the result independently of AI?
- Where did any time saved or capacity released go?
Practical application
Practical application
Build a one-page 30-Day Evidence Sheet before the pilot starts. Record the baseline, primary outcome, secondary quality and risk measures, comparison method, owner, review date, and known confounders. At day 30, record the evidence and make an explicit stop, revise, continue, or expand decision.
Ask Atria
Talk through this Card
Talk it through with us
If this Card describes your week, a free conversation is the fastest way to see what it means for your company and what to do next.
Book a free conversation