Writing / AI and Leadership
You Can’t Measure AI Performance Without Measuring Access
Ben Tartaglia · Published October 3, 2026
Imagine two people doing the same job. One gets an approved AI tool, access to useful information, training and permission to experiment. The other gets a warning about putting company data into chatbots.
Three months later, a manager compares their output and calls the difference performance.
I would want to ask a few questions before accepting that conclusion.
The workbench belongs in the evaluation
Individual capability matters. So do the conditions under which someone works.
If one employee can search a well-maintained knowledge base while another must reconstruct the answer from old email, we recognize the difference in support. AI should not make that distinction disappear.
Tool access is only one factor. The person also needs suitable work, relevant context, time to practice and enough feedback to recognize mistakes. A license is a purchase. Capability takes more work.
This is a leadership argument, not a claim that every organization is already mismeasuring its staff. It is a reason to examine what a productivity number actually contains.
Teaching judgment takes more than teaching prompts
Microsoft’s 2025 CHI study surveyed 319 knowledge workers. Greater confidence in generative AI was associated with less reported critical thinking; greater confidence in one’s own ability was associated with more. The findings are self-reported associations. They do not establish that AI training causes better judgment. the study
They do give us a useful reason to separate tool familiarity from the ability to assess an answer.
An employee may learn to generate a convincing result before learning to verify it. If we reward production alone, we may be rewarding the wrong part of the process.
I would include practice in checking source material, recognizing unsupported claims and deciding when the task exceeds the tool’s reliable use. Those skills deserve time in the rollout plan.
Make adoption a team capability
A sensible pilot begins with a shared task and a baseline. Record the support each participant receives. Compare accepted work, review effort and rework, then ask what helped and what created friction.
Do not assume that a gain in one role transfers to another. Do not treat a refusal to use an unsuitable tool as evidence of resistance.
Employees should be able to bring forward a failure without losing standing. The person who catches a bad recommendation contributes to the team’s capability, even when that contribution lowers the apparent speed of a workflow.
Share useful examples and the circumstances in which they worked. Give people a route to ask for access or explain why they cannot use the available tools. Where permissions differ, make the reasons and their effects on expectations clear.
Performance still matters
This does not excuse weak work or require everyone to use identical systems. Some jobs need tighter restrictions, and some people will develop stronger skills.
It asks leaders to make a fairer distinction: what did the person contribute, and what did the organization enable?
That distinction helps identify where to coach, where to improve the workflow and where an investment is producing value.
AI can expand what a team accomplishes. The expansion should become a shared capability rather than an invisible advantage that only some employees receive.
Before evaluating who is keeping up, check who was given a reasonable chance to start.