We're designing an AI teammate for business users: one that takes the laborious mental load out of their day, so they can focus on the judgment calls that actually require them. The design question is autonomy: what should the AI handle on its own, and what should it bring back to a person?
The team needed a way to think about it: which actions the AI could take by itself, which ones it should pause and confirm, and where a person had to stay in control.
I ran a study with 21 real participants, walking them through structured scenarios that simulated a six-month relationship with an AI teammate, to see how their trust changed as it suggested things, coached them, made decisions, and asked for access.
They rated it high on helpfulness, communication, and initiative, between 4.3 and 4.8 out of 5. One thing sat far below the rest. Handing over credentials scored 3.4. People would let it suggest, coach, even decide. They would not give it the keys.
It wasn't about effort. Handing over credentials takes one click, less work than most of the things people were happy to let the AI do. It still scored lowest, because there was no taking it back.
People judged each action by two things: how bad it would be if the AI got it wrong, and whether they could undo, limit, or fix it afterward. So the design problem moved. It's less about how capable the AI is, and more about how recoverable its actions are.
I used this to frame the problem as three modes, and to make the tradeoff clear to the team and to leadership:
Act: low-risk, reversible things the AI can do on its own, like drafting something only that person
would see.
Ask: high-impact or unclear actions it should pause and confirm first, like automating something the
rest of their team would see or build on.
Hand back: the moments a person needs to step in, to override it, pull back access, correct what it
remembered, or repair a mistake.
Today, every action still asks first. That's deliberate: trust is still being established. The design work shows where it could go: reversibility so Act can be undone, a clear confirmation moment for Ask, and a Hand back that doesn't just log a correction but feeds it into what the AI knows about that person's preferences. I built prototypes to make that concrete for the team, an override-and-learn loop among them.
Trust isn't a one-time setting. It builds up over time, one interaction at a time.
This is still exploratory, so there's no shipped product to point to. What it has done is shape decisions. I worked with a PM to turn the research and the vision into something leadership could act on, and it fed into how the team is deciding where to invest. We're still working out the specific vision and where it takes the product.