Short answer
An AI agent sandbox is an isolated environment where an AI agent can run code, call tools and take actions without reaching real systems, data or customers. Teams use sandboxes to contain what an agent can touch and test its behavior before launch.- Also called
- Agent sandbox, AI sandbox, execution sandbox, test environment
- Two main kinds
- Execution sandboxes for code; test sandboxes for business actions
- Isolates
- Files, network access, credentials and the systems an agent writes to
- Typically contains
- Test tools, test data and replayed conversations
- Not the same as
- Production, where every action is real
In software, a sandbox is a contained space where code can run without affecting anything outside it. The idea matters more for AI agents than for chatbots, because agents act. A chatbot that answers badly costs an apology. An agent that acts badly issues a refund, edits an order or emails the wrong customer.
What a sandbox means for an AI agent
An agent sandbox gives the agent everything it needs to work, except the ability to cause real damage.
Inside it, tool calls reach test systems or return simulated results, credentials only open test accounts, and anything the agent writes can be thrown away. The team can watch what the agent would have done, and fix it, before any customer is involved.
Types of AI agent sandboxes
The term covers three different setups, depending on what the agent does.
- Execution sandboxes. Isolated containers or virtual machines where an agent runs code, browses the web or edits files, with limits on network access, time and resources. Common for coding and research agents.
- Tool sandboxes. The agent's tools point at test versions of real systems: a development store instead of the live one, test API keys, a payment provider's test mode.
- Conversation sandboxes. A preview where the team chats with the agent as a customer would, or replays past conversations to see how it responds. This is what most chatbot builders call a sandbox or test mode.
A customer service agent usually needs the second and third kinds.
What a good agent sandbox isolates
A sandbox is only as safe as its weakest connection to the real world. Check six things:
- Credentials. Only test or read-only keys inside the sandbox.
- Writes. Every change lands in a test system, never in live orders or customer records.
- Messages. Nothing the agent sends reaches a real inbox or phone.
- Network. Access limited to the services the test needs.
- Limits. Caps on time, steps and cost, so a looping agent stops.
- Logs. A record of every step and tool call, and a way to reset to a clean state.
Sandbox testing vs production
A sandbox catches problems early, but it cannot reproduce everything real customers do.
| Sandbox | Staged rollout on live traffic | Full production | |
|---|---|---|---|
| Conversations | Test or replayed | Real | Real |
| Actions | Test systems only | Limited, or drafted for a person | Real |
| Finds | Logic errors, broken tools | The messy messages tests missed | What is left |
| Risk to customers | None | Small and contained | Full |
The step between the two matters. Running the agent on live conversations while it only drafts, then for a small share of customers, finds what a sandbox cannot. The practice this page rejects is skipping both and testing in production, with real customers as the test team.
In Aide, a change to a live procedure stays a draft while the live version keeps running, so the team can test the edit, read What changed against the live version, and restore an older version if the new one misbehaves. A topic set to Drafts keeps every action Aide proposes waiting for a person.
How to test a customer service agent in a sandbox
Good sandbox tests start from real conversations, not invented ones.
- Collect past conversations for the request types the agent will handle, including messy ones.
- Add edge cases: a wrong order number, two requests in one message, an angry customer, a request it should refuse.
- Run the agent's actions against a test store or simulated tools.
- Grade replies and actions against a written definition of what is acceptable.
- Run each case more than once, since the same input can give different results.
- Move to live traffic in small steps once the results hold.
With Aide, steps 1 and 2 happen inside the procedure: Check trigger lists which recent conversations the procedure would pick up and which it would decline, with a reason for each near miss. One click adds a near miss as a "Use for this" or "Don't use for this" example, so the edge cases you find become part of the procedure.
Where Aide fits
Aide gives teams several ways to test without reaching customers. The team can try the agent from its page with Preview, or run a procedure's draft in a test chat or replay a past conversation with it; preview conversations never reach customers. On a connected helpdesk, an inbox can start in Watch only, where Aide decides and drafts in its Activity log without writing anything, then move to test senders, a share of conversations and everyone. See testing before go-live.