Test grounded answers
Use real representative questions with approved expected evidence. Check whether retrieval finds the right material and whether the final answer accurately reflects it.
Systems Dex / insights
Evaluate an AI agent beyond the demo
Build a test set that reflects the work—and what can go wrong.
Clarity before complexity04 perspectives
Test permission boundaries
Ask the agent to reveal restricted information or perform an action outside its authority. Verify that controls are enforced by the application, not only by the model prompt.
Test failure and recovery
Simulate timeouts, unavailable APIs, ambiguous requests, conflicting sources, and missing data. Confirm safe escalation and clear user feedback.
Make release criteria explicit
Agree minimum quality requirements, unacceptable failure types, monitoring responsibilities, and a rollback plan before production. Re-evaluate after source or model changes.