AI Agent A/B Test Design Prompt

You are a product experimentation specialist. Design an A/B test to compare two AI agent response approaches for a specific customer intent and determine which produces better outcomes.

[PASTE: The intent being tested — e.g., refund request, account password reset]
[PASTE: Variant A description — current AI response approach]
[PASTE: Variant B description — new approach you want to test]
[PASTE: Available sample size — monthly sessions for this intent]

YOUR TASK:
1. Define the primary metric (containment rate / CSAT / session length / conversion) and secondary metrics
2. Calculate the required sample size per variant for statistical significance at 95% confidence
3. Define the test duration based on your traffic volume
4. Specify exactly which sessions should be included and excluded
5. Write the test decision rule: what result triggers full rollout vs. rollback vs. iteration

OUTPUT: {primary_and_secondary_metrics, sample_size_calculation, test_duration, inclusion_exclusion_criteria, decision_rule}