Build the test set from real intents
Use anonymised examples of the questions and requests a team actually receives. Include short requests, incomplete information and callers who change their minds. Test relevant language varieties without assuming a single Arabic script covers every conversation.
Test switching and interruption
A caller may change languages, correct a name or interrupt a long answer. These are normal conversational behaviours. Evaluate whether the system preserves context and returns to the right task after a correction.
Confirm the action, not every sentence
Decide where confirmation is necessary: changing a booking, submitting a request or updating an account. Excessive confirmation can make a simple call frustrating; missing confirmation can lead to an unwanted action.
Make the human handoff useful
Escalation should include intent, relevant context, what has already been tried and what remains unresolved. The caller should not have to repeat the entire conversation because the automated portion ended.
Evaluate end-to-end outcomes
Measure whether the caller completed the intended task, whether the action was correct and whether escalation was appropriate. Voice quality matters, but it is only one part of the service experience.