This article defines the following metrics in simple terms and gives you recommendations as to which you should place the most focus on:
Related articles:
Deflection rate
Any conversation that does not end in escalation is viewed as deflected. These are the number of conversations that come to the AI agent and do not make it to a human agent. This does also include conversations that end with the Default Reply.
AI agent-handled
The definition of AI agent-handled, on the other hand, is stricter and a conversation must match the following 3 criteria:
- Contains at least one meaningful intent
- Has no escalation attempt whatsoever - meaning if an escalation fails in a conversation, it will not be considered AI agent handled
- Last customer message was understood - meaning the default reply is not used as the last reply
However, this can include conversations where customers drop off before an escalation or were unhappy with the resolution, so requires checking the conversations to see if they ended in a positive outcome.
Automated resolutions rate
Automated resolutions apply to both messaging and email AI agents, and are calculated using the following logic:
- For messaging AI agents: An automated resolution is consumed for any conversation that is considered AI agent-handled and passes the verification performed by our LLM.
- For email AI agents: An automated resolution is consumed for any conversation that is considered answered (because a reply was sent) and passes verification performed by our LLM.
The verification process performed by the LLM evaluates the text of the conversation to ensure that the customer’s request was actually satisfactorily resolved without human-agent intervention. Conversations that don’t pass this verification are not considered automated and do not consume an automated resolution.
For additional information, see About automated resolutions for AI agents.
Custom resolution rate
This is the most powerful metric, however, it requires some work upfront as this is based on the Resolution States, specifically conversations that end in the state Informed or Resolved.
This is then going to be specific and provide actionable insights into how the conversations actually ended and whether there are flows that can be improved.
Check out Resolutions States Explained for more details and recommendations on how to use these states.
0 comments
Please sign in to leave a comment.