Creating AI prompt-based rating categories (EAP)

What's my plan?

Add-on

Quality Assurance (QA) or Workforce Engagement Management (WEM)

Quick Look: Zendesk QA > Settings > AI

Prompt-based categories are currently in an early access program (EAP). You can sign up for the EAP here.

Rating categories allow you to effectively assess customer conversations.

In addition to using the existing Zendesk QA system categories and creating manual, and exact text-match custom rating categories, you can create AI prompt-based rating categories tailored to your organization's needs.

By crafting specific AI-powered prompts, you can ask targeted questions about conversations and automatically score them based on your criteria.

This article contains the following topics:

• Understanding AI prompt-based categories
• Creating AI prompt-based categories

Related articles

• Zendesk QA prompt-based AI insights library
• About prompt-based AI insights

Understanding AI prompt-based categories

Similar to exact text-match custom categories, custom prompt-based rating categories also use autoscoring to identify and evaluate agent responses that meet or do not meet your quality standards.

You can find ideas and select ready-to-use prompts in our AI prompt library, or customize them to fit your needs. Additionally, you can create your own custom prompt categories asking specific questions about a conversation.

To ensure proper functionality, both the Automatic scoring with AutoQA and the LLM-based AutoQA options must be enabled in your account settings.

Creating AI prompt-based rating categories

Admins and account managers can create up to five custom AI prompt-based rating categories per account.

If you've reached the five active AI prompt-based categories limit, mark one as inactive or delete it so you can create a new category.

To create a new AI prompt-based rating category

1. In Quality Assurance, click your profile icon in the top-right corner.
2. Click the Settings icon ().
3. In the sidebar () under Account, click AI.
4. Click Create AI insight.
5. Select the Prompt based AI insight type.
6. Enter an AI insight name.
7. Under AI insight description, write your prompt by clearly describing what this insight should identify and accomplish. Enter a prompt of up to 2,000 characters specifying the behavior you want to detect, focusing on either agents or customers.

   Be sure to follow our best practices for Writing prompts for AI categories to ensure your insights are accurate and actionable.

   Tip: Click See examples to select a prompt from our ready-to-use AI prompt library. You can customize any prompt to fit your specific needs.
8. Under Detection conditions, enter an additional scoring prompt of up to 128 characters to evaluate your agents positively, so they receive a thumbs-up () rating in a category.
   Tip: Follow our Scoring custom prompt-based rating categories best practices to ensure accurate evaluations.
9. Under Absence conditions, enter an additional scoring prompt of up to 128 characters to evaluate your agent negatively. For example, if the agent did not offer a demo, and the text does not include “let’s schedule a demo,” the agent receives a thumbs-down () negative rating score for this category.
   Note: If AI can't find a clear answer based on the prompts, an N/A rating will be assigned automatically.
10. Click Validate to receive feedback and tips on how to improve your prompt.
11. Select Use as a category on a scorecard.
12. Click Create category.
    To confirm that your prompt is working correctly and finding your prompted criteria, filter the conversations based on the prompted category results.
13. Add the category to your scorecards.