Configuring GenerativeAgent
Conversation Explorer
Review and analyze GenerativeAgent conversations to improve performance.
As you add use cases to GenerativeAgent and refine its configuration, you will need to see how GenerativeAgent handles real customer interactions. Conversation Explorer is a powerful tool that enables you to fine-tune and review GenerativeAgent’s customer interactions.
For each conversation, you can see the full model actions GenerativeAgent took which includes its input, knowledge, reasoning, actions, and output back to the customer.
Conversation Explorer also shows conversations that have been flagged as having quality issues.
You can also share your current filtered view by copying the URL of your current page.
Our quality evaluators look for:
Using Conversation Explorer
To get started with Conversation Explorer:1
Access Conversation Explorer
Request access to Conversation Explorer from your admin.Once granted, you can access Conversation Explorer at the following URLs:
2
Find a conversation
Use the search and filter interface to locate specific interactions or patterns:
- Use date filters to narrow your search
- Search by conversation ID, customer name, or keywords
- Filter by specific tasks, functions, or model actions
3
Review the interaction
Once you have found a conversation, you can see exactly how GenerativeAgent makes decisions:
- Enable model actions to see GenerativeAgent’s reasoning
- Click on model actions for detailed function responses
- Check the quality tab for flagged interactions
Your admin must grant Conversation Explorer permissions before you can access the interface.
Find conversations
Conversation Explorer provides a search and filter interface to locate specific interactions or patterns.Search and filter options
Use the search bar to find conversations containing specific words or phrases. Enclose terms in quotes for exact matches.- Date range: Select specific time periods
- Task: Find conversations where specific tasks were performed
- Functions: Locate conversations that called particular APIs
- Conversation ID: Search for a specific conversation
Filter for flagged conversations
To find Quality Issues:- Add the “GenerativeAgent Flags” filter
- Review flagged interactions to understand quality alerts
Share a conversation
You can share a conversation with others by clicking the “Copy Link” button when viewing a conversation.Analyze Model Actions
Once you have found a conversation, you can see exactly how GenerativeAgent makes decisions by viewing its internal reasoning process via model actions. Model actions are the input, knowledge, api calls, reasoning, and output of GenerativeAgent’s model while handling the customer interaction. The information in the model actions can drive how you update the configuration of your tasks and functions.Model actions categories
Model actions are categorized into the following:When enabling a model action category, there may be multiple model actions with the same category that will be displayed. e.g. enabling Functions will show both a “Function Call” for the request and a “Function Response” for the response.
Authentication
Authentication
Actions occur when GenerativeAgent needs authentication data to call an API. Only used for API Connections that require client data.
Confirmation requests
Confirmation requests
Actions occur when GenerativeAgent needs to confirm an action with the customer.
Functions
Functions
Actions that occur when GenerativeAgent calls a function to handle the customer interaction.The model actions will show:
- The function name
- Input parameters
- Output
Errors
Errors
Actions occur when GenerativeAgent encounters an error.
HILA
HILA
Actions occur when GenerativeAgent needs consultation from a human agent as part of Human in the Loop.
Input variables
Input variables
The input variables that GenerativeAgent uses to call a function.
Knowledge
Knowledge
The Knowledgebase articles that GenerativeAgent was given to answer the customer’s question.
Out-of-scope Customer Message
Out-of-scope Customer Message
This occurs when GenerativeAgent determines that the customer’s question is out of scope for the current task.
Tasks
Tasks
The task that GenerativeAgent is entering or changing into in order to handle the customer interaction.
Thoughts
Thoughts
GenerativeAgent’s internal thoughts and reasoning process.
Transfer to agent
Transfer to agent
This occurs when GenerativeAgent performs a transfer to agent to escalate the conversation to a human agent.
Unsafe Customer Input
Unsafe Customer Input
This occurs when GenerativeAgent determines that the customer’s input is unsafe.
Review model actions
When looking at a conversation, model actions are displayed inline with the conversation flow. This allows you to understand exactly when and why the AI made specific decisions during the interaction. To review model actions:- Open a conversation
-
In the center panel, enable the model actions you want to review
- View the AI’s reasoning process inline with the conversation, showing the chronological flow of decisions
-
Click any model action to see detailed information.
This example shows a function response.
You can also see the “Raw” JSON interaction between GenerativeAgent and the function.
Quality issues
Our monitoring system can flag a conversation as having potential quality issues as determined by our quality evaluators. When quality issues are detected:- Inline indicators: Flagged messages appear with visual indicators directly in the conversation flow
- Quality tab: The “Quality” tab provides detailed information about each flagged utterance.
- Appropriate resource use
- Information groundedness
- Understanding of customer’s intent or assumptions
- Information misrepresentation or contradiction
- Indications of actions beyond its capabilities or false communication of action workflows
- Instruction adherence
- Internal system or unprofessional language