How to Use Thoughtly Analytics to Optimize Agent Performance

How to Use Thoughtly Analytics to Optimize Agent Performance

You built an AI voiceAI voiceAn artificially generated, natural-sounding voice produced by a TTS model. Thoughtly supports a library of AI voices and brand-specific cloning.Read full definition → agent. You deployed it. Calls are happening. But are those calls actually converting leads? Are they qualifying the right prospects? Are they losing people at step three because the prompt is too vague?

Most teams deploy an agent and then check back weeks later, hoping the numbers look good. That is not optimization — it is hope. Thoughtly gives you the instrumentation to move from hope to a feedback loop: measure what is happening on every call, identify where agents underperform, and make targeted changes that improve conversion rates across your entire lead funnel.

This guide walks you through Thoughtly's built-in analytics, call history, variableVariableA named value the voice agent stores during a conversation — caller name, intent, qualifying answers — and uses to drive routing and post-call actions.Read full definition → tracking, and post-call automations — and shows you how to combine them into a repeatable process for improving agent performance over time.

What you'll need

• A Thoughtly account with at least one deployed voice agent
• Enough call volume to establish baseline metrics (50+ completed calls recommended)
• Variables configured on key speak nodes to capture qualification data
• At least one post-call automation (On Call Completed trigger) for structured disposition tracking
• Access to your CRM or lead management system for closed-loop reporting

Understand the Analytics dashboard

Thoughtly's Analytics dashboard is your starting point. It lives in the Home tab of the primary navigation and gives you a workspace-level view of agent activity across five key areas.

The Analytics dashboard answers the first question in any optimization cycle: is the system running? If responses are flat, talk time is dropping, or usage skews heavily toward test calls, you have a deployment or configuration problem — not a performance problem. Fix infrastructure before tuning prompts.

Use History to diagnose individual calls

The History tab is where macro metrics become actionable. History provides a global, filterable log of every interaction across every agent — with recordings, transcripts, outcomes, and per-call status tracking.

Call statuses and what they mean

Every call in History carries a status. Understanding these statuses is the foundation for diagnosing agent performance issues.

How to run a call review session

1. Filter by agent and date range. Start with your highest-volume agent over the past seven days.
2. Sort by status. Look at Failed and No Answer calls first — these represent leads you never reached.
3. Listen to Completed calls. Sample five to ten calls and listen to the recordings. Pay attention to where callers disengage, ask confused questions, or go silent.
4. Read transcripts for variable extraction. Verify that your variables are capturing the right data. If a variable returns empty on calls where the information was clearly spoken, your extraction instructions need refinement.
5. Check Transferred calls. Are transfers going to the right destination? Are they happening at the right point in the conversation? Premature transfers waste agent capacity; late transfers frustrate callers.

Run this review weekly. Most optimization gains come from catching problems in the first 48 hours after a change, not from quarterly audits.

Track outcomes and variables that matter

Analytics and History tell you what happened at the call level. Variables and outcomes tell you what happened inside the conversation.

Define your qualification variables

Every agent should capture variables that map to your lead qualification criteria. For high-consideration consumer funnels — insurance, mortgage, education, healthcare, home services — these are typically:

• Intent or need — What is the caller looking for? A quote, an appointment, information?
• Urgency — How soon do they need to act? This week, this month, exploring?
• Location or service area — Are they in a geography you serve?
• Eligibility or fit — Do they meet basic criteria (age, income range, property type, coverage type)?
• Consent — Did they agree to follow-up contact?
• Preferred next step — Do they want a callback, a text with details, or a live transfer?

Configure each of these as a named variable on the relevant speak node. Use the Current speak node source for questions where you need the caller's most recent answer, and Conversation history when the information may have come up earlier in the call.

Map outcomes to business results

Outcomes determine where the conversation goes after each caller response. But they also serve as a measurement layer. When you review calls in History, you can see which outcome fired at each node — giving you a map of how conversations actually flow versus how you designed them to flow.

Common outcome patterns to track:

• Qualified and transferred: The caller met your criteria and was handed off to a human agent. This is your primary conversion event.
• Qualified but unavailable: The caller met criteria but could not be transferred immediately. This indicates a capacity or scheduling issue, not an agent issue.
• Not qualified: The caller did not meet your criteria. A high rate here may mean you are attracting the wrong leads or your qualification bar is too strict.
• Dropped or disengaged: The caller stopped responding or hung up before reaching a decision point. This is where prompt tuning has the biggest impact.
• Requested callback: The caller wanted to be contacted later. Track whether those callbacks actually happen and convert.

Build post-call automations for measurement

The most powerful measurement layer in Thoughtly is the On Call Completed automation trigger. It fires after every call ends and gives you access to rich call data: durations, outcomes, captured variables, action flags, transfer status, and voicemail detection.

Set up a measurement automation

1. Create a new automation. Go to Tools > Automations and click Create.
2. Select the On Call Completed trigger. Choose which agents should fire this automation. Use Specific Agents for targeted measurement or All Agents for workspace-wide tracking.
3. Add a Conditions step. Parse the trigger payload to route calls by outcome. For example: if the call was transferred, route to your CRM update branch. If the call ended without transfer, route to your follow-up branch.
4. Write dispositions and captured variables to your CRM. Use a HubSpot, Salesforce, GoHighLevel, or webhook step to push call results back to the lead record. Include the call outcome, key variables captured, talk time, and a link to the recording.
5. Activate the automation. Draft automations never execute. Click Activate when your configuration is tested.

What to send back to your CRM

Every completed call should write at least these fields back to the lead record:

For a detailed walkthrough on connecting post-call data to your CRMCRMThe system of record for leads, contacts, deals, and activity. Thoughtly reads from and writes to your CRM continuously.Read full definition →, see How to Set Up Post-Call Webhooks for CRM Automation.

Export and analyze data at scale

For teams running hundreds or thousands of calls per week, the History export feature lets you download filtered call data as a CSV for deeper analysis.

1. Apply filters in History. Narrow to the agent, date range, and status categories you want to analyze.
2. Click Export. For large datasets, Thoughtly may process the export asynchronously and deliver it by email.
3. Analyze in your preferred tool. Load the CSV into a spreadsheet, BI tool, or data warehouse. Calculate conversion rates, average talk times by outcome, and drop-off rates by call flow step.

Key metrics to calculate from exported data

The drop-off rate by step is the single most actionable metric. It tells you exactly which node in your agent flow is losing people. If 40% of callers hang up at step 3, that prompt needs work — regardless of how good the rest of your flow is.

Iterate based on what the data shows

Analytics without action is just dashboarding. Here is how to translate metrics into improvements.

Low answer rate

• Check your call timing. Are you calling during business hours in the lead's time zone?
• Enable branded calling if available. Unrecognized numbers get screened or ignored.
• For outbound campaigns, verify your phone numbers are not flagged as spam. Rotate numbers and register with carrier reputation services.
• Review your automation trigger timing. Speed-to-lead matters: calling within 60 seconds of a form fill dramatically improves pickup rates.

High drop-off at a specific node

• Listen to five calls that dropped at that node. Is the prompt too long? Too aggressive? Confusing?
• Check whether variables are extracting correctly at that node. Failed extraction can cause the agent to ask the same question repeatedly.
• If the node asks for sensitive information early in the call, consider moving it later in the flow after rapport is established.
• Test alternative prompt wording using Thoughtly's Test Agent before deploying changes to production.

Low qualification rate

• Review your qualification criteria. Are you filtering too aggressively for your lead source?
• Check whether your variable extraction instructions are too restrictive — returning empty values on calls where the information was actually provided.
• Examine the lead source itself. If most callers do not meet your criteria, the problem is upstream in your marketing, not in the agent.

Low transfer rate on qualified calls

• Verify your transfer node phone numbers are correct and the destination team is staffed during call hours.
• Check for transfer failures in History — look for calls with a Transferred status that show short duration after transfer.
• Consider adding a fallback path: if the transfer fails, offer to schedule a callback rather than ending the call.

Common mistakes

• Optimizing before you have enough data. Fifty calls is the minimum for meaningful patterns. Changing prompts after five calls is guessing, not optimizing.
• Looking only at completed calls. No Answer, Failed, and Suppressed calls are often a bigger opportunity than improving already-working conversations. A 10% improvement in answer rate can matter more than a 10% improvement in qualification rate.
• Ignoring the drop-off step. Teams often tweak the opening prompt or the closing CTA while ignoring the mid-conversation node where most callers actually leave. Check the data before deciding where to focus.
• Not writing results back to your CRM. If call outcomes do not appear on the lead record, your sales team cannot close the loop. Optimization without closed-loop reporting is flying blind.
• Treating every agent the same. Your inbound qualification agent and your re-engagement agent serve different purposes and need different success metrics. Analyze them separately.
• Forgetting to activate automations. Draft automations never execute, even when trigger events occur. If your measurement automation is not firing, check that it is set to Active.

Measuring success

After implementing these practices, track improvement over two- to four-week cycles:

• Week 1-2: Establish baselines for answer rate, qualification rate, transfer rate, talk time, and per-step drop-off.
• Week 3-4: Make one targeted change based on your biggest identified bottleneck. Measure impact against baseline.
• Ongoing: Repeat the cycle. Prioritize changes that affect the highest volume of calls first.

Teams that run weekly call reviews and monthly optimization cycles typically see 15-25% improvement in qualified transfer rates within the first 60 days of structured measurement. The key is consistency — reviewing data regularly, making small changes, and measuring their impact before moving on.

Frequently asked questions

How many calls do I need before analytics are meaningful?

At least 50 completed calls per agent gives you enough data to identify patterns. For statistically significant A/B comparisons between prompt versions, aim for 200+ calls per variant. Below that threshold, random variation can mislead you.

Can I compare two versions of an agent side by side?

Thoughtly does not have a built-in A/B test feature, but you can create two agents with different prompts, split traffic between them using automation routing, and compare their metrics in History and your CRM. Tag calls with an attribute or variable identifying the variant so you can filter accurately during analysis.

What is a good qualification rate?

It depends on your lead source and qualification criteria. For opted-in inbound leads from targeted marketing — rate inquiries for mortgage, quote requests for insurance, enrollment inquiries for education — 30-50% qualification rates are common. For broader lead sources like general website forms, 15-25% is more typical. The benchmark that matters is your own trend line: are you improving month over month?

Should I optimize the agent or the lead source first?

Look at your qualification rate. If fewer than 15% of completed calls meet your criteria, the problem is likely lead quality, not the agent. If 30%+ of callers qualify but few convert to transfers or bookings, the agent flow is where to focus. Analytics gives you the data to make this distinction rather than guessing.

How often should I review analytics?

Weekly call reviews for your highest-volume agents, monthly deep dives with exported data for cross-agent comparison, and immediate review after any significant change to prompts, variables, or agent flows. Do not wait for quarterly business reviews to look at agent performance data — by then, you have lost months of potential improvement.

Sources and further reading

Thoughtly Analytics documentation — official platform reference for the Analytics dashboard and workspace metrics.

Thoughtly History documentation — call filtering, transcripts, recordings, statuses, and export.

Thoughtly Variables documentation — capturing caller data for qualification, branching, and measurement.

Thoughtly Automation Triggers — setting up On Call Completed triggers for post-call measurement workflows.

How to Set Up Post-Call Webhooks for CRM Automation — related Thoughtly guide on connecting call data to your CRM.

How to Use Thoughtly Variables for Dynamic Call Personalization — related guide on variable configuration and best practices.

How to Measure Conversion Lift from AI Follow-Up — related guide on measuring the ROI of AI-driven lead follow-up.

Thoughtly Solutions: Speed to LeadSpeed to leadHow fast you respond to an inbound lead after they raise their hand. Conversion drops sharply past 5 minutes.Read full definition → — Thoughtly's approach to sub-60-second lead response for high-consideration industries.