Why can a lead-matching pilot trail the control group at first?

The matched agent did not take the call

A lead only tests matching if the agent it was matched to works it. Leads go elsewhere when the matched agent is busy or off shift, when unworked leads expire and are reassigned, or when agents swap leads. Every one of those leads measures your old way of allocating, and a pilot with low adherence shows only part of the change.

Track the share of matched leads worked by the assigned agent, every week. If it is low, clear rules on swaps and reassignment can lift it. In a live test at a North American collection agency, the recommended collector took about half of the calls. The case study.

The samples are still small

On a floor where most contacts do not end in a sale, a week’s difference between two groups can rest on a handful of sales. A low-volume batch can swing either way. A big early lift is as likely to be noise as an early dip, and one good month does not prove a model is stable. Judge on contacts and sales added up over the whole pilot, not on the latest week.

The early weeks are the roughest

The model is updated weekly in the first couple of weeks, then about once a month. An early model built on history can pick up patterns that do not hold on live calls. Live results show that, and it can take some weeks. Other things knock early results too:

• New leads with no history, such as a new lead source or a fresh batch of customers the model has not seen.
• Experienced agents leaving, taking the best-known pairs with them. Matching with high turnover.
• A pilot team of only the weakest or newest agents, with too little history to match on.

Something outside the routing tilted the comparison

• Late data. If sales are recorded days after the call, the latest weeks can look worse than they are.
• High-intent leads that close with any agent dilute the gap. Report them separately.
• Groups split by hand. If anyone chose which leads went where, the comparison is no longer fair. Split at random as leads arrive.

How to run it so the answer holds

1. Split leads at random as they arrive, before anyone dials, and keep every lead in its group whatever happens on the call.
2. Run both groups in the same weeks, with the same agents, dialer, scripts and offers.
3. Score on sales per contact, with your floor’s own definition of a sale, and track adherence alongside it.
4. Judge the result over several full months of clean adherence, not the first weeks, and agree the decision rule in advance.

The method is written for collection floors, and the same comparison works on a sales floor. How to prove a routing change worked.