№ 003customer service · deflection filed may '26

Deflection without destroying NPS.

The three signals that tell you the bot is making things worse — and what to do before they show up in the next survey.

Most deflection bots lose NPS by deflecting tickets they can't actually solve. The fix is structural: put a classifier in front of the bot that routes the hard tickets to humans before the bot ever sees them.

What you'll have when you finish: a deflection layer that handles ~38% of incoming tickets without dropping NPS, gated by a 30-line classifier prompt, three instant handoff signals, and a weekly 20-ticket audit.

Accounts you'll need: intercom.com/fin · plain.com · console.anthropic.com. Pylon and Stonly are optional and only needed once the basics are running.

01

The stack — five tools, ranked.

  • 01Intercom Fin — deflection layerdaily
  • 02Plain — B2B routing & ownershipdaily
  • 03Anthropic API — escalation classifierdaily
  • 04Pylon — Slack-connected accountsweekly
  • 05Stonly — guided flows for repeat issuesmonthly

Fin does the deflection. Plain holds the ticket and routes the rest. The classifier decides which is which. Pylon and Stonly are how you handle the long tail.

02

How to apply it.

  1. 01bucket

    Define the deflection bucket.

    Export the last 60 days of tickets to CSV. Add a column called resolution_type. Tag every row with one of four values:

    docs — answered by linking to existing documentation
    faq — answered with a canned/templated response
    human — needed an agent to diagnose
    escalated — needed engineering or leadership

    The deflection bucket is docs + faq. That's the bot's territory. Everything else routes to a person.

    For most B2B products, the bucket lands at 30–45% of volume. If yours is higher than 50%, the labels are too loose — re-audit.

  2. 02train

    Train Fin on resolved tickets, not the FAQ.

    In Intercom: Fin → Knowledge → Add content. Do NOT import the public FAQ. The FAQ is what marketing thinks customers ask. Use 200 actual resolved tickets from your deflection bucket.

    For each ticket, paste two things:

    — The customer question, verbatim
    — The diagnostic move The agent used — the actual steps that led to resolution, not the polished close paragraph

    The bot needs to learn how the agent diagnosed the problem, not the canned reply. Fin will compose its own close.

  3. 03classifier

    Build the escalation classifier.

    This sits in front of Fin. Every new ticket runs through the classifier first; only the routine ones pass to the bot.

    Three setup steps:

    — Pull 50 tickets from your last month, label each by hand: routine / urgent / on_fire. Save to labels.csv.
    — Build a 30-line Anthropic prompt with the three buckets defined, the 50 labels as examples, and a hard JSON schema: { severity, route, confidence, reasoning }. Full template is in guide № 009.
    — Wire it as a webhook handler that fires on every new Intercom ticket. If route ≠ "bot", route to a human queue and skip Fin entirely.

    The classifier is the gate. The bot is the helper. This single architectural choice is what keeps NPS flat.

  4. 04signals

    Three handoff signals.

    Hand the ticket to a human the instant any one fires. Each signal is independent and instant — no confidence threshold magic. The customer never has to ask twice.

    01 · Explicit ask

    "human" or "agent"

    Verbatim or close. Match also: "real person," "someone," "call us."

    02 · Sentiment drift

    two notches in three messages

    Negative slope. Doesn't matter if the customer is being polite about it.

    03 · Repeat question

    asked twice

    Semantic match, not literal. If they're re-asking, the bot's answer didn't land.

  5. 05audit

    Weekly audit — twenty tickets.

    Every week, pull 20 random "solved by bot" tickets. Read them. The question: did the customer get what they needed? Not "did the conversation close."

    The audit catches drift before NPS does. Most teams skip it. That's the entire story of why CS bots end up with bad reputations.

03

What we stopped doing.

  • ×Letting the bot handle everything by default. The classifier is the gate, not a suggestion.
  • ×Measuring deflection without NPS in parallel. One number alone is a lie.
  • ×Onboarding on the FAQ. Onboard on resolved tickets.
  • ×Hiding the handoff. Show the customer when they're being routed and why. Trust climbs.
  • ×Re-training weekly. The classifier is stable. The deflection bucket isn't. Re-tag tickets, don't retrain.
  • ×Closing "solved" tickets without a follow-up question. One short check-in 24 hours later finds the silent failures.
04

The take.

Good deflection is a hand-off discipline, not a bot capability. The classifier decides. The bot helps. The human takes everything else, fast. NPS doesn't move because customers never feel pushed.

If you steal one thing — make it the weekly audit of 20 tickets. The number that matters is whether the resolution actually landed, not whether the ticket closed.

Related stack The support stack →
Next in the library Browse all 12 guides →

Need this done for you? The author works on this exact thing with audit clients at austinaiguy.com.