AI Ticket Routing in Python with Strands Decider 2B

👁 114 reads

A decision model handles ticket routing far better than a chat interface: it selects a department from your predefined list, assigns a confidence score, and lets you automatically send ambiguous cases to a human agent. Below, we walk through a working Python script powered by the open-weight Strands Decider 2B model, complete with local benchmark results and two real-world pitfalls we encountered during testing. For an introduction to decision models and how they scored across our 30-ticket test suite, check out our guide on AI decision models.

What We Are Building

A script that processes each customer ticket and outputs:

• Department - one of 6 target queues (billing, shipping, returns, account, outage, sales) or “other”,
• Confidence - a score from 0 to 1,
• Queue - the selected department if confidence clears your threshold, or human_review,
• Urgency - a flag indicating whether the issue requires same-day resolution.

The output is a single JSON line per ticket, making it trivial to pipe into your helpdesk system, a spreadsheet, or an AI automation pipeline.

Prerequisites

• Python 3.10 or newer.
• PyTorch - with CUDA support if you have an NVIDIA GPU (the model takes under 4 GB of VRAM). On a CPU, it runs fine, just slower.
• The strands-decider package (Apache-2.0). On first run, it downloads roughly 4.4 GB: the Qwen3.5-2B-Base foundation model and the AWS adapter.

pip install strands-decider

On Windows, pip can sometimes overwrite an existing PyTorch install with a CPU-only wheel. Verify your setup with python -c "import torch; print(torch.cuda.is_available())". If it prints False, reinstall PyTorch directly from pytorch.org for your specific CUDA version. This exact issue occurred on our test machine.

The routing.py Script

# Ticket routing using the Strands Decider 2B decision model (local, Apache-2.0).
# pip install strands-decider   (plus PyTorch with CUDA if using an NVIDIA GPU)
import json, sys
from strands_decider.infer import load_engine
from strands_decider.schema import SystemOneRequest

MODEL = "StrandsAgents/strands-decider-2B-hobson-v19"
DEPARTMENTS = {
    "billing": "Payments, invoices, card charges, subscriptions",
    "shipping": "Delivery, courier, tracking numbers, shipping address",
    "returns": "Product returns, warranty claims, exchanges, damaged items",
    "account": "Logins, passwords, profile details, security, privacy compliance",
    "outage": "App or site bugs, server outages, integrations, performance issues",
    "sales": "Pricing inquiries, custom enterprise plans, discounts, demos",
    "other": "Ticket text lacks enough context to determine the issue",
}
CONFIDENCE_THRESHOLD = 0.60   # tickets below this confidence route to human review
URGENT_THRESHOLD = 0.40       # calibrated on our 30-ticket test set - tune on your own data

engine = load_engine(MODEL, device="cuda")  # "cpu" works as well, just slower

def route_ticket(text: str) -> dict:
    request = SystemOneRequest(state=text, questions={
        "department": {"type": "choice", "instructions": "Which department should handle this ticket?", "criteria": DEPARTMENTS},
        "urgent": {"type": "noul", "instructions": "Does this require same-day response (outage, financial loss, security breach, strict deadline)?"},
    })
    answers = engine.evaluate(request).answers
    dept, confidence, p_urgent = answers["department"].choice, answers["department"].confidence, answers["urgent"].noul
    queue = dept if confidence >= CONFIDENCE_THRESHOLD and dept != "other" else "human_review"
    return {"queue": queue, "department": dept, "confidence": round(confidence, 2),
            "urgent": p_urgent >= URGENT_THRESHOLD, "p_urgent": round(p_urgent, 2)}

if __name__ == "__main__":
    for line in sys.stdin:
        if line.strip():
            print(json.dumps({"text": line.strip()[:60], **route_ticket(line)}, ensure_ascii=False))

A few deliberate design choices in this implementation:

• Tested in Polish: We ran our evaluation benchmarks using Polish-language tickets and matching Polish criteria descriptions, finding that non-English prompts worked just as reliably as English ones.
• Two questions in a single request: The model processes ticket state tokens once, evaluating secondary questions at negligible computational overhead. Adding a third check (such as churn risk) does not noticeably affect latency.
• Business logic stays in code: The model outputs raw probabilities. Thresholds, fallbacks, and queue routing are handled entirely in Python, making adjustments easy without touching the model weights.

Running the Pipeline and Results

Pass tickets to standard input, one per line:

python routing.py < tickets.txt

Here are the evaluation results on an NVIDIA RTX 4090 across six unseen test tickets (we tested them in Polish; English translations provided for reference):

Per-ticket inference averaged approximately 135 ms on our GPU, following a 16-second initial model load. Every ticket that cleared the confidence threshold landed in the correct queue. Furthermore, the model triggered the urgent flag exclusively for the team-wide login outage, which matches expectations.

Pitfall 1: Overconfidence on Ambiguous Input

In our initial iteration without an “other” option, the vague ticket “Hej, mam pytanie. Coś mi nie gra.“ (“Hey, I have a question. Something seems off.”) was routed to outage with 0.80 confidence. Because this exceeded our threshold, it would have been sent straight to technical support. This is not a model hallucination: by forcing a choice among six categories, the softmax probabilities must sum to 1. A decision model cannot express uncertainty unless given an explicit fallback option.

The fix is adding an other class described as “Ticket text lacks enough context to determine the issue”, paired with a code rule routing other directly to human triage. With that option in place, confidence on vague text dropped to 0.41 and was diverted to a human agent.

Pitfall 2: The “Other” Class Dilutes Confidence

Adding an extra option carries a trade-off: probability mass spreads out, lowering confidence across unambiguous cases (shipping dropped from 0.96 to 0.90, returns from 0.94 to 0.84), while the VAT invoice request shifted into “other” rather than billing. To address this:

• Incorporate customer vocabulary into category descriptions: If “VAT invoice” ends up classified under “other”, update the billing description to explicitly mention “tax invoices, receipts, billing statements”.
• Tune thresholds against historical data: Pull 50 to 100 labeled tickets from your support archive, run them through the script, and identify the confidence threshold where misclassifications hit zero. Our 0.60 mark is a baseline, not a universal rule.
• Monitor human escalation rates: If your fallback queue exceeds 30-40% of incoming volume, your category descriptions are either too vague or overlapping.

Calibrating Urgency Thresholds

In our 30-ticket evaluation set, Strands Decider assigned urgency probabilities between 0.22 and 0.47 to truly urgent issues. Leaving the decision boundary at a naive 0.50 threshold meant zero urgent tickets were flagged, which is why we set URGENT_THRESHOLD = 0.40 in the script. By contrast, Cloudflare Clef-flash exhibits the opposite behavior, performing best with an urgency threshold around 0.95. Treat raw model outputs as ordinal rankings and set thresholds empirically on validation data.

Switching to Alternative Decision Models

Strands Decider, Cloudflare Clef, and TypeSafe Kev and Jev all implement the same System One schema (state plus questions). Migrating your pipeline to any of them requires only changing how you invoke inference:

• Local Clef-flash (24 GB VRAM required): Use the systemone() function from joint_schema_model.py in the Cloudflare/clef-flash repository. In our testing, it was slightly faster (112 ms) and firmer in its selections, but demands roughly 18 GB of VRAM. See our full coverage in Cloudflare Clef.
• Workers AI: Send a POST request to /ai/run/@cf/cloudflare/clef-flash with the identical JSON structure. It costs $0.09 per million input tokens with an included daily free tier (requires a Cloudflare account).
• Strands HTTP Server: Run strands-decider serve ... --port 8000 and post to /v1/systemone, decoupling model loading from your routing script.

For more architectural details and license terms, read our breakdown of Strands Decider 2B.

When Not to Use a Decision Model for Routing

• When you need to draft an immediate customer reply - that task requires a generative LLM. A decision model can, however, pick which canned response template to trigger.
• When volume is only a handful of tickets per day - prompting a large general model for JSON is simpler (as detailed in our guide to structured output).
• When department structures change weekly without labeled history - you will lack the historical validation set required to set reliable confidence cutoffs.

Google · Your sourcesQuantslant higher in your Google - one clickAdd to preferred sources →

References

• Strands Decider 2B on Hugging Face
• strands-labs/strands-decider on GitHub
• Benchmarked October 5, 2026 on an NVIDIA RTX 4090, Windows 11, Python 3.12, PyTorch 2.11, transformers 5.18, and strands-decider 0.1.0.

Kinetyra