AI checks support chats: more than half with no human

The QA team checked support chats by hand: about 60 chats a day per manager. We built an LLM into the check: it reads the chat, scores whether the issue was solved, empathy, critical errors and grammar, and tells the manager where the operator went wrong. Flawless chats it closes on its own. Personal data is masked before anything reaches the model, and chats the AI could not check go to people. Every time a manager disagrees with a hint, it becomes material for the next prompt version. Today more than half of chats are checked by AI with no human, and managers check the rest with its hints.

Input

Support chats that go to quality control: was the issue solved, empathy, critical errors, grammar.

Model

An LLM checks the chat against the QA criteria, points the manager to the operator's mistakes and closes flawless chats on its own.

Action

QA managers check the rest with a ready AI hint: where the operator went wrong and against which criterion. Disagreements feed the next prompt version.

Guards

Personal data is masked before a chat reaches the model. If the AI cannot check a chat, a person does, and the pipeline keeps running.

Impact

More than half of support chats are checked by AI with no human; QA managers check the rest with its hints.