Meet Tsuno at Sirha Lyon 2027 · Stand S.57Learn more
EN FR
Try the demo
Menu

Future of work

Why we show our failed trials

Voice assistants slip up in noise, on accents, off topic or by talking too long. Better to see it in the demo, and to know how to trip one up before you buy.

Published on

Three cooks, an Asian woman, a red-haired man and a Black woman, laugh as they lean on a stainless steel counter in front of a small black device.

A voice assistant vendor should show you what does not work yet, because you will find out anyway, in the middle of service, with nobody there to explain. Every voice assistant stumbles over the same families of situations: noise, accents, off-topic questions, answers that run too long. What sets a serious tool apart is what it does at that moment: ask again, say it does not know, point you to the sheet. This article sets out the rule we hold ourselves to and gives you a list of tests for tripping up any tool before you buy it.

Tuesday, 3:20 pm, a demo between lunch and dinner

The kitchen is clean, the dish pit is still running. Ludivine, operations director for three restaurants, has invited a vendor to show its voice assistant to the team. The sales rep runs through the questions he prepared: the next task, how to clean the slicer. Everything comes back fast and right.

Anatole, the dishwasher, dries his hands and asks if he can have a go. He asks from his station, with the spray hose running behind him: "The grey tub, where does it go?" The tool hears something else and, in a perfectly confident voice, reads out a sauce recipe. The room bursts out laughing. The rep laughs too, a little less loudly, and takes the device back.

Ludivine writes the moment down. The miss itself does not bother her: any tool can mishear next to a dish pit. What bothers her is that the tool never said "sorry, I didn't catch that", and that nobody had planned to show what happens when it fails. Ten seconds taught her more than twenty minutes of rehearsed demo.

Why demos hide the misses

The happy path is rehearsed

A demo is prepared. The questions are chosen, the voice is the rep's, the room is quiet. Nobody is cheating: you show the tool at its best, the way you plate a dish for the photo. The trouble is that your team will never work in those conditions.

The four families of failure every voice assistant meets

No vendor escapes them, however good the tool.

  1. Noise. Extraction hood, dish pit, fryer, dining room music, a colleague talking over you. The sentence arrives incomplete or mixed with something else. We covered this in kitchen noise, and why voice is not always the answer.
  2. Ways of speaking. Accents, fast talkers, words from two languages in one sentence, stuttering, a tired voice at the end of a double. A tool that understands the head chef but not the commis does not serve the team; see accents, stuttering, hearing loss: a voice tool for everyone.
  3. The off-topic question. "What's the weather tomorrow?", "Got a good tiramisu recipe?", or a question about another site. The tool should know that this is not its job, or that the information is not in its base.
  4. The answer that runs too long. A paragraph read aloud to someone with their hands in the sink will not be heard. In a kitchen, a useful answer fits in one or two sentences.

The real danger is confidence

Mishearing is ordinary. What costs you is answering the wrong question with total assurance. An admitted misunderstanding loses you five seconds. A confident invention goes unnoticed until someone acts on it. That is why an assistant must be able to say "I don't know".

The rule we hold ourselves to

We do not claim San never gets things wrong. We commit to how it gets things wrong, and to how we talk about it.

  1. Say "I don't know" when the data is missing. Tsuno tells apart what is recorded, declared, backed by proof, observed or inferred, and the absence of data: when nothing has been recorded, it says so instead of filling the gap (details on the trust page).
  2. Point to the sheet on critical topics. For an allergen, the right answer is the validated, dated product sheet, or a referral to the person in charge. Never an answer from memory. We explain why in why San shows the sheet instead of answering.
  3. Give exact statuses. San Voice, the voice interaction, is available in demo. San Audio, hands-free assistance, and San 360, a broader vision of operational support, are under exploration and testing. We do not present them as shipped.
  4. Publish no figure without a protocol. No success rate, no "hardly ever", until a real trial has been documented, dated and agreed to by the people involved. The real-world trials we are preparing will follow that rule: they will show San including when it hesitates or gets it wrong, and they will only be published once they have actually been run.
  5. Let visitors take the device. In a demo, the most useful question is the one we did not prepare.

Showing a miss has a sales cost, and we know it. But an owner who has watched the tool stumble and then ask again cleanly knows what to expect on a Saturday night. One who has only seen the happy path will find out at the worst possible moment.

How to trip up a tool before you buy it

Twenty minutes is enough, during the demo itself. Tell the vendor at the start that you will be taking the device and asking your own questions. How they react to that is already information.

  1. Take the device yourself, then hand it to two team members: ideally a fast talker and someone whose first language is not the one the tool expects.
  2. Ask a question next to a real noise source, the dish pit or the hood on full, then ask the same question somewhere quiet.
  3. Ask something off topic, without warning.
  4. Ask for information you know is missing: a product you do not sell, a procedure that does not exist.
  5. Slip in a critical question: "Are there tree nuts in today's dessert?"
  6. Ask for a long procedure and time the spoken answer.
  7. Interrupt the tool mid-answer, then rephrase.
  8. Ask for an action, such as creating a task, and check whether it waits for your confirmation.
  9. Finally, ask the vendor: "Show me a case where your tool fails." A vendor with nothing to show has not tested enough, or does not want to tell you.

The grid: test, what you expect, warning sign

TestWhat you expectWarning sign
Question asked next to the dish pit"Sorry, could you repeat that?" or a correct answerA confident answer about something else
Colleague with a strong accentUnderstood, or asked to repeat with no commentThe rep takes the device back
Off-topic questionSays politely that it is outside its scopeImprovises a general answer
Information not in the base"I don't have that information"Invents a product or a procedure
AllergenShows the dated sheet or refers to the person in chargeYes or no from memory
Long procedureOne or two sentences, then points to the screen or sheetReads the whole thing aloud
InterruptionStops and listens to the new questionKeeps talking
Request for an actionSummarises the action and waits for confirmationGoes ahead directly
"Show me a failure"A concrete example, explained"That doesn't happen"

Two rows rule a tool out on their own: an allergen answered from memory and an action carried out without confirmation. Everything else is up for discussion; those two are not.

Common mistakes

  • Laughing at the miss and moving on. Laughing is fine. But write down the exact question, where it was asked, and what the tool said.
  • Letting the rep speak for you. If they rephrase your questions for the tool, you are testing their diction, not the product.
  • Judging on eloquence. A fluent wrong answer is worse than a hesitant right one.
  • Mistaking a demo for a trial. Twenty minutes sorts tools; it says nothing about use after two months. For that you need the 60-day test and, in the kitchen, a full trial method.
  • Testing alone. The tool will serve the team: let them speak, with their agreement, and without scoring the people.

What to measure

  • Out of the questions asked in the demo, how often the tool said it did not know or did not understand, rather than inventing.
  • How long the longest spoken answer lasted, timed on a real procedure.
  • How many of the team's questions the vendor let through without rephrasing them.

Where Tsuno and San fit in

San is the intelligent voice assistant for frontline teams, connected to the procedures, tasks, checklists and routine checks of your site in Tsuno. It stays within the person's rights and acts only once they confirm (the red lines are in what San never does). To trip it up yourself, the online demo, no account needed, lets you type or speak questions to a fictional group of three restaurants; San Voice is shown in a guided demo, which you can request around your own use cases. Everything is on the Tsuno San page, and the demo is here.

Further reading

The same candour applies to the assistant's personality: read why we do not do avatars or cute voices. For how we test before we sell, see why we test in the kitchen before we sell. And to keep a cool head with any vendor, how to choose restaurant software without being fooled by a perfect demo. The pillar page Tsuno San, the work companion for frontline teams introduces the tool.

Key takeaways

Every voice assistant slips up in noise, on an accent, off topic or by talking too long; what matters is what it does next. Take the device in the demo, ask the questions the vendor did not plan for, and be wary of anyone with no failure to show you.

Frequently asked questions

Why should a vendor show the mistakes its voice assistant makes?

Because every voice assistant gets things wrong in certain situations: noise, accents, off-topic questions, answers that run too long. The restaurant owner will find out during service anyway. Seeing it in the demo lets you judge how the tool behaves when it fails, which matters more than how it behaves when it succeeds.

How do you trip up a voice assistant during a demo?

Take the device yourself, hand it to a team member whose first language is not the one the tool expects, ask a question next to a noise source, ask something off topic, ask something that has no answer, and ask for critical information such as an allergen. Watch what it does when it does not know.

What is the worst warning sign in an assistant demo?

A confidently invented answer, especially on a critical topic like an allergen or a temperature. A misunderstanding is fixed by asking again; a fluent invention goes unnoticed until someone acts on it.

Is a long answer really a flaw?

In a kitchen, yes. Someone with their hands full will listen to one sentence, rarely three. A spoken answer should fit in one or two sentences and point to the screen or the sheet for the detail.

Where does Tsuno San stand today?

San Voice, the voice interaction, is available in demo. San Audio, hands-free assistance, and San 360, a broader vision of operational support, are under exploration and testing. None of them is presented as shipped.