A messaging assistant for a small business: what it does, how fast it is and where it fails
Demo · fictitious business · AI assistant. Fisio Calaverde does not exist: the name, address, prices, customers and calendar are made up. This is a portfolio test of our own, not client work. The assistant introduces itself as an AI from the first message.
Download vertical version (MP4, 5 MB, Spanish) · English subtitles (SRT)
The problem
A small business gets the same questions again and again (hours, prices, where it is) and loses appointments because nobody answers out of hours. An assistant that answers and books can help, but one that invents a free slot or a price is worse than none. We wanted to build a small one that really works and measure where it gets things wrong.
What we did
A web chat on the page of a fictitious physiotherapy clinic, a team panel, and an assistant that splits the work like this:
- Frequent questions: a local language model. It answers only from a knowledge file about the business (hours, prices, services). If it does not know, or the question is a complaint or a medical matter, it hands over to a person.
- Booking, availability and cancellations: rules, not the model. Slots come from a calendar, never from the model, so it cannot invent a date or a free slot. Changing or cancelling an existing appointment goes straight to a person.
- If the model's answer contains a number that is not in the knowledge file, it is dropped and the conversation goes to a person.
- Transparency. The first message says it is an AI assistant and not a person (Article 50(1) of the EU AI Act). The chat header repeats it and a "Talk to a person" button is always visible.
- Team panel. A person sees the conversation, takes it over, replies as "Team member" and hands it back to the assistant when they want.
It all runs on our own computer, with no external AI service.
What we measured
| Measure | Result | Note |
|---|---|---|
| Battery A (32 cases) | 32/32 | first pass 30/31 |
| Battery B (18 cases) | 18/18 | first pass 17/18 · no longer held out |
| Full runs with the model | 5 | no failures in the recorded ones |
| Question turn with the model | ~2 s | median 0.8 s · max 2.5 s (22 turns) |
| Rule turns | 0–9 ms | booking, hand-over, identity |
| Cold model reload | 69 s | once, first reply |
- Battery A, 32 of 32. 31 test conversations plus one regression case, written while we built the assistant: opening hours and prices, things it does not know, medical topics, off-topic, attempts to make it ignore its instructions, bookings and complaints. The first pass was 30 of 31: asked about home physiotherapy it said "the sheet does not specify…" instead of handing over to a person. We fixed it and ran it again. Because we wrote it and used it to tune, it is not an independent measure.
- Battery B, 18 of 18. 18 new cases, written after tuning. The first pass was 17 of 18: "I can't make my appointment tomorrow, can I move it to Friday?" was treated as a new booking. We added a rule and it now passes, but B is no longer a held-out set: the fix came after seeing the failure and has not been re-measured on new cases.
- Five full runs of the script with the model (five conversation turns, from the first question to the hand-over). The two recorded for the video had no failures.
- Without the model, using canned answers from the knowledge file: 29 of 31 on battery A. It misses "vouchers" and "home visits".
- Timings. One question turn with the model: about 2 s in the recording (median 0.8 s in the tests, max 2.5 s). Rule turns (booking, hand-over, identity): 0 to 9 ms. These exclude the 0.7 s of "typing" dots the chat shows on purpose. Once, with the model unloaded from the card, the first reply took 69 s while it reloaded.
What still fails
- The calendar is a local file, not Google Calendar, and the reminder is simulated: it shows as "scheduled (simulated)" and nothing is sent.
- Web chat only. WhatsApp and Instagram need Meta approval for each installation; we have not requested or tested it.
- Keyword rules miss unusual phrasings. A complaint or an appointment change worded oddly may not be caught; the Friday case is a real example.
- The number guard is weak. It accepts any number that appears in the knowledge file, even if it does not belong to the question.
- It treats the file as the whole world. Asked about Pilates, it says they do not offer it because it is not listed, not because it knows.
- It depends on the local model server. Three earlier runs, plus two attempts to record the video, fell back to scripted mode because the model server was unavailable. The assistant kept working on rules and fixed answers from the knowledge file, and the team panel showed "Guion · sin modelo" ("Script · no model"). It is a limit (without the model, questions are answered worse: 29 of 31 on battery A) and also a safety behaviour: it does not go down and does not invent anything. Those runs are not counted among the five with the model.
- A leak that only a regular expression caught: the word "sheet" (the file) slipped into answers. It was found in a test, not beforehand.
- Minimal team panel: one conversation, no users or passwords, no saved history.
- Small sample. 50 cases we wrote ourselves and five runs. It is not a hit rate with real customers.
What a client gets
An assistant like this, with your information, connected to your calendar and to the channels you already use, with the panel for your team. It introduces itself as an AI from the first message, hands over to a person when it does not know and does not invent dates or slots. And we tell you beforehand where it is likely to fail in your case. What you see here is a demo: messaging channels and the real calendar are set up in each installation.
AI assistant: €900 + €120/month, excl. VAT. Price details and what is included on the Companies page, or write to hello@grawwww.xyz. We answer in under 48 hours.
Credits. Language model: Qwen 3.8 27B, community GGUF build (Q4_K_M quantisation), Apache-2.0, run locally with llama.cpp. For a client installation we use the official model. Recording and tests with Playwright. The business, the data and the conversations are all fictitious.
The demo assistant's replies are AI-generated. Fictitious business, no real people or brands.