Front Desk
An open-source demonstration project that builds a website customer service chatbot based on deterministic code review, letting the language model only draft answers while all output undergoes triple verification of sentences, numbers, and vocabulary by code, ensuring answers stay within the preset knowledge base and eliminating hallucinated content.
Timeline (last 90 days)
Front Desk is a website customer service chatbot demo project whose core idea is to let the language model only draft answers, while the content ultimately presented to the customer is decided by deterministic code review.
The virtual website in the Front Desk demo scenario has only 13 short entries, covering business hours, prices, cakes, allergens, delivery and payment information, and the chatbot only knows this content.
Front Desk's retrieval step is handled by code matching at most three relevant entries out of the 13 entries; if no entry matches, the model is not called and the customer is directly given a phone contact.
Front Desk requires the language model to generate a JSON-format answer based on the matched entries, including the answer text, the cited entry IDs and the supporting original sentences, and the model is restricted to read-only access of the selected entries with temperature set to zero.
Front Desk's validation step includes three independent checks: sentence validation (the cited sentence must appear verbatim in the corresponding entry), number validation (every number in the answer must be found in the cited entries, with form-equivalence rules such as '4 pm' equaling '16:00' but '$4' not equaling '$4.50'), and vocabulary validation (all semantically meaningful words must appear in the material the model received).
If any check fails, the draft is discarded; when the first step fails, the customer sees the original entry text or the contact information, and when a later sentence fails, that sentence is directly truncated; the chatbot does not try to repair the answer but falls back to displaying the original entry text.
The developer wrote 24 test questions with known answers, 6 of which involved content never mentioned on the website (pizza, loyalty card, Christmas business hours); the first round of testing had a correct rate of 17/24, and after adjusting to more precise rules, the second round improved to 21/24.
The second round of testing exposed a loophole: when asked 'Do you have a loyalty card?', the model answered 'No, we don't have a loyalty card', while the website said nothing about this at all; since the sentence had no numbers, number validation let it pass, which prompted the developer to add vocabulary validation ('loyalty' was not in the material, and the sentence was judged unsupported).
After adding the validation rules, the developer added 7 untuned questions, and of the 31 questions run online, 28 were correct; one of the failures came from the developer's own bug (the model output '$8.00' while the website wrote '$8', and number validation judged it a mismatch), and after the fix, the same draft passed 29/31.
The remaining two failure cases that the validation mechanism could not catch are: for 'Are you open on Christmas?', the chatbot gave regular business hours (every word was on the website but it was not the answer the user wanted); for 'Is the cinnamon roll vegan?', the chatbot correctly answered 'No', but the cited basis pointed to the pastry price entry rather than the vegan entry.
1 more timeline events
All Facts (11)
Front Desk offers a second 'citation mode': when no model endpoint is configured, the entire component needs only one script tag, and after finding a matching entry it displays it verbatim or provides a phone contact, so it cannot fabricate content and each question costs nothing; after a model endpoint is configured, it runs the drafting and validation process, and automatically falls back to citation mode when validation fails.
50%UnverifiedThe virtual website in the Front Desk demo scenario has only 13 short entries, covering business hours, prices, cakes, allergens, delivery and payment information, and the chatbot only knows this content.
50%UnverifiedFront Desk's retrieval step is handled by code matching at most three relevant entries out of the 13 entries; if no entry matches, the model is not called and the customer is directly given a phone contact.
50%UnverifiedFront Desk requires the language model to generate a JSON-format answer based on the matched entries, including the answer text, the cited entry IDs and the supporting original sentences, and the model is restricted to read-only access of the selected entries with temperature set to zero.
50%UnverifiedFront Desk's validation step includes three independent checks: sentence validation (the cited sentence must appear verbatim in the corresponding entry), number validation (every number in the answer must be found in the cited entries, with form-equivalence rules such as '4 pm' equaling '16:00' but '$4' not equaling '$4.50'), and vocabulary validation (all semantically meaningful words must appear in the material the model received).
50%UnverifiedIf any check fails, the draft is discarded; when the first step fails, the customer sees the original entry text or the contact information, and when a later sentence fails, that sentence is directly truncated; the chatbot does not try to repair the answer but falls back to displaying the original entry text.
50%UnverifiedThe developer wrote 24 test questions with known answers, 6 of which involved content never mentioned on the website (pizza, loyalty card, Christmas business hours); the first round of testing had a correct rate of 17/24, and after adjusting to more precise rules, the second round improved to 21/24.
50%UnverifiedAfter adding the validation rules, the developer added 7 untuned questions, and of the 31 questions run online, 28 were correct; one of the failures came from the developer's own bug (the model output '$8.00' while the website wrote '$8', and number validation judged it a mismatch), and after the fix, the same draft passed 29/31.
50%UnverifiedFront Desk is a website customer service chatbot demo project whose core idea is to let the language model only draft answers, while the content ultimately presented to the customer is decided by deterministic code review.
50%UnverifiedThe second round of testing exposed a loophole: when asked 'Do you have a loyalty card?', the model answered 'No, we don't have a loyalty card', while the website said nothing about this at all; since the sentence had no numbers, number validation let it pass, which prompted the developer to add vocabulary validation ('loyalty' was not in the material, and the sentence was judged unsupported).
50%UnverifiedThe remaining two failure cases that the validation mechanism could not catch are: for 'Are you open on Christmas?', the chatbot gave regular business hours (every word was on the website but it was not the answer the user wanted); for 'Is the cinnamon roll vegan?', the chatbot correctly answered 'No', but the cited basis pointed to the pastry price entry rather than the vegan entry.
50%