In this guide
A vendor shares a screen, types a question, and the AI answers perfectly in two seconds. You nod, because it looked real. But a polished demo and a working system are different objects, and the gap between them is where budgets disappear. This guide shows how to evaluate an AI demo: what a scripted demo is good for, what it can never tell you, what to ask before you pay for anything, and how to run a 30-day pilot that replaces a good impression with evidence.
What is a simulated AI demo, and what is it for?
A simulated demo is an interface wired to pre-planned behaviour. Instead of a model deciding what to say, the demo follows rules, canned answers or prepared examples. That is not a trick if it is labelled. It is a normal way to show a workflow before anything is built, the same way an architect shows a model of a building rather than pouring concrete.
A scripted demo is genuinely good at three things.
- Showing the workflow. Who asks, what happens next, where a person steps in and where the record is kept.
- Showing the interface. What staff would click, what a customer would see, how a trace or a queue looks.
- Starting a better conversation. It gives a business owner something concrete to react to: "yes, but our refunds also need a manager's sign-off."
What it cannot do is tell you how well a real system would perform, and that is the thing buyers most want to know.
What can a scripted demo never show?
Four things matter most in a purchase, and a scripted demo is silent on all four.
Accuracy
The demo's inputs were chosen because they work. Accuracy is only meaningful on a fair sample of your real inputs, including the awkward ones: misspelt names, mixed languages, photographs taken at an angle, customers who write three problems in one message.
Cost
Real AI services usually charge per use, in a way that depends on how much text goes in and out. A demo costs nothing to click, so it says nothing about the bill at your volume. As a made-up illustration: if a tool handled 2,000 requests a month at ₹2 each, the usage bill would be ₹4,000 a month. The question is what your figure really is, and whether it grows with your volume.
Speed
A canned answer arrives instantly. A real one may take a few seconds, or more when a tool lookup is involved, and a slow answer on a busy day feels different from a fast one in a meeting.
Edge cases and failure
The demo shows the happy path. What matters is what happens at the edges: the customer who tries to trick it, the question outside its scope, the document that contradicts another, the service that is down. A trustworthy system fails visibly and hands over to a person. A weak one fails confidently.
How does Zenocta label its own demos?
Zenocta builds interactive demos with sample companies, and the rule for every one is the same: say what is real and what is not, on the screen, before anyone has to ask.
Our own labelling rule
Every Zenocta demo uses a fictional company with sample data, saves nothing, resets on refresh, and states plainly when its AI is simulated, meaning scripted with no model behind it. The ten quick demos on the AI demos page carry that note above them. A demo is never a product with real users, and we do not quote accuracy figures for something that has none.
The AgentOcta support agent demo, for example, opens with a banner stating that the simulated agent is a deterministic rules engine and that no language model runs there. Its tool calls are real against a sample backend, which is a precise and honest way to separate the parts. The KnowOcta document assistant demo similarly separates what runs for real in the browser, such as retrieval and access filtering, from what is composed by rules rather than written by a model. You can judge any vendor by whether they volunteer that distinction.
Scripted demo, prototype on your data or production system?
Most confusion comes from treating three different things as one. Here is how they differ.
| Scripted demo | Prototype on your data | Production system | |
|---|---|---|---|
| Purpose | Show the workflow and interface | Test whether it works on your real inputs | Run the job every day |
| Data | Made-up samples | A copy of your real examples, handled under agreed rules | Live data with access controls |
| Accuracy evidence | None | Measured on a test set you helped build | Tracked continuously, with regular re-checks |
| Failure behaviour | Not tested | Tested with awkward and hostile inputs | Monitored, with alerts and a fallback to a person |
| Human approval | Shown as a screen | Real people approve real cases | Roles, queue, reasons and an audit record |
| Cost information | None | Measured usage cost per request | Budget, limits and monthly reporting |
| Typical time | Minutes to open | Days to a few weeks | Weeks to months, then ongoing |
Each stage answers a different question. The demo asks "is this the right shape?", the prototype asks "does it work on our work?", and production asks "can we rely on it?" Skipping the middle stage is the most expensive mistake in the field.
What should you ask any AI vendor?
These questions work for any supplier, big or small, and a good one will answer them without discomfort.
- Data. What data does it use, where is it stored, who can see it, and can you delete it? Check the current data protection rules for your country and sector.
- Evals. Do you have a repeatable test set with expected outcomes, and can I add my own cases and see the results?
- Human approval. Which actions need a person, who is allowed to approve, and is a reason recorded?
- Failure behaviour. What does it do when it is unsure, when the question is out of scope, or when a connected service is down?
- Privacy. Is my data used to improve anyone else's system, and is that written in the contract?
- Cost per use. What does one request cost at my expected volume, and what happens if volume doubles?
- Accountability. If it gives a wrong answer that costs a customer money, who is responsible, and how is the case handled?
- Change control. When the underlying service changes, how do you re-test before my customers feel it?
What are the red flags in an AI demo or pitch?
None of these proves a vendor is dishonest, but each is a reason to slow down and ask more.
- The demo never goes off script. If you cannot type your own question or upload your own example, you are watching a recording.
- Accuracy is a single big number with no method. A figure without a description of the test set, the sample size and who ran it is decoration.
- The word "autonomous" does the work of an answer. Ask who approves what, and how a mistake is reversed.
- No place to see reasons. If you cannot tell why it decided something, you cannot audit or improve it.
- Costs are vague. "Usage is minimal" is not a number.
- Privacy answers are hurried. Anything touching customers' personal data deserves a written answer.
- The reference customers are unnamed or unreachable. Real deployments can usually be discussed in general terms.
How do you run a 30-day pilot that settles the question?
A pilot turns a good impression into evidence. Keep it small: one job, one team, one measure.
- Days 1 to 3: choose the job and the measure. Pick one repetitive task, such as classifying support tickets or answering policy questions, and decide what "good" means in numbers you can count.
- Days 4 to 7: build the test set. Collect 50 real examples with the answer a careful person would give. Include ten awkward ones. Keep this set hidden from the builder until the test.
- Days 8 to 14: set the rules. Agree what the system may do alone, what needs approval and what it must never do. Write down the fallback to a person.
- Days 15 to 21: run beside the current process. The system suggests, people decide. Record every disagreement and its reason.
- Days 22 to 26: attack it. Try out-of-scope questions, conflicting documents, instructions hidden in text and a service outage, and note what happens each time.
- Days 27 to 30: decide with numbers. Compare results with your measure, count the cost per request, and decide to stop, extend or go live in a narrow scope.
As a made-up example of the arithmetic: if 46 of the 50 test cases come out right, that is 92 percent on that set, and the useful work is to read the four misses and ask whether they were harmless, fixable or dangerous. The percentage matters less than what the misses look like.
Which questions belong on the final checklist before you go live?
- The result is measured on examples you supplied, not only on the vendor's.
- Every action that spends money, changes a record or contacts a customer has a stated owner and an approval rule.
- There is a plain fallback to a person, and staff know how to use it.
- Failures are logged and someone reads the log each week.
- The monthly cost at your volume is written down, with a limit.
- Data handling has been checked against the current rules that apply to you.
- There is a re-test routine for when policies, prices or the underlying service change.
What do the AgentOcta and KnowOcta demos show?
Two Zenocta demos are built to make the questions above concrete. AgentOcta is an interactive support agent console for a fictional store: chat scenarios beside a live tool call trace, refund guardrails, an approval queue, evals and a red-team lab. Our guide to AI support agents with tool calls, guardrails and approval walks through each part. KnowOcta is a document assistant for a fictional retailer, with a retrieval trace, citations, role-based access and evals, explained in our guide to chatting with your company documents.
The demo caveat applies to both: they use sample data, nothing is saved, data resets on refresh, and the AI-like behaviour is simulated. Their value is not accuracy. It is that they show the parts a buyer should insist on: reasons you can read, limits enforced in code, approval before sensitive actions, tests you can run and a safe answer when nothing is found. When you meet a vendor, ask to see the equivalent of each.
What mistakes do buyers make when judging AI?
The most common is judging by fluency. A smooth answer is easy to confuse with a correct one, so judge by the examples that went wrong. The second is testing only the questions you know the answers to, and never the ones you would want escalated. The third is letting the pilot become the production system by accident, with no tests, no owner and no cost limit. The fourth is asking whether the tool is "accurate" rather than "accurate at what, measured how, and what happens when it is wrong". Our simulated demos make a good rehearsal for these habits, because you can see the reasons and break them yourself in a few minutes.
How do you get a pilot built the honest way?
If you have a job in mind, the useful first step is not a demo, it is a brief: the job, ten to fifty real examples, the answer you want for each, and the person who will check the output. Zenocta's AI and automation service works from that brief, sets out what is simulated, what is real and how it will be tested, and responds with a scoped, fixed-price proposal. Hosting, the domain and any model usage are billed separately and stated up front. When you are ready, send us the brief, and browse the demo apps first if you want to see our labelling in practice.
Try it live
Open the real thing: demos run in a new tab with sample data, apps link to their own page.
Live demoAgentOcta live demoA simulated support agent with live tool traces, refund guardrails, an approval queue, evals and a red-team lab.
Live demoKnowOcta live demoA document assistant demo with a retrieval trace, citations, role-based access and evals on a fictional retailer.
- ABrowse allAI and automation demosTen quick simulated demos and 12 fuller AI apps, each labelled as scripted.
- ABrowse allAI automation servicesHow Zenocta scopes, tests and builds AI and automation work.
- TBrowse allTalk about a pilotSend the job and ten real examples for a scoped, fixed-price proposal.
Frequently asked questions
01What is a simulated AI demo?
It is an interface driven by pre-planned behaviour, such as rules, canned answers or prepared examples, instead of a live model. It is a fair way to show a workflow when it is labelled clearly, and misleading when it is presented as proof of real performance.
02Can a vendor demo prove an AI tool is accurate?
No. Accuracy only means something when measured on a fair sample of your real inputs, including awkward ones, with the method explained. A demo uses inputs chosen to work, so treat it as a look at the workflow, not as evidence of accuracy.
03What should I ask an AI vendor before buying?
Ask about data storage and deletion, repeatable evals you can add cases to, which actions need human approval, behaviour when unsure or when a service fails, privacy, cost per use at your volume, and who is accountable for a wrong answer.
04How long should an AI pilot run?
Thirty days is enough for one narrow job: a few days to define the measure, a test set of about 50 real examples, a stretch running beside your current process, a round of deliberate attacks and a decision based on counted results and cost.
05Are Zenocta demos real AI?
No. Zenocta demos use fictional companies and sample data, save nothing, reset on refresh and state when their AI is simulated, meaning scripted with no model behind it. They show workflows and interfaces, and a custom build is scoped and tested separately.


