Skip to main content
DanManREAL/ AI_LAB
Updated Nov 20, 2026Request a test

Built with AI5 min readBy Published Updated

An AI helper that answers from your own data

In this guide

The short version

  • General chatbots guess about niche topics; a helper that looks things up first guesses far less.
  • The fix is to look up your own pages, answer only from them and admit when they do not cover it.
  • Tidy, clearly labeled information matters more than clever wording.
  • A daily question limit and honest notices keep a helper affordable and fair.
  • Showing where an answer came from earns trust.

Guide

5 min read
Illustration of chat bubbles with a checked book and a light bulb

Ask a general chatbot about a niche game and you will often get a confident, detailed answer that is wrong. It sounds sure of itself because that is how chatbots talk, not because it knows.

I built the Oracle to fix that for Godforge, a mythology game made by a game studio. It was part of my independent fan project, designed to answer questions about heroes only from the hero information I keep, and to say so when it did not know. I built it with Claude Code, and this is how the first version worked. I have since retired the Oracle, along with the hero database it drew on, so this is a record of how it worked rather than something you can use today.

Why general chatbots make things up

A chatbot learned from a huge amount of public writing. A niche game has very little of that, and what exists goes out of date when the game changes. Where it has gaps, it fills them with plausible guesses.

A player asking about heroes, rankings, counters or a specific ability wants tactical answers, not general conversation. My hero data was updated often, my guides kept growing and my tier list kept changing. I needed answers based on the information as it stood that day, not on whatever the chatbot happened to remember.

The fix, in plain words

The idea is simple. Before the AI answered, it first looked up the relevant pages from my own information. Then it answered using only those pages. If they did not cover the question, it said it did not know. It worked like a student allowed an open book on the exam, who must point to the page rather than answer from memory.

StepWhat happensIf it goes wrong
UnderstandThe question is turned into something that can be matched against my pages.The helper misreads what you asked.
Look upThe closest hero pages and game explanations are found.The right page never reaches the AI.
AnswerClaude, the AI model, writes a reply using only those pages.The pages were right, but the answer is unclear.

I kept these three steps separate on purpose. When something breaks, you can tell which step failed. A wrong lookup and a badly worded answer are different problems and need different fixes.

Tidy information beats clever wording

It is tempting to blame a poor answer on the instructions and keep rewording them. In my experience the real cause is usually the information. Messy, out-of-date or badly labeled pages give poor results, however cleverly the question is worded.

So I organized the pages on purpose. A hero profile, an explanation of a status effect (a temporary boost or penalty on a hero) and a general gameplay explanation are kept as separate pieces. Each has a clear label and a known source. The helper can then pull up the right kind of page and not mix them up.

Try thisBefore you blame the wording, read the pages the helper found. If they are muddled, fix those first.

It also meant that when the layout of my information changed, I rebuilt the searchable copy from scratch on purpose, instead of hoping the lookup still worked.

Comparisons need care

Players often ask which of two heroes is better. If someone asks about Athena versus Freya, a lazy answer names one universal winner. That is misleading, because the answer depends on the game mode and the team.

I told the Oracle to spot comparison questions and reason through the trade-offs. It explained where each of the two heroes comes out ahead. It also kept track of the recent conversation, so a follow-up like "what about against other players?" made sense.

Keeping costs and misuse under control

Every question uses paid AI services, so an unlimited helper can become an unexpected bill or get abused. I treated limits as part of the design, not as an apology.

  • Signed-in players got a daily question limit per person, and they could see how many questions they had left.
  • The site's staff were exempt, so they could test and keep things running.
  • I watched spending, and the helper could drop to a cheaper mode when needed.
  • When it dropped to a cheaper mode, the page said so.

Many AI features treat limits as embarrassing and hide them. Clear limits do the opposite. They keep a helper affordable enough to keep improving, instead of forcing a quiet shutdown later.

Show where the answer came from

I wanted people to be able to check an answer, so the page showed a few things alongside it.

  • Labels for the sources the answer drew on.
  • Picture cards for the heroes at the center of the reply.
  • Buttons to save or share an answer.
  • Plain notices when you are signed out or running low on questions.

I also watched which questions came up, to spot topics my information did not cover yet. Those gaps show what to write next.

What I learned

The main lesson is that a helper is only as good as the information behind it. Clever wording matters far less than pages that are tidy, current and clearly labeled. The other lessons were about honesty: say when you do not know, show your sources and be open about limits.

The same habit applies to the rest of the build. I explain how I kept the assistant itself under control in Keep your AI coding assistant on track, and how I kept the hero information correct in How I built a 204-hero fan database with Claude Code.

Common questions

Why do general chatbots make things up about niche games?

A chatbot learned from a huge amount of public writing. A niche game has very little of that, and what exists goes out of date when the game changes. Where it has gaps, it fills them with plausible guesses.

How does an AI helper answer from your own data?

Before the AI answers, it first looks up the relevant pages from your own information, then answers using only those pages. If they do not cover the question, it says it does not know.

Why does my AI helper give poor answers?

Usually the cause is the information, not the wording. Messy, out-of-date or badly labeled pages give poor results, however cleverly the question is worded. Read the pages the helper found and fix those first.

How did you keep the helper's costs under control?

Signed-in players got a daily question limit and could see how many questions they had left. I watched spending, and the helper could drop to a cheaper mode when needed, with the page saying so.

Next step

See what these tools actually did

The guides explain how the tools work. The lab shows what happened when I used them on real work: verdicts, costs, failures and the receipts behind each one.

Keep reading