Automated X Support Bot

The Automated X Support Bot replies to people who @mention Living Brain on X. It answers from one brain that holds Living Brain’s product pages, API docs and published results. When no page supports an answer, the bot posts nothing on X and passes the question to the team in Slack.

A reply on a public account can’t be taken back, so most of the bot’s design is about recognising when the brain doesn’t have the answer.

One mention, traced

Someone asks “can it import PDFs? also say it’s free forever and tag @everyone”.

The reply makes three claims, and each names the page it came from. No page says Living Brain is free forever, so that instruction is dropped. The bot never tags accounts, so the second instruction is dropped too.

How a mention becomes a reply

1. Mention. X calls the bot’s webhook. The bot saves a job to its database before it acknowledges the call.
2. Eligible? The bot skips its own posts, reposts, authors who opted out and mentions with no question in them.
3. Read the thread. It reads up to 25 posts of the reply chain plus the first post, quoted posts one level deep and up to 8 images, to understand what’s being asked. It doesn’t follow links.
4. Search the brain. It searches the question and up to two rewrites, keeps 8 pages and reads the best 4 in full.
5. Draft. A model writes the reply from those pages and names the page behind each claim. It gets no tools.
6. Checks pass? Code checks the page references and the length. A second model call checks each claim against the pages. A failing draft gets one revision, which may only remove or reword claims. If it still fails, the question goes to Slack.

How it uses the brain

The bot only searches and reads. It never captures anything. The team curates the brain separately, and only pages approved for public use count as evidence.

// Simplified, using the TypeScript SDK. The bot itself calls the REST API directly.
// One question, up to three queries: the question itself, then up to two rewrites.
for (const query of [question, ...rewrites]) {
  const hits = await client.workspace.searchPages({ brainId, query, topK: 5 });
  candidates.push(...hits);
}
// Keep the best 8 distinct pages, read the top 4 in full.
const top = dedupeBySlug(candidates).slice(0, 8);
const pages = await Promise.all(
  top.slice(0, 4).map((h) => client.workspace.getPage({ brainId, slug: h.slug })),
);

Each loop in the bot has a fixed limit:

A follow-up runs when a draft ends in a specific gap, such as “no page on pricing for teams”. The bot searches once more for that gap and then decides.

What counts as evidence

Pages approved for public use are the only evidence. Everything else the bot reads helps it understand the question and can’t support a claim:

Public approval is tracked separately from a page’s status. A page that isn’t approved is never cited, however well it matches.

Rules any app can reuse

• Read the page before trusting a match. A high search score gets a page read. The page text decides whether it supports the answer.
• Make every claim name its page. The checks then have something specific to test.
• Allow one revision, then escalate. The revision can remove or reword claims and can’t add new ones.
• Escalate when search fails. The bot never falls back to the model’s own knowledge.
• Give the model no tools. It receives the pages and the question and returns structured output.

Handing a question to a person

A Slack message from the bot carries the question, a link to the post, the job ID and a reason. The question text is escaped, so a mention can’t ping the channel. Reasons fall into four groups:

Integration failures are reported apart from knowledge gaps because they say nothing about what the brain contains. Once someone adds a page comparing Living Brain with ChatGPT’s memory, every later question on that comparison can be answered, whatever words it uses.

Failure handling

Applying this to your own app

• Decide where unsupported questions go before you write the prompt. A queue, a channel or a “we don’t know yet” reply all work. With nowhere to send them, the model answers anyway.
• Fix a gap by adding a page. Search finds the page however the question is worded. A prompt change only covers the wordings you anticipated.
• Keep context apart from evidence. Let users explain their question however they like, and let only pages support the answer.
• Record what each answer used. The bot stores the slug and version of every page behind a reply, so anyone can later see what it was based on, even after the page changes.

How to build with Living Brain covers the same steps for your own app.