A Reference Architecture for Domain-Specific AI Chatbots
Rosemary is a research-driven RAG chatbot architecture designed as a practical reference for building reliable domain-specific conversational AI systems using semantic retrieval, metadata-aware filtering, smart reranking, and LLM-powered response generation.
System Overview
In focused domains, a chatbot must do more than generate fluent answers. It must retrieve the right information, apply structured rules, and know when it should not answer.
A domain chatbot can sound convincing while returning the wrong menu, ignoring constraints, or inventing information that does not exist in its underlying data.
Rosemary is designed to make responses more grounded, structured, and domain-aware before the language model generates the final answer.
Semantic similarity alone cannot reliably enforce filters such as price, category, spice level, or availability.
Questions about promotions, prices, recommendations, and menu details should not be processed by one generic flow.
Retrieval alone is not enough; final answers still need validation, constrained formatting, and factual boundaries.
Rosemary is not designed to make a chatbot sound smarter. It is designed to make domain-specific responses more dependable through structured retrieval, explicit routing, and controlled generation.
Project Goals
Ground answers in retrieved menu and promotion data instead of relying on unconstrained generation.
Handle structured conditions such as category, price, temperature, spice level, and active promotions.
Route different question types through the right path and return concise, predictable output.
Core Concepts
Finds relevant candidates from menu or promotion knowledge using vector similarity search.
Applies structured constraints such as price, category, spice, temperature, and availability.
Distinguishes menu, promo, price, description, recommendation, and out-of-scope questions.
Refines candidate relevance and keeps final answers factual, concise, and consistently formatted.
Rosemary does not send every question directly to the language model. It validates, routes, retrieves, filters, ranks, and formats the result through purpose-built stages.
Tap to explore each stage of the Rosemary architecture.
Rosemary first determines whether the question belongs to the restaurant and promotion domain before any retrieval or generation begins.
A raw natural-language question from the user.
Domain keywords are checked first. If no keyword matches, the query is compared semantically against in-scope examples using embeddings.
Accepted queries continue through the pipeline; out-of-scope questions receive a controlled refusal response.
It prevents unrelated questions from reaching the retriever and reduces the chance of confident but irrelevant answers.
Rosemary identifies whether a question asks for a menu list, price, description, promotion, metadata-based recommendation, or a specific menu item.
List, price, descriptive, and exact menu-name questions can bypass unnecessary reranking or generation steps.
Promo-related terms route the query to the promotion index; other requests use the menu index.
The code distinguishes metadata questions, promotion questions, and possible specific-menu queries before ranking.
A price lookup and a recommendation query should not pay the same computational cost or follow the same logic.
Rosemary embeds menu and promotion documents separately, then stores them in distinct FAISS indexes for more focused retrieval.
Menu entries include names, descriptions, prices, categories, and attributes such as spicy, warm, soupy, and favorite status.
Promotion entries include descriptions, valid date ranges, requirements, and active status.
The implementation uses multilingual E5 embeddings with normalized vectors before performing similarity search.
Separate indexes avoid mixing two data types with different semantics, constraints, and time-sensitive fields.
Similarity search finds candidates; metadata filtering makes sure results obey explicit constraints contained in the question.
Filters cover category, warm or cold status, soup presence, spicy level, favorite status, and ingredient avoidance.
Results can be sorted for cheapest or most expensive options, or filtered using “below” and “above” price limits.
Promo queries can check active status, date range, next-month availability, and promotions nearing expiry.
This layer handles facts that semantic similarity alone cannot guarantee, such as “not spicy” or “under a certain price.”
Rosemary uses different ranking strategies based on the detected query type instead of applying one expensive reranker everywhere.
Exact and fuzzy menu-name checks identify likely menu references before CrossEncoder relevance scoring is applied.
Questions about spice, warmth, soup, favorites, or price primarily use deterministic filtering and skip CrossEncoder reranking.
For eligible queries, normalized CrossEncoder scores are blended with FAISS similarity scores before final ordering.
The architecture keeps costly semantic reranking for cases where it offers the most value, rather than using it indiscriminately.
Retrieved context is passed to Qwen with restricted prompt formats, then the response is cleaned and validated before it is returned.
Menu and promo prompts define required answer patterns, concise formatting, and limits on improvisation.
The implementation cleans prefixes, fixes menu-name spelling, enforces list punctuation, and normalizes output structure.
Promo output is checked so names and numeric values are preserved; fallback text is rebuilt from metadata when needed.
The language model is used as a constrained answer generator, not as the source of truth for menu or promotion facts.
Rosemary combines retrieval, rule-based control, and generation through decisions intended to improve reliability in a focused domain.
Tap to inspect the rationale and trade-offs behind each decision.
Rosemary creates independent FAISS indexes for menu documents and promotion documents, then routes the query to the appropriate one.
Menu data uses attributes such as category, price, warm, spicy, and soup status. Promo data uses date ranges, active status, and requirements.
The system needs explicit routing and maintains two indexes instead of one unified knowledge store.
Retrieval is more focused, and time-sensitive promotion checks can be handled without contaminating menu results.
Vector retrieval identifies candidate items, then filtering enforces conditions such as price, category, spice level, soup, warmth, favorite status, and promotion validity.
Similarity search does not guarantee that “not spicy”, “under 20,000”, or “currently active” will be respected.
The rule layer adds keyword parsing and fallback logic that must be maintained as supported query patterns grow.
Structured user constraints become enforceable instead of being left to probabilistic semantic similarity.
Rosemary skips CrossEncoder work for metadata and promotion paths, while menu-name and recommendation-like queries can receive additional relevance scoring.
A CrossEncoder can improve relevance, but it is slower than vector search and adds little value to deterministic filters.
Query classification becomes more complex because the system must decide when reranking is actually justified.
Expensive ranking is reserved for ambiguous semantic matching instead of being applied to every request.
List queries, prices, descriptions, and exact menu-name questions can bypass heavier processing when the answer is already explicit in the stored metadata.
Questions such as “berapa harga menu ini?” do not need broad reranking or creative LLM generation to be answered correctly.
Fast paths need careful handling so they do not conflict with more nuanced queries or incomplete menu-name matches.
The system reduces latency and lowers the chance that a simple fact is reformulated incorrectly by the language model.
A hybrid control layer combining vector retrieval, deterministic metadata filtering, and constrained generation.
Query-aware routing that treats price, descriptions, promos, recommendations, and out-of-scope questions differently.
A practical reliability pattern for narrow-domain chatbots built from accessible open-source components.
These examples show how Rosemary changes its handling based on query type: factual lookups take fast paths, constraints trigger metadata filters, promotion requests use a dedicated index, and unrelated questions stop at validation.
Tap to see how Rosemary handles each query type.
“Berapa harga [Nama Menu]?”
“Harga [Nama Menu] adalah [harga] rupiah.”
Output follows the controlled price-answer format defined in the generation logic.
A factual price question does not need a full reranking or free-form generation workflow when the answer exists directly in menu metadata.
“Ada makanan hangat yang tidak pedas?”
“Ada beberapa menu seperti [Menu A], [Menu B], dan [Menu C].”
The final list depends on entries that satisfy the requested metadata.
Semantic similarity finds candidates, but deterministic metadata rules enforce conditions such as warm, not spicy, category, and price.
“Promo apa yang sedang berlaku?”
“Ada promo [Nama Promo] dengan [deskripsi promo].”
Promotion descriptions and values are preserved from the stored promo data.
Promotions are handled independently because they carry time-sensitive fields such as start date, end date, active status, and conditions.
“Bagaimana cuaca hari ini?”
“Maaf, saya hanya bisa membantu pertanyaan seputar menu dan makanan di restoran ini.”
The request stops before retrieval, reranking, or LLM generation.
Guardrails are part of reliability: not answering is safer than producing a fluent but irrelevant response outside the system’s domain.