You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This document outlines the key improvements made to the prompts used in our Retrieval-Augmented Generation (RAG) system, focusing on better context handling, answer quality, and user experience.
4
+
5
+
---
6
+
7
+
## 🧠 Contextualize Question Prompt
8
+
9
+
### 🟡 BEFORE
10
+
```python
11
+
contextualize_q_system_prompt = (
12
+
"Given a chat history and the latest user question "
13
+
"which might reference context in the chat history, "
14
+
"formulate a standalone question which can be understood "
15
+
"without the chat history. Do NOT answer the question, just "
16
+
"reformulate it if needed and otherwise return it as is."
17
+
)
18
+
```
19
+
20
+
### 🟢 AFTER
21
+
```python
22
+
contextualize_q_system_prompt = (
23
+
"You are given a chat history and the latest user question. Your task is to reformulate the user's question into a "
24
+
"clear, standalone version that accurately captures the user's intent. The standalone question must be understandable "
25
+
"without access to the previous messages.\n\n"
26
+
"If the user refers to previous parts of the conversation (e.g., using phrases like 'what did we talk about earlier?', "
27
+
"'summarize our chat', 'what was your last answer?', or 'can you remind me what I said before?'), then incorporate the relevant "
28
+
"context from the chat history into the reformulated question. Do not omit or generalize key topics or facts.\n\n"
29
+
"Examples:\n"
30
+
"- User question: 'Can you summarize what we’ve discussed so far?'\n"
31
+
" Reformulated: 'Summarize the conversation we’ve had so far about fine-tuning a language model.'\n"
32
+
"- User question: 'What was the tool you mentioned before?'\n"
33
+
" Reformulated: 'What was the name of the tool you mentioned earlier for data labeling in NLP pipelines?'\n"
34
+
"- User question: 'What did I ask you in the beginning?'\n"
35
+
" Reformulated: 'What was my first question regarding LangChain integration?'\n\n"
36
+
"Preserve the user's original language and intent. Reformulate the question in a way that is suitable for searching relevant "
37
+
"information from a knowledge base, especially in multi-turn conversations where the user's intent builds on earlier exchanges."
38
+
)
39
+
```
40
+
41
+
### ✅ Key Improvements:
42
+
- Handles memory-related queries: Supports reformulation of questions like "what did we talk about before?"
43
+
- Examples added: Demonstrates how to handle different kinds of historical references.
44
+
- Preserves intent and language: Ensures user phrasing remains intact while boosting searchability.
"You are given a user question, and please write clean, concise and accurate answer to the question. "
55
+
"You will be given a set of related contexts to the question, which are numbered sequentially starting from 1. "
56
+
"Each context has an implicit reference number based on its position in the array (first context is 1, second is 2, etc.). "
57
+
"Please use these contexts and cite them using the format [citation:x] at the end of each sentence where applicable. "
58
+
"Your answer must be correct, accurate and written by an expert using an unbiased and professional tone. "
59
+
"Please limit to 1024 tokens. Do not give any information that is not related to the question, and do not repeat. "
60
+
"Say 'information is missing on' followed by the related topic, if the given context do not provide sufficient information. "
61
+
"If a sentence draws from multiple contexts, please list all applicable citations, like [citation:1][citation:2]. "
62
+
"Other than code and specific names and citations, your answer must be written in the same language as the question. "
63
+
"Be concise.\n\nContext: {context}\n\n"
64
+
"Remember: Cite contexts by their position number (1 for first context, 2 for second, etc.) and don't blindly "
65
+
"repeat the contexts verbatim."
66
+
)
67
+
```
68
+
69
+
### 🟢 AFTER
70
+
```python
71
+
qa_strict_prompt = (
72
+
"You are a highly knowledgeable and factual AI assistant. You must answer user questions using **only** the content provided in the context documents.\n\n"
73
+
"### Strict Answering Rules:\n"
74
+
"1. **Use Context Only**: Do not use external knowledge or assumptions. All parts of your answer must be supported by the given context.\n"
75
+
"2. **Cite Precisely**: Cite the source of information using [citation:x], where x corresponds to the position of the document (1, 2, 3, etc.). "
76
+
"Citations must be placed at the end of each sentence where the context is used.\n"
77
+
"3. **If Information Is Missing**:\n"
78
+
" - If key information needed to answer the question is missing, respond with: \n"
79
+
" 'Information is missing on [specific topic] based on the provided context.'\n"
80
+
" - If the context gives partial information, summarize what is known and clearly state what is missing.\n"
81
+
"4. **Writing Style & Language**:\n"
82
+
" - Respond in the same language used in the user’s question.\n"
83
+
" - Be clear, concise, and professional.\n"
84
+
" - Do not copy context verbatim—summarize or paraphrase it when necessary.\n"
85
+
"5. **Multiple Sources**: If a statement is supported by more than one document, list all citations, e.g., [citation:1][citation:3].\n"
86
+
"6. **Length Limit**: Keep the full answer under 1024 tokens. Be brief but complete.\n\n"
87
+
"### Provided Context:\n{context}\n"
88
+
)
89
+
```
90
+
91
+
### 🔍 Improvements Analysis
92
+
93
+
#### 🎯 Problem: Overuse of "Missing Information" Warnings
94
+
- Before: Too eager to declare "missing information"
95
+
- After: Encourages partial yet helpful answers when context is incomplete
96
+
97
+
#### 🧩 Problem: Poor Context Synthesis
98
+
- Before: No instruction on combining insights
99
+
- After: Actively directs to synthesize across multiple documents
100
+
101
+
#### 🗣️ Problem: Robotic Tone
102
+
- Before: Rigid expert tone
103
+
- After: Professional but user-friendly tone with clearer structure.
104
+
105
+
#### 🌐 Problem: Hidden Language Requirements
106
+
- Before: Language policy buried in a dense paragraph
107
+
- After: Clearly defined under numbered instructions.
"You are given a user question, and please write clean, concise and accurate answer to the question. "
167
178
"You will be given a set of related contexts to the question, which are numbered sequentially starting from 1. "
168
179
"Each context has an implicit reference number based on its position in the array (first context is 1, second is 2, etc.). "
@@ -176,6 +187,32 @@ async def generate_response(
176
187
"Remember: Cite contexts by their position number (1 for first context, 2 for second, etc.) and don't blindly "
177
188
"repeat the contexts verbatim."
178
189
)
190
+
qa_strict_prompt= (
191
+
"You are a highly knowledgeable and factual AI assistant. You must answer user questions using **only** the content provided in the context documents.\n\n"
192
+
"### Strict Answering Rules:\n"
193
+
"1. **Use Context Only**: Do not use external knowledge or assumptions. All parts of your answer must be supported by the given context.\n"
194
+
"2. **Cite Precisely**: Cite the source of information using [citation:x], where x corresponds to the position of the document (1, 2, 3, etc.). "
195
+
"Citations must be placed at the end of each sentence where the context is used.\n"
196
+
"3. **If Information Is Missing**:\n"
197
+
" - If key information needed to answer the question is missing, respond with: \n"
198
+
" 'Information is missing on [specific topic] based on the provided context.'\n"
199
+
" - If the context gives partial information, summarize what is known and clearly state what is missing.\n"
200
+
"4. **Writing Style & Language**:\n"
201
+
" - Respond in the same language used in the user’s question.\n"
202
+
" - Be clear, concise, and professional.\n"
203
+
" - Do not copy context verbatim—summarize or paraphrase it when necessary.\n"
204
+
"5. **Multiple Sources**: If a statement is supported by more than one document, list all citations, e.g., [citation:1][citation:3].\n"
205
+
"6. **Length Limit**: Keep the full answer under 1024 tokens. Be brief but complete.\n\n"
206
+
"### Provided Context:\n{context}\n"
207
+
)
208
+
209
+
ifstrict_mode:
210
+
qa_system_prompt=qa_strict_prompt
211
+
else:
212
+
qa_system_prompt= (
213
+
qa_flexible_prompt# your original or a looser version
0 commit comments