Make researcher, docs search AIs ask more diverse retrieval questions

Models weren't generating a diverse enough set of questions. They'd do
minor variations on the original query. What is required is asking
queries from a bunch of different lenses to retrieve the requisite
information.

This prompt updates shows the AIs the breadth of questions to by
example and instruction. Seem like performance improved based on vibes

src/khoj/processor/conversation/prompts.py

@@ -414,7 +414,7 @@ You are Khoj, an extremely smart and helpful document search assistant with only
 Construct search queries to retrieve relevant information to answer the user's question.
 - You will be provided past questions(Q) and answers(A) for context.
 - Add as much context from the previous questions and answers as required into your search queries.
-- Break messages into multiple search queries when required to retrieve the relevant information.
+- Break your search down into multiple search queries from a diverse set of lenses to retrieve all related documents.
 - Add date filters to your search queries from questions and answers when required to retrieve the relevant information.
 - When asked a meta, vague or random questions, search for a variety of broad topics to answer the user's question.
 {personality_context}
@@ -424,7 +424,7 @@ User's Location: {location}
 {username}
 
 Q: How was my trip to Cambodia?
-Khoj: {{"queries": ["How was my trip to Cambodia?"]}}
+Khoj: {{"queries": ["How was my trip to Cambodia?", "Angkor Wat temple visit", "Flight to Phnom Penh", "Expenses in Cambodia", "Stay in Cambodia"]}}
 A: The trip was amazing. You went to the Angkor Wat temple and it was beautiful.
 
 Q: Who did i visit that temple with?
@@ -471,7 +471,7 @@ You are Khoj, an extremely smart and helpful document search assistant with only
 Construct search queries to retrieve relevant information to answer the user's question.
 - You will be provided past questions(User), extracted queries(Assistant) and answers(A) for context.
 - Add as much context from the previous questions and answers as required into your search queries.
-- Break messages into multiple search queries when required to retrieve the relevant information.
+- Break your search down into multiple search queries from a diverse set of lenses to retrieve all related documents.
 - Add date filters to your search queries from questions and answers when required to retrieve the relevant information.
 - When asked a meta, vague or random questions, search for a variety of broad topics to answer the user's question.
 {personality_context}
@@ -484,7 +484,7 @@ User's Location: {location}
 Here are some examples of how you can construct search queries to answer the user's question:
 
 User: How was my trip to Cambodia?
-Assistant: {{"queries": ["How was my trip to Cambodia?"]}}
+Assistant: {{"queries": ["How was my trip to Cambodia?", "Angkor Wat temple visit", "Flight to Phnom Penh", "Expenses in Cambodia", "Stay in Cambodia"]}}
 A: The trip was amazing. You went to the Angkor Wat temple and it was beautiful.
 
 User: What national parks did I go to last year?
@@ -632,13 +632,13 @@ Khoj:
 
 plan_function_execution = PromptTemplate.from_template(
     """
-You are Khoj, a smart, methodical researcher agent. Use the provided tool AIs to answer my query.
+You are Khoj, a smart, creative and methodical researcher. Use the provided tool AIs to investigate information to answer query.
 Create a multi-step plan and intelligently iterate on the plan based on the retrieved information to find the requested information.
 {personality_context}
 
 # Instructions
 - Ask detailed queries to the tool AIs provided below, one at a time, to discover required information or run calculations. Their response will be shown to you in the next iteration.
-- Break down your research process into independent, self-contained steps that can be executed sequentially to answer my query.
+- Break down your research process into independent, self-contained steps that can be executed sequentially to answer my query. Write your step-by-step plan in the scratchpad.
 - Ask highly diverse, detailed queries to the tool AIs, one at a time, to discover required information or run calculations.
 - NEVER repeat the same query across iterations.
 - Ensure that all the required context is passed to the tool AIs for successful execution.
@@ -649,9 +649,9 @@ Create a multi-step plan and intelligently iterate on the plan based on the retr
 # Examples
 Assuming you can search my notes and the internet.
 - When I ask for the population of my hometown
-  1. Try look up my hometown in my notes.
+  1. Try look up my hometown in my notes. Ask the note search AI to search for my birth certificate, childhood memories, school, resume etc.
-  2. If not found in my notes, try infer my hometown from my online social media profiles.
+  2. If not found in my notes, try infer my hometown from my online social media profiles. Ask the online search AI to look for {username}'s biography, school, resume on linkedin, facebook, website etc.
-  3. Only then try find the latest population of my hometown by reading official websites.
+  3. Only then try find the latest population of my hometown by reading official websites with the help of the online search and web page reading AI.
 - When I ask for my computer's specs
   1. Try find my computer model in my notes.
   2. Now find webpages with my computer model's spec online and read them.