Commit graph

2838 commits

Author SHA1 Message Date
sabaimran
20617614ae Merge branch 'features/customize-chat-with-agents' of github.com:khoj-ai/khoj into features/add-agents-ui 2024-03-23 11:20:57 +05:30
sabaimran
2399d91f61 Merge migrations 2024-03-22 10:05:33 +05:30
sabaimran
d38089ab57 Merge with origin 2024-03-22 09:55:33 +05:30
Debanjum Singh Solanky
7416ca9ae1 Lower the default gunicorn workers running on prod 2024-03-21 04:35:52 +05:30
Debanjum Singh Solanky
aed4313cfc Fix updating specific conversation by id from the chat API endpoint
- Use the conversation id of the retrieved conversation rather than the
  potentially unset conversation id passed via API
- await creating new chat when no chat id provided and no existing
  conversations exist
2024-03-21 02:46:52 +05:30
Debanjum Singh Solanky
ec6dc0daaf Bump up the default gunicorn workers running on prod 2024-03-20 22:56:09 +05:30
sabaimran
6ba0d8e379 Add a connected notification if the websocket is connected 2024-03-20 20:53:28 +05:30
sabaimran
255b69dc58 Add a comma delimeter between outputted search queries 2024-03-20 19:43:35 +05:30
sabaimran
d84188b221 Scroll down when a message is added in the chat interface's handle stream response method 2024-03-20 15:04:41 +05:30
sabaimran
70ad78990a Use a common method for sending a generic message to the client from the server in the ws connection 2024-03-20 15:04:14 +05:30
sabaimran
d4e83b060a Update the web UI for the chat interface to establish a connection via a socket to the server
- Move some common methods into separate functions to make the UI components more efficient
- The normal HTTP-based chat connection will still work and serves as a fallback if the websocket is unavailable
2024-03-20 14:34:47 +05:30
sabaimran
a346f79b39 Add support for chatting via the web socket connection
- Convert to a model of calling the search API directly with a function call (rather than using the API method)
- Gracefully handle websocket connection disconnects
- Ensure that the rest of the response is still saved, as it is currently, if the user disconects from the client
- Setup unchangeable context at the beginning of the session when the connection is established (like location, username, etc)
2024-03-20 14:33:33 +05:30
sabaimran
36af9776e6 Add the websockets dependency to pyproject.toml 2024-03-20 14:11:18 +05:30
Debanjum Singh Solanky
62a83dc9bb Fix online search actor to use natural dates not after: operator
The recently added after: operator to online search actor was too
restrictive, gave worse results than when just use natural language
dates in search query
2024-03-15 21:50:14 +05:30
Debanjum Singh Solanky
4a1e6a2275 Convert deleted old user requests log line to debug from info 2024-03-15 20:50:10 +05:30
Debanjum Singh Solanky
9a068dadbf Fix extract questions prompt to use YYYY-MM-DD date filter format 2024-03-15 18:43:18 +05:30
Debanjum
bb2693c792
Improve Chat Session UX, Fix Login, Chat Message Truncation (#677)
### Improve
- Improve delete, rename chat session UX in Desktop, Web app
- Get conversation by title when requested via chat API

### Fix
- Allow unset locale for Google authenticating user
- Handle truncation when single long non-system chat message
- Fix setting chat session title from Desktop app
- Only create new chat on get if a specific chat id, slug isn't requested
2024-03-15 18:19:36 +05:30
Debanjum Singh Solanky
ecddf98430 Handle truncation when single long non-system chat message
Previously was assuming the system prompt is being always passed as
the first message. So expected there to be at least 2 messages in logs.

This broke chat actors querying with single long non system message.

A more robust way to extract system prompt is via the message role
instead
2024-03-15 15:58:39 +05:30
Debanjum Singh Solanky
ec0c35b7ed Improve delete, rename chat session UX in Desktop, Web app
- Ask for Confirmation before deleting chat session in Desktop, Web app
- Save chat session rename on hitting enter in title edit input box
- No need to flash previous conversation cleared status message
- Move chat session delete button after rename button in Desktop app
2024-03-15 15:58:19 +05:30
Debanjum Singh Solanky
924b1215ce Allow unset locale for Google authenticated user 2024-03-15 15:35:20 +05:30
Debanjum Singh Solanky
c792fa819f Fix setting chat session title from Desktop app
Pass auth headers to not have the chat session title update request fail
2024-03-15 15:19:20 +05:30
Debanjum Singh Solanky
c9e05dc184 Get conversation by title when requested via chat API 2024-03-15 12:31:50 +05:30
sabaimran
724557fc7b Merge branch 'master' of github.com:khoj-ai/khoj into features/add-agents-ui 2024-03-15 12:14:34 +05:30
sabaimran
7fc484ba7a Merge branch 'master' of github.com:khoj-ai/khoj into features/customize-chat-with-agents 2024-03-15 12:13:28 +05:30
Debanjum Singh Solanky
cac26dafe3 Only create new chat on get if a specific chat id, slug isn't requested 2024-03-15 11:58:39 +05:30
sabaimran
416feb13ef Fix layout of agent, agents pages 2024-03-15 11:17:40 +05:30
sabaimran
1b3fc68a87 Fix unit test for adding the filename to the compiled markdown entry 2024-03-15 11:01:48 +05:30
sabaimran
d734be61cf Rename agents_page -> agent_page 2024-03-15 10:17:51 +05:30
Debanjum Singh Solanky
8cdfaf41ec Update project URLs to show on pypi project page 2024-03-15 04:03:39 +05:30
Debanjum Singh Solanky
08993ff109 Add new, remove old known chat models from model to prompt size map 2024-03-15 04:02:25 +05:30
Debanjum Singh Solanky
fba0338787 Release Khoj version 1.7.0 2024-03-15 00:08:32 +05:30
sabaimran
345afec47e Resolve merge conflicts/ use agent_slug instead of agent_id for lookup 2024-03-14 16:16:07 +05:30
Debanjum Singh Solanky
6118d1ff57 Create chat actor for directly reading webpages based on user message
- Add prompt for the read webpages chat actor to extract, infer
  webpage links
- Make chat actor infer or extract webpage to read directly from user
  message
- Rename previous read_webpage function to more narrow
  read_webpage_at_url function
2024-03-14 14:58:37 +05:30
Debanjum
e549824fe2
Improve OpenAI Chat Actors and their prompts (#673)
### Major
- Enforce json mode response from OpenAI chat actors prev using string lists
- Use `gpt-4-turbo-preview' as default chat model, extract questions actor
- Make Khoj read khoj website to respond with accurate, up-to-date information about itself
- Dedupe query in notes prompt. Improve OAI chat actor, director tests

### Minor
- Test data source, output mode selector, web search query chat actors
- Improve notes search actor to always create a non-empty list of queries
- Construct available data sources, output modes as a bullet list in prompts
- Use consistent agent name across static and dynamic examples in prompts
- Add actor's name to extract questions prompt to improve context for guidance
2024-03-14 12:44:40 +05:30
sabaimran
3caf0a79d8 Spruce up the 404 page and improve the overall layout for agents pages 2024-03-14 11:26:49 +05:30
sabaimran
c45030af44 Fix agent view 2024-03-14 11:13:19 +05:30
Debanjum Singh Solanky
a1ce12296f Fix rendering online with note references post streaming chat response
Previously only the notes references would get rendered post response
streaming when when both online and notes references were used to
respond to the user's message
2024-03-14 03:40:40 +05:30
Debanjum Singh Solanky
1aeea3d854 Fix opening external links from confirmation dialog box on desktop app 2024-03-14 02:29:22 +05:30
Debanjum Singh Solanky
2e5cc49cb3 Enforce json response from OpenAI chat actors prev using string lists
- Allow passing response format type to OpenAI API via chat actors
- Convert in-context examples to use json objects instead of str lists
- Update actors outputting str list to request output to be json_object
  - OpenAI's json mode enforces the model to output valid json object
2024-03-14 01:22:33 +05:30
Debanjum Singh Solanky
7211eb9cf5 Default to gpt-4-turbo-preview for chat model, extract questions actor
GPT-4 is more expensive and generally less capable than gpt-4-turbo-preview
2024-03-14 01:22:33 +05:30
Debanjum Singh Solanky
dd883dc53a Dedupe query in notes prompt. Improve OAI chat actor, director tests
- Remove stale tests
- Improve tests to pass across gpt-3.5 and gpt-4-turbo
- The haiku creation director was failing because of duplicate query in
  instantiated prompt
2024-03-14 01:22:33 +05:30
Debanjum Singh Solanky
70b04d16c0 Test data source, output mode selector, web search query chat actors 2024-03-14 01:22:33 +05:30
Debanjum Singh Solanky
14682d5354 Improve notes search actor to always create a non-empty list of queries
- Remove the option for Notes search query generation actor to return
  no queries. Whether search should be performed is decided before,
  this step doesn't need to decide that
- But do not throw warning if the response is a list with no elements
2024-03-14 01:22:33 +05:30
Debanjum Singh Solanky
f5734826cb Improve pick data source prompt to look online for info about Khoj
- Add examples where user queries requesting information about Khoj
  results in the "online" data source being selected
- Add an example for "general" to select chat command prompt
2024-03-14 01:21:13 +05:30
Debanjum Singh Solanky
9a516bed47 Construct available data sources, output modes as a bullet list in prompts 2024-03-14 00:34:57 +05:30
Debanjum Singh Solanky
f28fb89af8 Use consistent agent name across static and dynamic examples in prompts
Previously the examples constructed from chat history used "Khoj" as
the agent's name but all 3 prompts using the func used static examples
with "AI:" as the pertinent agent's name
2024-03-14 00:34:57 +05:30
Debanjum Singh Solanky
f5793149a9 Add actor's name to extract questions prompt to improve context for guidance 2024-03-14 00:34:57 +05:30
Debanjum Singh Solanky
73ad444086 Make online search Actor read khoj.dev for docs, info about Khoj
- Add example to read khoj.dev website for up-to-date info to setup,
  use khoj, discover khoj features etc.
- Online search should use site: and after: google search operators
  - Show example of adding the after: date filter to google search
- Give local event lookup example using user's current location in
  query
- Remove unused select search content type prompt
2024-03-14 00:34:57 +05:30
sabaimran
290712c3fe Add web UI views for agents
- Add a page to view all agents
- Add slugs to manage agents
- Add a view to view single agent
- Display active agent when in chat window
- Fix post-login redirect issue
2024-03-14 00:07:36 +05:30
Debanjum
3abe7ccb26
Improve Online Search Speed and Context (#670)
### Major
- Read web pages in parallel to improve chat response time
- Read web pages directly when Olostep proxy not setup
- Include search results & web page content in online context for chat response

### Minor
- Simplify, modularize and add type hints to online search functions
2024-03-11 22:16:30 +05:30