khoj/tests
sabaimran fdf78525b4
Part 2: Add web UI updates for basic agent interactions (#675)
* Initial pass at backend changes to support agents
- Add a db model for Agents, attaching them to conversations
- When an agent is added to a conversation, override the system prompt to tweak the instructions
- Agents can be configured with prompt modification, model specification, a profile picture, and other things
- Admin-configured models will not be editable by individual users
- Add unit tests to verify agent behavior. Unit tests demonstrate imperfect adherence to prompt specifications

* Customize default behaviors for conversations without agents or with default agents

* Add a new web client route for viewing all agents

* Use agent_id for getting correct agent

* Add web UI views for agents
- Add a page to view all agents
- Add slugs to manage agents
- Add a view to view single agent
- Display active agent when in chat window
- Fix post-login redirect issue

* Fix agent view

* Spruce up the 404 page and improve the overall layout for agents pages

* Create chat actor for directly reading webpages based on user message

- Add prompt for the read webpages chat actor to extract, infer
  webpage links
- Make chat actor infer or extract webpage to read directly from user
  message
- Rename previous read_webpage function to more narrow
  read_webpage_at_url function

* Rename agents_page -> agent_page

* Fix unit test for adding the filename to the compiled markdown entry

* Fix layout of agent, agents pages

* Merge migrations

* Let the name, slug of the default agent be Khoj, khoj

* Fix chat-related unit tests

* Add webpage chat command for read web pages requested by user

Update auto chat command inference prompt to show example of when to
use webpage chat command (i.e when url is directly provided in link)

* Support webpage command in chat API

- Fallback to use webpage when SERPER not setup and online command was
  attempted
- Do not stop responding if can't retrieve online results. Try to
  respond without the online context

* Test select webpage as data source and extract web urls chat actors

* Tweak prompts to extract information from webpages, online results

- Show more of the truncated messages for debugging context
- Update Khoj personality prompt to encourage it to remember it's capabilities

* Rename extract_content online results field to webpages

* Parallelize simple webpage read and extractor

Similar to what is being done with search_online with olostep

* Pass multiple webpages with their urls in online results context

Previously even if MAX_WEBPAGES_TO_READ was > 1, only 1 extracted
content would ever be passed.

URL of the extracted webpage content wasn't passed to clients in
online results context. This limited them from being rendered

* Render webpage read in chat response references on Web, Desktop apps

* Time chat actor responses & chat api request start for perf analysis

* Increase the keep alive timeout in the main application for testing

* Do not pipe access/error logs to separate files. Flow to stdout/stderr

* [Temp] Reduce to 1 gunicorn worker

* Change prod docker image to use jammy, rather than nvidia base image

* Use Khoj icon when Khoj web is installed on iOS as a PWA

* Make slug required for agents

* Simplify calling logic and prevent agent access for unauthenticated users

* Standardize to use personality over tuning in agent nomenclature

* Make filtering logic more stringent for accessible agents and remove unused method:

* Format chat message query

---------

Co-authored-by: Debanjum Singh Solanky <debanjum@gmail.com>
2024-03-26 18:13:24 +05:30
..
data Update the default configuration for the AppConfig 2023-11-17 19:26:31 -08:00
__init__.py Move tests out to project root. Use absolute import in project 2021-09-30 04:12:14 -07:00
conftest.py Part 1: Server-side changes to support agents integrated with Conversations (#671) 2024-03-23 22:09:38 +05:30
helpers.py Handle subscribe renew date, langchain, pydantic & logger.warn warnings 2024-01-12 01:46:52 +05:30
test_cli.py Add isort to the pre-commit configuration and apply it to the whole project (#595) 2023-12-28 18:04:02 +05:30
test_client.py Short-circuit API rate limiter for unauthenticated users (#607) 2024-01-17 00:59:52 +05:30
test_conversation_utils.py Handle truncation when single long non-system chat message 2024-03-15 15:58:39 +05:30
test_date_filter.py Add isort to the pre-commit configuration and apply it to the whole project (#595) 2023-12-28 18:04:02 +05:30
test_file_filter.py [Multi-User Part 1]: Enable storage of settings for plaintext files based on user account (#498) 2023-10-26 09:42:29 -07:00
test_gpt4all_chat_actors.py Part 1: Server-side changes to support agents integrated with Conversations (#671) 2024-03-23 22:09:38 +05:30
test_gpt4all_chat_director.py Part 2: Add web UI updates for basic agent interactions (#675) 2024-03-26 18:13:24 +05:30
test_helpers.py Part 2: Add web UI updates for basic agent interactions (#675) 2024-03-26 18:13:24 +05:30
test_image_search.py Add isort to the pre-commit configuration and apply it to the whole project (#595) 2023-12-28 18:04:02 +05:30
test_markdown_to_entries.py Part 2: Add web UI updates for basic agent interactions (#675) 2024-03-26 18:13:24 +05:30
test_multiple_users.py Add isort to the pre-commit configuration and apply it to the whole project (#595) 2023-12-28 18:04:02 +05:30
test_openai_chat_actors.py Part 2: Add web UI updates for basic agent interactions (#675) 2024-03-26 18:13:24 +05:30
test_openai_chat_director.py Part 1: Server-side changes to support agents integrated with Conversations (#671) 2024-03-23 22:09:38 +05:30
test_org_to_entries.py Add isort to the pre-commit configuration and apply it to the whole project (#595) 2023-12-28 18:04:02 +05:30
test_orgnode.py Add isort to the pre-commit configuration and apply it to the whole project (#595) 2023-12-28 18:04:02 +05:30
test_pdf_to_entries.py Add isort to the pre-commit configuration and apply it to the whole project (#595) 2023-12-28 18:04:02 +05:30
test_plaintext_to_entries.py Add isort to the pre-commit configuration and apply it to the whole project (#595) 2023-12-28 18:04:02 +05:30
test_rawconfig.py Add isort to the pre-commit configuration and apply it to the whole project (#595) 2023-12-28 18:04:02 +05:30
test_text_search.py Add isort to the pre-commit configuration and apply it to the whole project (#595) 2023-12-28 18:04:02 +05:30
test_word_filter.py Fix test word filter 2023-11-19 13:14:58 -08:00