mirror of
https://github.com/khoj-ai/khoj.git
synced 2024-11-27 17:35:07 +01:00
Merge branch 'master' into migrate-to-llama-cpp-for-offline-chat
This commit is contained in:
Debanjum Singh Solanky
commit
886d49e3a4
51 changed files with 1708 additions and 145 deletions
12
.github/workflows/pypi.yml
vendored
12
.github/workflows/pypi.yml
vendored
|
|
@ -21,16 +21,18 @@ on:
|
||||||
jobs:
|
jobs:
|
||||||
publish:
|
publish:
|
||||||
name: Publish Python Package to PyPI
|
name: Publish Python Package to PyPI
|
||||||
runs-on: ubuntu-20.04
|
runs-on: ubuntu-latest
|
||||||
|
permissions:
|
||||||
|
id-token: write
|
||||||
steps:
|
steps:
|
||||||
- uses: actions/checkout@v3
|
- uses: actions/checkout@v3
|
||||||
with:
|
with:
|
||||||
fetch-depth: 0
|
fetch-depth: 0
|
||||||
|
|
||||||
- name: Set up Python 3.10
|
- name: Set up Python 3.11
|
||||||
uses: actions/setup-python@v4
|
uses: actions/setup-python@v4
|
||||||
with:
|
with:
|
||||||
python-version: '3.10'
|
python-version: '3.11'
|
||||||
|
|
||||||
- name: ⬇️ Install Application
|
- name: ⬇️ Install Application
|
||||||
run: python -m pip install --upgrade pip && pip install --upgrade .
|
run: python -m pip install --upgrade pip && pip install --upgrade .
|
||||||
|
|
@ -59,6 +61,4 @@ jobs:
|
||||||
|
|
||||||
- name: 📦 Publish Python Package to PyPI
|
- name: 📦 Publish Python Package to PyPI
|
||||||
if: startsWith(github.ref, 'refs/tags') || github.ref == 'refs/heads/master'
|
if: startsWith(github.ref, 'refs/tags') || github.ref == 'refs/heads/master'
|
||||||
uses: pypa/gh-action-pypi-publish@v1.6.4
|
uses: pypa/gh-action-pypi-publish@v1.8.14
|
||||||
with:
|
|
||||||
password: ${{ secrets.PYPI_API_KEY }}
|
|
||||||
|
|
|
||||||
BIN
documentation/assets/img/agents_demo.gif
Normal file
BIN
documentation/assets/img/agents_demo.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 14 MiB |
BIN
documentation/assets/img/dream_house.png
Normal file
BIN
documentation/assets/img/dream_house.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 3 MiB |
BIN
documentation/assets/img/plants_i_got.png
Normal file
BIN
documentation/assets/img/plants_i_got.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 3 MiB |
BIN
documentation/assets/img/using_khoj_for_studying.gif
Normal file
BIN
documentation/assets/img/using_khoj_for_studying.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 32 MiB |
15
documentation/docs/features/agents.md
Normal file
15
documentation/docs/features/agents.md
Normal file
|
|
@ -0,0 +1,15 @@
|
||||||
|
---
|
||||||
|
sidebar_position: 4
|
||||||
|
---
|
||||||
|
|
||||||
|
# Agents
|
||||||
|
|
||||||
|
You can use agents to setup custom system prompts with Khoj. The server host can setup their own agents, which are accessible to all users. You can see ours at https://app.khoj.dev/agents.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## Creating an Agent (Self-Hosted)
|
||||||
|
|
||||||
|
Go to `server/admin/database/agent` on your server and click `Add Agent` to create a new one. You have to set it to `public` in order for it to be accessible to all the users on your server. To limit access to a specific user, do not set the `public` flag and add the user in the `Creator` field.
|
||||||
|
|
||||||
|
Set your custom prompt in the `personality` field.
|
||||||
|
|
@ -2,7 +2,7 @@
|
||||||
sidebar_position: 1
|
sidebar_position: 1
|
||||||
---
|
---
|
||||||
|
|
||||||
# Features
|
# Overview
|
||||||
|
|
||||||
Khoj supports a variety of features, including search and chat with a wide range of data sources and interfaces.
|
Khoj supports a variety of features, including search and chat with a wide range of data sources and interfaces.
|
||||||
|
|
||||||
|
|
|
||||||
15
documentation/docs/features/image_generation.md
Normal file
15
documentation/docs/features/image_generation.md
Normal file
|
|
@ -0,0 +1,15 @@
|
||||||
|
# Image Generation
|
||||||
|
You can use Khoj to generate images from text prompts. You can get deeper into the details of our image generation flow in this blog post: https://blog.khoj.dev/posts/how-khoj-generates-images/.
|
||||||
|
|
||||||
|
To generate images, you just need to provide a prompt to Khoj in which the image generation is in the instructions. Khoj will automatically detect the image generation intent, augment your generation prompt, and then create the image. Here are some examples:
|
||||||
|
| Prompt | Image |
|
||||||
|
| --- | --- |
|
||||||
|
| Paint a picture of the plants I got last month, pixar-animation |  |
|
||||||
|
| Create a picture of my dream house, based on my interests |  |
|
||||||
|
|
||||||
|
|
||||||
|
## Setup (Self-Hosting)
|
||||||
|
|
||||||
|
Right now, we only support integration with OpenAI's DALL-E. You need to have an OpenAI API key to use this feature. Here's how you can set it up:
|
||||||
|
1. Setup your OpenAI API key. See instructions [here](/get-started/setup#2-configure)
|
||||||
|
2. Create a text to image config at http://localhost:42110/server/admin/database/texttoimagemodelconfig/. We recommend the value `dall-e-3`.
|
||||||
14
documentation/docs/features/voice_chat.md
Normal file
14
documentation/docs/features/voice_chat.md
Normal file
|
|
@ -0,0 +1,14 @@
|
||||||
|
# Voice
|
||||||
|
|
||||||
|
You can talk to Khoj using your voice. Khoj will respond to your queries using the same models as the chat feature. You can use voice chat on the web, Desktop, and Obsidian apps. Click on the little mic icon to send your voice message to Khoj. It will send back what it heard via text. You'll have some time to edit it before sending it, if required. Try it at https://app.khoj.dev/.
|
||||||
|
|
||||||
|
:::info[Voice Response]
|
||||||
|
Khoj doesn't yet respond with voice, but it will send back a text response. Let us know if you're interested in voice responses at team at khoj.dev.
|
||||||
|
:::
|
||||||
|
|
||||||
|
## Setup (Self-Hosting)
|
||||||
|
|
||||||
|
Voice chat will automatically be configured when you initialize the application. The default configuration will run locally. If you want to use the OpenAI whisper API for voice chat, you can set it up by following these steps:
|
||||||
|
|
||||||
|
1. Setup your OpenAI API key. See instructions [here](/get-started/setup#2-configure).
|
||||||
|
2. Create a new configuration at http://localhost:42110/server/admin/database/speechtotextmodeloptions/. We recommend the value `whisper-1` and model type `Openai`.
|
||||||
|
|
@ -37,9 +37,7 @@ Welcome to the Khoj Docs! This is the best place to get setup and explore Khoj's
|
||||||
- [Read these instructions](/get-started/setup) to self-host a private instance of Khoj
|
- [Read these instructions](/get-started/setup) to self-host a private instance of Khoj
|
||||||
|
|
||||||
## At a Glance
|
## At a Glance
|
||||||
<img src="https://docs.khoj.dev/img/khoj_search_on_web.png" width="400px" />
|
|
||||||
<span> </span>
|
|
||||||
<img src="https://docs.khoj.dev/img/khoj_chat_on_web.png" width="400px" />
|
|
||||||
|
|
||||||
#### [Search](/features/search)
|
#### [Search](/features/search)
|
||||||
- **Natural**: Use natural language queries to quickly find relevant notes and documents.
|
- **Natural**: Use natural language queries to quickly find relevant notes and documents.
|
||||||
|
|
|
||||||
|
|
@ -25,6 +25,10 @@ These are the general setup instructions for self-hosted Khoj.
|
||||||
|
|
||||||
For Installation, you can either use Docker or install the Khoj server locally.
|
For Installation, you can either use Docker or install the Khoj server locally.
|
||||||
|
|
||||||
|
:::info[Offline Model + GPU]
|
||||||
|
If you want to use the offline chat model and you have a GPU, you should use Installation Option 2 - local setup via the Python package directly. Our Docker image doesn't currently support running the offline chat model on GPU, making inference times really slow.
|
||||||
|
:::
|
||||||
|
|
||||||
### Installation Option 1 (Docker)
|
### Installation Option 1 (Docker)
|
||||||
|
|
||||||
#### Prerequisites
|
#### Prerequisites
|
||||||
|
|
@ -183,15 +187,15 @@ Khoj should now be running at http://localhost:42110. You can see the web UI in
|
||||||
Note: To start Khoj automatically in the background use [Task scheduler](https://www.windowscentral.com/how-create-automated-task-using-task-scheduler-windows-10) on Windows or [Cron](https://en.wikipedia.org/wiki/Cron) on Mac, Linux (e.g with `@reboot khoj`)
|
Note: To start Khoj automatically in the background use [Task scheduler](https://www.windowscentral.com/how-create-automated-task-using-task-scheduler-windows-10) on Windows or [Cron](https://en.wikipedia.org/wiki/Cron) on Mac, Linux (e.g with `@reboot khoj`)
|
||||||
|
|
||||||
|
|
||||||
### 2. Download the desktop client
|
### Setup Notes
|
||||||
|
|
||||||
You can use our desktop executables to select file paths and folders to index. You can simply select the folders or files, and they'll be automatically uploaded to the server. Once you specify a file or file path, you don't need to update the configuration again; it will grab any data diffs dynamically over time.
|
Optionally, you can use Khoj with a custom domain as well. To do so, you need to set the `KHOJ_DOMAIN` environment variable to your domain (e.g., `export KHOJ_DOMAIN=my-khoj-domain.com` or add it to your `docker-compose.yml`). By default, the Khoj server you set up will not be accessible outside of `localhost` or `127.0.0.1`.
|
||||||
|
|
||||||
**To download the latest desktop client, go to https://download.khoj.dev** and the correct executable for your OS will automatically start downloading. You can also go to https://khoj.dev/downloads to explicitly download your image of choice. Once downloaded, you can configure your folders for indexing using the settings tab. To set your chat configuration, you'll have to use the web interface for the Khoj server you setup in the previous step.
|
:::warning[Must use an SSL certificate]
|
||||||
|
If you're using a custom domain, you must use an SSL certificate. You can use [Let's Encrypt](https://letsencrypt.org/) to get a free SSL certificate for your domain.
|
||||||
|
:::
|
||||||
|
|
||||||
To use the desktop client, you need to go to your Khoj server's settings page (http://localhost:42110/config) and copy the API key. Then, paste it into the desktop client's settings page. Once you've done that, you can select files and folders to index. Set the desktop client settings to use `http://127.0.0.1:42110` as the host URL.
|
### 2. Configure
|
||||||
|
|
||||||
### 3. Configure
|
|
||||||
1. Go to http://localhost:42110/server/admin and login with your admin credentials.
|
1. Go to http://localhost:42110/server/admin and login with your admin credentials.
|
||||||
1. Go to [OpenAI settings](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/) in the server admin settings to add an OpenAI processor conversation config. This is where you set your API key. Alternatively, you can go to the [offline chat settings](http://localhost:42110/server/admin/database/offlinechatprocessorconversationconfig/) and simply create a new setting with `Enabled` set to `True`.
|
1. Go to [OpenAI settings](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/) in the server admin settings to add an OpenAI processor conversation config. This is where you set your API key. Alternatively, you can go to the [offline chat settings](http://localhost:42110/server/admin/database/offlinechatprocessorconversationconfig/) and simply create a new setting with `Enabled` set to `True`.
|
||||||
2. Go to the ChatModelOptions if you want to add additional models for chat.
|
2. Go to the ChatModelOptions if you want to add additional models for chat.
|
||||||
|
|
@ -207,6 +211,14 @@ To use the desktop client, you need to go to your Khoj server's settings page (h
|
||||||
Using Safari on Mac? You might not be able to login to the admin panel. Try using Chrome or Firefox instead.
|
Using Safari on Mac? You might not be able to login to the admin panel. Try using Chrome or Firefox instead.
|
||||||
:::
|
:::
|
||||||
|
|
||||||
|
### 3. Download the desktop client (Optional)
|
||||||
|
|
||||||
|
You can use our desktop executables to select file paths and folders to index. You can simply select the folders or files, and they'll be automatically uploaded to the server. Once you specify a file or file path, you don't need to update the configuration again; it will grab any data diffs dynamically over time.
|
||||||
|
|
||||||
|
**To download the latest desktop client, go to https://download.khoj.dev** and the correct executable for your OS will automatically start downloading. You can also go to https://khoj.dev/downloads to explicitly download your image of choice. Once downloaded, you can configure your folders for indexing using the settings tab. To set your chat configuration, you'll have to use the web interface for the Khoj server you setup in the previous step.
|
||||||
|
|
||||||
|
To use the desktop client, you need to go to your Khoj server's settings page (http://localhost:42110/config) and copy the API key. Then, paste it into the desktop client's settings page. Once you've done that, you can select files and folders to index. Set the desktop client settings to use `http://127.0.0.1:42110` as the host URL.
|
||||||
|
|
||||||
|
|
||||||
### 4. Install Client Plugins (Optional)
|
### 4. Install Client Plugins (Optional)
|
||||||
Khoj exposes a web interface to search, chat and configure by default.<br />
|
Khoj exposes a web interface to search, chat and configure by default.<br />
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
{
|
{
|
||||||
"id": "khoj",
|
"id": "khoj",
|
||||||
"name": "Khoj",
|
"name": "Khoj",
|
||||||
"version": "1.7.0",
|
"version": "1.8.0",
|
||||||
"minAppVersion": "0.15.0",
|
"minAppVersion": "0.15.0",
|
||||||
"description": "An AI copilot for your Second Brain",
|
"description": "An AI copilot for your Second Brain",
|
||||||
"author": "Khoj Inc.",
|
"author": "Khoj Inc.",
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,7 @@ name = "khoj-assistant"
|
||||||
description = "An AI copilot for your Second Brain"
|
description = "An AI copilot for your Second Brain"
|
||||||
readme = "README.md"
|
readme = "README.md"
|
||||||
license = "AGPL-3.0-or-later"
|
license = "AGPL-3.0-or-later"
|
||||||
requires-python = ">=3.8"
|
requires-python = ">=3.9"
|
||||||
authors = [
|
authors = [
|
||||||
{ name = "Debanjum Singh Solanky, Saba Imran" },
|
{ name = "Debanjum Singh Solanky, Saba Imran" },
|
||||||
]
|
]
|
||||||
|
|
@ -23,8 +23,8 @@ keywords = [
|
||||||
"pdf",
|
"pdf",
|
||||||
]
|
]
|
||||||
classifiers = [
|
classifiers = [
|
||||||
"Development Status :: 4 - Beta",
|
"Development Status :: 5 - Production/Stable",
|
||||||
"License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
|
"License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)",
|
||||||
"Operating System :: OS Independent",
|
"Operating System :: OS Independent",
|
||||||
"Programming Language :: Python :: 3",
|
"Programming Language :: Python :: 3",
|
||||||
"Programming Language :: Python :: 3.9",
|
"Programming Language :: Python :: 3.9",
|
||||||
|
|
@ -33,7 +33,7 @@ classifiers = [
|
||||||
"Topic :: Internet :: WWW/HTTP :: Indexing/Search",
|
"Topic :: Internet :: WWW/HTTP :: Indexing/Search",
|
||||||
"Topic :: Scientific/Engineering :: Artificial Intelligence",
|
"Topic :: Scientific/Engineering :: Artificial Intelligence",
|
||||||
"Topic :: Scientific/Engineering :: Human Machine Interfaces",
|
"Topic :: Scientific/Engineering :: Human Machine Interfaces",
|
||||||
"Topic :: Text Processing :: Linguistic",
|
"Intended Audience :: Information Technology",
|
||||||
]
|
]
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"beautifulsoup4 ~= 4.12.3",
|
"beautifulsoup4 ~= 4.12.3",
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
{
|
{
|
||||||
"name": "Khoj",
|
"name": "Khoj",
|
||||||
"version": "1.7.0",
|
"version": "1.8.0",
|
||||||
"description": "An AI copilot for your Second Brain",
|
"description": "An AI copilot for your Second Brain",
|
||||||
"author": "Saba Imran, Debanjum Singh Solanky <team@khoj.dev>",
|
"author": "Saba Imran, Debanjum Singh Solanky <team@khoj.dev>",
|
||||||
"license": "GPL-3.0-or-later",
|
"license": "GPL-3.0-or-later",
|
||||||
|
|
|
||||||
|
|
@ -6,7 +6,7 @@
|
||||||
;; Saba Imran <saba@khoj.dev>
|
;; Saba Imran <saba@khoj.dev>
|
||||||
;; Description: An AI copilot for your Second Brain
|
;; Description: An AI copilot for your Second Brain
|
||||||
;; Keywords: search, chat, org-mode, outlines, markdown, pdf, image
|
;; Keywords: search, chat, org-mode, outlines, markdown, pdf, image
|
||||||
;; Version: 1.7.0
|
;; Version: 1.8.0
|
||||||
;; Package-Requires: ((emacs "27.1") (transient "0.3.0") (dash "2.19.1"))
|
;; Package-Requires: ((emacs "27.1") (transient "0.3.0") (dash "2.19.1"))
|
||||||
;; URL: https://github.com/khoj-ai/khoj/tree/master/src/interface/emacs
|
;; URL: https://github.com/khoj-ai/khoj/tree/master/src/interface/emacs
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
{
|
{
|
||||||
"id": "khoj",
|
"id": "khoj",
|
||||||
"name": "Khoj",
|
"name": "Khoj",
|
||||||
"version": "1.7.0",
|
"version": "1.8.0",
|
||||||
"minAppVersion": "0.15.0",
|
"minAppVersion": "0.15.0",
|
||||||
"description": "An AI copilot for your Second Brain",
|
"description": "An AI copilot for your Second Brain",
|
||||||
"author": "Khoj Inc.",
|
"author": "Khoj Inc.",
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
{
|
{
|
||||||
"name": "Khoj",
|
"name": "Khoj",
|
||||||
"version": "1.7.0",
|
"version": "1.8.0",
|
||||||
"description": "An AI copilot for your Second Brain",
|
"description": "An AI copilot for your Second Brain",
|
||||||
"author": "Debanjum Singh Solanky, Saba Imran <team@khoj.dev>",
|
"author": "Debanjum Singh Solanky, Saba Imran <team@khoj.dev>",
|
||||||
"license": "GPL-3.0-or-later",
|
"license": "GPL-3.0-or-later",
|
||||||
|
|
|
||||||
|
|
@ -39,5 +39,6 @@
|
||||||
"1.6.0": "0.15.0",
|
"1.6.0": "0.15.0",
|
||||||
"1.6.1": "0.15.0",
|
"1.6.1": "0.15.0",
|
||||||
"1.6.2": "0.15.0",
|
"1.6.2": "0.15.0",
|
||||||
"1.7.0": "0.15.0"
|
"1.7.0": "0.15.0",
|
||||||
|
"1.8.0": "0.15.0"
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -21,6 +21,7 @@ from starlette.middleware.sessions import SessionMiddleware
|
||||||
from starlette.requests import HTTPConnection
|
from starlette.requests import HTTPConnection
|
||||||
|
|
||||||
from khoj.database.adapters import (
|
from khoj.database.adapters import (
|
||||||
|
AgentAdapters,
|
||||||
ClientApplicationAdapters,
|
ClientApplicationAdapters,
|
||||||
ConversationAdapters,
|
ConversationAdapters,
|
||||||
SubscriptionState,
|
SubscriptionState,
|
||||||
|
|
@ -229,11 +230,16 @@ def configure_server(
|
||||||
|
|
||||||
state.SearchType = configure_search_types()
|
state.SearchType = configure_search_types()
|
||||||
state.search_models = configure_search(state.search_models, state.config.search_type)
|
state.search_models = configure_search(state.search_models, state.config.search_type)
|
||||||
|
setup_default_agent()
|
||||||
initialize_content(regenerate, search_type, init, user)
|
initialize_content(regenerate, search_type, init, user)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
raise e
|
raise e
|
||||||
|
|
||||||
|
|
||||||
|
def setup_default_agent():
|
||||||
|
AgentAdapters.create_default_agent()
|
||||||
|
|
||||||
|
|
||||||
def initialize_content(regenerate: bool, search_type: Optional[SearchType] = None, init=False, user: KhojUser = None):
|
def initialize_content(regenerate: bool, search_type: Optional[SearchType] = None, init=False, user: KhojUser = None):
|
||||||
# Initialize Content from Config
|
# Initialize Content from Config
|
||||||
if state.search_models:
|
if state.search_models:
|
||||||
|
|
@ -262,6 +268,7 @@ def initialize_content(regenerate: bool, search_type: Optional[SearchType] = Non
|
||||||
def configure_routes(app):
|
def configure_routes(app):
|
||||||
# Import APIs here to setup search types before while configuring server
|
# Import APIs here to setup search types before while configuring server
|
||||||
from khoj.routers.api import api
|
from khoj.routers.api import api
|
||||||
|
from khoj.routers.api_agents import api_agents
|
||||||
from khoj.routers.api_chat import api_chat
|
from khoj.routers.api_chat import api_chat
|
||||||
from khoj.routers.api_config import api_config
|
from khoj.routers.api_config import api_config
|
||||||
from khoj.routers.indexer import indexer
|
from khoj.routers.indexer import indexer
|
||||||
|
|
@ -269,6 +276,7 @@ def configure_routes(app):
|
||||||
|
|
||||||
app.include_router(api, prefix="/api")
|
app.include_router(api, prefix="/api")
|
||||||
app.include_router(api_chat, prefix="/api/chat")
|
app.include_router(api_chat, prefix="/api/chat")
|
||||||
|
app.include_router(api_agents, prefix="/api/agents")
|
||||||
app.include_router(api_config, prefix="/api/config")
|
app.include_router(api_config, prefix="/api/config")
|
||||||
app.include_router(indexer, prefix="/api/v1/index")
|
app.include_router(indexer, prefix="/api/v1/index")
|
||||||
app.include_router(web_client)
|
app.include_router(web_client)
|
||||||
|
|
|
||||||
|
|
@ -16,6 +16,7 @@ from pgvector.django import CosineDistance
|
||||||
from torch import Tensor
|
from torch import Tensor
|
||||||
|
|
||||||
from khoj.database.models import (
|
from khoj.database.models import (
|
||||||
|
Agent,
|
||||||
ChatModelOptions,
|
ChatModelOptions,
|
||||||
ClientApplication,
|
ClientApplication,
|
||||||
Conversation,
|
Conversation,
|
||||||
|
|
@ -37,6 +38,7 @@ from khoj.database.models import (
|
||||||
UserRequests,
|
UserRequests,
|
||||||
UserSearchModelConfig,
|
UserSearchModelConfig,
|
||||||
)
|
)
|
||||||
|
from khoj.processor.conversation import prompts
|
||||||
from khoj.search_filter.date_filter import DateFilter
|
from khoj.search_filter.date_filter import DateFilter
|
||||||
from khoj.search_filter.file_filter import FileFilter
|
from khoj.search_filter.file_filter import FileFilter
|
||||||
from khoj.search_filter.word_filter import WordFilter
|
from khoj.search_filter.word_filter import WordFilter
|
||||||
|
|
@ -391,6 +393,80 @@ class ClientApplicationAdapters:
|
||||||
return await ClientApplication.objects.filter(client_id=client_id, client_secret=client_secret).afirst()
|
return await ClientApplication.objects.filter(client_id=client_id, client_secret=client_secret).afirst()
|
||||||
|
|
||||||
|
|
||||||
|
class AgentAdapters:
|
||||||
|
DEFAULT_AGENT_NAME = "Khoj"
|
||||||
|
DEFAULT_AGENT_AVATAR = "https://khoj-web-bucket.s3.amazonaws.com/lamp-128.png"
|
||||||
|
DEFAULT_AGENT_SLUG = "khoj"
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
async def aget_agent_by_slug(agent_slug: str, user: KhojUser):
|
||||||
|
return await Agent.objects.filter(
|
||||||
|
(Q(slug__iexact=agent_slug.lower())) & (Q(public=True) | Q(creator=user))
|
||||||
|
).afirst()
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get_agent_by_slug(slug: str, user: KhojUser = None):
|
||||||
|
if user:
|
||||||
|
return Agent.objects.filter((Q(slug__iexact=slug.lower())) & (Q(public=True) | Q(creator=user))).first()
|
||||||
|
return Agent.objects.filter(slug__iexact=slug.lower(), public=True).first()
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get_all_accessible_agents(user: KhojUser = None):
|
||||||
|
if user:
|
||||||
|
return Agent.objects.filter(Q(public=True) | Q(creator=user)).distinct().order_by("created_at")
|
||||||
|
return Agent.objects.filter(public=True).order_by("created_at")
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
async def aget_all_accessible_agents(user: KhojUser = None) -> List[Agent]:
|
||||||
|
agents = await sync_to_async(AgentAdapters.get_all_accessible_agents)(user)
|
||||||
|
return await sync_to_async(list)(agents)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get_conversation_agent_by_id(agent_id: int):
|
||||||
|
agent = Agent.objects.filter(id=agent_id).first()
|
||||||
|
if agent == AgentAdapters.get_default_agent():
|
||||||
|
# If the agent is set to the default agent, then return None and let the default application code be used
|
||||||
|
return None
|
||||||
|
return agent
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get_default_agent():
|
||||||
|
return Agent.objects.filter(name=AgentAdapters.DEFAULT_AGENT_NAME).first()
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def create_default_agent():
|
||||||
|
default_conversation_config = ConversationAdapters.get_default_conversation_config()
|
||||||
|
default_personality = prompts.personality.format(current_date="placeholder")
|
||||||
|
|
||||||
|
agent = Agent.objects.filter(name=AgentAdapters.DEFAULT_AGENT_NAME).first()
|
||||||
|
|
||||||
|
if agent:
|
||||||
|
agent.personality = default_personality
|
||||||
|
agent.chat_model = default_conversation_config
|
||||||
|
agent.slug = AgentAdapters.DEFAULT_AGENT_SLUG
|
||||||
|
agent.name = AgentAdapters.DEFAULT_AGENT_NAME
|
||||||
|
agent.save()
|
||||||
|
else:
|
||||||
|
# The default agent is public and managed by the admin. It's handled a little differently than other agents.
|
||||||
|
agent = Agent.objects.create(
|
||||||
|
name=AgentAdapters.DEFAULT_AGENT_NAME,
|
||||||
|
public=True,
|
||||||
|
managed_by_admin=True,
|
||||||
|
chat_model=default_conversation_config,
|
||||||
|
personality=default_personality,
|
||||||
|
tools=["*"],
|
||||||
|
avatar=AgentAdapters.DEFAULT_AGENT_AVATAR,
|
||||||
|
slug=AgentAdapters.DEFAULT_AGENT_SLUG,
|
||||||
|
)
|
||||||
|
Conversation.objects.filter(agent=None).update(agent=agent)
|
||||||
|
|
||||||
|
return agent
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
async def aget_default_agent():
|
||||||
|
return await Agent.objects.filter(name=AgentAdapters.DEFAULT_AGENT_NAME).afirst()
|
||||||
|
|
||||||
|
|
||||||
class ConversationAdapters:
|
class ConversationAdapters:
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def get_conversation_by_user(
|
def get_conversation_by_user(
|
||||||
|
|
@ -403,9 +479,10 @@ class ConversationAdapters:
|
||||||
.first()
|
.first()
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
|
agent = AgentAdapters.get_default_agent()
|
||||||
conversation = (
|
conversation = (
|
||||||
Conversation.objects.filter(user=user, client=client_application).order_by("-updated_at").first()
|
Conversation.objects.filter(user=user, client=client_application).order_by("-updated_at").first()
|
||||||
) or Conversation.objects.create(user=user, client=client_application)
|
) or Conversation.objects.create(user=user, client=client_application, agent=agent)
|
||||||
|
|
||||||
return conversation
|
return conversation
|
||||||
|
|
||||||
|
|
@ -431,8 +508,16 @@ class ConversationAdapters:
|
||||||
return Conversation.objects.filter(id=conversation_id).first()
|
return Conversation.objects.filter(id=conversation_id).first()
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
async def acreate_conversation_session(user: KhojUser, client_application: ClientApplication = None):
|
async def acreate_conversation_session(
|
||||||
return await Conversation.objects.acreate(user=user, client=client_application)
|
user: KhojUser, client_application: ClientApplication = None, agent_slug: str = None
|
||||||
|
):
|
||||||
|
if agent_slug:
|
||||||
|
agent = await AgentAdapters.aget_agent_by_slug(agent_slug, user)
|
||||||
|
if agent is None:
|
||||||
|
raise HTTPException(status_code=400, detail="No such agent currently exists.")
|
||||||
|
return await Conversation.objects.acreate(user=user, client=client_application, agent=agent)
|
||||||
|
agent = await AgentAdapters.aget_default_agent()
|
||||||
|
return await Conversation.objects.acreate(user=user, client=client_application, agent=agent)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
async def aget_conversation_by_user(
|
async def aget_conversation_by_user(
|
||||||
|
|
@ -443,6 +528,11 @@ class ConversationAdapters:
|
||||||
elif title:
|
elif title:
|
||||||
return await Conversation.objects.filter(user=user, client=client_application, title=title).afirst()
|
return await Conversation.objects.filter(user=user, client=client_application, title=title).afirst()
|
||||||
else:
|
else:
|
||||||
|
conversation = Conversation.objects.filter(user=user, client=client_application).order_by("-updated_at")
|
||||||
|
|
||||||
|
if await conversation.aexists():
|
||||||
|
return await conversation.prefetch_related("agent").afirst()
|
||||||
|
|
||||||
return await (
|
return await (
|
||||||
Conversation.objects.filter(user=user, client=client_application).order_by("-updated_at").afirst()
|
Conversation.objects.filter(user=user, client=client_application).order_by("-updated_at").afirst()
|
||||||
) or await Conversation.objects.acreate(user=user, client=client_application)
|
) or await Conversation.objects.acreate(user=user, client=client_application)
|
||||||
|
|
@ -603,9 +693,14 @@ class ConversationAdapters:
|
||||||
return random.sample(all_questions, max_results)
|
return random.sample(all_questions, max_results)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def get_valid_conversation_config(user: KhojUser):
|
def get_valid_conversation_config(user: KhojUser, conversation: Conversation):
|
||||||
offline_chat_config = ConversationAdapters.get_offline_chat_conversation_config()
|
offline_chat_config = ConversationAdapters.get_offline_chat_conversation_config()
|
||||||
|
|
||||||
|
if conversation.agent and conversation.agent.chat_model:
|
||||||
|
conversation_config = conversation.agent.chat_model
|
||||||
|
else:
|
||||||
conversation_config = ConversationAdapters.get_conversation_config(user)
|
conversation_config = ConversationAdapters.get_conversation_config(user)
|
||||||
|
|
||||||
if conversation_config is None:
|
if conversation_config is None:
|
||||||
conversation_config = ConversationAdapters.get_default_conversation_config()
|
conversation_config = ConversationAdapters.get_default_conversation_config()
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -6,6 +6,7 @@ from django.contrib.auth.admin import UserAdmin
|
||||||
from django.http import HttpResponse
|
from django.http import HttpResponse
|
||||||
|
|
||||||
from khoj.database.models import (
|
from khoj.database.models import (
|
||||||
|
Agent,
|
||||||
ChatModelOptions,
|
ChatModelOptions,
|
||||||
ClientApplication,
|
ClientApplication,
|
||||||
Conversation,
|
Conversation,
|
||||||
|
|
@ -50,6 +51,7 @@ admin.site.register(ReflectiveQuestion)
|
||||||
admin.site.register(UserSearchModelConfig)
|
admin.site.register(UserSearchModelConfig)
|
||||||
admin.site.register(TextToImageModelConfig)
|
admin.site.register(TextToImageModelConfig)
|
||||||
admin.site.register(ClientApplication)
|
admin.site.register(ClientApplication)
|
||||||
|
admin.site.register(Agent)
|
||||||
|
|
||||||
|
|
||||||
@admin.register(Entry)
|
@admin.register(Entry)
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,53 @@
|
||||||
|
# Generated by Django 4.2.10 on 2024-03-13 07:38
|
||||||
|
|
||||||
|
import django.db.models.deletion
|
||||||
|
from django.conf import settings
|
||||||
|
from django.db import migrations, models
|
||||||
|
|
||||||
|
|
||||||
|
class Migration(migrations.Migration):
|
||||||
|
dependencies = [
|
||||||
|
("database", "0030_conversation_slug_and_title"),
|
||||||
|
]
|
||||||
|
|
||||||
|
operations = [
|
||||||
|
migrations.CreateModel(
|
||||||
|
name="Agent",
|
||||||
|
fields=[
|
||||||
|
("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
|
||||||
|
("created_at", models.DateTimeField(auto_now_add=True)),
|
||||||
|
("updated_at", models.DateTimeField(auto_now=True)),
|
||||||
|
("name", models.CharField(max_length=200)),
|
||||||
|
("tuning", models.TextField()),
|
||||||
|
("avatar", models.URLField(blank=True, default=None, max_length=400, null=True)),
|
||||||
|
("tools", models.JSONField(default=list)),
|
||||||
|
("public", models.BooleanField(default=False)),
|
||||||
|
("managed_by_admin", models.BooleanField(default=False)),
|
||||||
|
("slug", models.CharField(max_length=200)),
|
||||||
|
(
|
||||||
|
"chat_model",
|
||||||
|
models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to="database.chatmodeloptions"),
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"creator",
|
||||||
|
models.ForeignKey(
|
||||||
|
blank=True,
|
||||||
|
default=None,
|
||||||
|
null=True,
|
||||||
|
on_delete=django.db.models.deletion.CASCADE,
|
||||||
|
to=settings.AUTH_USER_MODEL,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
],
|
||||||
|
options={
|
||||||
|
"abstract": False,
|
||||||
|
},
|
||||||
|
),
|
||||||
|
migrations.AddField(
|
||||||
|
model_name="conversation",
|
||||||
|
name="agent",
|
||||||
|
field=models.ForeignKey(
|
||||||
|
blank=True, default=None, null=True, on_delete=django.db.models.deletion.SET_NULL, to="database.agent"
|
||||||
|
),
|
||||||
|
),
|
||||||
|
]
|
||||||
|
|
@ -3,6 +3,18 @@
|
||||||
from django.db import migrations, models
|
from django.db import migrations, models
|
||||||
|
|
||||||
|
|
||||||
|
def set_default_locale(apps, schema_editor):
|
||||||
|
return
|
||||||
|
|
||||||
|
|
||||||
|
def reverse_set_default_locale(apps, schema_editor):
|
||||||
|
GoogleUser = apps.get_model("database", "GoogleUser")
|
||||||
|
for user in GoogleUser.objects.all():
|
||||||
|
if not user.locale:
|
||||||
|
user.locale = "en"
|
||||||
|
user.save()
|
||||||
|
|
||||||
|
|
||||||
class Migration(migrations.Migration):
|
class Migration(migrations.Migration):
|
||||||
dependencies = [
|
dependencies = [
|
||||||
("database", "0030_conversation_slug_and_title"),
|
("database", "0030_conversation_slug_and_title"),
|
||||||
|
|
@ -14,4 +26,5 @@ class Migration(migrations.Migration):
|
||||||
name="locale",
|
name="locale",
|
||||||
field=models.CharField(blank=True, default=None, max_length=200, null=True),
|
field=models.CharField(blank=True, default=None, max_length=200, null=True),
|
||||||
),
|
),
|
||||||
|
migrations.RunPython(set_default_locale, reverse_set_default_locale),
|
||||||
]
|
]
|
||||||
|
|
|
||||||
14
src/khoj/database/migrations/0032_merge_20240322_0427.py
Normal file
14
src/khoj/database/migrations/0032_merge_20240322_0427.py
Normal file
|
|
@ -0,0 +1,14 @@
|
||||||
|
# Generated by Django 4.2.10 on 2024-03-22 04:27
|
||||||
|
|
||||||
|
from typing import List
|
||||||
|
|
||||||
|
from django.db import migrations
|
||||||
|
|
||||||
|
|
||||||
|
class Migration(migrations.Migration):
|
||||||
|
dependencies = [
|
||||||
|
("database", "0031_agent_conversation_agent"),
|
||||||
|
("database", "0031_alter_googleuser_locale"),
|
||||||
|
]
|
||||||
|
|
||||||
|
operations: List[str] = []
|
||||||
|
|
@ -0,0 +1,17 @@
|
||||||
|
# Generated by Django 4.2.10 on 2024-03-23 16:01
|
||||||
|
|
||||||
|
from django.db import migrations
|
||||||
|
|
||||||
|
|
||||||
|
class Migration(migrations.Migration):
|
||||||
|
dependencies = [
|
||||||
|
("database", "0032_merge_20240322_0427"),
|
||||||
|
]
|
||||||
|
|
||||||
|
operations = [
|
||||||
|
migrations.RenameField(
|
||||||
|
model_name="agent",
|
||||||
|
old_name="tuning",
|
||||||
|
new_name="personality",
|
||||||
|
),
|
||||||
|
]
|
||||||
|
|
@ -1,7 +1,11 @@
|
||||||
import uuid
|
import uuid
|
||||||
|
from random import choice
|
||||||
|
|
||||||
from django.contrib.auth.models import AbstractUser
|
from django.contrib.auth.models import AbstractUser
|
||||||
|
from django.core.exceptions import ValidationError
|
||||||
from django.db import models
|
from django.db import models
|
||||||
|
from django.db.models.signals import pre_save
|
||||||
|
from django.dispatch import receiver
|
||||||
from pgvector.django import VectorField
|
from pgvector.django import VectorField
|
||||||
from phonenumber_field.modelfields import PhoneNumberField
|
from phonenumber_field.modelfields import PhoneNumberField
|
||||||
|
|
||||||
|
|
@ -69,6 +73,52 @@ class Subscription(BaseModel):
|
||||||
renewal_date = models.DateTimeField(null=True, default=None, blank=True)
|
renewal_date = models.DateTimeField(null=True, default=None, blank=True)
|
||||||
|
|
||||||
|
|
||||||
|
class ChatModelOptions(BaseModel):
|
||||||
|
class ModelType(models.TextChoices):
|
||||||
|
OPENAI = "openai"
|
||||||
|
OFFLINE = "offline"
|
||||||
|
|
||||||
|
max_prompt_size = models.IntegerField(default=None, null=True, blank=True)
|
||||||
|
tokenizer = models.CharField(max_length=200, default=None, null=True, blank=True)
|
||||||
|
chat_model = models.CharField(max_length=200, default="NousResearch/Hermes-2-Pro-Mistral-7B-GGUF")
|
||||||
|
model_type = models.CharField(max_length=200, choices=ModelType.choices, default=ModelType.OFFLINE)
|
||||||
|
|
||||||
|
|
||||||
|
class Agent(BaseModel):
|
||||||
|
creator = models.ForeignKey(
|
||||||
|
KhojUser, on_delete=models.CASCADE, default=None, null=True, blank=True
|
||||||
|
) # Creator will only be null when the agents are managed by admin
|
||||||
|
name = models.CharField(max_length=200)
|
||||||
|
personality = models.TextField()
|
||||||
|
avatar = models.URLField(max_length=400, default=None, null=True, blank=True)
|
||||||
|
tools = models.JSONField(default=list) # List of tools the agent has access to, like online search or notes search
|
||||||
|
public = models.BooleanField(default=False)
|
||||||
|
managed_by_admin = models.BooleanField(default=False)
|
||||||
|
chat_model = models.ForeignKey(ChatModelOptions, on_delete=models.CASCADE)
|
||||||
|
slug = models.CharField(max_length=200)
|
||||||
|
|
||||||
|
|
||||||
|
@receiver(pre_save, sender=Agent)
|
||||||
|
def verify_agent(sender, instance, **kwargs):
|
||||||
|
# check if this is a new instance
|
||||||
|
if instance._state.adding:
|
||||||
|
if Agent.objects.filter(name=instance.name, public=True).exists():
|
||||||
|
raise ValidationError(f"A public Agent with the name {instance.name} already exists.")
|
||||||
|
if Agent.objects.filter(name=instance.name, creator=instance.creator).exists():
|
||||||
|
raise ValidationError(f"A private Agent with the name {instance.name} already exists.")
|
||||||
|
|
||||||
|
slug = instance.name.lower().replace(" ", "-")
|
||||||
|
observed_random_numbers = set()
|
||||||
|
while Agent.objects.filter(slug=slug).exists():
|
||||||
|
try:
|
||||||
|
random_number = choice([i for i in range(0, 1000) if i not in observed_random_numbers])
|
||||||
|
except IndexError:
|
||||||
|
raise ValidationError("Unable to generate a unique slug for the Agent. Please try again later.")
|
||||||
|
observed_random_numbers.add(random_number)
|
||||||
|
slug = f"{slug}-{random_number}"
|
||||||
|
instance.slug = slug
|
||||||
|
|
||||||
|
|
||||||
class NotionConfig(BaseModel):
|
class NotionConfig(BaseModel):
|
||||||
token = models.CharField(max_length=200)
|
token = models.CharField(max_length=200)
|
||||||
user = models.ForeignKey(KhojUser, on_delete=models.CASCADE)
|
user = models.ForeignKey(KhojUser, on_delete=models.CASCADE)
|
||||||
|
|
@ -153,17 +203,6 @@ class SpeechToTextModelOptions(BaseModel):
|
||||||
model_type = models.CharField(max_length=200, choices=ModelType.choices, default=ModelType.OFFLINE)
|
model_type = models.CharField(max_length=200, choices=ModelType.choices, default=ModelType.OFFLINE)
|
||||||
|
|
||||||
|
|
||||||
class ChatModelOptions(BaseModel):
|
|
||||||
class ModelType(models.TextChoices):
|
|
||||||
OPENAI = "openai"
|
|
||||||
OFFLINE = "offline"
|
|
||||||
|
|
||||||
max_prompt_size = models.IntegerField(default=None, null=True, blank=True)
|
|
||||||
tokenizer = models.CharField(max_length=200, default=None, null=True, blank=True)
|
|
||||||
chat_model = models.CharField(max_length=200, default="NousResearch/Hermes-2-Pro-Mistral-7B-GGUF")
|
|
||||||
model_type = models.CharField(max_length=200, choices=ModelType.choices, default=ModelType.OFFLINE)
|
|
||||||
|
|
||||||
|
|
||||||
class UserConversationConfig(BaseModel):
|
class UserConversationConfig(BaseModel):
|
||||||
user = models.OneToOneField(KhojUser, on_delete=models.CASCADE)
|
user = models.OneToOneField(KhojUser, on_delete=models.CASCADE)
|
||||||
setting = models.ForeignKey(ChatModelOptions, on_delete=models.CASCADE, default=None, null=True, blank=True)
|
setting = models.ForeignKey(ChatModelOptions, on_delete=models.CASCADE, default=None, null=True, blank=True)
|
||||||
|
|
@ -180,6 +219,7 @@ class Conversation(BaseModel):
|
||||||
client = models.ForeignKey(ClientApplication, on_delete=models.CASCADE, default=None, null=True, blank=True)
|
client = models.ForeignKey(ClientApplication, on_delete=models.CASCADE, default=None, null=True, blank=True)
|
||||||
slug = models.CharField(max_length=200, default=None, null=True, blank=True)
|
slug = models.CharField(max_length=200, default=None, null=True, blank=True)
|
||||||
title = models.CharField(max_length=200, default=None, null=True, blank=True)
|
title = models.CharField(max_length=200, default=None, null=True, blank=True)
|
||||||
|
agent = models.ForeignKey(Agent, on_delete=models.SET_NULL, default=None, null=True, blank=True)
|
||||||
|
|
||||||
|
|
||||||
class ReflectiveQuestion(BaseModel):
|
class ReflectiveQuestion(BaseModel):
|
||||||
|
|
|
||||||
|
|
@ -2,14 +2,19 @@
|
||||||
<html>
|
<html>
|
||||||
<head>
|
<head>
|
||||||
<title>Khoj: An AI Personal Assistant for your digital brain</title>
|
<title>Khoj: An AI Personal Assistant for your digital brain</title>
|
||||||
<link rel=”stylesheet” href=”static/styles.css”>
|
<link rel="icon" type="image/png" sizes="128x128" href="/static/assets/icons/favicon-128x128.png?v={{ khoj_version }}">
|
||||||
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@picocss/pico@1/css/pico.min.css">
|
<link rel="manifest" href="/static/khoj.webmanifest?v={{ khoj_version }}">
|
||||||
|
<link rel="stylesheet" href="/static/assets/khoj.css?v={{ khoj_version }}">
|
||||||
</head>
|
</head>
|
||||||
<body class="not-found">
|
<body class="not-found">
|
||||||
|
<!--Add Header Logo and Nav Pane-->
|
||||||
|
{% import 'utils.html' as utils %}
|
||||||
|
{{ utils.heading_pane(user_photo, username, is_active, has_documents) }}
|
||||||
|
|
||||||
<header class=”header”>
|
<header class=”header”>
|
||||||
<h1>Oops, this is awkward. That page couldn't be found.</h1>
|
<h1>Oops, this is awkward. Looks like there's nothing here.</h1>
|
||||||
</header>
|
</header>
|
||||||
<a href="/config">Go Home</a>
|
<a class="redirect-link" href="/">Go Home</a>
|
||||||
|
|
||||||
<footer class=”footer”>
|
<footer class=”footer”>
|
||||||
</footer>
|
</footer>
|
||||||
|
|
@ -18,5 +23,34 @@
|
||||||
body.not-found {
|
body.not-found {
|
||||||
padding: 0 10%
|
padding: 0 10%
|
||||||
}
|
}
|
||||||
|
|
||||||
|
body {
|
||||||
|
background-color: var(--background-color);
|
||||||
|
color: var(--main-text-color);
|
||||||
|
text-align: center;
|
||||||
|
font-family: var(--font-family);
|
||||||
|
font-size: medium;
|
||||||
|
font-weight: 300;
|
||||||
|
line-height: 1.5em;
|
||||||
|
height: 100vh;
|
||||||
|
margin: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
body a.redirect-link {
|
||||||
|
font-size: 18px;
|
||||||
|
font-weight: bold;
|
||||||
|
background-color: var(--primary);
|
||||||
|
text-decoration: none;
|
||||||
|
border: 1px solid var(--main-text-color);
|
||||||
|
color: var(--main-text-color);
|
||||||
|
border-radius: 8px;
|
||||||
|
padding: 4px;
|
||||||
|
}
|
||||||
|
|
||||||
|
body a.redirect-link:hover {
|
||||||
|
background-color: var(--main-text-color);
|
||||||
|
color: var(--primary);
|
||||||
|
}
|
||||||
|
|
||||||
</style>
|
</style>
|
||||||
</html>
|
</html>
|
||||||
|
|
|
||||||
286
src/khoj/interface/web/agent.html
Normal file
286
src/khoj/interface/web/agent.html
Normal file
|
|
@ -0,0 +1,286 @@
|
||||||
|
<html>
|
||||||
|
<head>
|
||||||
|
<meta charset="utf-8">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1.0 maximum-scale=1.0">
|
||||||
|
<title>Khoj - Agents</title>
|
||||||
|
|
||||||
|
<link rel="icon" type="image/png" sizes="128x128" href="/static/assets/icons/favicon-128x128.png?v={{ khoj_version }}">
|
||||||
|
<link rel="manifest" href="/static/khoj.webmanifest?v={{ khoj_version }}">
|
||||||
|
<link rel="stylesheet" href="/static/assets/khoj.css?v={{ khoj_version }}">
|
||||||
|
</head>
|
||||||
|
<script type="text/javascript" src="/static/assets/utils.js?v={{ khoj_version }}"></script>
|
||||||
|
<body>
|
||||||
|
<!--Add Header Logo and Nav Pane-->
|
||||||
|
{% import 'utils.html' as utils %}
|
||||||
|
{{ utils.heading_pane(user_photo, username, is_active, has_documents) }}
|
||||||
|
<div id="agent-metadata-wrapper">
|
||||||
|
<div id="agent-metadata">
|
||||||
|
<div id="agent-avatar-wrapper">
|
||||||
|
<div id="agent-settings-header">Agent Settings</div>
|
||||||
|
</div>
|
||||||
|
<div class="divider"></div>
|
||||||
|
<div id="agent-data-wrapper">
|
||||||
|
<div id="agent-avatar-wrapper">
|
||||||
|
<img id="agent-avatar" src="{{ agent.avatar }}" alt="Agent Avatar">
|
||||||
|
<input type="text" id="agent-name-input" value="{{ agent.name }}" {% if agent.creator_not_self %} disabled {% endif %}>
|
||||||
|
</div>
|
||||||
|
<div id="agent-instructions">Personality</div>
|
||||||
|
<div id="agent-tuning">
|
||||||
|
<p>{{ agent.personality }}</p>
|
||||||
|
</div>
|
||||||
|
<div class="divider"></div>
|
||||||
|
<div id="agent-public">
|
||||||
|
<p>Public</p>
|
||||||
|
<label class="switch">
|
||||||
|
<input type="checkbox" {% if agent.public %} checked {% endif %} {% if agent.creator_not_self %} disabled {% endif %}>
|
||||||
|
<span class="slider round"></span>
|
||||||
|
</label>
|
||||||
|
</div>
|
||||||
|
<p id="agent-creator" style="display: none;">Creator: {{ agent.creator }}</p>
|
||||||
|
<p id="agent-managed-by-admin" style="display: none;">ⓘ This agent is managed by the administrator</p>
|
||||||
|
<button onclick="openChat('{{ agent.slug }}')">Chat</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div id="footer">
|
||||||
|
<a href="/agents">All Agents</a>
|
||||||
|
</div>
|
||||||
|
</body>
|
||||||
|
<style>
|
||||||
|
body {
|
||||||
|
background-color: var(--background-color);
|
||||||
|
display: grid;
|
||||||
|
color: var(--main-text-color);
|
||||||
|
text-align: center;
|
||||||
|
font-family: var(--font-family);
|
||||||
|
font-size: medium;
|
||||||
|
font-weight: 300;
|
||||||
|
line-height: 1.5em;
|
||||||
|
height: 100vh;
|
||||||
|
margin: 0;
|
||||||
|
grid-template-rows: auto 1fr auto;
|
||||||
|
}
|
||||||
|
|
||||||
|
div#agent-settings-header {
|
||||||
|
font-size: 24px;
|
||||||
|
font-weight: bold;
|
||||||
|
margin-top: auto;
|
||||||
|
margin-bottom: auto;
|
||||||
|
}
|
||||||
|
|
||||||
|
div.divider {
|
||||||
|
margin-top: 10px;
|
||||||
|
margin-bottom: 10px;
|
||||||
|
border-bottom: 2px solid var(--main-text-color);
|
||||||
|
}
|
||||||
|
|
||||||
|
div#footer {
|
||||||
|
width: auto;
|
||||||
|
padding: 10px;
|
||||||
|
background-color: var(--background-color);
|
||||||
|
border-top: 1px solid var(--main-text-color);
|
||||||
|
text-align: left;
|
||||||
|
margin-top: 12px;
|
||||||
|
margin-bottom: 12px;
|
||||||
|
}
|
||||||
|
|
||||||
|
div#footer a {
|
||||||
|
font-size: 18px;
|
||||||
|
font-weight: bold;
|
||||||
|
color: var(--primary-color);
|
||||||
|
}
|
||||||
|
|
||||||
|
div#agent-data-wrapper button {
|
||||||
|
font-size: 24px;
|
||||||
|
font-weight: bold;
|
||||||
|
padding: 10px;
|
||||||
|
border: none;
|
||||||
|
border-radius: 8px;
|
||||||
|
background-color: var(--primary);
|
||||||
|
font: inherit;
|
||||||
|
color: var(--main-text-color);
|
||||||
|
cursor: pointer;
|
||||||
|
transition: background-color 0.3s;
|
||||||
|
}
|
||||||
|
|
||||||
|
div#agent-data-wrapper button:hover {
|
||||||
|
background-color: var(--primary-hover);
|
||||||
|
box-shadow: 0 0 10px var(--primary-hover);
|
||||||
|
}
|
||||||
|
|
||||||
|
input#agent-name-input {
|
||||||
|
font-size: 24px;
|
||||||
|
font-weight: bold;
|
||||||
|
text-align: left;
|
||||||
|
background-color: #EEEEEE;
|
||||||
|
color: var(--main-text-color);
|
||||||
|
border-radius: 8px;
|
||||||
|
padding: 8px;
|
||||||
|
border: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
div#agent-instructions {
|
||||||
|
font-size: 24px;
|
||||||
|
font-weight: bold;
|
||||||
|
}
|
||||||
|
|
||||||
|
#agent-metadata {
|
||||||
|
padding: 10px;
|
||||||
|
background-color: #f8f9fa;
|
||||||
|
border-radius: 5px;
|
||||||
|
box-shadow: 0 1px 3px rgba(0, 0, 0, 0.2);
|
||||||
|
text-align: left;
|
||||||
|
padding: 20px;
|
||||||
|
}
|
||||||
|
|
||||||
|
#agent-avatar-wrapper {
|
||||||
|
margin-right: 10px;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: row;
|
||||||
|
}
|
||||||
|
|
||||||
|
#agent-avatar {
|
||||||
|
width: 50px;
|
||||||
|
height: 50px;
|
||||||
|
border-radius: 50%;
|
||||||
|
object-fit: cover;
|
||||||
|
margin-right: 10px;
|
||||||
|
}
|
||||||
|
|
||||||
|
#agent-name {
|
||||||
|
font-size: 18px;
|
||||||
|
font-weight: bold;
|
||||||
|
color: #333;
|
||||||
|
}
|
||||||
|
|
||||||
|
#agent-tuning, #agent-public, #agent-creator, #agent-managed-by-admin {
|
||||||
|
font-size: 14px;
|
||||||
|
color: #666;
|
||||||
|
}
|
||||||
|
|
||||||
|
#agent-tuning p {
|
||||||
|
white-space: pre-line;
|
||||||
|
}
|
||||||
|
|
||||||
|
#agent-metadata p {
|
||||||
|
margin: 0;
|
||||||
|
padding: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
#agent-public {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: auto 1fr;
|
||||||
|
grid-gap: 12px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.switch {
|
||||||
|
position: relative;
|
||||||
|
display: inline-block;
|
||||||
|
width: 50px;
|
||||||
|
height: 24px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.switch input {
|
||||||
|
opacity: 0;
|
||||||
|
width: 0;
|
||||||
|
height: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.slider {
|
||||||
|
position: absolute;
|
||||||
|
cursor: pointer;
|
||||||
|
top: 0;
|
||||||
|
left: 0;
|
||||||
|
right: 0;
|
||||||
|
bottom: 0;
|
||||||
|
background-color: #ccc;
|
||||||
|
-webkit-transition: .4s;
|
||||||
|
transition: .4s;
|
||||||
|
}
|
||||||
|
|
||||||
|
.slider:before {
|
||||||
|
position: absolute;
|
||||||
|
content: "";
|
||||||
|
height: 16px;
|
||||||
|
width: 16px;
|
||||||
|
left: 4px;
|
||||||
|
bottom: 4px;
|
||||||
|
background-color: white;
|
||||||
|
-webkit-transition: .4s;
|
||||||
|
transition: .4s;
|
||||||
|
}
|
||||||
|
|
||||||
|
input:checked + .slider {
|
||||||
|
background-color: var(--primary-hover);
|
||||||
|
}
|
||||||
|
|
||||||
|
input:focus + .slider {
|
||||||
|
box-shadow: 0 0 1px var(--primary-hover);
|
||||||
|
}
|
||||||
|
|
||||||
|
input:checked + .slider:before {
|
||||||
|
-webkit-transform: translateX(26px);
|
||||||
|
-ms-transform: translateX(26px);
|
||||||
|
transform: translateX(26px);
|
||||||
|
}
|
||||||
|
|
||||||
|
div#agent-data-wrapper {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: 1fr;
|
||||||
|
grid-gap: 10px;
|
||||||
|
text-align: left;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Rounded sliders */
|
||||||
|
.slider.round {
|
||||||
|
border-radius: 34px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.slider.round:before {
|
||||||
|
border-radius: 50%;
|
||||||
|
}
|
||||||
|
|
||||||
|
@media only screen and (min-width: 700px) {
|
||||||
|
body {
|
||||||
|
grid-template-columns: auto min(70vw, 100%) auto;
|
||||||
|
}
|
||||||
|
body > * {
|
||||||
|
grid-column: 2;
|
||||||
|
}
|
||||||
|
#agent-metadata-wrapper {
|
||||||
|
display: block;
|
||||||
|
width: min(30vw, 100%);
|
||||||
|
margin-left: auto;
|
||||||
|
margin-right: auto;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
</style>
|
||||||
|
<script>
|
||||||
|
async function openChat(agentSlug) {
|
||||||
|
let response = await fetch(`/api/chat/sessions?agent_slug=${agentSlug}`, { method: "POST" });
|
||||||
|
let data = await response.json();
|
||||||
|
if (response.status == 200) {
|
||||||
|
window.location.href = "/";
|
||||||
|
} else {
|
||||||
|
alert("Failed to start chat session");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Show the agent-managed-by-admin paragraph if the agent is managed by the admin
|
||||||
|
// compare agent.managed_by_admin as a lowercase string to "true"
|
||||||
|
let isManagedByAdmin = "{{ agent.managed_by_admin }}".toLowerCase() === "true";
|
||||||
|
if (isManagedByAdmin) {
|
||||||
|
document.getElementById("agent-managed-by-admin").style.display = "block";
|
||||||
|
} else {
|
||||||
|
document.getElementById("agent-creator").style.display = "block";
|
||||||
|
}
|
||||||
|
|
||||||
|
// Resize the input field based on the length of the value
|
||||||
|
let input = document.getElementById("agent-name-input");
|
||||||
|
input.addEventListener("input", resizeInput);
|
||||||
|
resizeInput.call(input);
|
||||||
|
function resizeInput() {
|
||||||
|
this.style.width = this.value.length + 1 + "ch";
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
</html>
|
||||||
203
src/khoj/interface/web/agents.html
Normal file
203
src/khoj/interface/web/agents.html
Normal file
|
|
@ -0,0 +1,203 @@
|
||||||
|
<html>
|
||||||
|
<head>
|
||||||
|
<meta charset="utf-8">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1.0 maximum-scale=1.0">
|
||||||
|
<title>Khoj - Agents</title>
|
||||||
|
|
||||||
|
<link rel="icon" type="image/png" sizes="128x128" href="/static/assets/icons/favicon-128x128.png?v={{ khoj_version }}">
|
||||||
|
<link rel="manifest" href="/static/khoj.webmanifest?v={{ khoj_version }}">
|
||||||
|
<link rel="stylesheet" href="/static/assets/khoj.css?v={{ khoj_version }}">
|
||||||
|
</head>
|
||||||
|
<script type="text/javascript" src="/static/assets/utils.js?v={{ khoj_version }}"></script>
|
||||||
|
<body>
|
||||||
|
<!--Add Header Logo and Nav Pane-->
|
||||||
|
{% import 'utils.html' as utils %}
|
||||||
|
{{ utils.heading_pane(user_photo, username, is_active, has_documents) }}
|
||||||
|
<!-- {{ agents }} -->
|
||||||
|
|
||||||
|
<div id="agents-list">
|
||||||
|
<div id="agents">
|
||||||
|
<div id="agents-header">
|
||||||
|
<h1 id="agents-list-title">Agents</h1>
|
||||||
|
<!-- <div id="create-agent">
|
||||||
|
<a href="/agents/create"><svg class="new-convo-button" viewBox="0 0 35 35" fill="#000000" viewBox="0 0 32 32" version="1.1" xmlns="http://www.w3.org/2000/svg">
|
||||||
|
<path d="M16 0c-8.836 0-16 7.163-16 16s7.163 16 16 16c8.837 0 16-7.163 16-16s-7.163-16-16-16zM16 30.032c-7.72 0-14-6.312-14-14.032s6.28-14 14-14 14 6.28 14 14-6.28 14.032-14 14.032zM23 15h-6v-6c0-0.552-0.448-1-1-1s-1 0.448-1 1v6h-6c-0.552 0-1 0.448-1 1s0.448 1 1 1h6v6c0 0.552 0.448 1 1 1s1-0.448 1-1v-6h6c0.552 0 1-0.448 1-1s-0.448-1-1-1z"></path>
|
||||||
|
</svg></a>
|
||||||
|
</div> -->
|
||||||
|
</div>
|
||||||
|
{% for agent in agents %}
|
||||||
|
<div class="agent">
|
||||||
|
<a href="/agent/{{ agent.slug }}">
|
||||||
|
<div class="agent-avatar">
|
||||||
|
<img src="{{ agent.avatar }}" alt="{{ agent.name }}">
|
||||||
|
</div>
|
||||||
|
</a>
|
||||||
|
<div class="agent-info">
|
||||||
|
<a href="/agent/{{ agent.slug }}">
|
||||||
|
<h2>{{ agent.name }}</h2>
|
||||||
|
</a>
|
||||||
|
<p>{{ agent.personality }}</p>
|
||||||
|
</div>
|
||||||
|
<div class="agent-info">
|
||||||
|
<button onclick="openChat('{{ agent.slug }}')">Talk</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
{% endfor %}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div id="footer">
|
||||||
|
<a href="/">Back to Chat</a>
|
||||||
|
</div>
|
||||||
|
</body>
|
||||||
|
<style>
|
||||||
|
body {
|
||||||
|
background-color: var(--background-color);
|
||||||
|
display: grid;
|
||||||
|
color: var(--main-text-color);
|
||||||
|
text-align: center;
|
||||||
|
font-family: var(--font-family);
|
||||||
|
font-size: medium;
|
||||||
|
font-weight: 300;
|
||||||
|
line-height: 1.5em;
|
||||||
|
height: 100vh;
|
||||||
|
margin: 0;
|
||||||
|
grid-template-rows: auto 1fr auto;
|
||||||
|
}
|
||||||
|
|
||||||
|
h1#agents-list-title {
|
||||||
|
margin: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.agent-info p {
|
||||||
|
height: 50px;
|
||||||
|
overflow: auto;
|
||||||
|
margin: 0px;
|
||||||
|
}
|
||||||
|
|
||||||
|
div.agent-info {
|
||||||
|
font-size: medium;
|
||||||
|
}
|
||||||
|
|
||||||
|
div.agent-info a,
|
||||||
|
div.agent-info h2 {
|
||||||
|
margin: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
div.agent img {
|
||||||
|
width: 78px;
|
||||||
|
height: 78px;
|
||||||
|
border-radius: 50%;
|
||||||
|
object-fit: cover;
|
||||||
|
}
|
||||||
|
|
||||||
|
div.agent a {
|
||||||
|
text-decoration: none;
|
||||||
|
color: var(--main-text-color);
|
||||||
|
}
|
||||||
|
|
||||||
|
div#agents-header {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: auto;
|
||||||
|
}
|
||||||
|
|
||||||
|
div#agents-header a,
|
||||||
|
div.agent-info button {
|
||||||
|
font-size: 24px;
|
||||||
|
font-weight: bold;
|
||||||
|
padding: 10px;
|
||||||
|
border: none;
|
||||||
|
border-radius: 8px;
|
||||||
|
background-color: var(--primary);
|
||||||
|
font: inherit;
|
||||||
|
color: var(--main-text-color);
|
||||||
|
cursor: pointer;
|
||||||
|
transition: background-color 0.3s;
|
||||||
|
}
|
||||||
|
|
||||||
|
div#agents-header a:hover,
|
||||||
|
div.agent-info button:hover {
|
||||||
|
background-color: var(--primary-hover);
|
||||||
|
box-shadow: 0 0 10px var(--primary-hover);
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
div#footer {
|
||||||
|
width: auto;
|
||||||
|
padding: 10px;
|
||||||
|
background-color: var(--background-color);
|
||||||
|
border-top: 1px solid var(--main-text-color);
|
||||||
|
text-align: left;
|
||||||
|
margin-top: 12px;
|
||||||
|
margin-bottom: 12px;
|
||||||
|
}
|
||||||
|
|
||||||
|
div#footer a {
|
||||||
|
font-size: 18px;
|
||||||
|
font-weight: bold;
|
||||||
|
color: var(--primary-color);
|
||||||
|
}
|
||||||
|
|
||||||
|
div.agent {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: auto 1fr auto;
|
||||||
|
gap: 20px;
|
||||||
|
align-items: center;
|
||||||
|
padding: 20px;
|
||||||
|
background-color: var(--frosted-background-color);
|
||||||
|
border-top: 1px solid var(--main-text-color);
|
||||||
|
}
|
||||||
|
|
||||||
|
div.agent-info {
|
||||||
|
text-align: left;
|
||||||
|
}
|
||||||
|
|
||||||
|
div#agents {
|
||||||
|
display: grid;
|
||||||
|
grid-auto-flow: row;
|
||||||
|
gap: 20px;
|
||||||
|
padding: 20px;
|
||||||
|
background-color: var(--frosted-background-color);
|
||||||
|
box-shadow: 0px 8px 16px 0px rgba(0,0,0,0.2);
|
||||||
|
border-radius: 8px;
|
||||||
|
width: 75%;
|
||||||
|
margin-right: auto;
|
||||||
|
margin-left: auto;
|
||||||
|
}
|
||||||
|
|
||||||
|
svg.new-convo-button {
|
||||||
|
width: 20px;
|
||||||
|
margin-left: 5px;
|
||||||
|
}
|
||||||
|
|
||||||
|
@media only screen and (min-width: 700px) {
|
||||||
|
body {
|
||||||
|
grid-template-columns: auto min(70vw, 100%) auto;
|
||||||
|
}
|
||||||
|
body > * {
|
||||||
|
grid-column: 2;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@media only screen and (max-width: 700px) {
|
||||||
|
div#agents {
|
||||||
|
width: 90%;
|
||||||
|
margin-right: auto;
|
||||||
|
margin-left: auto;
|
||||||
|
}
|
||||||
|
|
||||||
|
}
|
||||||
|
</style>
|
||||||
|
<script>
|
||||||
|
async function openChat(agentSlug) {
|
||||||
|
let response = await fetch(`/api/chat/sessions?agent_slug=${agentSlug}`, { method: "POST" });
|
||||||
|
let data = await response.json();
|
||||||
|
if (response.status == 200) {
|
||||||
|
window.location.href = "/";
|
||||||
|
} else if(response.status == 403 || response.status == 401) {
|
||||||
|
window.location.href = "/login?next=/agent/" + agentId;
|
||||||
|
} else {
|
||||||
|
alert("Failed to start chat session");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
</html>
|
||||||
|
|
@ -130,7 +130,7 @@ img.khoj-logo {
|
||||||
background-color: var(--background-color);
|
background-color: var(--background-color);
|
||||||
min-width: 160px;
|
min-width: 160px;
|
||||||
box-shadow: 0px 8px 16px 0px rgba(0,0,0,0.2);
|
box-shadow: 0px 8px 16px 0px rgba(0,0,0,0.2);
|
||||||
right: 15vw;
|
right: 5vw;
|
||||||
top: 64px;
|
top: 64px;
|
||||||
z-index: 1;
|
z-index: 1;
|
||||||
opacity: 0;
|
opacity: 0;
|
||||||
|
|
|
||||||
|
|
@ -162,7 +162,7 @@
|
||||||
height: 40px;
|
height: 40px;
|
||||||
}
|
}
|
||||||
.card-title {
|
.card-title {
|
||||||
font-size: 20px;
|
font-size: medium;
|
||||||
font-weight: normal;
|
font-weight: normal;
|
||||||
margin: 0;
|
margin: 0;
|
||||||
padding: 0;
|
padding: 0;
|
||||||
|
|
|
||||||
|
|
@ -13,15 +13,17 @@
|
||||||
<script type="text/javascript" src="/static/assets/markdown-it.min.js?v={{ khoj_version }}"></script>
|
<script type="text/javascript" src="/static/assets/markdown-it.min.js?v={{ khoj_version }}"></script>
|
||||||
<script>
|
<script>
|
||||||
let welcome_message = `
|
let welcome_message = `
|
||||||
Hi, I am Khoj, your open, personal AI 👋🏽. I can help:
|
Hi, I am Khoj, your open, personal AI 👋🏽. I can:
|
||||||
- 🧠 Answer general knowledge questions
|
- 🧠 Answer general knowledge questions
|
||||||
- 💡 Be a sounding board for your ideas
|
- 💡 Be a sounding board for your ideas
|
||||||
- 📜 Chat with your notes & documents
|
- 📜 Chat with your notes & documents
|
||||||
- 🌄 Generate images based on your messages
|
- 🌄 Generate images based on your messages
|
||||||
- 🔎 Search the web for answers to your questions
|
- 🔎 Search the web for answers to your questions
|
||||||
- 🎙️ Listen to your audio messages (use the mic by the input box to speak your message)
|
- 🎙️ Listen to your audio messages (use the mic by the input box to speak your message)
|
||||||
|
- 📚 Understand files you drag & drop here
|
||||||
|
- 👩🏾🚀 Be tuned to your conversation needs via [agents](./agents)
|
||||||
|
|
||||||
Get the Khoj [Desktop](https://khoj.dev/downloads), [Obsidian](https://docs.khoj.dev/clients/obsidian#setup), [Emacs](https://docs.khoj.dev/clients/emacs#setup) apps to search, chat with your 🖥️ computer docs.
|
Get the Khoj [Desktop](https://khoj.dev/downloads), [Obsidian](https://docs.khoj.dev/clients/obsidian#setup), [Emacs](https://docs.khoj.dev/clients/emacs#setup) apps to search, chat with your 🖥️ computer docs. You can manage all the files you've shared with me at any time by going to [your settings](/config/content-source/computer/).
|
||||||
|
|
||||||
To get started, just start typing below. You can also type / to see a list of commands.
|
To get started, just start typing below. You can also type / to see a list of commands.
|
||||||
`.trim()
|
`.trim()
|
||||||
|
|
@ -116,7 +118,6 @@ To get started, just start typing below. You can also type / to see a list of co
|
||||||
linkElement.setAttribute('href', link);
|
linkElement.setAttribute('href', link);
|
||||||
linkElement.setAttribute('target', '_blank');
|
linkElement.setAttribute('target', '_blank');
|
||||||
linkElement.setAttribute('rel', 'noopener noreferrer');
|
linkElement.setAttribute('rel', 'noopener noreferrer');
|
||||||
linkElement.classList.add("inline-chat-link");
|
|
||||||
linkElement.classList.add("reference-link");
|
linkElement.classList.add("reference-link");
|
||||||
linkElement.setAttribute('title', title);
|
linkElement.setAttribute('title', title);
|
||||||
linkElement.textContent = title;
|
linkElement.textContent = title;
|
||||||
|
|
@ -842,6 +843,33 @@ To get started, just start typing below. You can also type / to see a list of co
|
||||||
chatBody.dataset.conversationId = response.conversation_id;
|
chatBody.dataset.conversationId = response.conversation_id;
|
||||||
chatBody.dataset.conversationTitle = response.slug || `New conversation 🌱`;
|
chatBody.dataset.conversationTitle = response.slug || `New conversation 🌱`;
|
||||||
|
|
||||||
|
let agentMetadata = response.agent;
|
||||||
|
if (agentMetadata) {
|
||||||
|
let agentName = agentMetadata.name;
|
||||||
|
let agentAvatar = agentMetadata.avatar;
|
||||||
|
let agentOwnedByUser = agentMetadata.isCreator;
|
||||||
|
|
||||||
|
let agentAvatarElement = document.getElementById("agent-avatar");
|
||||||
|
let agentNameElement = document.getElementById("agent-name");
|
||||||
|
|
||||||
|
let agentLinkElement = document.getElementById("agent-link");
|
||||||
|
|
||||||
|
agentAvatarElement.src = agentAvatar;
|
||||||
|
agentNameElement.textContent = agentName;
|
||||||
|
agentLinkElement.setAttribute("href", `/agent/${agentMetadata.slug}`);
|
||||||
|
|
||||||
|
if (agentOwnedByUser) {
|
||||||
|
let agentOwnedByUserElement = document.getElementById("agent-owned-by-user");
|
||||||
|
agentOwnedByUserElement.style.display = "block";
|
||||||
|
}
|
||||||
|
|
||||||
|
let agentMetadataElement = document.getElementById("agent-metadata");
|
||||||
|
agentMetadataElement.style.display = "block";
|
||||||
|
} else {
|
||||||
|
let agentMetadataElement = document.getElementById("agent-metadata");
|
||||||
|
agentMetadataElement.style.display = "none";
|
||||||
|
}
|
||||||
|
|
||||||
let chatBodyWrapper = document.getElementById("chat-body-wrapper");
|
let chatBodyWrapper = document.getElementById("chat-body-wrapper");
|
||||||
const fullChatLog = response.chat || [];
|
const fullChatLog = response.chat || [];
|
||||||
|
|
||||||
|
|
@ -934,12 +962,100 @@ To get started, just start typing below. You can also type / to see a list of co
|
||||||
}
|
}
|
||||||
|
|
||||||
function createNewConversation() {
|
function createNewConversation() {
|
||||||
|
// Create a modal that appears in the middle of the entire screen. It should have a form to create a new conversation.
|
||||||
|
let modal = document.createElement('div');
|
||||||
|
modal.classList.add("modal");
|
||||||
|
modal.id = "new-conversation-modal";
|
||||||
|
let modalContent = document.createElement('div');
|
||||||
|
modalContent.classList.add("modal-content");
|
||||||
|
let modalHeader = document.createElement('div');
|
||||||
|
modalHeader.classList.add("modal-header");
|
||||||
|
let modalTitle = document.createElement('h2');
|
||||||
|
modalTitle.textContent = "New Conversation";
|
||||||
|
let modalCloseButton = document.createElement('button');
|
||||||
|
modalCloseButton.classList.add("modal-close-button");
|
||||||
|
modalCloseButton.innerHTML = "×";
|
||||||
|
modalCloseButton.addEventListener('click', function() {
|
||||||
|
modal.remove();
|
||||||
|
});
|
||||||
|
modalHeader.appendChild(modalTitle);
|
||||||
|
modalHeader.appendChild(modalCloseButton);
|
||||||
|
modalContent.appendChild(modalHeader);
|
||||||
|
let modalBody = document.createElement('div');
|
||||||
|
modalBody.classList.add("modal-body");
|
||||||
|
|
||||||
|
let agentDropDownPicker = document.createElement('select');
|
||||||
|
agentDropDownPicker.setAttribute("id", "agent-dropdown-picker");
|
||||||
|
agentDropDownPicker.setAttribute("name", "agent-dropdown-picker");
|
||||||
|
|
||||||
|
let agentDropDownLabel = document.createElement('label');
|
||||||
|
agentDropDownLabel.setAttribute("for", "agent-dropdown-picker");
|
||||||
|
agentDropDownLabel.textContent = "Who do you want to talk to?";
|
||||||
|
|
||||||
|
fetch('/api/agents')
|
||||||
|
.then(response => response.json())
|
||||||
|
.then(data => {
|
||||||
|
if (data.length > 0) {
|
||||||
|
data.forEach((agent) => {
|
||||||
|
let agentOption = document.createElement('option');
|
||||||
|
agentOption.setAttribute("value", agent.slug);
|
||||||
|
agentOption.textContent = agent.name;
|
||||||
|
agentDropDownPicker.appendChild(agentOption);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
})
|
||||||
|
.catch(err => {
|
||||||
|
return;
|
||||||
|
});
|
||||||
|
|
||||||
|
let seeAllAgentsLink = document.createElement('a');
|
||||||
|
seeAllAgentsLink.setAttribute("href", "/agents");
|
||||||
|
seeAllAgentsLink.setAttribute("target", "_blank");
|
||||||
|
seeAllAgentsLink.textContent = "See all agents";
|
||||||
|
|
||||||
|
let newConversationSubmitButton = document.createElement('button');
|
||||||
|
newConversationSubmitButton.setAttribute("type", "submit");
|
||||||
|
newConversationSubmitButton.textContent = "Go";
|
||||||
|
newConversationSubmitButton.id = "new-conversation-submit-button";
|
||||||
|
|
||||||
|
newConversationSubmitButton.addEventListener('click', function(event) {
|
||||||
|
event.preventDefault();
|
||||||
|
let agentSlug = agentDropDownPicker.value;
|
||||||
|
let createURL = `/api/chat/sessions?client=web&agent_slug=${agentSlug}`;
|
||||||
let chatBody = document.getElementById("chat-body");
|
let chatBody = document.getElementById("chat-body");
|
||||||
chatBody.innerHTML = "";
|
fetch(createURL, { method: "POST" })
|
||||||
flashStatusInChatInput("📝 New conversation started");
|
.then(response => response.json())
|
||||||
chatBody.dataset.conversationId = "";
|
.then(data => {
|
||||||
chatBody.dataset.conversationTitle = "";
|
chatBody.dataset.conversationId = data.conversation_id;
|
||||||
renderMessage(welcome_message, "khoj");
|
modal.remove();
|
||||||
|
loadChat();
|
||||||
|
})
|
||||||
|
.catch(err => {
|
||||||
|
return;
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
let closeButton = document.createElement('button');
|
||||||
|
closeButton.id = "close-button";
|
||||||
|
closeButton.innerHTML = "Close";
|
||||||
|
closeButton.classList.add("close-button");
|
||||||
|
closeButton.addEventListener('click', function() {
|
||||||
|
modal.remove();
|
||||||
|
});
|
||||||
|
|
||||||
|
modalBody.appendChild(agentDropDownLabel);
|
||||||
|
modalBody.appendChild(agentDropDownPicker);
|
||||||
|
modalBody.appendChild(seeAllAgentsLink);
|
||||||
|
|
||||||
|
let modalFooter = document.createElement('div');
|
||||||
|
modalFooter.classList.add("modal-footer");
|
||||||
|
modalFooter.appendChild(closeButton);
|
||||||
|
modalFooter.appendChild(newConversationSubmitButton);
|
||||||
|
modalBody.appendChild(modalFooter);
|
||||||
|
|
||||||
|
modalContent.appendChild(modalBody);
|
||||||
|
modal.appendChild(modalContent);
|
||||||
|
document.body.appendChild(modal);
|
||||||
}
|
}
|
||||||
|
|
||||||
function refreshChatSessionsPanel() {
|
function refreshChatSessionsPanel() {
|
||||||
|
|
@ -1194,8 +1310,6 @@ To get started, just start typing below. You can also type / to see a list of co
|
||||||
document.getElementById('new-conversation').classList.toggle('collapsed');
|
document.getElementById('new-conversation').classList.toggle('collapsed');
|
||||||
document.getElementById('existing-conversations').classList.toggle('collapsed');
|
document.getElementById('existing-conversations').classList.toggle('collapsed');
|
||||||
document.getElementById('side-panel-collapse').style.transform = document.getElementById('side-panel').classList.contains('collapsed') ? 'rotate(0deg)' : 'rotate(180deg)';
|
document.getElementById('side-panel-collapse').style.transform = document.getElementById('side-panel').classList.contains('collapsed') ? 'rotate(0deg)' : 'rotate(180deg)';
|
||||||
|
|
||||||
document.getElementById('chat-section-wrapper').classList.toggle('mobile-friendly');
|
|
||||||
}
|
}
|
||||||
</script>
|
</script>
|
||||||
<body>
|
<body>
|
||||||
|
|
@ -1215,13 +1329,27 @@ To get started, just start typing below. You can also type / to see a list of co
|
||||||
<path d="M16 0c-8.836 0-16 7.163-16 16s7.163 16 16 16c8.837 0 16-7.163 16-16s-7.163-16-16-16zM16 30.032c-7.72 0-14-6.312-14-14.032s6.28-14 14-14 14 6.28 14 14-6.28 14.032-14 14.032zM23 15h-6v-6c0-0.552-0.448-1-1-1s-1 0.448-1 1v6h-6c-0.552 0-1 0.448-1 1s0.448 1 1 1h6v6c0 0.552 0.448 1 1 1s1-0.448 1-1v-6h6c0.552 0 1-0.448 1-1s-0.448-1-1-1z"></path>
|
<path d="M16 0c-8.836 0-16 7.163-16 16s7.163 16 16 16c8.837 0 16-7.163 16-16s-7.163-16-16-16zM16 30.032c-7.72 0-14-6.312-14-14.032s6.28-14 14-14 14 6.28 14 14-6.28 14.032-14 14.032zM23 15h-6v-6c0-0.552-0.448-1-1-1s-1 0.448-1 1v6h-6c-0.552 0-1 0.448-1 1s0.448 1 1 1h6v6c0 0.552 0.448 1 1 1s1-0.448 1-1v-6h6c0.552 0 1-0.448 1-1s-0.448-1-1-1z"></path>
|
||||||
</svg>
|
</svg>
|
||||||
</button>
|
</button>
|
||||||
|
<div id="conversation-list-header" style="display: none;">Conversations</div>
|
||||||
</div>
|
</div>
|
||||||
<div id="existing-conversations">
|
<div id="existing-conversations">
|
||||||
<div id="conversation-list">
|
<div id="conversation-list">
|
||||||
<div id="conversation-list-header" style="display: none;">Recent Conversations</div>
|
|
||||||
<div id="conversation-list-body"></div>
|
<div id="conversation-list-body"></div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
<a class="inline-chat-link" id="agent-link" href="">
|
||||||
|
<div id="agent-metadata" style="display: none;">
|
||||||
|
Active
|
||||||
|
<div id="agent-metadata-content">
|
||||||
|
<div id="agent-avatar-wrapper">
|
||||||
|
<img id="agent-avatar" src="" alt="Agent Avatar" />
|
||||||
|
</div>
|
||||||
|
<div id="agent-name-wrapper">
|
||||||
|
<div id="agent-name"></div>
|
||||||
|
<div id="agent-owned-by-user" style="display: none;">Edit</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</a>
|
||||||
</div>
|
</div>
|
||||||
<div id="collapse-side-panel">
|
<div id="collapse-side-panel">
|
||||||
<button
|
<button
|
||||||
|
|
@ -1297,7 +1425,7 @@ To get started, just start typing below. You can also type / to see a list of co
|
||||||
color: var(--main-text-color);
|
color: var(--main-text-color);
|
||||||
text-align: center;
|
text-align: center;
|
||||||
font-family: var(--font-family);
|
font-family: var(--font-family);
|
||||||
font-size: 20px;
|
font-size: medium;
|
||||||
font-weight: 300;
|
font-weight: 300;
|
||||||
line-height: 1.5em;
|
line-height: 1.5em;
|
||||||
height: 100vh;
|
height: 100vh;
|
||||||
|
|
@ -1448,10 +1576,6 @@ To get started, just start typing below. You can also type / to see a list of co
|
||||||
overflow-y: scroll;
|
overflow-y: scroll;
|
||||||
}
|
}
|
||||||
|
|
||||||
#chat-section-wrapper.mobile-friendly {
|
|
||||||
grid-template-columns: auto auto;
|
|
||||||
}
|
|
||||||
|
|
||||||
#chat-body-wrapper {
|
#chat-body-wrapper {
|
||||||
display: flex;
|
display: flex;
|
||||||
flex-direction: column;
|
flex-direction: column;
|
||||||
|
|
@ -1464,10 +1588,15 @@ To get started, just start typing below. You can also type / to see a list of co
|
||||||
background: var(--background-color);
|
background: var(--background-color);
|
||||||
border-radius: 5px;
|
border-radius: 5px;
|
||||||
box-shadow: 0 0 11px #aaa;
|
box-shadow: 0 0 11px #aaa;
|
||||||
overflow-y: scroll;
|
|
||||||
text-align: left;
|
text-align: left;
|
||||||
transition: width 0.3s ease-in-out;
|
transition: width 0.3s ease-in-out;
|
||||||
max-height: 85vh;
|
max-height: 100%;
|
||||||
|
display: grid;
|
||||||
|
grid-template-rows: auto 1fr auto;
|
||||||
|
}
|
||||||
|
|
||||||
|
div#existing-conversations {
|
||||||
|
max-height: 95%;
|
||||||
overflow-y: auto;
|
overflow-y: auto;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -1489,8 +1618,12 @@ To get started, just start typing below. You can also type / to see a list of co
|
||||||
grid-gap: 8px;
|
grid-gap: 8px;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
div#conversation-list {
|
||||||
|
height: 1;
|
||||||
|
}
|
||||||
|
|
||||||
div#side-panel-wrapper {
|
div#side-panel-wrapper {
|
||||||
display: flex
|
display: flex;
|
||||||
}
|
}
|
||||||
|
|
||||||
#chat-body {
|
#chat-body {
|
||||||
|
|
@ -1880,7 +2013,7 @@ To get started, just start typing below. You can also type / to see a list of co
|
||||||
}
|
}
|
||||||
@media only screen and (min-width: 700px) {
|
@media only screen and (min-width: 700px) {
|
||||||
body {
|
body {
|
||||||
grid-template-columns: auto min(70vw, 100%) auto;
|
grid-template-columns: auto min(90vw, 100%) auto;
|
||||||
grid-template-rows: auto auto minmax(80px, 100%) auto;
|
grid-template-rows: auto auto minmax(80px, 100%) auto;
|
||||||
}
|
}
|
||||||
body > * {
|
body > * {
|
||||||
|
|
@ -1901,6 +2034,7 @@ To get started, just start typing below. You can also type / to see a list of co
|
||||||
div#new-conversation {
|
div#new-conversation {
|
||||||
text-align: left;
|
text-align: left;
|
||||||
border-bottom: 1px solid var(--main-text-color);
|
border-bottom: 1px solid var(--main-text-color);
|
||||||
|
margin-top: 8px;
|
||||||
margin-bottom: 8px;
|
margin-bottom: 8px;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -2056,6 +2190,170 @@ To get started, just start typing below. You can also type / to see a list of co
|
||||||
animation-delay: -0.5s;
|
animation-delay: -0.5s;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#agent-metadata-content {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: auto 1fr;
|
||||||
|
padding: 10px;
|
||||||
|
background-color: var(--primary);
|
||||||
|
border-radius: 5px;
|
||||||
|
box-shadow: 0 1px 3px rgba(0, 0, 0, 0.2);
|
||||||
|
margin-bottom: 20px;
|
||||||
|
}
|
||||||
|
|
||||||
|
#agent-metadata {
|
||||||
|
border-top: 1px solid black;
|
||||||
|
padding-top: 10px;
|
||||||
|
}
|
||||||
|
|
||||||
|
#agent-avatar-wrapper {
|
||||||
|
margin-right: 10px;
|
||||||
|
}
|
||||||
|
|
||||||
|
#agent-avatar {
|
||||||
|
width: 50px;
|
||||||
|
height: 50px;
|
||||||
|
border-radius: 50%;
|
||||||
|
object-fit: cover;
|
||||||
|
}
|
||||||
|
|
||||||
|
#agent-name-wrapper {
|
||||||
|
display: grid;
|
||||||
|
align-items: center;
|
||||||
|
}
|
||||||
|
|
||||||
|
#agent-name {
|
||||||
|
font-size: 18px;
|
||||||
|
font-weight: bold;
|
||||||
|
color: #333;
|
||||||
|
}
|
||||||
|
|
||||||
|
#agent-instructions {
|
||||||
|
font-size: 14px;
|
||||||
|
color: #666;
|
||||||
|
height: 50px;
|
||||||
|
overflow: auto;
|
||||||
|
}
|
||||||
|
|
||||||
|
#agent-owned-by-user {
|
||||||
|
font-size: 12px;
|
||||||
|
color: #007BFF;
|
||||||
|
margin-top: 5px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.modal {
|
||||||
|
position: fixed; /* Stay in place */
|
||||||
|
z-index: 1; /* Sit on top */
|
||||||
|
left: 0;
|
||||||
|
top: 0;
|
||||||
|
width: 100%; /* Full width */
|
||||||
|
height: 100%; /* Full height */
|
||||||
|
background-color: rgba(0,0,0,0.4); /* Black w/ opacity */
|
||||||
|
margin: 0px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.modal-content {
|
||||||
|
margin: 15% auto; /* 15% from the top and centered */
|
||||||
|
padding: 20px;
|
||||||
|
border: 1px solid #888;
|
||||||
|
width: 250px;
|
||||||
|
text-align: left;
|
||||||
|
background: var(--background-color);
|
||||||
|
border-radius: 5px;
|
||||||
|
box-shadow: 0 0 11px #aaa;
|
||||||
|
text-align: left;
|
||||||
|
}
|
||||||
|
|
||||||
|
.modal-header {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: 1fr auto;
|
||||||
|
color: var(--main-text-color);
|
||||||
|
align-items: baseline;
|
||||||
|
}
|
||||||
|
|
||||||
|
.modal-header h2 {
|
||||||
|
margin: 0;
|
||||||
|
text-align: left;
|
||||||
|
}
|
||||||
|
|
||||||
|
.modal-body {
|
||||||
|
display: grid;
|
||||||
|
grid-auto-flow: row;
|
||||||
|
gap: 8px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.modal-body a {
|
||||||
|
/* text-decoration: none; */
|
||||||
|
color: var(--summer-sun);
|
||||||
|
}
|
||||||
|
|
||||||
|
.modal-close-button {
|
||||||
|
margin: 0;
|
||||||
|
font-size: 20px;
|
||||||
|
background: none;
|
||||||
|
border: none;
|
||||||
|
color: var(--summer-sun);
|
||||||
|
}
|
||||||
|
|
||||||
|
.modal-close-button:hover,
|
||||||
|
.modal-close-button:focus {
|
||||||
|
color: #000;
|
||||||
|
text-decoration: none;
|
||||||
|
cursor: pointer;
|
||||||
|
}
|
||||||
|
|
||||||
|
#new-conversation-form {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
}
|
||||||
|
|
||||||
|
#new-conversation-form label,
|
||||||
|
#new-conversation-form input,
|
||||||
|
#new-conversation-form button {
|
||||||
|
margin-bottom: 10px;
|
||||||
|
}
|
||||||
|
|
||||||
|
#new-conversation-form button {
|
||||||
|
cursor: pointer;
|
||||||
|
}
|
||||||
|
|
||||||
|
.modal-footer {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: 1fr 1fr;
|
||||||
|
grid-gap: 12px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.modal-body button {
|
||||||
|
cursor: pointer;
|
||||||
|
border-radius: 12px;
|
||||||
|
padding: 8px;
|
||||||
|
border: 1px solid var(--main-text-color);
|
||||||
|
}
|
||||||
|
|
||||||
|
button#new-conversation-submit-button {
|
||||||
|
background: var(--summer-sun);
|
||||||
|
transition: background 0.2s ease-in-out;
|
||||||
|
}
|
||||||
|
|
||||||
|
button#close-button {
|
||||||
|
background: var(--background-color);
|
||||||
|
transition: background 0.2s ease-in-out;
|
||||||
|
}
|
||||||
|
|
||||||
|
button#new-conversation-submit-button:hover {
|
||||||
|
background: var(--primary);
|
||||||
|
}
|
||||||
|
|
||||||
|
button#close-button:hover {
|
||||||
|
background: var(--primary-hover);
|
||||||
|
}
|
||||||
|
|
||||||
|
.modal-body select {
|
||||||
|
padding: 8px;
|
||||||
|
border-radius: 12px;
|
||||||
|
border: 1px solid var(--main-text-color);
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
@keyframes lds-ripple {
|
@keyframes lds-ripple {
|
||||||
0% {
|
0% {
|
||||||
top: 36px;
|
top: 36px;
|
||||||
|
|
|
||||||
|
|
@ -4,11 +4,12 @@
|
||||||
<img class="khoj-logo" src="/static/assets/icons/khoj-logo-sideways-500.png?v={{ khoj_version }}" alt="Khoj"></img>
|
<img class="khoj-logo" src="/static/assets/icons/khoj-logo-sideways-500.png?v={{ khoj_version }}" alt="Khoj"></img>
|
||||||
</a>
|
</a>
|
||||||
<nav class="khoj-nav">
|
<nav class="khoj-nav">
|
||||||
|
<a id="agents-nav" class="khoj-nav" href="/agents">Agents</a>
|
||||||
{% if has_documents %}
|
{% if has_documents %}
|
||||||
<a id="chat-nav" class="khoj-nav" href="/chat">💬 Chat</a>
|
<a id="search-nav" class="khoj-nav" href="/search">Search</a>
|
||||||
<a id="search-nav" class="khoj-nav" href="/search">🔎 Search</a>
|
|
||||||
{% endif %}
|
{% endif %}
|
||||||
<!-- Dropdown Menu -->
|
<!-- Dropdown Menu -->
|
||||||
|
{% if username %}
|
||||||
<div id="khoj-nav-menu-container" class="khoj-nav dropdown">
|
<div id="khoj-nav-menu-container" class="khoj-nav dropdown">
|
||||||
{% if user_photo and user_photo != "None" %}
|
{% if user_photo and user_photo != "None" %}
|
||||||
{% if is_active %}
|
{% if is_active %}
|
||||||
|
|
@ -29,6 +30,7 @@
|
||||||
<a class="khoj-nav" href="/auth/logout">🔑 Logout</a>
|
<a class="khoj-nav" href="/auth/logout">🔑 Logout</a>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
{% endif %}
|
||||||
</nav>
|
</nav>
|
||||||
</div>
|
</div>
|
||||||
{%- endmacro %}
|
{%- endmacro %}
|
||||||
|
|
|
||||||
|
|
@ -160,7 +160,9 @@ def start_server(app, host=None, port=None, socket=None):
|
||||||
if socket:
|
if socket:
|
||||||
uvicorn.run(app, proxy_headers=True, uds=socket, log_level="debug", use_colors=True, log_config=None)
|
uvicorn.run(app, proxy_headers=True, uds=socket, log_level="debug", use_colors=True, log_config=None)
|
||||||
else:
|
else:
|
||||||
uvicorn.run(app, host=host, port=port, log_level="debug", use_colors=True, log_config=None)
|
uvicorn.run(
|
||||||
|
app, host=host, port=port, log_level="debug", use_colors=True, log_config=None, timeout_keep_alive=60
|
||||||
|
)
|
||||||
logger.info("🌒 Stopping Khoj")
|
logger.info("🌒 Stopping Khoj")
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -114,7 +114,7 @@ class MarkdownToEntries(TextToEntries):
|
||||||
# Append base filename to compiled entry for context to model
|
# Append base filename to compiled entry for context to model
|
||||||
# Increment heading level for heading entries and make filename as its top level heading
|
# Increment heading level for heading entries and make filename as its top level heading
|
||||||
prefix = f"# {stem}\n#" if heading else f"# {stem}\n"
|
prefix = f"# {stem}\n#" if heading else f"# {stem}\n"
|
||||||
compiled_entry = f"{prefix}{parsed_entry}"
|
compiled_entry = f"{entry_filename}\n{prefix}{parsed_entry}"
|
||||||
entries.append(
|
entries.append(
|
||||||
Entry(
|
Entry(
|
||||||
compiled=compiled_entry,
|
compiled=compiled_entry,
|
||||||
|
|
|
||||||
|
|
@ -7,6 +7,7 @@ from typing import Any, Iterator, List, Union
|
||||||
from langchain.schema import ChatMessage
|
from langchain.schema import ChatMessage
|
||||||
from llama_cpp import Llama
|
from llama_cpp import Llama
|
||||||
|
|
||||||
|
from khoj.database.models import Agent
|
||||||
from khoj.processor.conversation import prompts
|
from khoj.processor.conversation import prompts
|
||||||
from khoj.processor.conversation.offline.utils import download_model
|
from khoj.processor.conversation.offline.utils import download_model
|
||||||
from khoj.processor.conversation.utils import (
|
from khoj.processor.conversation.utils import (
|
||||||
|
|
@ -131,6 +132,7 @@ def converse_offline(
|
||||||
tokenizer_name=None,
|
tokenizer_name=None,
|
||||||
location_data: LocationData = None,
|
location_data: LocationData = None,
|
||||||
user_name: str = None,
|
user_name: str = None,
|
||||||
|
agent: Agent = None,
|
||||||
) -> Union[ThreadedGenerator, Iterator[str]]:
|
) -> Union[ThreadedGenerator, Iterator[str]]:
|
||||||
"""
|
"""
|
||||||
Converse with user using Llama
|
Converse with user using Llama
|
||||||
|
|
@ -140,6 +142,15 @@ def converse_offline(
|
||||||
offline_chat_model = loaded_model or download_model(model)
|
offline_chat_model = loaded_model or download_model(model)
|
||||||
compiled_references_message = "\n\n".join({f"{item}" for item in references})
|
compiled_references_message = "\n\n".join({f"{item}" for item in references})
|
||||||
|
|
||||||
|
current_date = datetime.now().strftime("%Y-%m-%d")
|
||||||
|
|
||||||
|
if agent and agent.personality:
|
||||||
|
system_prompt = prompts.custom_system_prompt_offline_chat.format(
|
||||||
|
name=agent.name, bio=agent.personality, current_date=current_date
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
system_prompt = prompts.system_prompt_offline_chat.format(current_date=current_date)
|
||||||
|
|
||||||
conversation_primer = prompts.query_prompt.format(query=user_query)
|
conversation_primer = prompts.query_prompt.format(query=user_query)
|
||||||
|
|
||||||
if location_data:
|
if location_data:
|
||||||
|
|
@ -169,10 +180,9 @@ def converse_offline(
|
||||||
conversation_primer = f"{prompts.notes_conversation_offline.format(references=compiled_references_message)}\n{conversation_primer}"
|
conversation_primer = f"{prompts.notes_conversation_offline.format(references=compiled_references_message)}\n{conversation_primer}"
|
||||||
|
|
||||||
# Setup Prompt with Primer or Conversation History
|
# Setup Prompt with Primer or Conversation History
|
||||||
current_date = datetime.now().strftime("%Y-%m-%d")
|
|
||||||
messages = generate_chatml_messages_with_context(
|
messages = generate_chatml_messages_with_context(
|
||||||
conversation_primer,
|
conversation_primer,
|
||||||
prompts.system_prompt_offline_chat.format(current_date=current_date),
|
system_prompt,
|
||||||
conversation_log,
|
conversation_log,
|
||||||
model_name=model,
|
model_name=model,
|
||||||
loaded_model=offline_chat_model,
|
loaded_model=offline_chat_model,
|
||||||
|
|
|
||||||
|
|
@ -5,6 +5,7 @@ from typing import Dict, Optional
|
||||||
|
|
||||||
from langchain.schema import ChatMessage
|
from langchain.schema import ChatMessage
|
||||||
|
|
||||||
|
from khoj.database.models import Agent
|
||||||
from khoj.processor.conversation import prompts
|
from khoj.processor.conversation import prompts
|
||||||
from khoj.processor.conversation.openai.utils import (
|
from khoj.processor.conversation.openai.utils import (
|
||||||
chat_completion_with_backoff,
|
chat_completion_with_backoff,
|
||||||
|
|
@ -115,6 +116,7 @@ def converse(
|
||||||
tokenizer_name=None,
|
tokenizer_name=None,
|
||||||
location_data: LocationData = None,
|
location_data: LocationData = None,
|
||||||
user_name: str = None,
|
user_name: str = None,
|
||||||
|
agent: Agent = None,
|
||||||
):
|
):
|
||||||
"""
|
"""
|
||||||
Converse with user using OpenAI's ChatGPT
|
Converse with user using OpenAI's ChatGPT
|
||||||
|
|
@ -125,6 +127,13 @@ def converse(
|
||||||
|
|
||||||
conversation_primer = prompts.query_prompt.format(query=user_query)
|
conversation_primer = prompts.query_prompt.format(query=user_query)
|
||||||
|
|
||||||
|
if agent and agent.personality:
|
||||||
|
system_prompt = prompts.custom_personality.format(
|
||||||
|
name=agent.name, bio=agent.personality, current_date=current_date
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
system_prompt = prompts.personality.format(current_date=current_date)
|
||||||
|
|
||||||
if location_data:
|
if location_data:
|
||||||
location = f"{location_data.city}, {location_data.region}, {location_data.country}"
|
location = f"{location_data.city}, {location_data.region}, {location_data.country}"
|
||||||
location_prompt = prompts.user_location.format(location=location)
|
location_prompt = prompts.user_location.format(location=location)
|
||||||
|
|
@ -152,7 +161,7 @@ def converse(
|
||||||
# Setup Prompt with Primer or Conversation History
|
# Setup Prompt with Primer or Conversation History
|
||||||
messages = generate_chatml_messages_with_context(
|
messages = generate_chatml_messages_with_context(
|
||||||
conversation_primer,
|
conversation_primer,
|
||||||
prompts.personality.format(current_date=current_date),
|
system_prompt,
|
||||||
conversation_log,
|
conversation_log,
|
||||||
model,
|
model,
|
||||||
max_prompt_size,
|
max_prompt_size,
|
||||||
|
|
|
||||||
|
|
@ -21,6 +21,24 @@ Today is {current_date} in UTC.
|
||||||
""".strip()
|
""".strip()
|
||||||
)
|
)
|
||||||
|
|
||||||
|
custom_personality = PromptTemplate.from_template(
|
||||||
|
"""
|
||||||
|
You are {name}, a personal agent on Khoj.
|
||||||
|
Use your general knowledge and past conversation with the user as context to inform your responses.
|
||||||
|
You were created by Khoj Inc. with the following capabilities:
|
||||||
|
|
||||||
|
- You *CAN REMEMBER ALL NOTES and PERSONAL INFORMATION FOREVER* that the user ever shares with you.
|
||||||
|
- Users can share files and other information with you using the Khoj Desktop, Obsidian or Emacs app. They can also drag and drop their files into the chat window.
|
||||||
|
- Say "I don't know" or "I don't understand" if you don't know what to say or if you don't know the answer to a question.
|
||||||
|
- Ask crisp follow-up questions to get additional context, when the answer cannot be inferred from the provided notes or past conversations.
|
||||||
|
- Sometimes the user will share personal information that needs to be remembered, like an account ID or a residential address. These can be acknowledged with a simple "Got it" or "Okay".
|
||||||
|
|
||||||
|
Today is {current_date} in UTC.
|
||||||
|
|
||||||
|
Instructions:\n{bio}
|
||||||
|
""".strip()
|
||||||
|
)
|
||||||
|
|
||||||
## General Conversation
|
## General Conversation
|
||||||
## --
|
## --
|
||||||
general_conversation = PromptTemplate.from_template(
|
general_conversation = PromptTemplate.from_template(
|
||||||
|
|
@ -61,6 +79,20 @@ Today is {current_date} in UTC.
|
||||||
""".strip()
|
""".strip()
|
||||||
)
|
)
|
||||||
|
|
||||||
|
custom_system_prompt_offline_chat = PromptTemplate.from_template(
|
||||||
|
"""
|
||||||
|
You are {name}, a personal agent on Khoj.
|
||||||
|
- Use your general knowledge and past conversation with the user as context to inform your responses.
|
||||||
|
- If you do not know the answer, say 'I don't know.'
|
||||||
|
- Think step-by-step and ask questions to get the necessary information to answer the user's question.
|
||||||
|
- Do not print verbatim Notes unless necessary.
|
||||||
|
|
||||||
|
Today is {current_date} in UTC.
|
||||||
|
|
||||||
|
Instructions:\n{bio}
|
||||||
|
""".strip()
|
||||||
|
)
|
||||||
|
|
||||||
## Notes Conversation
|
## Notes Conversation
|
||||||
## --
|
## --
|
||||||
notes_conversation = PromptTemplate.from_template(
|
notes_conversation = PromptTemplate.from_template(
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@ from fastapi.requests import Request
|
||||||
from fastapi.responses import Response
|
from fastapi.responses import Response
|
||||||
from starlette.authentication import requires
|
from starlette.authentication import requires
|
||||||
|
|
||||||
from khoj.configure import configure_server, initialize_content
|
from khoj.configure import initialize_content
|
||||||
from khoj.database.adapters import (
|
from khoj.database.adapters import (
|
||||||
ConversationAdapters,
|
ConversationAdapters,
|
||||||
EntryAdapters,
|
EntryAdapters,
|
||||||
|
|
|
||||||
43
src/khoj/routers/api_agents.py
Normal file
43
src/khoj/routers/api_agents.py
Normal file
|
|
@ -0,0 +1,43 @@
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Request
|
||||||
|
from fastapi.requests import Request
|
||||||
|
from fastapi.responses import Response
|
||||||
|
|
||||||
|
from khoj.database.adapters import AgentAdapters
|
||||||
|
from khoj.database.models import KhojUser
|
||||||
|
from khoj.routers.helpers import CommonQueryParams
|
||||||
|
|
||||||
|
# Initialize Router
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
api_agents = APIRouter()
|
||||||
|
|
||||||
|
|
||||||
|
@api_agents.get("", response_class=Response)
|
||||||
|
async def all_agents(
|
||||||
|
request: Request,
|
||||||
|
common: CommonQueryParams,
|
||||||
|
) -> Response:
|
||||||
|
user: KhojUser = request.user.object if request.user.is_authenticated else None
|
||||||
|
agents = await AgentAdapters.aget_all_accessible_agents(user)
|
||||||
|
agents_packet = list()
|
||||||
|
for agent in agents:
|
||||||
|
agents_packet.append(
|
||||||
|
{
|
||||||
|
"slug": agent.slug,
|
||||||
|
"avatar": agent.avatar,
|
||||||
|
"name": agent.name,
|
||||||
|
"personality": agent.personality,
|
||||||
|
"public": agent.public,
|
||||||
|
"creator": agent.creator.username if agent.creator else None,
|
||||||
|
"managed_by_admin": agent.managed_by_admin,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
# Make sure that the agent named 'khoj' is first in the list. Everything else is sorted by name.
|
||||||
|
agents_packet.sort(key=lambda x: x["name"])
|
||||||
|
agents_packet.sort(key=lambda x: x["slug"] == "khoj", reverse=True)
|
||||||
|
return Response(content=json.dumps(agents_packet), media_type="application/json", status_code=200)
|
||||||
|
|
@ -12,7 +12,11 @@ from starlette.authentication import requires
|
||||||
|
|
||||||
from khoj.database.adapters import ConversationAdapters, EntryAdapters, aget_user_name
|
from khoj.database.adapters import ConversationAdapters, EntryAdapters, aget_user_name
|
||||||
from khoj.database.models import KhojUser
|
from khoj.database.models import KhojUser
|
||||||
from khoj.processor.conversation.prompts import help_message, no_entries_found
|
from khoj.processor.conversation.prompts import (
|
||||||
|
help_message,
|
||||||
|
no_entries_found,
|
||||||
|
no_notes_found,
|
||||||
|
)
|
||||||
from khoj.processor.conversation.utils import save_to_conversation_log
|
from khoj.processor.conversation.utils import save_to_conversation_log
|
||||||
from khoj.processor.tools.online_search import (
|
from khoj.processor.tools.online_search import (
|
||||||
online_search_enabled,
|
online_search_enabled,
|
||||||
|
|
@ -85,9 +89,22 @@ def chat_history(
|
||||||
status_code=404,
|
status_code=404,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
agent_metadata = None
|
||||||
|
if conversation.agent:
|
||||||
|
agent_metadata = {
|
||||||
|
"slug": conversation.agent.slug,
|
||||||
|
"name": conversation.agent.name,
|
||||||
|
"avatar": conversation.agent.avatar,
|
||||||
|
"isCreator": conversation.agent.creator == user,
|
||||||
|
}
|
||||||
|
|
||||||
meta_log = conversation.conversation_log
|
meta_log = conversation.conversation_log
|
||||||
meta_log.update(
|
meta_log.update(
|
||||||
{"conversation_id": conversation.id, "slug": conversation.title if conversation.title else conversation.slug}
|
{
|
||||||
|
"conversation_id": conversation.id,
|
||||||
|
"slug": conversation.title if conversation.title else conversation.slug,
|
||||||
|
"agent": agent_metadata,
|
||||||
|
}
|
||||||
)
|
)
|
||||||
|
|
||||||
update_telemetry_state(
|
update_telemetry_state(
|
||||||
|
|
@ -152,18 +169,24 @@ def chat_sessions(
|
||||||
async def create_chat_session(
|
async def create_chat_session(
|
||||||
request: Request,
|
request: Request,
|
||||||
common: CommonQueryParams,
|
common: CommonQueryParams,
|
||||||
|
agent_slug: Optional[str] = None,
|
||||||
):
|
):
|
||||||
user = request.user.object
|
user = request.user.object
|
||||||
|
|
||||||
# Create new Conversation Session
|
# Create new Conversation Session
|
||||||
conversation = await ConversationAdapters.acreate_conversation_session(user, request.user.client_app)
|
conversation = await ConversationAdapters.acreate_conversation_session(user, request.user.client_app, agent_slug)
|
||||||
|
|
||||||
response = {"conversation_id": conversation.id}
|
response = {"conversation_id": conversation.id}
|
||||||
|
|
||||||
|
conversation_metadata = {
|
||||||
|
"agent": agent_slug,
|
||||||
|
}
|
||||||
|
|
||||||
update_telemetry_state(
|
update_telemetry_state(
|
||||||
request=request,
|
request=request,
|
||||||
telemetry_type="api",
|
telemetry_type="api",
|
||||||
api="create_chat_sessions",
|
api="create_chat_sessions",
|
||||||
|
metadata=conversation_metadata,
|
||||||
**common.__dict__,
|
**common.__dict__,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -242,7 +265,7 @@ async def chat(
|
||||||
) -> Response:
|
) -> Response:
|
||||||
user: KhojUser = request.user.object
|
user: KhojUser = request.user.object
|
||||||
q = unquote(q)
|
q = unquote(q)
|
||||||
logger.info("Chat request by {user.username}: {q}")
|
logger.info(f"Chat request by {user.username}: {q}")
|
||||||
|
|
||||||
await is_ready_to_chat(user)
|
await is_ready_to_chat(user)
|
||||||
conversation_commands = [get_conversation_command(query=q, any_references=True)]
|
conversation_commands = [get_conversation_command(query=q, any_references=True)]
|
||||||
|
|
@ -295,6 +318,14 @@ async def chat(
|
||||||
response_obj = {"response": no_entries_found_format}
|
response_obj = {"response": no_entries_found_format}
|
||||||
return Response(content=json.dumps(response_obj), media_type="text/plain", status_code=200)
|
return Response(content=json.dumps(response_obj), media_type="text/plain", status_code=200)
|
||||||
|
|
||||||
|
if conversation_commands == [ConversationCommand.Notes] and is_none_or_empty(compiled_references):
|
||||||
|
no_notes_found_format = no_notes_found.format()
|
||||||
|
if stream:
|
||||||
|
return StreamingResponse(iter([no_notes_found_format]), media_type="text/event-stream", status_code=200)
|
||||||
|
else:
|
||||||
|
response_obj = {"response": no_notes_found_format}
|
||||||
|
return Response(content=json.dumps(response_obj), media_type="text/plain", status_code=200)
|
||||||
|
|
||||||
if ConversationCommand.Notes in conversation_commands and is_none_or_empty(compiled_references):
|
if ConversationCommand.Notes in conversation_commands and is_none_or_empty(compiled_references):
|
||||||
conversation_commands.remove(ConversationCommand.Notes)
|
conversation_commands.remove(ConversationCommand.Notes)
|
||||||
|
|
||||||
|
|
@ -356,6 +387,7 @@ async def chat(
|
||||||
llm_response, chat_metadata = await agenerate_chat_response(
|
llm_response, chat_metadata = await agenerate_chat_response(
|
||||||
defiltered_query,
|
defiltered_query,
|
||||||
meta_log,
|
meta_log,
|
||||||
|
conversation,
|
||||||
compiled_references,
|
compiled_references,
|
||||||
online_results,
|
online_results,
|
||||||
inferred_queries,
|
inferred_queries,
|
||||||
|
|
@ -369,6 +401,7 @@ async def chat(
|
||||||
|
|
||||||
cmd_set = set([cmd.value for cmd in conversation_commands])
|
cmd_set = set([cmd.value for cmd in conversation_commands])
|
||||||
chat_metadata["conversation_command"] = cmd_set
|
chat_metadata["conversation_command"] = cmd_set
|
||||||
|
chat_metadata["agent"] = conversation.agent.slug if conversation.agent else None
|
||||||
|
|
||||||
update_telemetry_state(
|
update_telemetry_state(
|
||||||
request=request,
|
request=request,
|
||||||
|
|
|
||||||
|
|
@ -7,6 +7,7 @@ from starlette.authentication import requires
|
||||||
from starlette.config import Config
|
from starlette.config import Config
|
||||||
from starlette.requests import Request
|
from starlette.requests import Request
|
||||||
from starlette.responses import HTMLResponse, RedirectResponse, Response
|
from starlette.responses import HTMLResponse, RedirectResponse, Response
|
||||||
|
from starlette.status import HTTP_302_FOUND
|
||||||
|
|
||||||
from khoj.database.adapters import (
|
from khoj.database.adapters import (
|
||||||
create_khoj_token,
|
create_khoj_token,
|
||||||
|
|
@ -90,6 +91,7 @@ async def delete_token(request: Request, token: str) -> str:
|
||||||
@auth_router.post("/redirect")
|
@auth_router.post("/redirect")
|
||||||
async def auth(request: Request):
|
async def auth(request: Request):
|
||||||
form = await request.form()
|
form = await request.form()
|
||||||
|
next_url = request.query_params.get("next", "/")
|
||||||
credential = form.get("credential")
|
credential = form.get("credential")
|
||||||
|
|
||||||
csrf_token_cookie = request.cookies.get("g_csrf_token")
|
csrf_token_cookie = request.cookies.get("g_csrf_token")
|
||||||
|
|
@ -117,9 +119,9 @@ async def auth(request: Request):
|
||||||
metadata={"user_id": str(khoj_user.uuid)},
|
metadata={"user_id": str(khoj_user.uuid)},
|
||||||
)
|
)
|
||||||
logger.log(logging.INFO, f"New User Created: {khoj_user.uuid}")
|
logger.log(logging.INFO, f"New User Created: {khoj_user.uuid}")
|
||||||
RedirectResponse(url="/?status=welcome")
|
return RedirectResponse(url=f"{next_url}", status_code=HTTP_302_FOUND)
|
||||||
|
|
||||||
return RedirectResponse(url="/")
|
return RedirectResponse(url=f"{next_url}")
|
||||||
|
|
||||||
|
|
||||||
@auth_router.get("/logout")
|
@auth_router.get("/logout")
|
||||||
|
|
|
||||||
|
|
@ -10,10 +10,11 @@ import openai
|
||||||
from fastapi import Depends, Header, HTTPException, Request, UploadFile
|
from fastapi import Depends, Header, HTTPException, Request, UploadFile
|
||||||
from starlette.authentication import has_required_scope
|
from starlette.authentication import has_required_scope
|
||||||
|
|
||||||
from khoj.database.adapters import ConversationAdapters, EntryAdapters
|
from khoj.database.adapters import AgentAdapters, ConversationAdapters, EntryAdapters
|
||||||
from khoj.database.models import (
|
from khoj.database.models import (
|
||||||
ChatModelOptions,
|
ChatModelOptions,
|
||||||
ClientApplication,
|
ClientApplication,
|
||||||
|
Conversation,
|
||||||
KhojUser,
|
KhojUser,
|
||||||
Subscription,
|
Subscription,
|
||||||
TextToImageModelConfig,
|
TextToImageModelConfig,
|
||||||
|
|
@ -407,6 +408,7 @@ async def send_message_to_model_wrapper(
|
||||||
def generate_chat_response(
|
def generate_chat_response(
|
||||||
q: str,
|
q: str,
|
||||||
meta_log: dict,
|
meta_log: dict,
|
||||||
|
conversation: Conversation,
|
||||||
compiled_references: List[str] = [],
|
compiled_references: List[str] = [],
|
||||||
online_results: Dict[str, Dict] = {},
|
online_results: Dict[str, Dict] = {},
|
||||||
inferred_queries: List[str] = [],
|
inferred_queries: List[str] = [],
|
||||||
|
|
@ -422,6 +424,7 @@ def generate_chat_response(
|
||||||
logger.debug(f"Conversation Types: {conversation_commands}")
|
logger.debug(f"Conversation Types: {conversation_commands}")
|
||||||
|
|
||||||
metadata = {}
|
metadata = {}
|
||||||
|
agent = AgentAdapters.get_conversation_agent_by_id(conversation.agent.id) if conversation.agent else None
|
||||||
|
|
||||||
try:
|
try:
|
||||||
partial_completion = partial(
|
partial_completion = partial(
|
||||||
|
|
@ -436,7 +439,7 @@ def generate_chat_response(
|
||||||
conversation_id=conversation_id,
|
conversation_id=conversation_id,
|
||||||
)
|
)
|
||||||
|
|
||||||
conversation_config = ConversationAdapters.get_valid_conversation_config(user)
|
conversation_config = ConversationAdapters.get_valid_conversation_config(user, conversation)
|
||||||
if conversation_config.model_type == "offline":
|
if conversation_config.model_type == "offline":
|
||||||
if state.offline_chat_processor_config is None or state.offline_chat_processor_config.loaded_model is None:
|
if state.offline_chat_processor_config is None or state.offline_chat_processor_config.loaded_model is None:
|
||||||
state.offline_chat_processor_config = OfflineChatProcessorModel(conversation_config.chat_model)
|
state.offline_chat_processor_config = OfflineChatProcessorModel(conversation_config.chat_model)
|
||||||
|
|
@ -455,6 +458,7 @@ def generate_chat_response(
|
||||||
tokenizer_name=conversation_config.tokenizer,
|
tokenizer_name=conversation_config.tokenizer,
|
||||||
location_data=location_data,
|
location_data=location_data,
|
||||||
user_name=user_name,
|
user_name=user_name,
|
||||||
|
agent=agent,
|
||||||
)
|
)
|
||||||
|
|
||||||
elif conversation_config.model_type == "openai":
|
elif conversation_config.model_type == "openai":
|
||||||
|
|
@ -474,6 +478,7 @@ def generate_chat_response(
|
||||||
tokenizer_name=conversation_config.tokenizer,
|
tokenizer_name=conversation_config.tokenizer,
|
||||||
location_data=location_data,
|
location_data=location_data,
|
||||||
user_name=user_name,
|
user_name=user_name,
|
||||||
|
agent=agent,
|
||||||
)
|
)
|
||||||
|
|
||||||
metadata.update({"chat_model": conversation_config.chat_model})
|
metadata.update({"chat_model": conversation_config.chat_model})
|
||||||
|
|
|
||||||
|
|
@ -10,6 +10,7 @@ from starlette.authentication import has_required_scope, requires
|
||||||
|
|
||||||
from khoj.database import adapters
|
from khoj.database import adapters
|
||||||
from khoj.database.adapters import (
|
from khoj.database.adapters import (
|
||||||
|
AgentAdapters,
|
||||||
ConversationAdapters,
|
ConversationAdapters,
|
||||||
EntryAdapters,
|
EntryAdapters,
|
||||||
get_user_github_config,
|
get_user_github_config,
|
||||||
|
|
@ -114,8 +115,8 @@ def chat_page(request: Request):
|
||||||
|
|
||||||
@web_client.get("/login", response_class=FileResponse)
|
@web_client.get("/login", response_class=FileResponse)
|
||||||
def login_page(request: Request):
|
def login_page(request: Request):
|
||||||
if request.user.is_authenticated:
|
|
||||||
next_url = request.query_params.get("next", "/")
|
next_url = request.query_params.get("next", "/")
|
||||||
|
if request.user.is_authenticated:
|
||||||
return RedirectResponse(url=next_url)
|
return RedirectResponse(url=next_url)
|
||||||
google_client_id = os.environ.get("GOOGLE_CLIENT_ID")
|
google_client_id = os.environ.get("GOOGLE_CLIENT_ID")
|
||||||
redirect_uri = str(request.app.url_path_for("auth"))
|
redirect_uri = str(request.app.url_path_for("auth"))
|
||||||
|
|
@ -124,7 +125,85 @@ def login_page(request: Request):
|
||||||
context={
|
context={
|
||||||
"request": request,
|
"request": request,
|
||||||
"google_client_id": google_client_id,
|
"google_client_id": google_client_id,
|
||||||
"redirect_uri": redirect_uri,
|
"redirect_uri": f"{redirect_uri}?next={next_url}",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@web_client.get("/agents", response_class=HTMLResponse)
|
||||||
|
def agents_page(request: Request):
|
||||||
|
user: KhojUser = request.user.object if request.user.is_authenticated else None
|
||||||
|
user_picture = request.session.get("user", {}).get("picture") if user else None
|
||||||
|
agents = AgentAdapters.get_all_accessible_agents(user)
|
||||||
|
agents_packet = list()
|
||||||
|
for agent in agents:
|
||||||
|
agents_packet.append(
|
||||||
|
{
|
||||||
|
"slug": agent.slug,
|
||||||
|
"avatar": agent.avatar,
|
||||||
|
"name": agent.name,
|
||||||
|
"personality": agent.personality,
|
||||||
|
"public": agent.public,
|
||||||
|
"creator": agent.creator.username if agent.creator else None,
|
||||||
|
"managed_by_admin": agent.managed_by_admin,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return templates.TemplateResponse(
|
||||||
|
"agents.html",
|
||||||
|
context={
|
||||||
|
"request": request,
|
||||||
|
"agents": agents_packet,
|
||||||
|
"khoj_version": state.khoj_version,
|
||||||
|
"username": user.username if user else None,
|
||||||
|
"has_documents": False,
|
||||||
|
"is_active": has_required_scope(request, ["premium"]),
|
||||||
|
"user_photo": user_picture,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@web_client.get("/agent/{agent_slug}", response_class=HTMLResponse)
|
||||||
|
def agent_page(request: Request, agent_slug: str):
|
||||||
|
user: KhojUser = request.user.object if request.user.is_authenticated else None
|
||||||
|
user_picture = request.session.get("user", {}).get("picture") if user else None
|
||||||
|
|
||||||
|
agent = AgentAdapters.get_agent_by_slug(agent_slug)
|
||||||
|
|
||||||
|
if agent == None:
|
||||||
|
return templates.TemplateResponse(
|
||||||
|
"404.html",
|
||||||
|
context={
|
||||||
|
"request": request,
|
||||||
|
"khoj_version": state.khoj_version,
|
||||||
|
"username": user.username if user else None,
|
||||||
|
"has_documents": False,
|
||||||
|
"is_active": has_required_scope(request, ["premium"]),
|
||||||
|
"user_photo": user_picture,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
agent_metadata = {
|
||||||
|
"slug": agent.slug,
|
||||||
|
"avatar": agent.avatar,
|
||||||
|
"name": agent.name,
|
||||||
|
"personality": agent.personality,
|
||||||
|
"public": agent.public,
|
||||||
|
"creator": agent.creator.username if agent.creator else None,
|
||||||
|
"managed_by_admin": agent.managed_by_admin,
|
||||||
|
"chat_model": agent.chat_model.chat_model,
|
||||||
|
"creator_not_self": agent.creator != user,
|
||||||
|
}
|
||||||
|
|
||||||
|
return templates.TemplateResponse(
|
||||||
|
"agent.html",
|
||||||
|
context={
|
||||||
|
"request": request,
|
||||||
|
"agent": agent_metadata,
|
||||||
|
"khoj_version": state.khoj_version,
|
||||||
|
"username": user.username if user else None,
|
||||||
|
"has_documents": False,
|
||||||
|
"is_active": has_required_scope(request, ["premium"]),
|
||||||
|
"user_photo": user_picture,
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -12,6 +12,7 @@ from khoj.configure import (
|
||||||
configure_search_types,
|
configure_search_types,
|
||||||
)
|
)
|
||||||
from khoj.database.models import (
|
from khoj.database.models import (
|
||||||
|
Agent,
|
||||||
GithubConfig,
|
GithubConfig,
|
||||||
GithubRepoConfig,
|
GithubRepoConfig,
|
||||||
KhojApiUser,
|
KhojApiUser,
|
||||||
|
|
@ -181,6 +182,28 @@ def api_user4(default_user4):
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.django_db
|
||||||
|
@pytest.fixture
|
||||||
|
def offline_agent():
|
||||||
|
chat_model = ChatModelOptionsFactory()
|
||||||
|
return Agent.objects.create(
|
||||||
|
name="Accountant",
|
||||||
|
chat_model=chat_model,
|
||||||
|
personality="You are a certified CPA. You are able to tell me how much I've spent based on my notes. Regardless of what I ask, you should always respond with the total amount I've spent. ALWAYS RESPOND WITH A SUMMARY TOTAL OF HOW MUCH MONEY I HAVE SPENT.",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.django_db
|
||||||
|
@pytest.fixture
|
||||||
|
def openai_agent():
|
||||||
|
chat_model = ChatModelOptionsFactory(chat_model="gpt-3.5-turbo", model_type="openai")
|
||||||
|
return Agent.objects.create(
|
||||||
|
name="Accountant",
|
||||||
|
chat_model=chat_model,
|
||||||
|
personality="You are a certified CPA. You are able to tell me how much I've spent based on my notes. Regardless of what I ask, you should always respond with the total amount I've spent.",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(scope="session")
|
@pytest.fixture(scope="session")
|
||||||
def search_models(search_config: SearchConfig):
|
def search_models(search_config: SearchConfig):
|
||||||
search_models = SearchModels()
|
search_models = SearchModels()
|
||||||
|
|
|
||||||
|
|
@ -34,7 +34,9 @@ def test_markdown_file_with_no_headings_to_jsonl(tmp_path):
|
||||||
# Ensure raw entry with no headings do not get heading prefix prepended
|
# Ensure raw entry with no headings do not get heading prefix prepended
|
||||||
assert not jsonl_data[0]["raw"].startswith("#")
|
assert not jsonl_data[0]["raw"].startswith("#")
|
||||||
# Ensure compiled entry has filename prepended as top level heading
|
# Ensure compiled entry has filename prepended as top level heading
|
||||||
assert jsonl_data[0]["compiled"].startswith(expected_heading)
|
assert expected_heading in jsonl_data[0]["compiled"]
|
||||||
|
# Ensure compiled entry also includes the file name
|
||||||
|
assert str(tmp_path) in jsonl_data[0]["compiled"]
|
||||||
|
|
||||||
|
|
||||||
def test_single_markdown_entry_to_jsonl(tmp_path):
|
def test_single_markdown_entry_to_jsonl(tmp_path):
|
||||||
|
|
|
||||||
|
|
@ -467,6 +467,47 @@ My sister, Aiyla is married to Tolga. They have 3 kids, Yildiz, Ali and Ahmet.""
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------------------------------------------
|
||||||
|
@pytest.mark.chatquality
|
||||||
|
def test_agent_prompt_should_be_used(loaded_model, offline_agent):
|
||||||
|
"Chat actor should ask be tuned to think like an accountant based on the agent definition"
|
||||||
|
# Arrange
|
||||||
|
context = [
|
||||||
|
f"""I went to the store and bought some bananas for 2.20""",
|
||||||
|
f"""I went to the store and bought some apples for 1.30""",
|
||||||
|
f"""I went to the store and bought some oranges for 6.00""",
|
||||||
|
]
|
||||||
|
|
||||||
|
# Act
|
||||||
|
response_gen = converse_offline(
|
||||||
|
references=context, # Assume context retrieved from notes for the user_query
|
||||||
|
user_query="What did I buy?",
|
||||||
|
loaded_model=loaded_model,
|
||||||
|
)
|
||||||
|
response = "".join([response_chunk for response_chunk in response_gen])
|
||||||
|
|
||||||
|
# Assert that the model without the agent prompt does not include the summary of purchases
|
||||||
|
expected_responses = ["9.50", "9.5"]
|
||||||
|
assert all([expected_response not in response for expected_response in expected_responses]), (
|
||||||
|
"Expected chat actor to summarize values of purchases" + response
|
||||||
|
)
|
||||||
|
|
||||||
|
# Act
|
||||||
|
response_gen = converse_offline(
|
||||||
|
references=context, # Assume context retrieved from notes for the user_query
|
||||||
|
user_query="What did I buy?",
|
||||||
|
loaded_model=loaded_model,
|
||||||
|
agent=offline_agent,
|
||||||
|
)
|
||||||
|
response = "".join([response_chunk for response_chunk in response_gen])
|
||||||
|
|
||||||
|
# Assert that the model with the agent prompt does include the summary of purchases
|
||||||
|
expected_responses = ["9.50", "9.5"]
|
||||||
|
assert any([expected_response in response for expected_response in expected_responses]), (
|
||||||
|
"Expected chat actor to summarize values of purchases" + response
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
# ----------------------------------------------------------------------------------------------------
|
# ----------------------------------------------------------------------------------------------------
|
||||||
def test_chat_does_not_exceed_prompt_size(loaded_model):
|
def test_chat_does_not_exceed_prompt_size(loaded_model):
|
||||||
"Ensure chat context and response together do not exceed max prompt size for the model"
|
"Ensure chat context and response together do not exceed max prompt size for the model"
|
||||||
|
|
|
||||||
|
|
@ -6,6 +6,7 @@ import pytest
|
||||||
from faker import Faker
|
from faker import Faker
|
||||||
from freezegun import freeze_time
|
from freezegun import freeze_time
|
||||||
|
|
||||||
|
from khoj.database.models import Agent, KhojUser
|
||||||
from khoj.processor.conversation import prompts
|
from khoj.processor.conversation import prompts
|
||||||
from khoj.processor.conversation.utils import message_to_log
|
from khoj.processor.conversation.utils import message_to_log
|
||||||
from khoj.routers.helpers import aget_relevant_information_sources
|
from khoj.routers.helpers import aget_relevant_information_sources
|
||||||
|
|
@ -26,20 +27,20 @@ def generate_history(message_list):
|
||||||
# Generate conversation logs
|
# Generate conversation logs
|
||||||
conversation_log = {"chat": []}
|
conversation_log = {"chat": []}
|
||||||
for user_message, gpt_message, context in message_list:
|
for user_message, gpt_message, context in message_list:
|
||||||
conversation_log["chat"] += message_to_log(
|
message_to_log(
|
||||||
user_message,
|
user_message,
|
||||||
gpt_message,
|
gpt_message,
|
||||||
{"context": context, "intent": {"query": user_message, "inferred-queries": f'["{user_message}"]'}},
|
{"context": context, "intent": {"query": user_message, "inferred-queries": f'["{user_message}"]'}},
|
||||||
|
conversation_log=conversation_log.get("chat", []),
|
||||||
)
|
)
|
||||||
return conversation_log
|
return conversation_log
|
||||||
|
|
||||||
|
|
||||||
def populate_chat_history(message_list, user):
|
def create_conversation(message_list, user, agent=None):
|
||||||
# Generate conversation logs
|
# Generate conversation logs
|
||||||
conversation_log = generate_history(message_list)
|
conversation_log = generate_history(message_list)
|
||||||
|
|
||||||
# Update Conversation Metadata Logs in Database
|
# Update Conversation Metadata Logs in Database
|
||||||
ConversationFactory(user=user, conversation_log=conversation_log)
|
return ConversationFactory(user=user, conversation_log=conversation_log, agent=agent)
|
||||||
|
|
||||||
|
|
||||||
# Tests
|
# Tests
|
||||||
|
|
@ -114,7 +115,7 @@ def test_answer_from_chat_history(client_offline_chat, default_user2):
|
||||||
("Hello, my name is Testatron. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
("Hello, my name is Testatron. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
||||||
("When was I born?", "You were born on 1st April 1984.", []),
|
("When was I born?", "You were born on 1st April 1984.", []),
|
||||||
]
|
]
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = client_offline_chat.get(f'/api/chat?q="What is my name?"&stream=true')
|
response = client_offline_chat.get(f'/api/chat?q="What is my name?"&stream=true')
|
||||||
|
|
@ -141,7 +142,7 @@ def test_answer_from_currently_retrieved_content(client_offline_chat, default_us
|
||||||
["Testatron was born on 1st April 1984 in Testville."],
|
["Testatron was born on 1st April 1984 in Testville."],
|
||||||
),
|
),
|
||||||
]
|
]
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = client_offline_chat.get(f'/api/chat?q="Where was Xi Li born?"')
|
response = client_offline_chat.get(f'/api/chat?q="Where was Xi Li born?"')
|
||||||
|
|
@ -165,7 +166,7 @@ def test_answer_from_chat_history_and_previously_retrieved_content(client_offlin
|
||||||
["Testatron was born on 1st April 1984 in Testville."],
|
["Testatron was born on 1st April 1984 in Testville."],
|
||||||
),
|
),
|
||||||
]
|
]
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = client_offline_chat.get(f'/api/chat?q="Where was I born?"')
|
response = client_offline_chat.get(f'/api/chat?q="Where was I born?"')
|
||||||
|
|
@ -191,7 +192,7 @@ def test_answer_from_chat_history_and_currently_retrieved_content(client_offline
|
||||||
("Hello, my name is Xi Li. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
("Hello, my name is Xi Li. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
||||||
("When was I born?", "You were born on 1st April 1984.", []),
|
("When was I born?", "You were born on 1st April 1984.", []),
|
||||||
]
|
]
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = client_offline_chat.get(f'/api/chat?q="Where was I born?"')
|
response = client_offline_chat.get(f'/api/chat?q="Where was I born?"')
|
||||||
|
|
@ -217,7 +218,7 @@ def test_no_answer_in_chat_history_or_retrieved_content(client_offline_chat, def
|
||||||
("Hello, my name is Testatron. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
("Hello, my name is Testatron. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
||||||
("When was I born?", "You were born on 1st April 1984.", []),
|
("When was I born?", "You were born on 1st April 1984.", []),
|
||||||
]
|
]
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = client_offline_chat.get(f'/api/chat?q="Where was I born?"&stream=true')
|
response = client_offline_chat.get(f'/api/chat?q="Where was I born?"&stream=true')
|
||||||
|
|
@ -238,7 +239,7 @@ def test_answer_using_general_command(client_offline_chat, default_user2):
|
||||||
# Arrange
|
# Arrange
|
||||||
query = urllib.parse.quote("/general Where was Xi Li born?")
|
query = urllib.parse.quote("/general Where was Xi Li born?")
|
||||||
message_list = []
|
message_list = []
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = client_offline_chat.get(f"/api/chat?q={query}&stream=true")
|
response = client_offline_chat.get(f"/api/chat?q={query}&stream=true")
|
||||||
|
|
@ -256,7 +257,7 @@ def test_answer_from_retrieved_content_using_notes_command(client_offline_chat,
|
||||||
# Arrange
|
# Arrange
|
||||||
query = urllib.parse.quote("/notes Where was Xi Li born?")
|
query = urllib.parse.quote("/notes Where was Xi Li born?")
|
||||||
message_list = []
|
message_list = []
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = client_offline_chat.get(f"/api/chat?q={query}&stream=true")
|
response = client_offline_chat.get(f"/api/chat?q={query}&stream=true")
|
||||||
|
|
@ -275,7 +276,7 @@ def test_answer_using_file_filter(client_offline_chat, default_user2):
|
||||||
no_answer_query = urllib.parse.quote('Where was Xi Li born? file:"Namita.markdown"')
|
no_answer_query = urllib.parse.quote('Where was Xi Li born? file:"Namita.markdown"')
|
||||||
answer_query = urllib.parse.quote('Where was Xi Li born? file:"Xi Li.markdown"')
|
answer_query = urllib.parse.quote('Where was Xi Li born? file:"Xi Li.markdown"')
|
||||||
message_list = []
|
message_list = []
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
no_answer_response = client_offline_chat.get(f"/api/chat?q={no_answer_query}&stream=true").content.decode("utf-8")
|
no_answer_response = client_offline_chat.get(f"/api/chat?q={no_answer_query}&stream=true").content.decode("utf-8")
|
||||||
|
|
@ -293,7 +294,7 @@ def test_answer_not_known_using_notes_command(client_offline_chat, default_user2
|
||||||
# Arrange
|
# Arrange
|
||||||
query = urllib.parse.quote("/notes Where was Testatron born?")
|
query = urllib.parse.quote("/notes Where was Testatron born?")
|
||||||
message_list = []
|
message_list = []
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = client_offline_chat.get(f"/api/chat?q={query}&stream=true")
|
response = client_offline_chat.get(f"/api/chat?q={query}&stream=true")
|
||||||
|
|
@ -351,7 +352,7 @@ def test_answer_general_question_not_in_chat_history_or_retrieved_content(client
|
||||||
("When was I born?", "You were born on 1st April 1984.", []),
|
("When was I born?", "You were born on 1st April 1984.", []),
|
||||||
("Where was I born?", "You were born Testville.", []),
|
("Where was I born?", "You were born Testville.", []),
|
||||||
]
|
]
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = client_offline_chat.get(
|
response = client_offline_chat.get(
|
||||||
|
|
@ -394,14 +395,14 @@ def test_ask_for_clarification_if_not_enough_context_in_question(client_offline_
|
||||||
@pytest.mark.xfail(reason="Chat director not capable of answering this question yet")
|
@pytest.mark.xfail(reason="Chat director not capable of answering this question yet")
|
||||||
@pytest.mark.chatquality
|
@pytest.mark.chatquality
|
||||||
@pytest.mark.django_db(transaction=True)
|
@pytest.mark.django_db(transaction=True)
|
||||||
def test_answer_in_chat_history_beyond_lookback_window(client_offline_chat, default_user2):
|
def test_answer_in_chat_history_beyond_lookback_window(client_offline_chat, default_user2: KhojUser):
|
||||||
# Arrange
|
# Arrange
|
||||||
message_list = [
|
message_list = [
|
||||||
("Hello, my name is Testatron. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
("Hello, my name is Testatron. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
||||||
("When was I born?", "You were born on 1st April 1984.", []),
|
("When was I born?", "You were born on 1st April 1984.", []),
|
||||||
("Where was I born?", "You were born Testville.", []),
|
("Where was I born?", "You were born Testville.", []),
|
||||||
]
|
]
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = client_offline_chat.get(f'/api/chat?q="What is my name?"&stream=true')
|
response = client_offline_chat.get(f'/api/chat?q="What is my name?"&stream=true')
|
||||||
|
|
@ -415,13 +416,77 @@ def test_answer_in_chat_history_beyond_lookback_window(client_offline_chat, defa
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------------------------------------------
|
||||||
|
@pytest.mark.chatquality
|
||||||
|
@pytest.mark.django_db(transaction=True)
|
||||||
|
def test_answer_in_chat_history_by_conversation_id(client_offline_chat, default_user2: KhojUser):
|
||||||
|
# Arrange
|
||||||
|
message_list = [
|
||||||
|
("Hello, my name is Testatron. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
||||||
|
("When was I born?", "You were born on 1st April 1984.", []),
|
||||||
|
("What's my favorite color", "Your favorite color is green.", []),
|
||||||
|
("Where was I born?", "You were born Testville.", []),
|
||||||
|
]
|
||||||
|
message_list2 = [
|
||||||
|
("Hello, my name is Julia. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
||||||
|
("When was I born?", "You were born on 14th August 1947.", []),
|
||||||
|
("What's my favorite color", "Your favorite color is maroon.", []),
|
||||||
|
("Where was I born?", "You were born in a potato farm.", []),
|
||||||
|
]
|
||||||
|
conversation = create_conversation(message_list, default_user2)
|
||||||
|
create_conversation(message_list2, default_user2)
|
||||||
|
|
||||||
|
# Act
|
||||||
|
response = client_offline_chat.get(
|
||||||
|
f'/api/chat?q="What is my favorite color?"&conversation_id={conversation.id}&stream=true'
|
||||||
|
)
|
||||||
|
response_message = response.content.decode("utf-8")
|
||||||
|
|
||||||
|
# Assert
|
||||||
|
expected_responses = ["green"]
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert any([expected_response in response_message.lower() for expected_response in expected_responses]), (
|
||||||
|
"Expected green in response, but got: " + response_message
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------------------------------------------
|
||||||
|
@pytest.mark.xfail(reason="Chat director not great at adhering to agent instructions yet")
|
||||||
|
@pytest.mark.chatquality
|
||||||
|
@pytest.mark.django_db(transaction=True)
|
||||||
|
def test_answer_in_chat_history_by_conversation_id_with_agent(
|
||||||
|
client_offline_chat, default_user2: KhojUser, offline_agent: Agent
|
||||||
|
):
|
||||||
|
# Arrange
|
||||||
|
message_list = [
|
||||||
|
("Hello, my name is Testatron. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
||||||
|
("When was I born?", "You were born on 1st April 1984.", []),
|
||||||
|
("What's my favorite color", "Your favorite color is green.", []),
|
||||||
|
("Where was I born?", "You were born Testville.", []),
|
||||||
|
("What did I buy?", "You bought an apple for 2.00, an orange for 3.00, and a potato for 8.00", []),
|
||||||
|
]
|
||||||
|
conversation = create_conversation(message_list, default_user2, offline_agent)
|
||||||
|
|
||||||
|
# Act
|
||||||
|
query = urllib.parse.quote("/general What did I eat for breakfast?")
|
||||||
|
response = client_offline_chat.get(f"/api/chat?q={query}&conversation_id={conversation.id}&stream=true")
|
||||||
|
response_message = response.content.decode("utf-8")
|
||||||
|
|
||||||
|
# Assert that agent only responds with the summary of spending
|
||||||
|
expected_responses = ["13.00", "13", "13.0", "thirteen"]
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert any([expected_response in response_message.lower() for expected_response in expected_responses]), (
|
||||||
|
"Expected green in response, but got: " + response_message
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.chatquality
|
@pytest.mark.chatquality
|
||||||
@pytest.mark.django_db(transaction=True)
|
@pytest.mark.django_db(transaction=True)
|
||||||
def test_answer_chat_history_very_long(client_offline_chat, default_user2):
|
def test_answer_chat_history_very_long(client_offline_chat, default_user2):
|
||||||
# Arrange
|
# Arrange
|
||||||
message_list = [(" ".join([fake.paragraph() for _ in range(50)]), fake.sentence(), []) for _ in range(10)]
|
message_list = [(" ".join([fake.paragraph() for _ in range(50)]), fake.sentence(), []) for _ in range(10)]
|
||||||
|
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = client_offline_chat.get(f'/api/chat?q="What is my name?"&stream=true')
|
response = client_offline_chat.get(f'/api/chat?q="What is my name?"&stream=true')
|
||||||
|
|
@ -525,7 +590,7 @@ async def test_get_correct_tools_with_chat_history(client_offline_chat, default_
|
||||||
),
|
),
|
||||||
("What's up in New York City?", "A Pride parade has recently been held in New York City, on July 31st.", []),
|
("What's up in New York City?", "A Pride parade has recently been held in New York City, on July 31st.", []),
|
||||||
]
|
]
|
||||||
chat_history = populate_chat_history(chat_log, default_user2)
|
chat_history = create_conversation(chat_log, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
tools = await aget_relevant_information_sources(user_query, chat_history)
|
tools = await aget_relevant_information_sources(user_query, chat_history)
|
||||||
|
|
|
||||||
|
|
@ -414,6 +414,42 @@ My sister, Aiyla is married to Tolga. They have 3 kids, Yildiz, Ali and Ahmet.""
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------------------------------------------
|
||||||
|
@pytest.mark.chatquality
|
||||||
|
def test_agent_prompt_should_be_used(openai_agent):
|
||||||
|
"Chat actor should ask be tuned to think like an accountant based on the agent definition"
|
||||||
|
# Arrange
|
||||||
|
context = [
|
||||||
|
f"""I went to the store and bought some bananas for 2.20""",
|
||||||
|
f"""I went to the store and bought some apples for 1.30""",
|
||||||
|
f"""I went to the store and bought some oranges for 6.00""",
|
||||||
|
]
|
||||||
|
expected_responses = ["9.50", "9.5"]
|
||||||
|
|
||||||
|
# Act
|
||||||
|
response_gen = converse(
|
||||||
|
references=context, # Assume context retrieved from notes for the user_query
|
||||||
|
user_query="What did I buy?",
|
||||||
|
api_key=api_key,
|
||||||
|
)
|
||||||
|
no_agent_response = "".join([response_chunk for response_chunk in response_gen])
|
||||||
|
response_gen = converse(
|
||||||
|
references=context, # Assume context retrieved from notes for the user_query
|
||||||
|
user_query="What did I buy?",
|
||||||
|
api_key=api_key,
|
||||||
|
agent=openai_agent,
|
||||||
|
)
|
||||||
|
agent_response = "".join([response_chunk for response_chunk in response_gen])
|
||||||
|
|
||||||
|
# Assert that the model without the agent prompt does not include the summary of purchases
|
||||||
|
assert all([expected_response not in no_agent_response for expected_response in expected_responses]), (
|
||||||
|
"Expected chat actor to summarize values of purchases" + no_agent_response
|
||||||
|
)
|
||||||
|
assert any([expected_response in agent_response for expected_response in expected_responses]), (
|
||||||
|
"Expected chat actor to summarize values of purchases" + agent_response
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
# ----------------------------------------------------------------------------------------------------
|
# ----------------------------------------------------------------------------------------------------
|
||||||
@pytest.mark.anyio
|
@pytest.mark.anyio
|
||||||
@pytest.mark.django_db(transaction=True)
|
@pytest.mark.django_db(transaction=True)
|
||||||
|
|
|
||||||
|
|
@ -5,13 +5,10 @@ from urllib.parse import quote
|
||||||
import pytest
|
import pytest
|
||||||
from freezegun import freeze_time
|
from freezegun import freeze_time
|
||||||
|
|
||||||
from khoj.database.models import KhojUser
|
from khoj.database.models import Agent, KhojUser
|
||||||
from khoj.processor.conversation import prompts
|
from khoj.processor.conversation import prompts
|
||||||
from khoj.processor.conversation.utils import message_to_log
|
from khoj.processor.conversation.utils import message_to_log
|
||||||
from khoj.routers.helpers import (
|
from khoj.routers.helpers import aget_relevant_information_sources
|
||||||
aget_relevant_information_sources,
|
|
||||||
aget_relevant_output_modes,
|
|
||||||
)
|
|
||||||
from tests.helpers import ConversationFactory
|
from tests.helpers import ConversationFactory
|
||||||
|
|
||||||
# Initialize variables for tests
|
# Initialize variables for tests
|
||||||
|
|
@ -29,20 +26,21 @@ def generate_history(message_list):
|
||||||
# Generate conversation logs
|
# Generate conversation logs
|
||||||
conversation_log = {"chat": []}
|
conversation_log = {"chat": []}
|
||||||
for user_message, gpt_message, context in message_list:
|
for user_message, gpt_message, context in message_list:
|
||||||
conversation_log["chat"] += message_to_log(
|
message_to_log(
|
||||||
user_message,
|
user_message,
|
||||||
gpt_message,
|
gpt_message,
|
||||||
{"context": context, "intent": {"query": user_message, "inferred-queries": f'["{user_message}"]'}},
|
{"context": context, "intent": {"query": user_message, "inferred-queries": f'["{user_message}"]'}},
|
||||||
|
conversation_log=conversation_log.get("chat", []),
|
||||||
)
|
)
|
||||||
return conversation_log
|
return conversation_log
|
||||||
|
|
||||||
|
|
||||||
def populate_chat_history(message_list, user):
|
def create_conversation(message_list, user, agent=None):
|
||||||
# Generate conversation logs
|
# Generate conversation logs
|
||||||
conversation_log = generate_history(message_list)
|
conversation_log = generate_history(message_list)
|
||||||
|
|
||||||
# Update Conversation Metadata Logs in Database
|
# Update Conversation Metadata Logs in Database
|
||||||
ConversationFactory(user=user, conversation_log=conversation_log)
|
return ConversationFactory(user=user, conversation_log=conversation_log, agent=agent)
|
||||||
|
|
||||||
|
|
||||||
# Tests
|
# Tests
|
||||||
|
|
@ -116,7 +114,7 @@ def test_answer_from_chat_history(chat_client, default_user2: KhojUser):
|
||||||
("Hello, my name is Testatron. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
("Hello, my name is Testatron. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
||||||
("When was I born?", "You were born on 1st April 1984.", []),
|
("When was I born?", "You were born on 1st April 1984.", []),
|
||||||
]
|
]
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = chat_client.get(f'/api/chat?q="What is my name?"&stream=true')
|
response = chat_client.get(f'/api/chat?q="What is my name?"&stream=true')
|
||||||
|
|
@ -143,7 +141,7 @@ def test_answer_from_currently_retrieved_content(chat_client, default_user2: Kho
|
||||||
["Testatron was born on 1st April 1984 in Testville."],
|
["Testatron was born on 1st April 1984 in Testville."],
|
||||||
),
|
),
|
||||||
]
|
]
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = chat_client.get(f'/api/chat?q="Where was Xi Li born?"')
|
response = chat_client.get(f'/api/chat?q="Where was Xi Li born?"')
|
||||||
|
|
@ -167,7 +165,7 @@ def test_answer_from_chat_history_and_previously_retrieved_content(chat_client_n
|
||||||
["Testatron was born on 1st April 1984 in Testville."],
|
["Testatron was born on 1st April 1984 in Testville."],
|
||||||
),
|
),
|
||||||
]
|
]
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = chat_client_no_background.get(f'/api/chat?q="Where was I born?"')
|
response = chat_client_no_background.get(f'/api/chat?q="Where was I born?"')
|
||||||
|
|
@ -190,7 +188,7 @@ def test_answer_from_chat_history_and_currently_retrieved_content(chat_client, d
|
||||||
("Hello, my name is Xi Li. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
("Hello, my name is Xi Li. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
||||||
("When was I born?", "You were born on 1st April 1984.", []),
|
("When was I born?", "You were born on 1st April 1984.", []),
|
||||||
]
|
]
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = chat_client.get(f'/api/chat?q="Where was I born?"')
|
response = chat_client.get(f'/api/chat?q="Where was I born?"')
|
||||||
|
|
@ -215,7 +213,7 @@ def test_no_answer_in_chat_history_or_retrieved_content(chat_client, default_use
|
||||||
("Hello, my name is Testatron. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
("Hello, my name is Testatron. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
||||||
("When was I born?", "You were born on 1st April 1984.", []),
|
("When was I born?", "You were born on 1st April 1984.", []),
|
||||||
]
|
]
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = chat_client.get(f'/api/chat?q="Where was I born?"&stream=true')
|
response = chat_client.get(f'/api/chat?q="Where was I born?"&stream=true')
|
||||||
|
|
@ -244,7 +242,7 @@ def test_answer_using_general_command(chat_client, default_user2: KhojUser):
|
||||||
# Arrange
|
# Arrange
|
||||||
query = urllib.parse.quote("/general Where was Xi Li born?")
|
query = urllib.parse.quote("/general Where was Xi Li born?")
|
||||||
message_list = []
|
message_list = []
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = chat_client.get(f"/api/chat?q={query}&stream=true")
|
response = chat_client.get(f"/api/chat?q={query}&stream=true")
|
||||||
|
|
@ -262,7 +260,7 @@ def test_answer_from_retrieved_content_using_notes_command(chat_client, default_
|
||||||
# Arrange
|
# Arrange
|
||||||
query = urllib.parse.quote("/notes Where was Xi Li born?")
|
query = urllib.parse.quote("/notes Where was Xi Li born?")
|
||||||
message_list = []
|
message_list = []
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = chat_client.get(f"/api/chat?q={query}&stream=true")
|
response = chat_client.get(f"/api/chat?q={query}&stream=true")
|
||||||
|
|
@ -280,7 +278,7 @@ def test_answer_not_known_using_notes_command(chat_client_no_background, default
|
||||||
# Arrange
|
# Arrange
|
||||||
query = urllib.parse.quote("/notes Where was Testatron born?")
|
query = urllib.parse.quote("/notes Where was Testatron born?")
|
||||||
message_list = []
|
message_list = []
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = chat_client_no_background.get(f"/api/chat?q={query}&stream=true")
|
response = chat_client_no_background.get(f"/api/chat?q={query}&stream=true")
|
||||||
|
|
@ -335,7 +333,7 @@ def test_answer_general_question_not_in_chat_history_or_retrieved_content(chat_c
|
||||||
("When was I born?", "You were born on 1st April 1984.", []),
|
("When was I born?", "You were born on 1st April 1984.", []),
|
||||||
("Where was I born?", "You were born Testville.", []),
|
("Where was I born?", "You were born Testville.", []),
|
||||||
]
|
]
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = chat_client.get(f'/api/chat?q="Write a haiku about unit testing. Do not say anything else."&stream=true')
|
response = chat_client.get(f'/api/chat?q="Write a haiku about unit testing. Do not say anything else."&stream=true')
|
||||||
|
|
@ -387,7 +385,7 @@ def test_answer_in_chat_history_beyond_lookback_window(chat_client, default_user
|
||||||
("When was I born?", "You were born on 1st April 1984.", []),
|
("When was I born?", "You were born on 1st April 1984.", []),
|
||||||
("Where was I born?", "You were born Testville.", []),
|
("Where was I born?", "You were born Testville.", []),
|
||||||
]
|
]
|
||||||
populate_chat_history(message_list, default_user2)
|
create_conversation(message_list, default_user2)
|
||||||
|
|
||||||
# Act
|
# Act
|
||||||
response = chat_client.get(f'/api/chat?q="What is my name?"&stream=true')
|
response = chat_client.get(f'/api/chat?q="What is my name?"&stream=true')
|
||||||
|
|
@ -401,6 +399,68 @@ def test_answer_in_chat_history_beyond_lookback_window(chat_client, default_user
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------------------------------------------
|
||||||
|
@pytest.mark.chatquality
|
||||||
|
@pytest.mark.django_db(transaction=True)
|
||||||
|
def test_answer_in_chat_history_by_conversation_id(chat_client, default_user2: KhojUser):
|
||||||
|
# Arrange
|
||||||
|
message_list = [
|
||||||
|
("Hello, my name is Testatron. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
||||||
|
("When was I born?", "You were born on 1st April 1984.", []),
|
||||||
|
("What's my favorite color", "Your favorite color is green.", []),
|
||||||
|
("Where was I born?", "You were born Testville.", []),
|
||||||
|
]
|
||||||
|
message_list2 = [
|
||||||
|
("Hello, my name is Julia. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
||||||
|
("When was I born?", "You were born on 14th August 1947.", []),
|
||||||
|
("What's my favorite color", "Your favorite color is maroon.", []),
|
||||||
|
("Where was I born?", "You were born in a potato farm.", []),
|
||||||
|
]
|
||||||
|
conversation = create_conversation(message_list, default_user2)
|
||||||
|
create_conversation(message_list2, default_user2)
|
||||||
|
|
||||||
|
# Act
|
||||||
|
query = urllib.parse.quote("/general What is my favorite color?")
|
||||||
|
response = chat_client.get(f"/api/chat?q={query}&conversation_id={conversation.id}&stream=true")
|
||||||
|
response_message = response.content.decode("utf-8")
|
||||||
|
|
||||||
|
# Assert
|
||||||
|
expected_responses = ["green"]
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert any([expected_response in response_message.lower() for expected_response in expected_responses]), (
|
||||||
|
"Expected green in response, but got: " + response_message
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------------------------------------------
|
||||||
|
@pytest.mark.chatquality
|
||||||
|
@pytest.mark.django_db(transaction=True)
|
||||||
|
def test_answer_in_chat_history_by_conversation_id_with_agent(
|
||||||
|
chat_client, default_user2: KhojUser, openai_agent: Agent
|
||||||
|
):
|
||||||
|
# Arrange
|
||||||
|
message_list = [
|
||||||
|
("Hello, my name is Testatron. Who are you?", "Hi, I am Khoj, a personal assistant. How can I help?", []),
|
||||||
|
("When was I born?", "You were born on 1st April 1984.", []),
|
||||||
|
("What's my favorite color", "Your favorite color is green.", []),
|
||||||
|
("Where was I born?", "You were born Testville.", []),
|
||||||
|
("What did I buy?", "You bought an apple for 2.00, an orange for 3.00, and a potato for 8.00", []),
|
||||||
|
]
|
||||||
|
conversation = create_conversation(message_list, default_user2, openai_agent)
|
||||||
|
|
||||||
|
# Act
|
||||||
|
query = urllib.parse.quote("/general What did I eat for breakfast?")
|
||||||
|
response = chat_client.get(f"/api/chat?q={query}&conversation_id={conversation.id}&stream=true")
|
||||||
|
response_message = response.content.decode("utf-8")
|
||||||
|
|
||||||
|
# Assert that agent only responds with the summary of spending
|
||||||
|
expected_responses = ["13.00", "13", "13.0", "thirteen"]
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert any([expected_response in response_message.lower() for expected_response in expected_responses]), (
|
||||||
|
"Expected green in response, but got: " + response_message
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
# ----------------------------------------------------------------------------------------------------
|
# ----------------------------------------------------------------------------------------------------
|
||||||
@pytest.mark.django_db(transaction=True)
|
@pytest.mark.django_db(transaction=True)
|
||||||
@pytest.mark.chatquality
|
@pytest.mark.chatquality
|
||||||
|
|
|
||||||
|
|
@ -39,5 +39,6 @@
|
||||||
"1.6.0": "0.15.0",
|
"1.6.0": "0.15.0",
|
||||||
"1.6.1": "0.15.0",
|
"1.6.1": "0.15.0",
|
||||||
"1.6.2": "0.15.0",
|
"1.6.2": "0.15.0",
|
||||||
"1.7.0": "0.15.0"
|
"1.7.0": "0.15.0",
|
||||||
|
"1.8.0": "0.15.0"
|
||||||
}
|
}
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue