-An AI personal assistant for your digital brain
+An AI copilot for your Second Brain
@@ -24,30 +24,29 @@
## Introduction
-Welcome to the Khoj Docs! This is the best place to [get started](./setup.md) with Khoj.
+Welcome to the Khoj Docs! This is the best place to get setup and explore Khoj's features.
-- Khoj is a desktop application to [search](./search.md) and [chat](./chat.md) with your notes, documents and images
-- It is an offline-first, open source AI personal assistant accessible from your [Emacs](./emacs.md), [Obsidian](./obsidian.md) or [Web browser](./web.md)
-- It works with jpeg, markdown, [notion](./notion_integration.md) org-mode, pdf files and [github repositories](./github_integration.md)
-- If you have more questions, check out the [FAQ](https://faq.khoj.dev/) - it's a live Khoj instance indexing our Github repository!
+- Khoj is an open source, personal AI
+- You can [chat](chat.md) with it about anything. When relevant, it'll use any notes or documents you shared with it to respond
+- Quickly [find](search.md) relevant notes and documents using natural language
+- It understands pdf, plaintext, markdown, org-mode files, [notion pages](notion_integration.md) and [github repositories](github_integration.md)
+- Access it from your [Emacs](emacs.md), [Obsidian](obsidian.md), [Web browser](web.md) or the [Khoj Desktop app](desktop.md)
+- You can self-host Khoj on your consumer hardware or share it with your family, friends or team from your private cloud
## Quickstart
-[Click here](./setup.md) for full setup instructions
-
-```shell
-pip install khoj-assistant && khoj
-```
+- [Try Khoj Cloud](https://app.khoj.dev) to get started quickly
+- [Read these instructions](./setup.md) to self-host a private instance of Khoj
## Overview
-#### [Search](./search.md)
- - **Local**: Your personal data stays local. All search and indexing is done on your machine.
+#### [Search](search.md)
+ - **Natural**: Use natural language queries to quickly find relevant notes and documents.
- **Incremental**: Incremental search for a fast, search-as-you-type experience
-#### [Chat](./chat.md)
+#### [Chat](chat.md)
- **Faster answers**: Find answers faster, smoother than search. No need to manually scan through your notes to find answers.
- **Iterative discovery**: Iteratively explore and (re-)discover your notes
- **Assisted creativity**: Smoothly weave across answers retrieval and content generation
diff --git a/docs/_sidebar.md b/docs/_sidebar.md
index 9e0b8849..348b785a 100644
--- a/docs/_sidebar.md
+++ b/docs/_sidebar.md
@@ -1,12 +1,13 @@
- Get Started
- [Overview](README.md)
- - [Install](setup.md)
+ - [Self-Host](setup.md)
- [Demos](demos.md)
- Use
- [Features](features.md)
- [Chat](chat.md)
- [Search](search.md)
- - Interfaces
+ - Clients
+ - [Desktop](desktop.md)
- [Obsidian](obsidian.md)
- [Emacs](emacs.md)
- [Web](web.md)
diff --git a/docs/advanced.md b/docs/advanced.md
index a567783f..95dacf30 100644
--- a/docs/advanced.md
+++ b/docs/advanced.md
@@ -1,63 +1,11 @@
## Advanced Usage
-### Search across Different Languages
+
+### Search across Different Languages (Self-Hosting)
To search for notes in multiple, different languages, you can use a [multi-lingual model](https://www.sbert.net/docs/pretrained_models.html#multi-lingual-models).For example, the [paraphrase-multilingual-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2) supports [50+ languages](https://www.sbert.net/docs/pretrained_models.html#:~:text=we%20used%20the%20following%2050%2B%20languages), has good search quality and speed. To use it: -1. Manually update `search-type > asymmetric > encoder` to `paraphrase-multilingual-MiniLM-L12-v2` in your `~/.khoj/khoj.yml` file for now. See diff of `khoj.yml` below for illustration: - - ```diff - asymmetric: - - encoder: sentence-transformers/multi-qa-MiniLM-L6-cos-v1 - + encoder: paraphrase-multilingual-MiniLM-L12-v2 - cross-encoder: cross-encoder/ms-marco-MiniLM-L-6-v2 - model_directory: "~/.khoj/search/asymmetric/" - ``` - -2. Regenerate your content index. For example, by opening [\
It should be `http://tailscale-ip-of-server:42110` or `http://name-of-server:42110` if you've setup [MagicDNS](https://tailscale.com/kb/1081/magicdns/) -4. Click the [Add to Homescreen](https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps/Add_to_home_screen) button -5. Enjoy exploring your notes, documents and images from your phone! - - - -### Use OpenAI Models for Search -#### Setup -1. Set `encoder-type`, `encoder` and `model-directory` under `asymmetric` and/or `symmetric` `search-type` in your `khoj.yml` (at `~/.khoj/khoj.yml`): - ```diff - asymmetric: - - encoder: "sentence-transformers/multi-qa-MiniLM-L6-cos-v1" - + encoder: text-embedding-ada-002 - + encoder-type: khoj.utils.models.OpenAI - cross-encoder: "cross-encoder/ms-marco-MiniLM-L-6-v2" - - encoder-type: sentence_transformers.SentenceTransformer - - model_directory: "~/.khoj/search/asymmetric/" - + model-directory: null - ``` -2. [Setup your OpenAI API key in Khoj](/#/chat?id=setup) -3. Restart Khoj server to generate embeddings. It will take longer than with the offline search models. - -#### Warnings - This configuration *uses an online model* - - It will **send all notes to OpenAI** to generate embeddings - - **All queries will be sent to OpenAI** when you search with Khoj - - You will be **charged by OpenAI** based on the total tokens processed - - It *requires an active internet connection* to search and index - -### Bootstrap Khoj Search for Offline Usage later - -You can bootstrap Khoj pre-emptively to run on machines that do not have internet access. An example use-case would be to run Khoj on an air-gapped machine. -Note: *Only search can currently run in fully offline mode, not chat.* - -- With Internet - 1. Manually download the [asymmetric text](https://huggingface.co/sentence-transformers/multi-qa-MiniLM-L6-cos-v1), [symmetric text](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) and [image search](https://huggingface.co/sentence-transformers/clip-ViT-B-32) models from HuggingFace - 2. Pip install khoj (and dependencies) in an associated virtualenv. E.g `python -m venv .venv && source .venv/bin/activate && pip install khoj-assistant` -- Without Internet - 1. Copy each of the search models into their respective folders, `asymmetric`, `symmetric` and `image` under the `~/.khoj/search/` directory on the air-gapped machine - 2. Copy the khoj virtual environment directory onto the air-gapped machine, activate the environment and start and khoj as normal. E.g `source .venv/bin/activate && khoj` +1. Manually update the search config in server's admin settings page. Go to [the search config](http://localhost:42110/server/admin/database/searchmodelconfig/). Either create a new one, if none exists, or update the existing one. Set the bi_encoder to `sentence-transformers/multi-qa-MiniLM-L6-cos-v1` and the cross_encoder to `cross-encoder/ms-marco-MiniLM-L-6-v2`. +2. Regenerate your content index from all the relevant clients. This step is very important, as you'll need to re-encode all your content with the new model. ### Query Filters diff --git a/docs/assets/khoj_chat_on_desktop.png b/docs/assets/khoj_chat_on_desktop.png new file mode 100644 index 00000000..e8c10718 Binary files /dev/null and b/docs/assets/khoj_chat_on_desktop.png differ diff --git a/docs/assets/khoj_search_on_desktop.png b/docs/assets/khoj_search_on_desktop.png new file mode 100644 index 00000000..1dc3231d Binary files /dev/null and b/docs/assets/khoj_search_on_desktop.png differ diff --git a/docs/chat.md b/docs/chat.md index 2efd7b1b..4ea64c3f 100644 --- a/docs/chat.md +++ b/docs/chat.md @@ -1,13 +1,13 @@ -### Khoj Chat -#### Overview +## Khoj Chat +### Overview - Creates a personal assistant for you to inquire and engage with your notes - You can choose to use Online or Offline Chat depending on your requirements - Supports multi-turn conversations with the relevant notes for context - Shows reference notes used to generate a response -### Setup +### Setup (Self-Hosting) #### Offline Chat -Offline chat stays completely private and works without internet. But it is slower, lower quality and more compute intensive. +Offline chat stays completely private and works without internet using open-source models. > **System Requirements**: > - Minimum 8 GB RAM. Recommend **16Gb VRAM** @@ -15,9 +15,10 @@ Offline chat stays completely private and works without internet. But it is slow > - A CPU supporting [AVX or AVX2 instructions](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions) is required > - A Mac M1+ or [Vulcan supported GPU](https://vulkan.gpuinfo.org/) should significantly speed up chat response times -- Open your [Khoj settings](http://localhost:42110/config/) and click *Enable* on the Offline Chat card +1. Open your [Khoj offline settings](http://localhost:42110/server/admin/database/offlinechatprocessorconversationconfig/) and click *Enable* on the Offline Chat configuration. +2. Open your [Chat model options](http://localhost:42110/server/admin/database/chatmodeloptions/) and add a new option for the offline chat model you want to use. Make sure to use `Offline` as its type. We currently only support offline models that use the [Llama chat prompt](https://replicate.com/blog/how-to-prompt-llama#wrap-user-input-with-inst-inst-tags) format. We recommend using `mistral-7b-instruct-v0.1.Q4_0.gguf`. - +!> **Note**: Offline chat is not supported for a multi-user scenario. The host machine will encounter segmentation faults if multiple users try to use offline chat at the same time. #### Online Chat Online chat requires internet to use ChatGPT but is faster, higher quality and less compute intensive. @@ -25,14 +26,12 @@ Online chat requires internet to use ChatGPT but is faster, higher quality and l !> **Warning**: This will enable Khoj to send your chat queries and query relevant notes to OpenAI for processing 1. Get your [OpenAI API Key](https://platform.openai.com/account/api-keys) -2. Open your [Khoj Online Chat settings](http://localhost:42110/config/processor/conversation), add your OpenAI API key, and click *Save*. Then go to your [Khoj settings](http://localhost:42110/config) and click `Configure`. This will refresh Khoj with your OpenAI API key. - - - +2. Open your [Khoj Online Chat settings](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/). Add a new setting with your OpenAI API key, and click *Save*. Only one configuration will be used, so make sure that's the only one you have. +3. Open your [Chat model options](http://localhost:42110/server/admin/database/chatmodeloptions/) and add a new option for the OpenAI chat model you want to use. Make sure to use `OpenAI` as its type. ### Use 1. Open Khoj Chat - - **On Web**: Open [/chat](http://localhost:42110/chat) in your web browser + - **On Web**: Open [/chat](https://app.khoj.dev/chat) in your web browser - **On Obsidian**: Search for *Khoj: Chat* in the [Command Palette](https://help.obsidian.md/Plugins/Command+palette) - **On Emacs**: Run `M-x khoj
Desktop
+
+> An AI copilot for your Second Brain
+
+## Features
+- **Chat**
+ - **Faster answers**: Find answers quickly, from your private notes or the public internet
+ - **Assisted creativity**: Smoothly weave across retrieving answers and generating content
+ - **Iterative discovery**: Iteratively explore and re-discover your notes
+- **Search**
+ - **Natural**: Advanced natural language understanding using Transformer based ML Models
+ - **Incremental**: Incremental search for a fast, search-as-you-type experience
+
+## Setup
+
+1. Install the [Khoj Desktop app](https://khoj.dev/downloads) for your OS
+2. Generate an API key on the [Khoj Web App](https://app.khoj.dev/config#clients)
+3. Set your Khoj API Key on the *Settings* page of the Khoj Desktop app
+4. [Optional] Add any files, folders you'd like Khoj to be aware of on the *Settings* page and Click *Save*
+
+## Interface
+
+
diff --git a/docs/desktop_installation.md b/docs/desktop_installation.md
index d79a282f..42a89383 100644
--- a/docs/desktop_installation.md
+++ b/docs/desktop_installation.md
@@ -28,5 +28,5 @@ For the Linux installation, you have to have `glibc` version 2.35 or higher. You
If you decide you want to uninstall the application, you can uninstall it like any other application on your system. For example, on MacOS, you can drag the application to the trash. On Windows, you can uninstall it from the `Add or Remove Programs` menu. On Linux, you can uninstall it with `sudo apt remove khoj`.
In addition to that, you might want to `rm -rf` the following directories:
-- `~/.khoj`
-- `~/.cache/gpt4all`
+ - `~/.khoj`
+ - `~/.cache/gpt4all`
diff --git a/docs/development.md b/docs/development.md
index dd1aad46..0d715dc4 100644
--- a/docs/development.md
+++ b/docs/development.md
@@ -25,13 +25,7 @@ pip install -e .'[dev]'
khoj -vv
```
2. Configure Khoj
- - **Via the Settings UI**: Add files, directories to index the [Khoj settings](http://localhost:42110/config) UI once Khoj has started up. Once you've saved all your settings, click `Configure`.
- - **Manually**:
- - Copy the `config/khoj_sample.yml` to `~/.khoj/khoj.yml`
- - Set `input-files` or `input-filter` in each relevant `content-type` section of `~/.khoj/khoj.yml`
- - Set `input-directories` field in `image` `content-type` section
- - Delete `content-type` and `processor` sub-section(s) irrelevant for your use-case
- - Restart khoj
+ - **Via the Desktop application**: Add files, directories to index using the settings page of your desktop application. Click "Save" to immediately trigger indexing.
Note: Wait after configuration for khoj to Load ML model, generate embeddings and expose API to query notes, images, documents etc specified in config YAML
diff --git a/docs/emacs.md b/docs/emacs.md
index 6492ecc4..d8e7e682 100644
--- a/docs/emacs.md
+++ b/docs/emacs.md
@@ -1,6 +1,6 @@
-Emacs
+ Emacs
-> An AI personal assistance for your digital brain
+> An AI copilot for your Second Brain in Emacs
@@ -10,14 +10,13 @@
## Features
+- **Chat**
+ - **Faster answers**: Find answers quickly, from your private notes or the public internet
+ - **Assisted creativity**: Smoothly weave across retrieving answers and generating content
+ - **Iterative discovery**: Iteratively explore and re-discover your notes
- **Search**
- **Natural**: Advanced natural language understanding using Transformer based ML Models
- - **Local**: Your personal data stays local. All search, indexing is done on your machine*
- **Incremental**: Incremental search for a fast, search-as-you-type experience
-- **Chat**
- - **Faster answers**: Find answers faster than search
- - **Iterative discovery**: Iteratively explore and (re-)discover your notes
- - **Assisted creativity**: Smoothly weave across answer retrieval and content generation
## Interface
#### Search
@@ -27,79 +26,76 @@
## Setup
-- *Make sure [python](https://realpython.com/installing-python/) and [pip](https://pip.pypa.io/en/stable/installation/) are installed on your machine*
+1. Generate an API key on the [Khoj Web App](https://app.khoj.dev/config#clients)
+2. Add below snippet to your Emacs config file, usually at `~/.emacs.d/init.el`
-- *khoj.el attempts to automatically install, start and configure the khoj server.*
- If this fails, follow [these instructions](/setup) to manually setup the khoj server.
-### Direct Install
+
+
+#### **Direct Install**
+*Khoj will index your org-agenda files, by default*
+
```elisp
+;; Install Khoj.el
M-x package-install khoj
+
+; Set your Khoj API key
+(setq khoj-api-key "YOUR_KHOJ_CLOUD_API_KEY")
```
-### Minimal Install
-Add below snippet to your Emacs config file.
-Indexes your org-agenda files, by default.
+#### **Minimal Install**
+*Khoj will index your org-agenda files, by default*
```elisp
- ;; Install Khoj Package from MELPA Stable
- (use-package khoj
- :ensure t
- :pin melpa-stable
- :bind ("C-c s" . 'khoj))
-```
-
-- Note: Install `khoj.el` from MELPA (instead of MELPA Stable) if you installed the pre-release version of khoj
- - That is, use `:pin melpa` to install khoj.el in above snippet if khoj server was installed with `--pre` flag, i.e `pip install --pre khoj-assistant`
- - Else use `:pin melpa-stable` to install khoj.el in above snippet if khoj was installed with `pip install khoj-assistant`
- - This ensures both khoj.el and khoj app are from the same version (git tagged or latest)
-
-### Standard Install
- Add below snippet to your Emacs config file.
- Indexes the specified org files, directories. Sets up OpenAI API key for Khoj Chat
-
-```elisp
-;; Install Khoj Package from MELPA Stable
+;; Install Khoj client from MELPA Stable
(use-package khoj
:ensure t
:pin melpa-stable
:bind ("C-c s" . 'khoj)
- :config (setq khoj-org-directories '("~/docs/org-roam" "~/docs/notes")
- khoj-org-files '("~/docs/todo.org" "~/docs/work.org")
- khoj-openai-api-key "YOUR_OPENAI_API_KEY")) ; required to enable chat
+ :config (setq khoj-api-key "YOUR_KHOJ_CLOUD_API_KEY"))
```
-### With [Straight.el](https://github.com/raxod502/straight.el)
-Add below snippet to your Emacs config file.
-Indexes the specified org files, directories. Sets up OpenAI API key for Khoj Chat
+#### **Standard Install**
+*Configures the specified org files, directories to be indexed by Khoj*
```elisp
- ;; Install Khoj Package using Straight.el
- (use-package khoj
- :after org
- :straight (khoj :type git :host github :repo "khoj-ai/khoj" :files (:defaults "src/interface/emacs/khoj.el"))
- :bind ("C-c s" . 'khoj)
- :config (setq khoj-org-directories '("~/docs/org-roam" "~/docs/notes")
- khoj-org-files '("~/docs/todo.org" "~/docs/work.org")
- khoj-openai-api-key "YOUR_OPENAI_API_KEY" ; required to enable chat)
- ```
+;; Install Khoj client from MELPA Stable
+(use-package khoj
+ :ensure t
+ :pin melpa-stable
+ :bind ("C-c s" . 'khoj)
+ :config (setq khoj-api-key "YOUR_KHOJ_CLOUD_API_KEY"
+ khoj-org-directories '("~/docs/org-roam" "~/docs/notes")
+ khoj-org-files '("~/docs/todo.org" "~/docs/work.org")))
+```
+#### **Straight.el**
+*Configures the specified org files, directories to be indexed by Khoj*
+
+```elisp
+;; Install Khoj client using Straight.el
+(use-package khoj
+ :after org
+ :straight (khoj :type git :host github :repo "khoj-ai/khoj" :files (:defaults "src/interface/emacs/khoj.el"))
+ :bind ("C-c s" . 'khoj)
+ :config (setq khoj-api-key "YOUR_KHOJ_CLOUD_API_KEY"
+ khoj-org-directories '("~/docs/org-roam" "~/docs/notes")
+ khoj-org-files '("~/docs/todo.org" "~/docs/work.org")))
+```
+
+
## Use
### Search
+See [Khoj Search](search.md) for details
1. Hit `C-c s s` (or `M-x khoj RET s`) to open khoj search
-
-2. Enter your query in natural language
-
- e.g "What is the meaning of life?", "My life goals for 2023"
+2. Enter your query in natural language+ E.g *"What is the meaning of life?"*, *"My life goals for 2023"* ### Chat +See [Khoj Chat](chat.md) for details 1. Hit `C-c s c` (or `M-x khoj RET c`) to open khoj chat - -2. Ask questions in a natural, conversational style - - E.g "When did I file my taxes last year?" - - See [Khoj Chat](/#/chat) for more details +2. Ask questions in a natural, conversational style
+ E.g *"When did I file my taxes last year?"* ### Find Similar Entries This feature finds entries similar to the one you are currently on. @@ -108,7 +104,6 @@ This feature finds entries similar to the one you are currently on. ### Advanced Usage - Add [query filters](https://github.com/khoj-ai/khoj/#query-filters) during search to narrow down results further - e.g `What is the meaning of life? -"god" +"none" dt>"last week"` - Use `C-c C-o 2` to open the current result at cursor in its source org file @@ -121,31 +116,21 @@ This feature finds entries similar to the one you are currently on.  Hit `C-c s` (or `M-x khoj`) to open the khoj menu above. Then: - Hit `t` until you preferred content type is selected in the khoj menu - `Content Type` specifies the content to perform `Search`, `Update` or `Find Similar` actions on - Hit `n` twice and then enter number of results you want to see - `Results Count` is used by the `Search` and `Find Similar` actions - Hit `-f u` to `force` update the khoj content index - The `Force Update` switch is only used by the `Update` action ## Upgrade -### Upgrade Khoj Backend -```bash -pip install --upgrade khoj-assistant -``` -### Upgrade Khoj.el Use your Emacs package manager to upgrade `khoj.el` + -- For `khoj.el` from MELPA - - Method 1 - - Run `M-x package-list-packages` to list all packages - - Press `U` on `khoj` to mark it for upgrade - - Press `x` to execute the marked actions - - Method 2 - - Run `M-x package-refresh-content` - - Run `M-x package-reinstall khoj` +#### **With MELPA** +1. Run `M-x package-refresh-content` +2. Run `M-x package-reinstall khoj` -- For `khoj.el` from Straight - - Run `M-x straight-pull-package khoj` +#### **With Straight.el** +- Run `M-x straight-pull-package khoj` + + diff --git a/docs/features.md b/docs/features.md index 3bd8939f..f59e0657 100644 --- a/docs/features.md +++ b/docs/features.md @@ -1,10 +1,10 @@ ## Features -#### [Search](./search.md) +#### [Search](search.md) - **Local**: Your personal data stays local. All search and indexing is done on your machine. - **Incremental**: Incremental search for a fast, search-as-you-type experience -#### [Chat](./chat.md) +#### [Chat](chat.md) - **Faster answers**: Find answers faster, smoother than search. No need to manually scan through your notes to find answers. - **Iterative discovery**: Iteratively explore and (re-)discover your notes - **Assisted creativity**: Smoothly weave across answers retrieval and content generation diff --git a/docs/github_integration.md b/docs/github_integration.md index 6b8dce48..b7c8a4fe 100644 --- a/docs/github_integration.md +++ b/docs/github_integration.md @@ -1,14 +1,14 @@ -# Setup the Github integration +# 🧑🏾💻 Setup the Github integration The Github integration allows you to index as many repositories as you want. It's currently default configured to index Issues, Commits, and all Markdown/Org files in each repository. For large repositories, this takes a fairly long time, but it works well for smaller projects. # Configure your settings -1. Go to [http://localhost:42110/config](http://localhost:42110/config) and enter in settings for the data sources you want to index. You'll have to specify the file paths. +1. Go to [https://app.khoj.dev/config](https://app.khoj.dev/config) and enter in settings for the data sources you want to index. You'll have to specify the file paths. ## Use the Github plugin 1. Generate a [classic PAT (personal access token)](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) from [Github](https://github.com/settings/tokens) with `repo` and `admin:org` scopes at least. -2. Navigate to [http://localhost:42110/config/content_type/github](http://localhost:42110/config/content_type/github) to configure your Github settings. Enter in your PAT, along with details for each repository you want to index. +2. Navigate to [https://app.khoj.dev/config/content-source/github](https://app.khoj.dev/config/content-source/github) to configure your Github settings. Enter in your PAT, along with details for each repository you want to index. 3. Click `Save`. Go back to the settings page and click `Configure`. -4. Go to [http://localhost:42110/](http://localhost:42110/) and start searching! +4. Go to [https://app.khoj.dev/](https://app.khoj.dev/) and start searching! diff --git a/docs/index.html b/docs/index.html index 33ba0735..5c1d3466 100644 --- a/docs/index.html +++ b/docs/index.html @@ -5,6 +5,15 @@
Obsidian
+ Obsidian
-> An AI personal assistant for your Digital Brain in Obsidian
+> An AI copilot for your Second Brain in Obsidian
## Features
+- **Chat**
+ - **Faster answers**: Find answers quickly, from your private notes or the public internet
+ - **Assisted creativity**: Smoothly weave across retrieving answers and generating content
+ - **Iterative discovery**: Iteratively explore and re-discover your notes
- **Search**
- **Natural**: Advanced natural language understanding using Transformer based ML Models
- - **Local**: Your personal data stays local. All search and indexing is done on your machine. *Unlike chat which requires access to GPT.*
- **Incremental**: Incremental search for a fast, search-as-you-type experience
-- **Chat**
- - **Faster answers**: Find answers faster and with less effort than search
- - **Iterative discovery**: Iteratively explore and (re-)discover your notes
- - **Assisted creativity**: Smoothly weave across answers retrieval and content generation
## Interface
@@ -18,102 +17,37 @@
## Setup
-- *Make sure [python](https://realpython.com/installing-python/) and [pip](https://pip.pypa.io/en/stable/installation/) are installed on your machine*
-- *Ensure you follow the ordering of the setup steps. Install the plugin after starting the khoj backend. This allows the plugin to configure the khoj backend*
-### 1. Setup Backend
-Open terminal/cmd and run below command to install and start the khoj backend
-- On Linux/MacOS
- ```shell
- python -m pip install khoj-assistant && khoj
- ```
-
-- On Windows
- ```shell
- py -m pip install khoj-assistant && khoj
- ```
-
-### 2. Setup Plugin
1. Open [Khoj](https://obsidian.md/plugins?id=khoj) from the *Community plugins* tab in Obsidian settings panel
2. Click *Install*, then *Enable* on the Khoj plugin page in Obsidian
- 3. [Optional] To enable Khoj Chat, set your [OpenAI API key](https://platform.openai.com/account/api-keys) in the Khoj plugin settings
+ 3. Generate an API key on the [Khoj Web App](https://app.khoj.dev/config#clients)
+ 4. Set your Khoj API Key in the Khoj plugin settings in Obsidian
-See [official Obsidian plugin docs](https://help.obsidian.md/Extending+Obsidian/Community+plugins) for details
+See the official [Obsidian Plugin Docs](https://help.obsidian.md/Extending+Obsidian/Community+plugins) for more details on installing Obsidian plugins.
## Use
### Chat
Run *Khoj: Chat* from the [Command Palette](https://help.obsidian.md/Plugins/Command+palette) and ask questions in a natural, conversational style.-E.g "When did I file my taxes last year?" - -Notes: -- *Using Khoj Chat will result in query relevant notes being shared with OpenAI for ChatGPT to respond.* -- *To use Khoj Chat, ensure you've set your [OpenAI API key](https://platform.openai.com/account/api-keys) in the Khoj plugin settings.* +E.g *"When did I file my taxes last year?"* See [Khoj Chat](/chat) for more details -### Search -Click the *Khoj search* icon 🔎 on the [Ribbon](https://help.obsidian.md/User+interface/Workspace/Ribbon) or run *Khoj: Search* from the [Command Palette](https://help.obsidian.md/Plugins/Command+palette) - -*Note: Ensure the khoj server is running in the background before searching. Execute `khoj` in your terminal if it is not already running* - -[search_demo](https://user-images.githubusercontent.com/6413477/218801155-cd67e8b4-a770-404a-8179-d6b61caa0f93.mp4 ':include :type=mp4') - -#### Query Filters - -Use structured query syntax to filter the natural language search results -- **Word Filter**: Get entries that include/exclude a specified term - - Entries that contain term_to_include: `+"term_to_include"` - - Entries that contain term_to_exclude: `-"term_to_exclude"` -- **Date Filter**: Get entries containing dates in YYYY-MM-DD format from specified date (range) - - Entries from April 1st 1984: `dt:"1984-04-01"` - - Entries after March 31st 1984: `dt>="1984-04-01"` - - Entries before April 2nd 1984 : `dt<="1984-04-01"` -- **File Filter**: Get entries from a specified file - - Entries from incoming.org file: `file:"incoming.org"` -- Combined Example - - `what is the meaning of life? file:"1984.org" dt>="1984-01-01" dt<="1985-01-01" -"big" -"brother"` - - Adds all filters to the natural language query. It should return entries - - from the file *1984.org* - - containing dates from the year *1984* - - excluding words *"big"* and *"brother"* - - that best match the natural language query *"what is the meaning of life?"* - ### Find Similar Notes To see other notes similar to the current one, run *Khoj: Find Similar Notes* from the [Command Palette](https://help.obsidian.md/Plugins/Command+palette) +### Search +Click the *Khoj search* icon 🔎 on the [Ribbon](https://help.obsidian.md/User+interface/Workspace/Ribbon) or run *Khoj: Search* from the [Command Palette](https://help.obsidian.md/Plugins/Command+palette) + +See [Khoj Search](/search) for more details. Use [query filters](/advanced#query-filters) to limit entries to search + +[search_demo](https://user-images.githubusercontent.com/6413477/218801155-cd67e8b4-a770-404a-8179-d6b61caa0f93.mp4 ':include :type=mp4') + ## Upgrade -### 1. Upgrade Backend - ```shell - pip install --upgrade khoj-assistant - ``` -### 2. Upgrade Plugin 1. Open *Community plugins* tab in Obsidian settings 2. Click the *Check for updates* button 3. Click the *Update* button next to Khoj, if available -## Demo -### Search Demo -[demo](https://github-production-user-asset-6210df.s3.amazonaws.com/6413477/240061700-3e33d8ea-25bb-46c8-a3bf-c92f78d0f56b.mp4 ':include :type=mp4') - -#### Description - -1. Install Khoj via `pip` and start Khoj backend - ```shell - python -m pip install khoj-assistant && khoj - ``` -2. Install Khoj plugin via Community Plugins settings pane on Obsidian app - - Check the new Khoj plugin settings - - Wait for Khoj backend to index markdown, PDF files in the current Vault - - Open Khoj plugin on Obsidian via Search button on Left Pane - - Search \"*Announce plugin to folks*\" in the [Obsidian Plugin docs](https://marcus.se.net/obsidian-plugin-docs/) - - Jump to the [search result](https://marcus.se.net/obsidian-plugin-docs/publishing/submit-your-plugin) - - ## Troubleshooting - Open the Khoj plugin settings pane, to configure Khoj - Toggle Enable/Disable Khoj, if setting changes have not applied - Click *Update* button to force index to refresh, if results are failing or stale - -## Current Limitations -- The plugin loads the index of only one vault at a time.
- So notes across multiple vaults **cannot** be searched at the same time diff --git a/docs/search.md b/docs/search.md index 579034ec..f2387f06 100644 --- a/docs/search.md +++ b/docs/search.md @@ -1,7 +1,7 @@ ## Khoj Search ### Use 1. Open Khoj Search - - **On Web**: Open
- Its simpler as it can skip the server *install*, *run* and *configure* step below. + It's simpler as it can skip the server *install*, *run* and *configure* step below. - Check the [Khoj Obsidian docs](/obsidian?id=_2-setup-plugin) to setup Khoj with Obsidian
Its simpler as it can skip the *configure* step below. -### 1. Install +For Installation, you can either use Docker or install Khoj locally. -#### 1.1 Local Server Setup -Run the following command in your terminal to install the Khoj backend. +### 1. Installation (Docker) -- On Linux/MacOS - ```shell - python -m pip install khoj-assistant - ``` - -- On Windows - ```shell - py -m pip install khoj-assistant - ``` -For more detailed Windows installation and troubleshooting, see [Windows Install](./windows_install.md). - - -##### 1.1.1 Local Server Start - -Run the following command from your terminal to start the Khoj backend and open Khoj in your browser. - -```shell -khoj -``` - -Khoj should now be running at http://localhost:42110. You can see the web UI in your browser. - -Note: To start Khoj automatically in the background use [Task scheduler](https://www.windowscentral.com/how-create-automated-task-using-task-scheduler-windows-10) on Windows or [Cron](https://en.wikipedia.org/wiki/Cron) on Mac, Linux (e.g with `@reboot khoj`) - -#### 1.2 Local Docker Setup -Use the sample docker-compose [in Github](https://github.com/khoj-ai/khoj/blob/master/docker-compose.yml) to run Khoj in Docker. To start the container, run the following command in the same directory as the docker-compose.yml file. You'll have to configure the mounted directories to match your local knowledge base. +Use the sample docker-compose [in Github](https://github.com/khoj-ai/khoj/blob/master/docker-compose.yml) to run Khoj in Docker. Start by configuring all the environment variables to your choosing. Your admin account will automatically be created based on the admin credentials in that file, so pay attention to those. To start the container, run the following command in the same directory as the docker-compose.yml file. This will automatically setup the database and run the Khoj server. ```shell docker-compose up @@ -45,27 +19,131 @@ docker-compose up Khoj should now be running at http://localhost:42110. You can see the web UI in your browser. -#### 1.3 Download the desktop client [Optional] +### 1. Installation (Local) -You can use our desktop executables to select file paths and folders to index. You can simply select the folders or files, and they'll be automatically uploaded to the server. Once you specify a file or file path, you don't need to update the configuration again; it will grab any data diffs dynamically over time. This part is currently optional, but may make setup and configuration slightly easier. It removes the need for setting up custom file paths for your Khoj data configurations. +#### Prerequisites -**To download the desktop client, go to https://download.khoj.dev** and the correct executable for your OS will automatically start downloading. Once downloaded, you can configure your folders for indexing using the settings tab. To set your chat configuration, you'll have to use the web interface for the Khoj server you setup in the previous step. +##### Install Postgres (with PgVector) -### 1.4 Use (deprecated) desktop builds +Khoj uses the `pgvector` package to store embeddings of your index in a Postgres database. In order to use this, you need to have Postgres installed. -Before `v0.12.0``, we had self-contained desktop builds that included both the server and the client. These were difficult to maintain, but are still available as part of earlier releases. To find setup instructions, see here: + -- [Desktop Installation](desktop_installation.md) -- [Windows Installation](windows_install.md) +#### **MacOS** -### 2. Configure -1. Set `File`, `Folder` and hit `Save` in each Plugins you want to enable for Search on the Khoj config page -2. Add your OpenAI API key to Chat Feature settings if you want to use Chat -3. Click `Configure` and wait. The app will download ML models and index the content for search and (optionally) chat +Install [Postgres.app](https://postgresapp.com/). This comes pre-installed with `pgvector` and relevant dependencies. - +#### **Windows** -### 3. Install Interface Plugins (Optional) +Use the [recommended installer](https://www.postgresql.org/download/windows/) + +#### **Linux** +From [official instructions](https://wiki.postgresql.org/wiki/Apt) + +```bash +sudo apt install -y postgresql-common +sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh +sudo apt install postgres-16 postgresql-16-pgvector +``` + +##### **From Source** +1. Follow instructions to [Install Postgres](https://www.postgresql.org/download/) +2. Follow instructions to [Install PgVector](https://github.com/pgvector/pgvector#installation) in case you need to manually install it. Reproduced instructions below for convenience. + +```bash +cd /tmp +git clone --branch v0.5.1 https://github.com/pgvector/pgvector.git +cd pgvector +make +make install # may need sudo +``` + + + + +##### Create the Khoj database + +Make sure to update your environment variables to match your Postgres configuration if you're using a different name. The default values should work for most people. + + + +#### **MacOS** +```bash +createdb khoj -U postgres +``` + +#### **Windows** +```bash +createdb khoj -U postgres +``` + +#### **Linux** +```bash +sudo -u postgres createdb khoj +``` + + + +#### Install package + +##### Local Server Setup +- *Make sure [python](https://realpython.com/installing-python/) and [pip](https://pip.pypa.io/en/stable/installation/) are installed on your machine* + +Run the following command in your terminal to install the Khoj backend. + + + +#### **MacOS** + +```shell +python -m pip install khoj-assistant +``` + +#### **Windows** + +```shell +py -m pip install khoj-assistant +``` +For more detailed Windows installation and troubleshooting, see [Windows Install](./windows_install.md). + +#### **Linux** + +```shell +python -m pip install khoj-assistant +``` + + + +##### Local Server Start + +Run the following command from your terminal to start the Khoj backend and open Khoj in your browser. + +```shell +khoj --anonymous-mode +``` +`--anonymous-mode` allows you to run the server without setting up Google credentials for login. This allows you to use any of the clients without a login wall. If you want to use Google login, you can skip this flag, but you will have to add your Google developer credentials. + +On the first run, you will be prompted to input credentials for your admin account and do some basic configuration for your chat model settings. Once created, you can go to http://localhost:42110/server/admin and login with the credentials you just created. + +Khoj should now be running at http://localhost:42110. You can see the web UI in your browser. + +Note: To start Khoj automatically in the background use [Task scheduler](https://www.windowscentral.com/how-create-automated-task-using-task-scheduler-windows-10) on Windows or [Cron](https://en.wikipedia.org/wiki/Cron) on Mac, Linux (e.g with `@reboot khoj`) + + +### 2. Download the desktop client + +You can use our desktop executables to select file paths and folders to index. You can simply select the folders or files, and they'll be automatically uploaded to the server. Once you specify a file or file path, you don't need to update the configuration again; it will grab any data diffs dynamically over time. + +**To download the latest desktop client, go to https://download.khoj.dev** and the correct executable for your OS will automatically start downloading. Once downloaded, you can configure your folders for indexing using the settings tab. To set your chat configuration, you'll have to use the web interface for the Khoj server you setup in the previous step. + +To use the desktop client, you need to go to your Khoj server's settings page (http://localhost:42110/config) and copy the API key. Then, paste it into the desktop client's settings page. Once you've done that, you can select files and folders to index. + +### 3. Configure +1. Go to http://localhost:42110/server/admin and login with your admin credentials. Go to the ChatModelOptions if you want to add additional models for chat. +1. Select files and folders to index [using the desktop client](./setup.md?id=_2-download-the-desktop-client). When you click 'Save', the files will be sent to your server for indexing. + - Select Notion workspaces and Github repositories to index using the web interface. + +### 4. Install Client Plugins (Optional) Khoj exposes a web interface to search, chat and configure by default.
The optional steps below allow using Khoj from within an existing application like Obsidian or Emacs. @@ -75,9 +153,17 @@ The optional steps below allow using Khoj from within an existing application li - **Khoj Emacs**:
[Install](/emacs?id=setup) khoj.el +### 5. Use Khoj 🚀 + +You can head to http://localhost:42110 to use the web interface. You can also use the desktop client to search and chat. ## Upgrade ### Upgrade Khoj Server + + + +#### **Local Setup** + ```shell pip install --upgrade khoj-assistant ``` @@ -88,6 +174,16 @@ pip install --upgrade khoj-assistant pip install --upgrade --pre khoj-assistant ``` +#### **Docker** +From the same directory where you have your `docker-compose` file, this will fetch the latest build and upgrade your server. + +```shell +docker-compose up --build +``` + + + + ### Upgrade Khoj on Emacs - Use your Emacs Package Manager to Upgrade - See [khoj.el package setup](/emacs?id=setup) for details @@ -100,8 +196,8 @@ pip install --upgrade --pre khoj-assistant 1. (Optional) Hit `Ctrl-C` in the terminal running the khoj server to stop it 2. Delete the khoj directory in your home folder (i.e `~/.khoj` on Linux, Mac or `C:\Users\
Web
+ Web
-> An AI personal assistant for your Digital Brain
+> An AI copilot for your Second Brain
## Features
+- **Chat**
+ - **Faster answers**: Find answers quickly, from your private notes or the public internet
+ - **Assisted creativity**: Smoothly weave across retrieving answers and generating content
+ - **Iterative discovery**: Iteratively explore and re-discover your notes
- **Search**
- **Natural**: Advanced natural language understanding using Transformer based ML Models
- - **Local**: Your personal data stays local. All search and indexing is done on your machine. *Unlike chat which requires access to GPT.*
- **Incremental**: Incremental search for a fast, search-as-you-type experience
-- **Chat**
- - **Faster answers**: Find answers faster and with less effort than search
- - **Iterative discovery**: Iteratively explore and (re-)discover your notes
- - **Assisted creativity**: Smoothly weave across answers retrieval and content generation
## Setup
-The Khoj web interface is the default interface. It comes packaged with the khoj server.
+No setup required. The Khoj web app is the default interface to Khoj. You can access it from any web browser. Try it on [Khoj Cloud](https://app.khoj.dev)
## Interface
diff --git a/gunicorn-config.py b/gunicorn-config.py
new file mode 100644
index 00000000..1760ae38
--- /dev/null
+++ b/gunicorn-config.py
@@ -0,0 +1,10 @@
+import multiprocessing
+
+bind = "0.0.0.0:42110"
+workers = 4
+worker_class = "uvicorn.workers.UvicornWorker"
+timeout = 120
+keep_alive = 60
+accesslog = "access.log"
+errorlog = "error.log"
+loglevel = "debug"
diff --git a/manifest.json b/manifest.json
index a1c1f913..4d019834 100644
--- a/manifest.json
+++ b/manifest.json
@@ -1,7 +1,7 @@
{
"id": "khoj",
"name": "Khoj",
- "version": "0.14.0",
+ "version": "1.0.0",
"minAppVersion": "0.15.0",
"description": "An AI copilot for your Second Brain",
"author": "Khoj Inc.",
diff --git a/prod.Dockerfile b/prod.Dockerfile
new file mode 100644
index 00000000..693a3a8b
--- /dev/null
+++ b/prod.Dockerfile
@@ -0,0 +1,30 @@
+# Use Nvidia's latest Ubuntu 22.04 image as the base image
+FROM nvidia/cuda:12.2.0-devel-ubuntu22.04
+
+LABEL org.opencontainers.image.source https://github.com/khoj-ai/khoj
+
+# Install System Dependencies
+RUN apt update -y && apt -y install python3-pip git libsqlite3-0 ffmpeg libsm6 libxext6
+
+WORKDIR /app
+
+# Install Application
+COPY pyproject.toml .
+COPY README.md .
+RUN sed -i 's/dynamic = \["version"\]/version = "0.0.0"/' pyproject.toml && \
+ TMPDIR=/home/cache/ pip install --cache-dir=/home/cache/ -e .
+
+# Copy Source Code
+COPY . .
+
+RUN apt install vim -y
+
+# Set the PYTHONPATH environment variable in order for it to find the Django app.
+ENV PYTHONPATH=/app/src:$PYTHONPATH
+
+# Run the Application
+# There are more arguments required for the application to run,
+# but these should be passed in through the docker-compose.yml file.
+ARG PORT
+EXPOSE ${PORT}
+ENTRYPOINT [ "gunicorn", "-c", "gunicorn-config.py", "src.khoj.main:app" ]
diff --git a/pyproject.toml b/pyproject.toml
index bac662a4..a457aec6 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -39,7 +39,7 @@ dependencies = [
"bs4 >= 0.0.1",
"dateparser >= 1.1.1",
"defusedxml == 0.7.1",
- "fastapi == 0.77.1",
+ "fastapi >= 0.104.1",
"python-multipart >= 0.0.5",
"jinja2 == 3.1.2",
"openai >= 0.27.0, < 1.0.0",
@@ -54,14 +54,27 @@ dependencies = [
"transformers >= 4.28.0",
"torch == 2.0.1",
"uvicorn == 0.17.6",
- "aiohttp == 3.8.5",
- "langchain >= 0.0.187",
+ "aiohttp == 3.8.6",
+ "langchain >= 0.0.331",
"requests >= 2.26.0",
"bs4 >= 0.0.1",
"anyio == 3.7.1",
- "pymupdf >= 1.23.3",
+ "pymupdf >= 1.23.5",
+ "django == 4.2.7",
+ "authlib == 1.2.1",
"gpt4all >= 2.0.0; platform_system == 'Linux' and platform_machine == 'x86_64'",
"gpt4all >= 2.0.0; platform_system == 'Windows' or platform_system == 'Darwin'",
+ "itsdangerous == 2.1.2",
+ "httpx == 0.25.0",
+ "pgvector == 0.2.3",
+ "psycopg2-binary == 2.9.9",
+ "google-auth == 2.23.3",
+ "python-multipart == 0.0.6",
+ "gunicorn == 21.2.0",
+ "lxml == 4.9.3",
+ "tzdata == 2023.3",
+ "rapidocr-onnxruntime == 1.3.8",
+ "stripe == 7.3.0",
]
dynamic = ["version"]
@@ -81,12 +94,15 @@ test = [
"factory-boy >= 3.2.1",
"trio >= 0.22.0",
"pytest-xdist",
+ "psutil >= 5.8.0",
]
dev = [
"khoj-assistant[test]",
"mypy >= 1.0.1",
"black >= 23.1.0",
"pre-commit >= 3.0.4",
+ "pytest-django == 4.5.2",
+ "pytest-asyncio == 0.21.1",
]
[tool.hatch.version]
diff --git a/pytest.ini b/pytest.ini
new file mode 100644
index 00000000..b3e418d0
--- /dev/null
+++ b/pytest.ini
@@ -0,0 +1,6 @@
+[pytest]
+DJANGO_SETTINGS_MODULE = app.settings
+pythonpath = . src
+testpaths = tests
+markers =
+ chatquality: marks tests as chatquality (deselect with '-m "not chatquality"')
diff --git a/src/app/README.md b/src/app/README.md
new file mode 100644
index 00000000..14fc8501
--- /dev/null
+++ b/src/app/README.md
@@ -0,0 +1,94 @@
+# Django App
+
+Khoj uses Django as the backend framework primarily for its powerful ORM and the admin interface. The Django app is located in the `src/app` directory. We have one installed app, under the `/database/` directory. This app is responsible for all the database related operations and holds all of our models. You can find the extensive Django documentation [here](https://docs.djangoproject.com/en/4.2/) 🌈.
+
+## Setup (Docker)
+
+### Prerequisites
+1. Ensure you have [Docker](https://docs.docker.com/get-docker/) installed.
+2. Ensure you have [Docker Compose](https://docs.docker.com/compose/install/) installed.
+
+### Run
+
+Using the `docker-compose.yml` file in the root directory, you can run the Khoj app using the following command:
+```bash
+docker-compose up
+```
+
+## Setup (Local)
+
+### Install Postgres (with PgVector)
+
+#### MacOS
+- Install the [Postgres.app](https://postgresapp.com/).
+
+#### Debian, Ubuntu
+From [official instructions](https://wiki.postgresql.org/wiki/Apt)
+
+```bash
+sudo apt install -y postgresql-common
+sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh
+sudo apt install postgres-16 postgresql-16-pgvector
+```
+
+#### Windows
+- Use the [recommended installer](https://www.postgresql.org/download/windows/)
+
+#### From Source
+1. Follow instructions to [Install Postgres](https://www.postgresql.org/download/)
+2. Follow instructions to [Install PgVector](https://github.com/pgvector/pgvector#installation) in case you need to manually install it. Reproduced instructions below for convenience.
+
+```bash
+cd /tmp
+git clone --branch v0.5.1 https://github.com/pgvector/pgvector.git
+cd pgvector
+make
+make install # may need sudo
+```
+
+### Create the Khoj database
+
+#### MacOS
+```bash
+createdb khoj -U postgres
+```
+
+#### Debian, Ubuntu
+```bash
+sudo -u postgres createdb khoj
+```
+
+- [Optional] To set default postgres user's password
+ - Execute `ALTER USER postgres PASSWORD 'my_secure_password';` using `psql`
+ - Run `export $POSTGRES_PASSWORD=my_secure_password` in your terminal for Khoj to use it later
+
+### Install Khoj
+
+```bash
+pip install -e '.[dev]'
+```
+
+### Make Khoj DB migrations
+
+This command will create the migrations for the database app. This command should be run whenever a new db model is added to the database app or an existing db model is modified (updated or deleted).
+
+```bash
+python3 src/manage.py makemigrations
+```
+
+### Run Khoj DB migrations
+
+This command will run any pending migrations in your application.
+```bash
+python3 src/manage.py migrate
+```
+
+### Start Khoj Server
+
+While we're using Django for the ORM, we're still using the FastAPI server for the API. This command automatically scaffolds the Django application in the backend.
+
+*Note: Anonymous mode bypasses authentication for local, single-user usage.*
+
+```bash
+python3 src/khoj/main.py --anonymous-mode
+```
diff --git a/src/khoj/processor/jsonl/__init__.py b/src/app/__init__.py
similarity index 100%
rename from src/khoj/processor/jsonl/__init__.py
rename to src/app/__init__.py
diff --git a/src/app/settings.py b/src/app/settings.py
new file mode 100644
index 00000000..0803081d
--- /dev/null
+++ b/src/app/settings.py
@@ -0,0 +1,152 @@
+"""
+Django settings for app project.
+
+Generated by 'django-admin startproject' using Django 4.2.5.
+
+For more information on this file, see
+https://docs.djangoproject.com/en/4.2/topics/settings/
+
+For the full list of settings and their values, see
+https://docs.djangoproject.com/en/4.2/ref/settings/
+"""
+
+from pathlib import Path
+import os
+
+# Build paths inside the project like this: BASE_DIR / 'subdir'.
+BASE_DIR = Path(__file__).resolve().parent.parent.parent
+
+
+# Quick-start development settings - unsuitable for production
+# See https://docs.djangoproject.com/en/4.2/howto/deployment/checklist/
+
+# SECURITY WARNING: keep the secret key used in production secret!
+SECRET_KEY = os.getenv("KHOJ_DJANGO_SECRET_KEY")
+
+# SECURITY WARNING: don't run with debug turned on in production!
+DEBUG = os.getenv("KHOJ_DEBUG", "False") == "True"
+
+ALLOWED_HOSTS = [".khoj.dev", "localhost", "127.0.0.1", "[::1]", "beta.khoj.dev"]
+
+CSRF_TRUSTED_ORIGINS = [
+ "https://app.khoj.dev",
+ "https://beta.khoj.dev",
+ "https://khoj.dev",
+ "https://*.khoj.dev",
+]
+
+COOKIE_SAMESITE = "None"
+if DEBUG:
+ SESSION_COOKIE_DOMAIN = "localhost"
+ CSRF_COOKIE_DOMAIN = "localhost"
+else:
+ SESSION_COOKIE_DOMAIN = "khoj.dev"
+ CSRF_COOKIE_DOMAIN = "khoj.dev"
+
+SESSION_COOKIE_SECURE = True
+CSRF_COOKIE_SECURE = True
+COOKIE_SAMESITE = "None"
+SESSION_COOKIE_SAMESITE = "None"
+
+# Application definition
+
+INSTALLED_APPS = [
+ "django.contrib.auth",
+ "django.contrib.contenttypes",
+ "database.apps.DatabaseConfig",
+ "django.contrib.admin",
+ "django.contrib.sessions",
+ "django.contrib.messages",
+ "django.contrib.staticfiles",
+]
+
+MIDDLEWARE = [
+ "django.middleware.security.SecurityMiddleware",
+ "django.contrib.sessions.middleware.SessionMiddleware",
+ "django.middleware.common.CommonMiddleware",
+ "django.middleware.csrf.CsrfViewMiddleware",
+ "django.contrib.auth.middleware.AuthenticationMiddleware",
+ "django.contrib.messages.middleware.MessageMiddleware",
+ "django.middleware.clickjacking.XFrameOptionsMiddleware",
+]
+
+ROOT_URLCONF = "app.urls"
+
+TEMPLATES = [
+ {
+ "BACKEND": "django.template.backends.django.DjangoTemplates",
+ "APP_DIRS": True,
+ "DIRS": [os.path.join(BASE_DIR, "templates"), os.path.join(BASE_DIR, "templates", "account")],
+ "OPTIONS": {
+ "context_processors": [
+ "django.template.context_processors.debug",
+ "django.template.context_processors.request",
+ "django.contrib.auth.context_processors.auth",
+ "django.contrib.messages.context_processors.messages",
+ ],
+ },
+ },
+]
+
+WSGI_APPLICATION = "app.wsgi.application"
+
+
+# Database
+# https://docs.djangoproject.com/en/4.2/ref/settings/#databases
+
+DATABASES = {
+ "default": {
+ "ENGINE": "django.db.backends.postgresql",
+ "HOST": os.getenv("POSTGRES_HOST", "localhost"),
+ "PORT": os.getenv("POSTGRES_PORT", "5432"),
+ "USER": os.getenv("POSTGRES_USER", "postgres"),
+ "NAME": os.getenv("POSTGRES_DB", "khoj"),
+ "PASSWORD": os.getenv("POSTGRES_PASSWORD", "postgres"),
+ }
+}
+
+# User Settings
+AUTH_USER_MODEL = "database.KhojUser"
+
+# Password validation
+# https://docs.djangoproject.com/en/4.2/ref/settings/#auth-password-validators
+
+AUTH_PASSWORD_VALIDATORS = [
+ {
+ "NAME": "django.contrib.auth.password_validation.UserAttributeSimilarityValidator",
+ },
+ {
+ "NAME": "django.contrib.auth.password_validation.MinimumLengthValidator",
+ },
+ {
+ "NAME": "django.contrib.auth.password_validation.CommonPasswordValidator",
+ },
+ {
+ "NAME": "django.contrib.auth.password_validation.NumericPasswordValidator",
+ },
+]
+
+
+# Internationalization
+# https://docs.djangoproject.com/en/4.2/topics/i18n/
+
+LANGUAGE_CODE = "en-us"
+
+TIME_ZONE = "UTC"
+
+USE_I18N = True
+
+USE_TZ = True
+
+
+# Static files (CSS, JavaScript, Images)
+# https://docs.djangoproject.com/en/4.2/howto/static-files/
+
+STATIC_ROOT = BASE_DIR / "static"
+STATICFILES_DIRS = [BASE_DIR / "src/khoj/interface/web"]
+STATIC_URL = "/static/"
+
+# Default primary key field type
+# https://docs.djangoproject.com/en/4.2/ref/settings/#default-auto-field
+
+DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"
diff --git a/src/app/urls.py b/src/app/urls.py
new file mode 100644
index 00000000..39b4b1ef
--- /dev/null
+++ b/src/app/urls.py
@@ -0,0 +1,25 @@
+"""
+URL configuration for app project.
+
+The `urlpatterns` list routes URLs to views. For more information please see:
+ https://docs.djangoproject.com/en/4.2/topics/http/urls/
+Examples:
+Function views
+ 1. Add an import: from my_app import views
+ 2. Add a URL to urlpatterns: path('', views.home, name='home')
+Class-based views
+ 1. Add an import: from other_app.views import Home
+ 2. Add a URL to urlpatterns: path('', Home.as_view(), name='home')
+Including another URLconf
+ 1. Import the include() function: from django.urls import include, path
+ 2. Add a URL to urlpatterns: path('blog/', include('blog.urls'))
+"""
+from django.contrib import admin
+from django.urls import path
+from django.contrib.staticfiles.urls import staticfiles_urlpatterns
+
+urlpatterns = [
+ path("admin/", admin.site.urls),
+]
+
+urlpatterns += staticfiles_urlpatterns()
diff --git a/src/app/wsgi.py b/src/app/wsgi.py
new file mode 100644
index 00000000..cbdf4342
--- /dev/null
+++ b/src/app/wsgi.py
@@ -0,0 +1,16 @@
+"""
+WSGI config for app project.
+
+It exposes the WSGI callable as a module-level variable named ``application``.
+
+For more information on this file, see
+https://docs.djangoproject.com/en/4.2/howto/deployment/wsgi/
+"""
+
+import os
+
+from django.core.wsgi import get_wsgi_application
+
+os.environ.setdefault("DJANGO_SETTINGS_MODULE", "app.settings")
+
+application = get_wsgi_application()
diff --git a/src/database/__init__.py b/src/database/__init__.py
new file mode 100644
index 00000000..e69de29b
diff --git a/src/database/adapters/__init__.py b/src/database/adapters/__init__.py
new file mode 100644
index 00000000..4141d3bb
--- /dev/null
+++ b/src/database/adapters/__init__.py
@@ -0,0 +1,481 @@
+import math
+from typing import Optional, Type, List
+from datetime import date, datetime
+import secrets
+from typing import Type, List
+from datetime import date, timezone
+
+from django.db import models
+from django.contrib.sessions.backends.db import SessionStore
+from pgvector.django import CosineDistance
+from django.db.models.manager import BaseManager
+from django.db.models import Q
+from torch import Tensor
+
+# Import sync_to_async from Django Channels
+from asgiref.sync import sync_to_async
+
+from fastapi import HTTPException
+
+from database.models import (
+ KhojUser,
+ GoogleUser,
+ KhojApiUser,
+ NotionConfig,
+ GithubConfig,
+ Entry,
+ GithubRepoConfig,
+ Conversation,
+ ChatModelOptions,
+ SearchModelConfig,
+ Subscription,
+ UserConversationConfig,
+ OpenAIProcessorConversationConfig,
+ OfflineChatProcessorConversationConfig,
+)
+from khoj.utils.helpers import generate_random_name
+from khoj.search_filter.word_filter import WordFilter
+from khoj.search_filter.file_filter import FileFilter
+from khoj.search_filter.date_filter import DateFilter
+
+
+async def set_notion_config(token: str, user: KhojUser):
+ notion_config = await NotionConfig.objects.filter(user=user).afirst()
+ if not notion_config:
+ notion_config = await NotionConfig.objects.acreate(token=token, user=user)
+ else:
+ notion_config.token = token
+ await notion_config.asave()
+ return notion_config
+
+
+async def create_khoj_token(user: KhojUser, name=None):
+ "Create Khoj API key for user"
+ token = f"kk-{secrets.token_urlsafe(32)}"
+ name = name or f"{generate_random_name().title()}"
+ return await KhojApiUser.objects.acreate(token=token, user=user, name=name)
+
+
+def get_khoj_tokens(user: KhojUser):
+ "Get all Khoj API keys for user"
+ return list(KhojApiUser.objects.filter(user=user))
+
+
+async def delete_khoj_token(user: KhojUser, token: str):
+ "Delete Khoj API Key for user"
+ await KhojApiUser.objects.filter(token=token, user=user).adelete()
+
+
+async def get_or_create_user(token: dict) -> KhojUser:
+ user = await get_user_by_token(token)
+ if not user:
+ user = await create_user_by_google_token(token)
+ return user
+
+
+async def create_user_by_google_token(token: dict) -> KhojUser:
+ user, _ = await KhojUser.objects.filter(email=token.get("email")).aupdate_or_create(
+ defaults={"username": token.get("email"), "email": token.get("email")}
+ )
+ await user.asave()
+
+ await GoogleUser.objects.acreate(
+ sub=token.get("sub"),
+ azp=token.get("azp"),
+ email=token.get("email"),
+ name=token.get("name"),
+ given_name=token.get("given_name"),
+ family_name=token.get("family_name"),
+ picture=token.get("picture"),
+ locale=token.get("locale"),
+ user=user,
+ )
+
+ await Subscription.objects.acreate(user=user, type="trial")
+
+ return user
+
+
+def get_user_subscription(email: str) -> Optional[Subscription]:
+ return Subscription.objects.filter(user__email=email).first()
+
+
+async def set_user_subscription(
+ email: str, is_recurring=None, renewal_date=None, type="standard"
+) -> Optional[Subscription]:
+ user_subscription = await Subscription.objects.filter(user__email=email).afirst()
+ if not user_subscription:
+ user = await get_user_by_email(email)
+ if not user:
+ return None
+ user_subscription = await Subscription.objects.acreate(
+ user=user, type=type, is_recurring=is_recurring, renewal_date=renewal_date
+ )
+ return user_subscription
+ elif user_subscription:
+ user_subscription.type = type
+ if is_recurring is not None:
+ user_subscription.is_recurring = is_recurring
+ if renewal_date is False:
+ user_subscription.renewal_date = None
+ elif renewal_date is not None:
+ user_subscription.renewal_date = renewal_date
+ await user_subscription.asave()
+ return user_subscription
+ else:
+ return None
+
+
+def get_user_subscription_state(email: str) -> str:
+ """Get subscription state of user
+ Valid state transitions: trial -> subscribed <-> unsubscribed OR expired
+ """
+ user_subscription = Subscription.objects.filter(user__email=email).first()
+ if not user_subscription:
+ return "trial"
+ elif user_subscription.type == Subscription.Type.TRIAL:
+ return "trial"
+ elif user_subscription.is_recurring and user_subscription.renewal_date >= datetime.now(tz=timezone.utc):
+ return "subscribed"
+ elif not user_subscription.is_recurring and user_subscription.renewal_date >= datetime.now(tz=timezone.utc):
+ return "unsubscribed"
+ elif not user_subscription.is_recurring and user_subscription.renewal_date < datetime.now(tz=timezone.utc):
+ return "expired"
+ return "invalid"
+
+
+async def get_user_by_email(email: str) -> KhojUser:
+ return await KhojUser.objects.filter(email=email).afirst()
+
+
+async def get_user_by_token(token: dict) -> KhojUser:
+ google_user = await GoogleUser.objects.filter(sub=token.get("sub")).select_related("user").afirst()
+ if not google_user:
+ return None
+ return google_user.user
+
+
+async def retrieve_user(session_id: str) -> KhojUser:
+ session = SessionStore(session_key=session_id)
+ if not await sync_to_async(session.exists)(session_key=session_id):
+ raise HTTPException(status_code=401, detail="Invalid session")
+ session_data = await sync_to_async(session.load)()
+ user = await KhojUser.objects.filter(id=session_data.get("_auth_user_id")).afirst()
+ if not user:
+ raise HTTPException(status_code=401, detail="Invalid user")
+ return user
+
+
+def get_all_users() -> BaseManager[KhojUser]:
+ return KhojUser.objects.all()
+
+
+def get_user_github_config(user: KhojUser):
+ config = GithubConfig.objects.filter(user=user).prefetch_related("githubrepoconfig").first()
+ return config
+
+
+def get_user_notion_config(user: KhojUser):
+ config = NotionConfig.objects.filter(user=user).first()
+ return config
+
+
+async def set_text_content_config(user: KhojUser, object: Type[models.Model], updated_config):
+ deduped_files = list(set(updated_config.input_files)) if updated_config.input_files else None
+ deduped_filters = list(set(updated_config.input_filter)) if updated_config.input_filter else None
+ await object.objects.filter(user=user).adelete()
+ await object.objects.acreate(
+ input_files=deduped_files,
+ input_filter=deduped_filters,
+ index_heading_entries=updated_config.index_heading_entries,
+ user=user,
+ )
+
+
+async def set_user_github_config(user: KhojUser, pat_token: str, repos: list):
+ config = await GithubConfig.objects.filter(user=user).afirst()
+
+ if not config:
+ config = await GithubConfig.objects.acreate(pat_token=pat_token, user=user)
+ else:
+ config.pat_token = pat_token
+ await config.asave()
+ await config.githubrepoconfig.all().adelete()
+
+ for repo in repos:
+ await GithubRepoConfig.objects.acreate(
+ name=repo["name"], owner=repo["owner"], branch=repo["branch"], github_config=config
+ )
+ return config
+
+
+def get_or_create_search_model():
+ search_model = SearchModelConfig.objects.filter().first()
+ if not search_model:
+ search_model = SearchModelConfig.objects.create()
+
+ return search_model
+
+
+class ConversationAdapters:
+ @staticmethod
+ def get_conversation_by_user(user: KhojUser):
+ conversation = Conversation.objects.filter(user=user)
+ if conversation.exists():
+ return conversation.first()
+ return Conversation.objects.create(user=user)
+
+ @staticmethod
+ async def aget_conversation_by_user(user: KhojUser):
+ conversation = Conversation.objects.filter(user=user)
+ if await conversation.aexists():
+ return await conversation.afirst()
+ return await Conversation.objects.acreate(user=user)
+
+ @staticmethod
+ def has_any_conversation_config(user: KhojUser):
+ return ChatModelOptions.objects.filter(user=user).exists()
+
+ @staticmethod
+ def get_openai_conversation_config():
+ return OpenAIProcessorConversationConfig.objects.filter().first()
+
+ @staticmethod
+ async def aget_openai_conversation_config():
+ return await OpenAIProcessorConversationConfig.objects.filter().afirst()
+
+ @staticmethod
+ def get_offline_chat_conversation_config():
+ return OfflineChatProcessorConversationConfig.objects.filter().first()
+
+ @staticmethod
+ async def aget_offline_chat_conversation_config():
+ return await OfflineChatProcessorConversationConfig.objects.filter().afirst()
+
+ @staticmethod
+ def has_valid_offline_conversation_config():
+ return OfflineChatProcessorConversationConfig.objects.filter(enabled=True).exists()
+
+ @staticmethod
+ def has_valid_openai_conversation_config():
+ return OpenAIProcessorConversationConfig.objects.filter().exists()
+
+ @staticmethod
+ async def aset_user_conversation_processor(user: KhojUser, conversation_processor_config_id: int):
+ config = await ChatModelOptions.objects.filter(id=conversation_processor_config_id).afirst()
+ if not config:
+ return None
+ new_config = await UserConversationConfig.objects.aupdate_or_create(user=user, defaults={"setting": config})
+ return new_config
+
+ @staticmethod
+ def get_conversation_config(user: KhojUser):
+ config = UserConversationConfig.objects.filter(user=user).first()
+ if not config:
+ return None
+ return config.setting
+
+ @staticmethod
+ async def aget_conversation_config(user: KhojUser):
+ config = await UserConversationConfig.objects.filter(user=user).prefetch_related("setting").afirst()
+ if not config:
+ return None
+ return config.setting
+
+ @staticmethod
+ def get_default_conversation_config():
+ return ChatModelOptions.objects.filter().first()
+
+ @staticmethod
+ async def aget_default_conversation_config():
+ return await ChatModelOptions.objects.filter().afirst()
+
+ @staticmethod
+ def save_conversation(user: KhojUser, conversation_log: dict):
+ conversation = Conversation.objects.filter(user=user)
+ if conversation.exists():
+ conversation.update(conversation_log=conversation_log)
+ else:
+ Conversation.objects.create(user=user, conversation_log=conversation_log)
+
+ @staticmethod
+ def get_conversation_processor_options():
+ return ChatModelOptions.objects.all()
+
+ @staticmethod
+ def set_conversation_processor_config(user: KhojUser, new_config: ChatModelOptions):
+ user_conversation_config, _ = UserConversationConfig.objects.get_or_create(user=user)
+ user_conversation_config.setting = new_config
+ user_conversation_config.save()
+
+ @staticmethod
+ def has_offline_chat():
+ return OfflineChatProcessorConversationConfig.objects.filter(enabled=True).exists()
+
+ @staticmethod
+ async def ahas_offline_chat():
+ return await OfflineChatProcessorConversationConfig.objects.filter(enabled=True).aexists()
+
+ @staticmethod
+ async def get_offline_chat():
+ return await ChatModelOptions.objects.filter(model_type="offline").afirst()
+
+ @staticmethod
+ async def aget_user_conversation_config(user: KhojUser):
+ config = await UserConversationConfig.objects.filter(user=user).prefetch_related("setting").afirst()
+ if not config:
+ return None
+ return config.setting
+
+ @staticmethod
+ async def has_openai_chat():
+ return await OpenAIProcessorConversationConfig.objects.filter().aexists()
+
+ @staticmethod
+ async def get_openai_chat():
+ return await ChatModelOptions.objects.filter(model_type="openai").afirst()
+
+ @staticmethod
+ async def get_openai_chat_config():
+ return await OpenAIProcessorConversationConfig.objects.filter().afirst()
+
+
+class EntryAdapters:
+ word_filer = WordFilter()
+ file_filter = FileFilter()
+ date_filter = DateFilter()
+
+ @staticmethod
+ def does_entry_exist(user: KhojUser, hashed_value: str) -> bool:
+ return Entry.objects.filter(user=user, hashed_value=hashed_value).exists()
+
+ @staticmethod
+ def delete_entry_by_file(user: KhojUser, file_path: str):
+ deleted_count, _ = Entry.objects.filter(user=user, file_path=file_path).delete()
+ return deleted_count
+
+ @staticmethod
+ def delete_all_entries_by_type(user: KhojUser, file_type: str = None):
+ if file_type is None:
+ deleted_count, _ = Entry.objects.filter(user=user).delete()
+ else:
+ deleted_count, _ = Entry.objects.filter(user=user, file_type=file_type).delete()
+ return deleted_count
+
+ @staticmethod
+ def delete_all_entries(user: KhojUser, file_source: str = None):
+ if file_source is None:
+ deleted_count, _ = Entry.objects.filter(user=user).delete()
+ else:
+ deleted_count, _ = Entry.objects.filter(user=user, file_source=file_source).delete()
+ return deleted_count
+
+ @staticmethod
+ def get_existing_entry_hashes_by_file(user: KhojUser, file_path: str):
+ return Entry.objects.filter(user=user, file_path=file_path).values_list("hashed_value", flat=True)
+
+ @staticmethod
+ def delete_entry_by_hash(user: KhojUser, hashed_values: List[str]):
+ Entry.objects.filter(user=user, hashed_value__in=hashed_values).delete()
+
+ @staticmethod
+ def get_entries_by_date_filter(entry: BaseManager[Entry], start_date: date, end_date: date):
+ return entry.filter(
+ entrydates__date__gte=start_date,
+ entrydates__date__lte=end_date,
+ )
+
+ @staticmethod
+ def user_has_entries(user: KhojUser):
+ return Entry.objects.filter(user=user).exists()
+
+ @staticmethod
+ async def auser_has_entries(user: KhojUser):
+ return await Entry.objects.filter(user=user).aexists()
+
+ @staticmethod
+ async def adelete_entry_by_file(user: KhojUser, file_path: str):
+ return await Entry.objects.filter(user=user, file_path=file_path).adelete()
+
+ @staticmethod
+ def aget_all_filenames_by_source(user: KhojUser, file_source: str):
+ return (
+ Entry.objects.filter(user=user, file_source=file_source)
+ .distinct("file_path")
+ .values_list("file_path", flat=True)
+ )
+
+ @staticmethod
+ async def adelete_all_entries(user: KhojUser):
+ return await Entry.objects.filter(user=user).adelete()
+
+ @staticmethod
+ def apply_filters(user: KhojUser, query: str, file_type_filter: str = None):
+ q_filter_terms = Q()
+
+ explicit_word_terms = EntryAdapters.word_filer.get_filter_terms(query)
+ file_filters = EntryAdapters.file_filter.get_filter_terms(query)
+ date_filters = EntryAdapters.date_filter.get_query_date_range(query)
+
+ if len(explicit_word_terms) == 0 and len(file_filters) == 0 and len(date_filters) == 0:
+ return Entry.objects.filter(user=user)
+
+ for term in explicit_word_terms:
+ if term.startswith("+"):
+ q_filter_terms &= Q(raw__icontains=term[1:])
+ elif term.startswith("-"):
+ q_filter_terms &= ~Q(raw__icontains=term[1:])
+
+ q_file_filter_terms = Q()
+
+ if len(file_filters) > 0:
+ for term in file_filters:
+ q_file_filter_terms |= Q(file_path__regex=term)
+
+ q_filter_terms &= q_file_filter_terms
+
+ if len(date_filters) > 0:
+ min_date, max_date = date_filters
+ if min_date is not None:
+ # Convert the min_date timestamp to yyyy-mm-dd format
+ formatted_min_date = date.fromtimestamp(min_date).strftime("%Y-%m-%d")
+ q_filter_terms &= Q(embeddings_dates__date__gte=formatted_min_date)
+ if max_date is not None:
+ # Convert the max_date timestamp to yyyy-mm-dd format
+ formatted_max_date = date.fromtimestamp(max_date).strftime("%Y-%m-%d")
+ q_filter_terms &= Q(embeddings_dates__date__lte=formatted_max_date)
+
+ relevant_entries = Entry.objects.filter(user=user).filter(
+ q_filter_terms,
+ )
+ if file_type_filter:
+ relevant_entries = relevant_entries.filter(file_type=file_type_filter)
+ return relevant_entries
+
+ @staticmethod
+ def search_with_embeddings(
+ user: KhojUser,
+ embeddings: Tensor,
+ max_results: int = 10,
+ file_type_filter: str = None,
+ raw_query: str = None,
+ max_distance: float = math.inf,
+ ):
+ relevant_entries = EntryAdapters.apply_filters(user, raw_query, file_type_filter)
+ relevant_entries = relevant_entries.filter(user=user).annotate(
+ distance=CosineDistance("embeddings", embeddings)
+ )
+ relevant_entries = relevant_entries.filter(distance__lte=max_distance)
+
+ if file_type_filter:
+ relevant_entries = relevant_entries.filter(file_type=file_type_filter)
+ relevant_entries = relevant_entries.order_by("distance")
+ return relevant_entries[:max_results]
+
+ @staticmethod
+ def get_unique_file_types(user: KhojUser):
+ return Entry.objects.filter(user=user).values_list("file_type", flat=True).distinct()
+
+ @staticmethod
+ def get_unique_file_sources(user: KhojUser):
+ return Entry.objects.filter(user=user).values_list("file_source", flat=True).distinct().all()
diff --git a/src/database/admin.py b/src/database/admin.py
new file mode 100644
index 00000000..8d2130ba
--- /dev/null
+++ b/src/database/admin.py
@@ -0,0 +1,21 @@
+from django.contrib import admin
+from django.contrib.auth.admin import UserAdmin
+
+# Register your models here.
+
+from database.models import (
+ KhojUser,
+ ChatModelOptions,
+ OpenAIProcessorConversationConfig,
+ OfflineChatProcessorConversationConfig,
+ SearchModelConfig,
+ Subscription,
+)
+
+admin.site.register(KhojUser, UserAdmin)
+
+admin.site.register(ChatModelOptions)
+admin.site.register(OpenAIProcessorConversationConfig)
+admin.site.register(OfflineChatProcessorConversationConfig)
+admin.site.register(SearchModelConfig)
+admin.site.register(Subscription)
diff --git a/src/database/apps.py b/src/database/apps.py
new file mode 100644
index 00000000..a3b71b13
--- /dev/null
+++ b/src/database/apps.py
@@ -0,0 +1,6 @@
+from django.apps import AppConfig
+
+
+class DatabaseConfig(AppConfig):
+ default_auto_field = "django.db.models.BigAutoField"
+ name = "database"
diff --git a/src/database/migrations/0001_khojuser.py b/src/database/migrations/0001_khojuser.py
new file mode 100644
index 00000000..f1420575
--- /dev/null
+++ b/src/database/migrations/0001_khojuser.py
@@ -0,0 +1,98 @@
+# Generated by Django 4.2.5 on 2023-09-14 19:00
+
+import django.contrib.auth.models
+import django.contrib.auth.validators
+from django.db import migrations, models
+import django.utils.timezone
+
+
+class Migration(migrations.Migration):
+ initial = True
+
+ dependencies = [
+ ("auth", "0012_alter_user_first_name_max_length"),
+ ]
+
+ run_before = [
+ ("admin", "0001_initial"),
+ ]
+
+ operations = [
+ migrations.CreateModel(
+ name="KhojUser",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("password", models.CharField(max_length=128, verbose_name="password")),
+ ("last_login", models.DateTimeField(blank=True, null=True, verbose_name="last login")),
+ (
+ "is_superuser",
+ models.BooleanField(
+ default=False,
+ help_text="Designates that this user has all permissions without explicitly assigning them.",
+ verbose_name="superuser status",
+ ),
+ ),
+ (
+ "username",
+ models.CharField(
+ error_messages={"unique": "A user with that username already exists."},
+ help_text="Required. 150 characters or fewer. Letters, digits and @/./+/-/_ only.",
+ max_length=150,
+ unique=True,
+ validators=[django.contrib.auth.validators.UnicodeUsernameValidator()],
+ verbose_name="username",
+ ),
+ ),
+ ("first_name", models.CharField(blank=True, max_length=150, verbose_name="first name")),
+ ("last_name", models.CharField(blank=True, max_length=150, verbose_name="last name")),
+ ("email", models.EmailField(blank=True, max_length=254, verbose_name="email address")),
+ (
+ "is_staff",
+ models.BooleanField(
+ default=False,
+ help_text="Designates whether the user can log into this admin site.",
+ verbose_name="staff status",
+ ),
+ ),
+ (
+ "is_active",
+ models.BooleanField(
+ default=True,
+ help_text="Designates whether this user should be treated as active. Unselect this instead of deleting accounts.",
+ verbose_name="active",
+ ),
+ ),
+ ("date_joined", models.DateTimeField(default=django.utils.timezone.now, verbose_name="date joined")),
+ (
+ "groups",
+ models.ManyToManyField(
+ blank=True,
+ help_text="The groups this user belongs to. A user will get all permissions granted to each of their groups.",
+ related_name="user_set",
+ related_query_name="user",
+ to="auth.group",
+ verbose_name="groups",
+ ),
+ ),
+ (
+ "user_permissions",
+ models.ManyToManyField(
+ blank=True,
+ help_text="Specific permissions for this user.",
+ related_name="user_set",
+ related_query_name="user",
+ to="auth.permission",
+ verbose_name="user permissions",
+ ),
+ ),
+ ],
+ options={
+ "verbose_name": "user",
+ "verbose_name_plural": "users",
+ "abstract": False,
+ },
+ managers=[
+ ("objects", django.contrib.auth.models.UserManager()),
+ ],
+ ),
+ ]
diff --git a/src/database/migrations/0002_googleuser.py b/src/database/migrations/0002_googleuser.py
new file mode 100644
index 00000000..478770d6
--- /dev/null
+++ b/src/database/migrations/0002_googleuser.py
@@ -0,0 +1,32 @@
+# Generated by Django 4.2.4 on 2023-09-18 23:24
+
+from django.conf import settings
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("database", "0001_khojuser"),
+ ]
+
+ operations = [
+ migrations.CreateModel(
+ name="GoogleUser",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("sub", models.CharField(max_length=200)),
+ ("azp", models.CharField(max_length=200)),
+ ("email", models.CharField(max_length=200)),
+ ("name", models.CharField(max_length=200)),
+ ("given_name", models.CharField(max_length=200)),
+ ("family_name", models.CharField(max_length=200)),
+ ("picture", models.CharField(max_length=200)),
+ ("locale", models.CharField(max_length=200)),
+ (
+ "user",
+ models.OneToOneField(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL),
+ ),
+ ],
+ ),
+ ]
diff --git a/src/database/migrations/0003_vector_extension.py b/src/database/migrations/0003_vector_extension.py
new file mode 100644
index 00000000..9de01df2
--- /dev/null
+++ b/src/database/migrations/0003_vector_extension.py
@@ -0,0 +1,10 @@
+from django.db import migrations
+from pgvector.django import VectorExtension
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("database", "0002_googleuser"),
+ ]
+
+ operations = [VectorExtension()]
diff --git a/src/database/migrations/0004_content_types_and_more.py b/src/database/migrations/0004_content_types_and_more.py
new file mode 100644
index 00000000..ec704e1f
--- /dev/null
+++ b/src/database/migrations/0004_content_types_and_more.py
@@ -0,0 +1,180 @@
+# Generated by Django 4.2.5 on 2023-10-11 22:24
+
+from django.conf import settings
+from django.db import migrations, models
+import django.db.models.deletion
+import pgvector.django
+import uuid
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("database", "0003_vector_extension"),
+ ]
+
+ operations = [
+ migrations.CreateModel(
+ name="GithubConfig",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("created_at", models.DateTimeField(auto_now_add=True)),
+ ("updated_at", models.DateTimeField(auto_now=True)),
+ ("pat_token", models.CharField(max_length=200)),
+ ],
+ options={
+ "abstract": False,
+ },
+ ),
+ migrations.AddField(
+ model_name="khojuser",
+ name="uuid",
+ field=models.UUIDField(default=1234, verbose_name=models.UUIDField(default=uuid.uuid4, editable=False)),
+ preserve_default=False,
+ ),
+ migrations.CreateModel(
+ name="NotionConfig",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("created_at", models.DateTimeField(auto_now_add=True)),
+ ("updated_at", models.DateTimeField(auto_now=True)),
+ ("token", models.CharField(max_length=200)),
+ ("user", models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
+ ],
+ options={
+ "abstract": False,
+ },
+ ),
+ migrations.CreateModel(
+ name="LocalPlaintextConfig",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("created_at", models.DateTimeField(auto_now_add=True)),
+ ("updated_at", models.DateTimeField(auto_now=True)),
+ ("input_files", models.JSONField(default=list, null=True)),
+ ("input_filter", models.JSONField(default=list, null=True)),
+ ("index_heading_entries", models.BooleanField(default=False)),
+ ("user", models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
+ ],
+ options={
+ "abstract": False,
+ },
+ ),
+ migrations.CreateModel(
+ name="LocalPdfConfig",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("created_at", models.DateTimeField(auto_now_add=True)),
+ ("updated_at", models.DateTimeField(auto_now=True)),
+ ("input_files", models.JSONField(default=list, null=True)),
+ ("input_filter", models.JSONField(default=list, null=True)),
+ ("index_heading_entries", models.BooleanField(default=False)),
+ ("user", models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
+ ],
+ options={
+ "abstract": False,
+ },
+ ),
+ migrations.CreateModel(
+ name="LocalOrgConfig",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("created_at", models.DateTimeField(auto_now_add=True)),
+ ("updated_at", models.DateTimeField(auto_now=True)),
+ ("input_files", models.JSONField(default=list, null=True)),
+ ("input_filter", models.JSONField(default=list, null=True)),
+ ("index_heading_entries", models.BooleanField(default=False)),
+ ("user", models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
+ ],
+ options={
+ "abstract": False,
+ },
+ ),
+ migrations.CreateModel(
+ name="LocalMarkdownConfig",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("created_at", models.DateTimeField(auto_now_add=True)),
+ ("updated_at", models.DateTimeField(auto_now=True)),
+ ("input_files", models.JSONField(default=list, null=True)),
+ ("input_filter", models.JSONField(default=list, null=True)),
+ ("index_heading_entries", models.BooleanField(default=False)),
+ ("user", models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
+ ],
+ options={
+ "abstract": False,
+ },
+ ),
+ migrations.CreateModel(
+ name="GithubRepoConfig",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("created_at", models.DateTimeField(auto_now_add=True)),
+ ("updated_at", models.DateTimeField(auto_now=True)),
+ ("name", models.CharField(max_length=200)),
+ ("owner", models.CharField(max_length=200)),
+ ("branch", models.CharField(max_length=200)),
+ (
+ "github_config",
+ models.ForeignKey(
+ on_delete=django.db.models.deletion.CASCADE,
+ related_name="githubrepoconfig",
+ to="database.githubconfig",
+ ),
+ ),
+ ],
+ options={
+ "abstract": False,
+ },
+ ),
+ migrations.AddField(
+ model_name="githubconfig",
+ name="user",
+ field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL),
+ ),
+ migrations.CreateModel(
+ name="Embeddings",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("created_at", models.DateTimeField(auto_now_add=True)),
+ ("updated_at", models.DateTimeField(auto_now=True)),
+ ("embeddings", pgvector.django.VectorField(dimensions=384)),
+ ("raw", models.TextField()),
+ ("compiled", models.TextField()),
+ ("heading", models.CharField(blank=True, default=None, max_length=1000, null=True)),
+ (
+ "file_type",
+ models.CharField(
+ choices=[
+ ("image", "Image"),
+ ("pdf", "Pdf"),
+ ("plaintext", "Plaintext"),
+ ("markdown", "Markdown"),
+ ("org", "Org"),
+ ("notion", "Notion"),
+ ("github", "Github"),
+ ("conversation", "Conversation"),
+ ],
+ default="plaintext",
+ max_length=30,
+ ),
+ ),
+ ("file_path", models.CharField(blank=True, default=None, max_length=400, null=True)),
+ ("file_name", models.CharField(blank=True, default=None, max_length=400, null=True)),
+ ("url", models.URLField(blank=True, default=None, max_length=400, null=True)),
+ ("hashed_value", models.CharField(max_length=100)),
+ (
+ "user",
+ models.ForeignKey(
+ blank=True,
+ default=None,
+ null=True,
+ on_delete=django.db.models.deletion.CASCADE,
+ to=settings.AUTH_USER_MODEL,
+ ),
+ ),
+ ],
+ options={
+ "abstract": False,
+ },
+ ),
+ ]
diff --git a/src/database/migrations/0005_embeddings_corpus_id.py b/src/database/migrations/0005_embeddings_corpus_id.py
new file mode 100644
index 00000000..984953d6
--- /dev/null
+++ b/src/database/migrations/0005_embeddings_corpus_id.py
@@ -0,0 +1,18 @@
+# Generated by Django 4.2.5 on 2023-10-13 02:39
+
+from django.db import migrations, models
+import uuid
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("database", "0004_content_types_and_more"),
+ ]
+
+ operations = [
+ migrations.AddField(
+ model_name="embeddings",
+ name="corpus_id",
+ field=models.UUIDField(default=uuid.uuid4, editable=False),
+ ),
+ ]
diff --git a/src/database/migrations/0006_embeddingsdates.py b/src/database/migrations/0006_embeddingsdates.py
new file mode 100644
index 00000000..9d988ed8
--- /dev/null
+++ b/src/database/migrations/0006_embeddingsdates.py
@@ -0,0 +1,33 @@
+# Generated by Django 4.2.5 on 2023-10-13 19:28
+
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("database", "0005_embeddings_corpus_id"),
+ ]
+
+ operations = [
+ migrations.CreateModel(
+ name="EmbeddingsDates",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("created_at", models.DateTimeField(auto_now_add=True)),
+ ("updated_at", models.DateTimeField(auto_now=True)),
+ ("date", models.DateField()),
+ (
+ "embeddings",
+ models.ForeignKey(
+ on_delete=django.db.models.deletion.CASCADE,
+ related_name="embeddings_dates",
+ to="database.embeddings",
+ ),
+ ),
+ ],
+ options={
+ "indexes": [models.Index(fields=["date"], name="database_em_date_a1ba47_idx")],
+ },
+ ),
+ ]
diff --git a/src/database/migrations/0007_add_conversation.py b/src/database/migrations/0007_add_conversation.py
new file mode 100644
index 00000000..167b6cab
--- /dev/null
+++ b/src/database/migrations/0007_add_conversation.py
@@ -0,0 +1,27 @@
+# Generated by Django 4.2.5 on 2023-10-18 05:31
+
+from django.conf import settings
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("database", "0006_embeddingsdates"),
+ ]
+
+ operations = [
+ migrations.CreateModel(
+ name="Conversation",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("created_at", models.DateTimeField(auto_now_add=True)),
+ ("updated_at", models.DateTimeField(auto_now=True)),
+ ("conversation_log", models.JSONField()),
+ ("user", models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
+ ],
+ options={
+ "abstract": False,
+ },
+ ),
+ ]
diff --git a/src/database/migrations/0008_alter_conversation_conversation_log.py b/src/database/migrations/0008_alter_conversation_conversation_log.py
new file mode 100644
index 00000000..00f37385
--- /dev/null
+++ b/src/database/migrations/0008_alter_conversation_conversation_log.py
@@ -0,0 +1,17 @@
+# Generated by Django 4.2.5 on 2023-10-18 16:46
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("database", "0007_add_conversation"),
+ ]
+
+ operations = [
+ migrations.AlterField(
+ model_name="conversation",
+ name="conversation_log",
+ field=models.JSONField(default=dict),
+ ),
+ ]
diff --git a/src/database/migrations/0009_khojapiuser.py b/src/database/migrations/0009_khojapiuser.py
new file mode 100644
index 00000000..86b09ab3
--- /dev/null
+++ b/src/database/migrations/0009_khojapiuser.py
@@ -0,0 +1,24 @@
+# Generated by Django 4.2.5 on 2023-10-26 17:02
+
+from django.conf import settings
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("database", "0008_alter_conversation_conversation_log"),
+ ]
+
+ operations = [
+ migrations.CreateModel(
+ name="KhojApiUser",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("token", models.CharField(max_length=50, unique=True)),
+ ("name", models.CharField(max_length=50)),
+ ("accessed_at", models.DateTimeField(default=None, null=True)),
+ ("user", models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
+ ],
+ ),
+ ]
diff --git a/src/database/migrations/0010_chatmodeloptions_and_more.py b/src/database/migrations/0010_chatmodeloptions_and_more.py
new file mode 100644
index 00000000..9f3a491a
--- /dev/null
+++ b/src/database/migrations/0010_chatmodeloptions_and_more.py
@@ -0,0 +1,83 @@
+# Generated by Django 4.2.4 on 2023-11-01 17:41
+
+from django.conf import settings
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("database", "0009_khojapiuser"),
+ ]
+
+ operations = [
+ migrations.CreateModel(
+ name="ChatModelOptions",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("created_at", models.DateTimeField(auto_now_add=True)),
+ ("updated_at", models.DateTimeField(auto_now=True)),
+ ("max_prompt_size", models.IntegerField(blank=True, default=None, null=True)),
+ ("tokenizer", models.CharField(blank=True, default=None, max_length=200, null=True)),
+ ("chat_model", models.CharField(blank=True, default=None, max_length=200, null=True)),
+ (
+ "model_type",
+ models.CharField(
+ choices=[("openai", "Openai"), ("offline", "Offline")], default="openai", max_length=200
+ ),
+ ),
+ ],
+ options={
+ "abstract": False,
+ },
+ ),
+ migrations.CreateModel(
+ name="OfflineChatProcessorConversationConfig",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("created_at", models.DateTimeField(auto_now_add=True)),
+ ("updated_at", models.DateTimeField(auto_now=True)),
+ ("enabled", models.BooleanField(default=False)),
+ ],
+ options={
+ "abstract": False,
+ },
+ ),
+ migrations.CreateModel(
+ name="OpenAIProcessorConversationConfig",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("created_at", models.DateTimeField(auto_now_add=True)),
+ ("updated_at", models.DateTimeField(auto_now=True)),
+ ("api_key", models.CharField(max_length=200)),
+ ],
+ options={
+ "abstract": False,
+ },
+ ),
+ migrations.CreateModel(
+ name="UserConversationConfig",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("created_at", models.DateTimeField(auto_now_add=True)),
+ ("updated_at", models.DateTimeField(auto_now=True)),
+ (
+ "setting",
+ models.ForeignKey(
+ blank=True,
+ default=None,
+ null=True,
+ on_delete=django.db.models.deletion.CASCADE,
+ to="database.chatmodeloptions",
+ ),
+ ),
+ (
+ "user",
+ models.OneToOneField(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL),
+ ),
+ ],
+ options={
+ "abstract": False,
+ },
+ ),
+ ]
diff --git a/src/database/migrations/0010_rename_embeddings_entry_and_more.py b/src/database/migrations/0010_rename_embeddings_entry_and_more.py
new file mode 100644
index 00000000..f86b2caa
--- /dev/null
+++ b/src/database/migrations/0010_rename_embeddings_entry_and_more.py
@@ -0,0 +1,30 @@
+# Generated by Django 4.2.5 on 2023-10-26 23:52
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("database", "0009_khojapiuser"),
+ ]
+
+ operations = [
+ migrations.RenameModel(
+ old_name="Embeddings",
+ new_name="Entry",
+ ),
+ migrations.RenameModel(
+ old_name="EmbeddingsDates",
+ new_name="EntryDates",
+ ),
+ migrations.RenameField(
+ model_name="entrydates",
+ old_name="embeddings",
+ new_name="entry",
+ ),
+ migrations.RenameIndex(
+ model_name="entrydates",
+ new_name="database_en_date_8d823c_idx",
+ old_name="database_em_date_a1ba47_idx",
+ ),
+ ]
diff --git a/src/database/migrations/0011_merge_20231102_0138.py b/src/database/migrations/0011_merge_20231102_0138.py
new file mode 100644
index 00000000..112c76a2
--- /dev/null
+++ b/src/database/migrations/0011_merge_20231102_0138.py
@@ -0,0 +1,12 @@
+# Generated by Django 4.2.5 on 2023-11-02 01:38
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("database", "0010_chatmodeloptions_and_more"),
+ ("database", "0010_rename_embeddings_entry_and_more"),
+ ]
+
+ operations = []
diff --git a/src/database/migrations/0012_entry_file_source.py b/src/database/migrations/0012_entry_file_source.py
new file mode 100644
index 00000000..187136ae
--- /dev/null
+++ b/src/database/migrations/0012_entry_file_source.py
@@ -0,0 +1,21 @@
+# Generated by Django 4.2.5 on 2023-11-07 07:24
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("database", "0011_merge_20231102_0138"),
+ ]
+
+ operations = [
+ migrations.AddField(
+ model_name="entry",
+ name="file_source",
+ field=models.CharField(
+ choices=[("computer", "Computer"), ("notion", "Notion"), ("github", "Github")],
+ default="computer",
+ max_length=30,
+ ),
+ ),
+ ]
diff --git a/src/database/migrations/0013_subscription.py b/src/database/migrations/0013_subscription.py
new file mode 100644
index 00000000..931cea12
--- /dev/null
+++ b/src/database/migrations/0013_subscription.py
@@ -0,0 +1,37 @@
+# Generated by Django 4.2.5 on 2023-11-09 01:27
+
+from django.conf import settings
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("database", "0012_entry_file_source"),
+ ]
+
+ operations = [
+ migrations.CreateModel(
+ name="Subscription",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("created_at", models.DateTimeField(auto_now_add=True)),
+ ("updated_at", models.DateTimeField(auto_now=True)),
+ (
+ "type",
+ models.CharField(
+ choices=[("trial", "Trial"), ("standard", "Standard")], default="trial", max_length=20
+ ),
+ ),
+ ("is_recurring", models.BooleanField(default=False)),
+ ("renewal_date", models.DateTimeField(default=None, null=True)),
+ (
+ "user",
+ models.OneToOneField(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL),
+ ),
+ ],
+ options={
+ "abstract": False,
+ },
+ ),
+ ]
diff --git a/src/database/migrations/0014_alter_googleuser_picture.py b/src/database/migrations/0014_alter_googleuser_picture.py
new file mode 100644
index 00000000..a3d2ce05
--- /dev/null
+++ b/src/database/migrations/0014_alter_googleuser_picture.py
@@ -0,0 +1,17 @@
+# Generated by Django 4.2.5 on 2023-11-09 08:11
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("database", "0013_subscription"),
+ ]
+
+ operations = [
+ migrations.AlterField(
+ model_name="googleuser",
+ name="picture",
+ field=models.CharField(default=None, max_length=200, null=True),
+ ),
+ ]
diff --git a/src/database/migrations/0015_alter_subscription_user.py b/src/database/migrations/0015_alter_subscription_user.py
new file mode 100644
index 00000000..e4ba6ab0
--- /dev/null
+++ b/src/database/migrations/0015_alter_subscription_user.py
@@ -0,0 +1,21 @@
+# Generated by Django 4.2.5 on 2023-11-11 05:39
+
+from django.conf import settings
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("database", "0014_alter_googleuser_picture"),
+ ]
+
+ operations = [
+ migrations.AlterField(
+ model_name="subscription",
+ name="user",
+ field=models.OneToOneField(
+ on_delete=django.db.models.deletion.CASCADE, related_name="subscription", to=settings.AUTH_USER_MODEL
+ ),
+ ),
+ ]
diff --git a/src/database/migrations/0016_alter_subscription_renewal_date.py b/src/database/migrations/0016_alter_subscription_renewal_date.py
new file mode 100644
index 00000000..bc7c5ada
--- /dev/null
+++ b/src/database/migrations/0016_alter_subscription_renewal_date.py
@@ -0,0 +1,17 @@
+# Generated by Django 4.2.5 on 2023-11-11 06:15
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("database", "0015_alter_subscription_user"),
+ ]
+
+ operations = [
+ migrations.AlterField(
+ model_name="subscription",
+ name="renewal_date",
+ field=models.DateTimeField(blank=True, default=None, null=True),
+ ),
+ ]
diff --git a/src/database/migrations/0017_searchmodel.py b/src/database/migrations/0017_searchmodel.py
new file mode 100644
index 00000000..f150e12b
--- /dev/null
+++ b/src/database/migrations/0017_searchmodel.py
@@ -0,0 +1,32 @@
+# Generated by Django 4.2.5 on 2023-11-14 23:25
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("database", "0016_alter_subscription_renewal_date"),
+ ]
+
+ operations = [
+ migrations.CreateModel(
+ name="SearchModel",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("created_at", models.DateTimeField(auto_now_add=True)),
+ ("updated_at", models.DateTimeField(auto_now=True)),
+ ("name", models.CharField(default="default", max_length=200)),
+ ("model_type", models.CharField(choices=[("text", "Text")], default="text", max_length=200)),
+ ("bi_encoder", models.CharField(default="thenlper/gte-small", max_length=200)),
+ (
+ "cross_encoder",
+ models.CharField(
+ blank=True, default="cross-encoder/ms-marco-MiniLM-L-6-v2", max_length=200, null=True
+ ),
+ ),
+ ],
+ options={
+ "abstract": False,
+ },
+ ),
+ ]
diff --git a/src/database/migrations/0018_searchmodelconfig_delete_searchmodel.py b/src/database/migrations/0018_searchmodelconfig_delete_searchmodel.py
new file mode 100644
index 00000000..a8100370
--- /dev/null
+++ b/src/database/migrations/0018_searchmodelconfig_delete_searchmodel.py
@@ -0,0 +1,30 @@
+# Generated by Django 4.2.5 on 2023-11-16 01:13
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("database", "0017_searchmodel"),
+ ]
+
+ operations = [
+ migrations.CreateModel(
+ name="SearchModelConfig",
+ fields=[
+ ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+ ("created_at", models.DateTimeField(auto_now_add=True)),
+ ("updated_at", models.DateTimeField(auto_now=True)),
+ ("name", models.CharField(default="default", max_length=200)),
+ ("model_type", models.CharField(choices=[("text", "Text")], default="text", max_length=200)),
+ ("bi_encoder", models.CharField(default="thenlper/gte-small", max_length=200)),
+ ("cross_encoder", models.CharField(default="cross-encoder/ms-marco-MiniLM-L-6-v2", max_length=200)),
+ ],
+ options={
+ "abstract": False,
+ },
+ ),
+ migrations.DeleteModel(
+ name="SearchModel",
+ ),
+ ]
diff --git a/src/database/migrations/__init__.py b/src/database/migrations/__init__.py
new file mode 100644
index 00000000..e69de29b
diff --git a/src/database/models/__init__.py b/src/database/models/__init__.py
new file mode 100644
index 00000000..92848e5c
--- /dev/null
+++ b/src/database/models/__init__.py
@@ -0,0 +1,181 @@
+import uuid
+
+from django.db import models
+from django.contrib.auth.models import AbstractUser
+from pgvector.django import VectorField
+
+
+class BaseModel(models.Model):
+ created_at = models.DateTimeField(auto_now_add=True)
+ updated_at = models.DateTimeField(auto_now=True)
+
+ class Meta:
+ abstract = True
+
+
+class KhojUser(AbstractUser):
+ uuid = models.UUIDField(models.UUIDField(default=uuid.uuid4, editable=False))
+
+ def save(self, *args, **kwargs):
+ if not self.uuid:
+ self.uuid = uuid.uuid4()
+ super().save(*args, **kwargs)
+
+
+class GoogleUser(models.Model):
+ user = models.OneToOneField(KhojUser, on_delete=models.CASCADE)
+ sub = models.CharField(max_length=200)
+ azp = models.CharField(max_length=200)
+ email = models.CharField(max_length=200)
+ name = models.CharField(max_length=200)
+ given_name = models.CharField(max_length=200)
+ family_name = models.CharField(max_length=200)
+ picture = models.CharField(max_length=200, null=True, default=None)
+ locale = models.CharField(max_length=200)
+
+ def __str__(self):
+ return self.name
+
+
+class KhojApiUser(models.Model):
+ """User issued API tokens to authenticate Khoj clients"""
+
+ user = models.ForeignKey(KhojUser, on_delete=models.CASCADE)
+ token = models.CharField(max_length=50, unique=True)
+ name = models.CharField(max_length=50)
+ accessed_at = models.DateTimeField(null=True, default=None)
+
+
+class Subscription(BaseModel):
+ class Type(models.TextChoices):
+ TRIAL = "trial"
+ STANDARD = "standard"
+
+ user = models.OneToOneField(KhojUser, on_delete=models.CASCADE, related_name="subscription")
+ type = models.CharField(max_length=20, choices=Type.choices, default=Type.TRIAL)
+ is_recurring = models.BooleanField(default=False)
+ renewal_date = models.DateTimeField(null=True, default=None, blank=True)
+
+
+class NotionConfig(BaseModel):
+ token = models.CharField(max_length=200)
+ user = models.ForeignKey(KhojUser, on_delete=models.CASCADE)
+
+
+class GithubConfig(BaseModel):
+ pat_token = models.CharField(max_length=200)
+ user = models.ForeignKey(KhojUser, on_delete=models.CASCADE)
+
+
+class GithubRepoConfig(BaseModel):
+ name = models.CharField(max_length=200)
+ owner = models.CharField(max_length=200)
+ branch = models.CharField(max_length=200)
+ github_config = models.ForeignKey(GithubConfig, on_delete=models.CASCADE, related_name="githubrepoconfig")
+
+
+class LocalOrgConfig(BaseModel):
+ input_files = models.JSONField(default=list, null=True)
+ input_filter = models.JSONField(default=list, null=True)
+ index_heading_entries = models.BooleanField(default=False)
+ user = models.ForeignKey(KhojUser, on_delete=models.CASCADE)
+
+
+class LocalMarkdownConfig(BaseModel):
+ input_files = models.JSONField(default=list, null=True)
+ input_filter = models.JSONField(default=list, null=True)
+ index_heading_entries = models.BooleanField(default=False)
+ user = models.ForeignKey(KhojUser, on_delete=models.CASCADE)
+
+
+class LocalPdfConfig(BaseModel):
+ input_files = models.JSONField(default=list, null=True)
+ input_filter = models.JSONField(default=list, null=True)
+ index_heading_entries = models.BooleanField(default=False)
+ user = models.ForeignKey(KhojUser, on_delete=models.CASCADE)
+
+
+class LocalPlaintextConfig(BaseModel):
+ input_files = models.JSONField(default=list, null=True)
+ input_filter = models.JSONField(default=list, null=True)
+ index_heading_entries = models.BooleanField(default=False)
+ user = models.ForeignKey(KhojUser, on_delete=models.CASCADE)
+
+
+class SearchModelConfig(BaseModel):
+ class ModelType(models.TextChoices):
+ TEXT = "text"
+
+ name = models.CharField(max_length=200, default="default")
+ model_type = models.CharField(max_length=200, choices=ModelType.choices, default=ModelType.TEXT)
+ bi_encoder = models.CharField(max_length=200, default="thenlper/gte-small")
+ cross_encoder = models.CharField(max_length=200, default="cross-encoder/ms-marco-MiniLM-L-6-v2")
+
+
+class OpenAIProcessorConversationConfig(BaseModel):
+ api_key = models.CharField(max_length=200)
+
+
+class OfflineChatProcessorConversationConfig(BaseModel):
+ enabled = models.BooleanField(default=False)
+
+
+class ChatModelOptions(BaseModel):
+ class ModelType(models.TextChoices):
+ OPENAI = "openai"
+ OFFLINE = "offline"
+
+ max_prompt_size = models.IntegerField(default=None, null=True, blank=True)
+ tokenizer = models.CharField(max_length=200, default=None, null=True, blank=True)
+ chat_model = models.CharField(max_length=200, default=None, null=True, blank=True)
+ model_type = models.CharField(max_length=200, choices=ModelType.choices, default=ModelType.OPENAI)
+
+
+class UserConversationConfig(BaseModel):
+ user = models.OneToOneField(KhojUser, on_delete=models.CASCADE)
+ setting = models.ForeignKey(ChatModelOptions, on_delete=models.CASCADE, default=None, null=True, blank=True)
+
+
+class Conversation(BaseModel):
+ user = models.ForeignKey(KhojUser, on_delete=models.CASCADE)
+ conversation_log = models.JSONField(default=dict)
+
+
+class Entry(BaseModel):
+ class EntryType(models.TextChoices):
+ IMAGE = "image"
+ PDF = "pdf"
+ PLAINTEXT = "plaintext"
+ MARKDOWN = "markdown"
+ ORG = "org"
+ NOTION = "notion"
+ GITHUB = "github"
+ CONVERSATION = "conversation"
+
+ class EntrySource(models.TextChoices):
+ COMPUTER = "computer"
+ NOTION = "notion"
+ GITHUB = "github"
+
+ user = models.ForeignKey(KhojUser, on_delete=models.CASCADE, default=None, null=True, blank=True)
+ embeddings = VectorField(dimensions=384)
+ raw = models.TextField()
+ compiled = models.TextField()
+ heading = models.CharField(max_length=1000, default=None, null=True, blank=True)
+ file_source = models.CharField(max_length=30, choices=EntrySource.choices, default=EntrySource.COMPUTER)
+ file_type = models.CharField(max_length=30, choices=EntryType.choices, default=EntryType.PLAINTEXT)
+ file_path = models.CharField(max_length=400, default=None, null=True, blank=True)
+ file_name = models.CharField(max_length=400, default=None, null=True, blank=True)
+ url = models.URLField(max_length=400, default=None, null=True, blank=True)
+ hashed_value = models.CharField(max_length=100)
+ corpus_id = models.UUIDField(default=uuid.uuid4, editable=False)
+
+
+class EntryDates(BaseModel):
+ date = models.DateField()
+ entry = models.ForeignKey(Entry, on_delete=models.CASCADE, related_name="embeddings_dates")
+
+ class Meta:
+ indexes = [
+ models.Index(fields=["date"]),
+ ]
diff --git a/src/database/tests.py b/src/database/tests.py
new file mode 100644
index 00000000..7ce503c2
--- /dev/null
+++ b/src/database/tests.py
@@ -0,0 +1,3 @@
+from django.test import TestCase
+
+# Create your tests here.
diff --git a/src/interface/desktop/about.html b/src/interface/desktop/about.html
new file mode 100644
index 00000000..deff03e1
--- /dev/null
+++ b/src/interface/desktop/about.html
@@ -0,0 +1,88 @@
+
+
+
+
+ 
+ Khoj for Desktop +
+
+
+
+
+
+
+
diff --git a/src/interface/desktop/assets/icons/favicon-20x20.png b/src/interface/desktop/assets/icons/favicon-20x20.png
new file mode 100644
index 00000000..1a4ee0be
Binary files /dev/null and b/src/interface/desktop/assets/icons/favicon-20x20.png differ
diff --git a/src/interface/desktop/assets/icons/key.svg b/src/interface/desktop/assets/icons/key.svg
new file mode 100644
index 00000000..437688fb
--- /dev/null
+++ b/src/interface/desktop/assets/icons/key.svg
@@ -0,0 +1,4 @@
+
+
diff --git a/src/interface/desktop/assets/icons/khoj-logo-sideways-500.png b/src/interface/desktop/assets/icons/khoj-logo-sideways-500.png
index 56648932..765d6e33 100644
Binary files a/src/interface/desktop/assets/icons/khoj-logo-sideways-500.png and b/src/interface/desktop/assets/icons/khoj-logo-sideways-500.png differ
diff --git a/src/interface/desktop/assets/icons/link.svg b/src/interface/desktop/assets/icons/link.svg
index ef484368..43852d95 100644
--- a/src/interface/desktop/assets/icons/link.svg
+++ b/src/interface/desktop/assets/icons/link.svg
@@ -1,5 +1,4 @@
-
+
diff --git a/src/interface/desktop/assets/khoj.css b/src/interface/desktop/assets/khoj.css
index 02f2493f..b2e048c5 100644
--- a/src/interface/desktop/assets/khoj.css
+++ b/src/interface/desktop/assets/khoj.css
@@ -2,29 +2,44 @@
/* Can be forced with data-theme="light" */
[data-theme="light"],
:root:not([data-theme="dark"]) {
- --primary: #ffb300;
- --primary-hover: #ffa000;
+ --primary: #fee285;
+ --primary-hover: #fcc50b;
--primary-focus: rgba(255, 179, 0, 0.125);
--primary-inverse: rgba(0, 0, 0, 0.75);
+ --background-color: #f5f4f3;
+ --main-text-color: #475569;
+ --water: #44b9da;
+ --leaf: #7b990a;
+ --flower: #d1684e;
}
/* Amber Dark scheme (Auto) */
/* Automatically enabled if user has Dark mode enabled */
@media only screen and (prefers-color-scheme: dark) {
:root:not([data-theme]) {
- --primary: #ffb300;
- --primary-hover: #ffc107;
+ --primary: #fee285;
+ --primary-hover: #fcc50b;
--primary-focus: rgba(255, 179, 0, 0.25);
--primary-inverse: rgba(0, 0, 0, 0.75);
+ --background-color: #f5f4f3;
+ --main-text-color: #475569;
+ --water: #44b9da;
+ --leaf: #7b990a;
+ --flower: #d1684e;
}
}
/* Amber Dark scheme (Forced) */
/* Enabled if forced with data-theme="dark" */
[data-theme="dark"] {
- --primary: #ffb300;
- --primary-hover: #ffc107;
+ --primary: #fee285;
+ --primary-hover: #fcc50b;
--primary-focus: rgba(255, 179, 0, 0.25);
--primary-inverse: rgba(0, 0, 0, 0.75);
+ --background-color: #f5f4f3;
+ --main-text-color: #475569;
+ --water: #44b9da;
+ --leaf: #7b990a;
+ --flower: #d1684e;
}
/* Amber (Common styles) */
:root {
@@ -37,8 +52,10 @@
.khoj-configure {
display: grid;
grid-template-columns: 1fr;
- padding: 0 24px;
+ font-family: roboto, karma, segoe ui, sans-serif;
+ font-weight: 300;
}
+
.khoj-header {
display: grid;
grid-auto-flow: column;
@@ -64,7 +81,7 @@ a.khoj-logo {
}
.khoj-nav a {
- color: #333;
+ color: var(--main-text-color);
text-decoration: none;
font-size: small;
font-weight: normal;
@@ -75,8 +92,9 @@ a.khoj-logo {
}
.khoj-nav a:hover {
background-color: var(--primary-hover);
+ color: var(--main-text-color);
}
-.khoj-nav-selected {
+a.khoj-nav-selected {
background-color: var(--primary);
}
img.khoj-logo {
@@ -85,21 +103,6 @@ img.khoj-logo {
justify-self: center;
}
-a.khoj-banner {
- color: black;
- text-decoration: none;
-}
-
-p.khoj-banner {
- font-size: small;
- margin: 0;
- padding: 10px;
-}
-
-p#khoj-banner {
- display: inline;
-}
-
@media only screen and (max-width: 600px) {
div.khoj-header {
display: grid;
diff --git a/src/interface/desktop/assets/three.min.js b/src/interface/desktop/assets/three.min.js
new file mode 100644
index 00000000..57018496
--- /dev/null
+++ b/src/interface/desktop/assets/three.min.js
@@ -0,0 +1,991 @@
+// threejs.org/license
+'use strict';var THREE={REVISION:"77"};"function"===typeof define&&define.amd?define("three",THREE):"undefined"!==typeof exports&&"undefined"!==typeof module&&(module.exports=THREE);void 0===Number.EPSILON&&(Number.EPSILON=Math.pow(2,-52));void 0===Math.sign&&(Math.sign=function(a){return 0>a?-1:0
+
+
@@ -284,8 +399,7 @@
@@ -298,8 +412,8 @@
}
body {
display: grid;
- background: #fff;
- color: #475569;
+ background: var(--background-color);
+ color: var(--main-text-color);
text-align: center;
font-family: roboto, karma, segoe ui, sans-serif;
font-size: small;
@@ -433,6 +547,83 @@
box-shadow: 0 0 12px rgb(119, 156, 46);
}
+ div.collapsed {
+ display: none;
+ }
+
+ div.expanded {
+ display: block;
+ }
+
+ div.reference {
+ display: grid;
+ grid-template-rows: auto;
+ grid-auto-flow: row;
+ grid-column-gap: 10px;
+ grid-row-gap: 10px;
+ margin: 10px;
+ }
+
+ div.expanded.reference-section {
+ display: grid;
+ grid-template-rows: auto;
+ grid-auto-flow: row;
+ grid-column-gap: 10px;
+ grid-row-gap: 10px;
+ margin: 10px;
+ }
+
+ button.reference-button {
+ background: var(--background-color);
+ color: var(--main-text-color);
+ border: 1px solid var(--main-text-color);
+ border-radius: 5px;
+ padding: 5px;
+ font-size: 14px;
+ font-weight: 300;
+ line-height: 1.5em;
+ cursor: pointer;
+ transition: background 0.2s ease-in-out;
+ text-align: left;
+ max-height: 75px;
+ transition: max-height 0.3s ease-in-out;
+ overflow: hidden;
+ }
+ button.reference-button.expanded {
+ max-height: none;
+ }
+
+ button.reference-button::before {
+ content: "▶";
+ margin-right: 5px;
+ display: inline-block;
+ transition: transform 0.3s ease-in-out;
+ }
+
+ button.reference-button:active:before,
+ button.reference-button[aria-expanded="true"]::before {
+ transform: rotate(90deg);
+ }
+
+ button.reference-expand-button {
+ background: var(--background-color);
+ color: var(--main-text-color);
+ border: 1px dotted var(--main-text-color);
+ border-radius: 5px;
+ padding: 5px;
+ font-size: 14px;
+ font-weight: 300;
+ line-height: 1.5em;
+ cursor: pointer;
+ transition: background 0.4s ease-in-out;
+ text-align: left;
+ }
+
+ button.reference-expand-button:hover {
+ background: var(--primary-hover);
+ }
+
+
.option-enabled:focus {
outline: none !important;
border:1px solid #475569;
@@ -445,6 +636,11 @@
border-bottom: 1px dotted #475569;
}
+ div.khoj-empty-container {
+ padding: 0;
+ margin: 0;
+ }
+
@media (pointer: coarse), (hover: none) {
abbr[title] {
position: relative;
@@ -481,12 +677,6 @@
margin: 4px;
grid-template-columns: auto;
}
- a.khoj-banner {
- display: block;
- }
- p.khoj-banner {
- padding: 0;
- }
}
@media only screen and (min-width: 600px) {
body {
@@ -498,11 +688,6 @@
}
}
- div.khoj-banner-container {
- padding: 0px;
- margin: 0px;
- }
-
div#chat-tooltip {
text-align: left;
font-size: medium;
@@ -524,23 +709,6 @@
text-align: center;
}
- button#khoj-banner-submit,
- input#khoj-banner-email {
- padding: 10px;
- border-radius: 5px;
- border: 1px solid #475569;
- background: #f9fafc;
- }
-
- button#khoj-banner-submit:hover,
- input#khoj-banner-email:hover {
- box-shadow: 0 0 11px #aaa;
- }
- div.khoj-banner-container-hidden {
- margin: 0px;
- padding: 0px;
- }
-
div.programmatic-output {
background-color: #f5f5f5;
border: 1px solid #ddd;
diff --git a/src/interface/desktop/config.html b/src/interface/desktop/config.html
index 04599bb1..fb39fbb8 100644
--- a/src/interface/desktop/config.html
+++ b/src/interface/desktop/config.html
@@ -2,89 +2,105 @@
- Khoj - Search
+ Khoj - Settings
-
-
+
-
-
-
-
-
-
-
-
+
+
-
-
-
+
+
+
+
+
+
-
-
- 
-
- - Host -
+
+
+
-
+
+
+
+ + Server URL +
+
+
+
+
+ 
+
+
+ + API Key +
+
+
+
-
-
-
- 
-
+
- - Files -
+
@@ -93,7 +109,7 @@
body {
display: grid;
grid-template-columns: 1fr;
- grid-template-rows: 1fr auto auto auto minmax(80px, 100%);
+ grid-template-rows: 1fr auto;
font-size: small!important;
}
body > * {
@@ -104,8 +120,7 @@
body {
display: grid;
grid-template-columns: 1fr min(70vw, 100%) 1fr;
- grid-template-rows: 1fr auto auto auto minmax(80px, 100%);
- padding-top: 60vw;
+ grid-template-rows: 80px auto;
}
body > * {
grid-column: 2;
@@ -114,7 +129,7 @@
body, input {
padding: 0px;
margin: 0px;
- background: #fff;
+ background: var(--background-color);
color: #475569;
font-family: roboto, karma, segoe ui, sans-serif;
font-size: small;
@@ -126,11 +141,6 @@
margin: 10px;
}
- div.page {
- padding: 0px;
- margin: 0px;
- }
-
svg {
transition: transform 0.3s ease-in-out;
}
@@ -167,19 +177,19 @@
}
}
- #khoj-host-url {
+ .card-input {
padding: 4px;
- box-shadow: 0 0 2px 1px rgba(0, 0, 0, 0.2);
+ box-shadow: 0 0 2px 1px rgba(0, 0, 0, 0.3);
border: none;
+ width: 450px;
}
.card {
display: grid;
- /* grid-template-rows: repeat(3, 1fr); */
gap: 8px;
padding: 24px 16px;
- width: 100%;
- background: white;
+ width: 450px;
+ background: var(--background-color);
border: 1px solid rgb(229, 229, 229);
border-radius: 4px;
box-shadow: 0px 1px 3px 0px rgba(0,0,0,0.1),0px 1px 2px -1px rgba(0,0,0,0.1);
@@ -188,15 +198,15 @@
.section-cards {
display: grid;
- grid-template-columns: repeat(1, 1fr);
gap: 16px;
- justify-items: start;
+ justify-items: center;
margin: 0;
- width: auto;
}
-
- div.configuration {
- width: auto;
+ .section-action-row {
+ display: grid;
+ grid-auto-flow: column;
+ gap: 16px;
+ height: fit-content;
}
.card-title-row {
@@ -247,7 +257,7 @@
}
.primary-button {
border: none;
- color: white;
+ color: var(--background-color);
padding: 15px 32px;
text-align: center;
text-decoration: none;
@@ -256,7 +266,7 @@
}
button.card-button.disabled {
- color: rgb(255, 136, 136);
+ color: var(--flower);
background: transparent;
font-size: small;
cursor: pointer;
@@ -268,11 +278,7 @@
}
button.card-button.happy {
- color: rgb(0, 146, 0);
- }
-
- button.card-button.happy {
- color: rgb(0, 146, 0);
+ color: var(--leaf);
}
img.configured-icon {
@@ -296,13 +302,14 @@
div.folder-element {
display: grid;
grid-template-columns: auto 1fr;
- box-shadow: 0 0 2px 1px rgba(0, 0, 0, 0.2);
+ border: 1px solid rgb(229, 229, 229);
+ border-radius: 4px;
+ box-shadow: 0px 1px 3px 0px rgba(0,0,0,0.1),0px 1px 2px -1px rgba(0,0,0,0.8);
padding: 4px;
margin-bottom: 8px;
}
div.content-name {
- width: 500px;
overflow-wrap: break-word;
}
@@ -315,7 +322,7 @@
background-color: rgb(253 214 214);
border-radius: 3px;
border: none;
- color: rgb(207, 67, 59);
+ color: var(--flower);
padding: 4px;
}
@@ -324,14 +331,14 @@
background-color: rgb(255 235 235);
border-radius: 3px;
border: none;
- color: rgb(207, 67, 59);
+ color: var(--flower);
padding: 4px;
cursor: pointer;
}
- #sync-data {
- background-color: #ffb300;
+ button.sync-data {
+ background-color: var(--primary);
border: none;
- color: white;
+ color: var(--main-text-color);
padding: 12px;
text-align: center;
text-decoration: none;
@@ -340,43 +347,20 @@
border-radius: 4px;
cursor: pointer;
transition: background-color 0.3s ease;
- box-shadow: 0px 5px 0px #f9f5de;
+ box-shadow: 0px 5px 0px var(--background-color);
}
- #sync-data:hover {
- background-color: #ffcc00;
- box-shadow: 0px 3px 0px #f9f5de;
+ button.sync-data:hover {
+ background-color: var(--primary-hover);
+ box-shadow: 0px 3px 0px var(--background-color);
+ }
+ .sync-force-toggle {
+ align-content: center;
+ display: grid;
+ grid-auto-flow: column;
+ gap: 4px;
}
-
+
-
+
+
+
+ + Files + +
+
+
+
+
+
-
+
-
-
-
-
-
-
- 
-
+
- - Folders -
+
+
+
+
+
+
+
+ + Folders + +
+
+
+
+
+
-
+
+
+
+
+
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+