sij/khoj
1
0
Fork 0
mirror of https://github.com/khoj-ai/khoj.git synced 2024-11-30 10:53:02 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

Simplify setup, features since Khoj cloud in docs

Browse source 
- No Khoj server setup required to start using Khoj from Obsidian, Emacs
- Use tabs for install, upgrade in Emacs with different package
  managers
- Use default subtitles in Khoj Docs
- Deduplicate query filters, remove backend setup instructions in
  plugin pages
- Remove stale Setup demo on Khoj Obsidian plugin docs
This commit is contained in:
Debanjum Singh Solanky 2023-11-18 17:25:52 -08:00
parent e1bf1f0e86
commit 35b469e488
5 changed files with 99 additions and 173 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

25
docs/README.md
View file

@ -9,7 +9,7 @@
</div> </div>
<div align="center"> <div align="center">
<b>An AI personal assistant for your digital brain</b> <b>An AI copilot for your Second Brain</b>
</div> </div>
@ -24,20 +24,19 @@
</div> </div>
## Introduction ## Introduction
Welcome to the Khoj Docs! This is the best place to [get started](./setup.md) with Khoj. We have instructions on self-hosting, using Khoj with Emacs, Obsidian, and the Web, and more. We also include setup instructions for users on the hosted instance at [app.khoj.dev](https://app.khoj.dev). Welcome to the Khoj Docs! This is the best place to get setup and explore Khoj's features.
- Khoj is an application to dynamically engage with your notes, documents and images. We support APIs for [semantic search](./search.md) and [chat](./chat.md). - Khoj is an open source, personal AI
- It can be easily self-hosted and run on your consumer hardware or private cloud. - You can [chat](./chat.md) with it about anything. When relevant, it'll use any notes or documents you shared with it to respond
- It provides an open source, AI personal assistant accessible from your [Emacs](./emacs.md), [Obsidian](./obsidian.md) or [Web browser](./web.md), or our [desktop app](https://khoj.dev/downloads). - Use natural language to quickly [find](./search.md) relevant notes and documents
- It works with plaintext, markdown, [notion](./notion_integration.md) org-mode, pdf files and [github repositories](./github_integration.md) - Access it from your [Emacs](./emacs.md), [Obsidian](./obsidian.md), [Web browser](./web.md) or the [Khoj Desktop app](https://khoj.dev/downloads)
- It can support use with multiple users. If you're self-hosting, your family, friends, or team can have a shared assistance server. You'll the the suite of server admin settings at `/server/admin`. - It understands pdf, plaintext, markdown, org-mode files, [notion pages](./notion_integration.md) and [github repositories](./github_integration.md)
- You can self-host and run Khoj on your consumer hardware or private cloud
- It supports multiple users. If you want, you can self-host, a shared assistance server for your family, friends, or team
## Quickstart ## Quickstart
[Click here](./setup.md) for full setup instructions - [Try Khoj Cloud](https://app.khoj.dev) to get started quickly
- [Read these instructions](./setup.md) to self-host a private instance of Khoj
```shell
pip install khoj-assistant && khoj
```
## Overview ## Overview
<img src="https://docs.khoj.dev/assets/khoj_search_on_web.png" width="400px"> <img src="https://docs.khoj.dev/assets/khoj_search_on_web.png" width="400px">
@ -45,7 +44,7 @@ pip install khoj-assistant && khoj
<img src="https://docs.khoj.dev/assets/khoj_chat_on_web.png" width="400px"> <img src="https://docs.khoj.dev/assets/khoj_chat_on_web.png" width="400px">
#### [Search](./search.md) #### [Search](./search.md)
- **Local**: Your personal data stays local. All search and indexing is done on your machine. - **Natural**: Use natural language queries to quickly find relevant notes and documents.
- **Incremental**: Incremental search for a fast, search-as-you-type experience - **Incremental**: Incremental search for a fast, search-as-you-type experience
#### [Chat](./chat.md) #### [Chat](./chat.md)

4
docs/chat.md
View file

@ -1,5 +1,5 @@
### Khoj Chat ## Khoj Chat
#### Overview ### Overview
- Creates a personal assistant for you to inquire and engage with your notes - Creates a personal assistant for you to inquire and engage with your notes
- You can choose to use Online or Offline Chat depending on your requirements - You can choose to use Online or Offline Chat depending on your requirements
- Supports multi-turn conversations with the relevant notes for context - Supports multi-turn conversations with the relevant notes for context

137
docs/emacs.md
View file

@ -1,6 +1,6 @@
<h1><img src="./assets/khoj-logo-sideways-500.png" width="200" alt="Khoj Logo">Emacs</h1> <h1><img src="./assets/khoj-logo-sideways-500.png" width="200" alt="Khoj Logo"> Emacs</h1>
> An AI personal assistance for your digital brain > An AI copilot for your Second Brain in Emacs
<img src="https://stable.melpa.org/packages/khoj-badge.svg" width="150" alt="Melpa Stable Badge"> <img src="https://stable.melpa.org/packages/khoj-badge.svg" width="150" alt="Melpa Stable Badge">
<img src="https://melpa.org/packages/khoj-badge.svg" width="150" alt="Melpa Badge"> <img src="https://melpa.org/packages/khoj-badge.svg" width="150" alt="Melpa Badge">
@ -10,14 +10,13 @@
## Features ## Features
- **Chat**
- **Faster answers**: Find answers quickly, from your private notes or the public internet
- **Assisted creativity**: Smoothly weave across retrieving answers and generating content
- **Iterative discovery**: Iteratively explore and re-discover your notes
- **Search** - **Search**
- **Natural**: Advanced natural language understanding using Transformer based ML Models - **Natural**: Advanced natural language understanding using Transformer based ML Models
- **Local**: Your personal data stays local. All search, indexing is done on your machine*
- **Incremental**: Incremental search for a fast, search-as-you-type experience - **Incremental**: Incremental search for a fast, search-as-you-type experience
- **Chat**
- **Faster answers**: Find answers faster than search
- **Iterative discovery**: Iteratively explore and (re-)discover your notes
- **Assisted creativity**: Smoothly weave across answer retrieval and content generation
## Interface ## Interface
#### Search #### Search
@ -27,79 +26,76 @@
![khoj chat on emacs](./assets/khoj_chat_on_emacs.png ':size=400px') ![khoj chat on emacs](./assets/khoj_chat_on_emacs.png ':size=400px')
## Setup ## Setup
- *Make sure [python](https://realpython.com/installing-python/) and [pip](https://pip.pypa.io/en/stable/installation/) are installed on your machine* 1. Generate an API key on the [Khoj Web App](https://app.khoj.dev/config#clients)
2. Add below snippet to your Emacs config file, usually at `~/.emacs.d/init.el`
- *khoj.el attempts to automatically install, start and configure the khoj server.*
If this fails, follow [these instructions](/setup) to manually setup the khoj server.
### Direct Install <!-- tabs:start -->
#### **Direct Install**
*Khoj will index your org-agenda files, by default*
```elisp ```elisp
;; Install Khoj.el
M-x package-install khoj M-x package-install khoj
; Set your Khoj API key
(setq khoj-api-key "YOUR_KHOJ_CLOUD_API_KEY")
``` ```
### Minimal Install #### **Minimal Install**
Add below snippet to your Emacs config file. *Khoj will index your org-agenda files, by default*
Indexes your org-agenda files, by default.
```elisp ```elisp
;; Install Khoj Package from MELPA Stable ;; Install Khoj client from MELPA Stable
(use-package khoj
:ensure t
:pin melpa-stable
:bind ("C-c s" . 'khoj))
```
- Note: Install `khoj.el` from MELPA (instead of MELPA Stable) if you installed the pre-release version of khoj
- That is, use `:pin melpa` to install khoj.el in above snippet if khoj server was installed with `--pre` flag, i.e `pip install --pre khoj-assistant`
- Else use `:pin melpa-stable` to install khoj.el in above snippet if khoj was installed with `pip install khoj-assistant`
- This ensures both khoj.el and khoj app are from the same version (git tagged or latest)
### Standard Install
Add below snippet to your Emacs config file.
Indexes the specified org files, directories. Sets up OpenAI API key for Khoj Chat
```elisp
;; Install Khoj Package from MELPA Stable
(use-package khoj (use-package khoj
:ensure t :ensure t
:pin melpa-stable :pin melpa-stable
:bind ("C-c s" . 'khoj) :bind ("C-c s" . 'khoj)
:config (setq khoj-org-directories '("~/docs/org-roam" "~/docs/notes") :config (setq khoj-api-key "YOUR_KHOJ_CLOUD_API_KEY"))
khoj-org-files '("~/docs/todo.org" "~/docs/work.org")
khoj-openai-api-key "YOUR_OPENAI_API_KEY")) ; required to enable chat
``` ```
### With [ Straight.el](https://github.com/raxod502/straight.el) #### **Standard Install**
Add below snippet to your Emacs config file. *Configures the specified org files, directories to be indexed by Khoj*
Indexes the specified org files, directories. Sets up OpenAI API key for Khoj Chat
```elisp ```elisp
;; Install Khoj Package using Straight.el ;; Install Khoj client from MELPA Stable
(use-package khoj (use-package khoj
:after org :ensure t
:straight (khoj :type git :host github :repo "khoj-ai/khoj" :files (:defaults "src/interface/emacs/khoj.el")) :pin melpa-stable
:bind ("C-c s" . 'khoj) :bind ("C-c s" . 'khoj)
:config (setq khoj-org-directories '("~/docs/org-roam" "~/docs/notes") :config (setq khoj-api-key "YOUR_KHOJ_CLOUD_API_KEY"
khoj-org-files '("~/docs/todo.org" "~/docs/work.org") khoj-org-directories '("~/docs/org-roam" "~/docs/notes")
khoj-openai-api-key "YOUR_OPENAI_API_KEY" ; required to enable chat) khoj-org-files '("~/docs/todo.org" "~/docs/work.org")))
``` ```
#### **Straight.el**
*Configures the specified org files, directories to be indexed by Khoj*
```elisp
;; Install Khoj client using Straight.el
(use-package khoj
:after org
:straight (khoj :type git :host github :repo "khoj-ai/khoj" :files (:defaults "src/interface/emacs/khoj.el"))
:bind ("C-c s" . 'khoj)
:config (setq khoj-api-key "YOUR_KHOJ_CLOUD_API_KEY"
khoj-org-directories '("~/docs/org-roam" "~/docs/notes")
khoj-org-files '("~/docs/todo.org" "~/docs/work.org")))
```
<!-- tabs:end -->
## Use ## Use
### Search ### Search
See [Khoj Search](/search) for details
1. Hit `C-c s s` (or `M-x khoj RET s`) to open khoj search 1. Hit `C-c s s` (or `M-x khoj RET s`) to open khoj search
2. Enter your query in natural language<br/>
2. Enter your query in natural language E.g *"What is the meaning of life?"*, *"My life goals for 2023"*
e.g "What is the meaning of life?", "My life goals for 2023"
### Chat ### Chat
See [Khoj Chat](/chat) for details
1. Hit `C-c s c` (or `M-x khoj RET c`) to open khoj chat 1. Hit `C-c s c` (or `M-x khoj RET c`) to open khoj chat
2. Ask questions in a natural, conversational style<br/>
2. Ask questions in a natural, conversational style E.g *"When did I file my taxes last year?"*
E.g "When did I file my taxes last year?"
See [Khoj Chat](/#/chat) for more details
### Find Similar Entries ### Find Similar Entries
This feature finds entries similar to the one you are currently on. This feature finds entries similar to the one you are currently on.
@ -108,7 +104,6 @@ This feature finds entries similar to the one you are currently on.
### Advanced Usage ### Advanced Usage
- Add [query filters](https://github.com/khoj-ai/khoj/#query-filters) during search to narrow down results further - Add [query filters](https://github.com/khoj-ai/khoj/#query-filters) during search to narrow down results further
e.g `What is the meaning of life? -"god" +"none" dt>"last week"` e.g `What is the meaning of life? -"god" +"none" dt>"last week"`
- Use `C-c C-o 2` to open the current result at cursor in its source org file - Use `C-c C-o 2` to open the current result at cursor in its source org file
@ -121,31 +116,21 @@ This feature finds entries similar to the one you are currently on.
![](./assets/khoj_emacs_menu.png) ![](./assets/khoj_emacs_menu.png)
Hit `C-c s` (or `M-x khoj`) to open the khoj menu above. Then: Hit `C-c s` (or `M-x khoj`) to open the khoj menu above. Then:
- Hit `t` until you preferred content type is selected in the khoj menu - Hit `t` until you preferred content type is selected in the khoj menu
`Content Type` specifies the content to perform `Search`, `Update` or `Find Similar` actions on `Content Type` specifies the content to perform `Search`, `Update` or `Find Similar` actions on
- Hit `n` twice and then enter number of results you want to see - Hit `n` twice and then enter number of results you want to see
`Results Count` is used by the `Search` and `Find Similar` actions `Results Count` is used by the `Search` and `Find Similar` actions
- Hit `-f u` to `force` update the khoj content index - Hit `-f u` to `force` update the khoj content index
The `Force Update` switch is only used by the `Update` action The `Force Update` switch is only used by the `Update` action
## Upgrade ## Upgrade
### Upgrade Khoj Backend
```bash
pip install --upgrade khoj-assistant
```
### Upgrade Khoj.el
Use your Emacs package manager to upgrade `khoj.el` Use your Emacs package manager to upgrade `khoj.el`
<!-- tabs:start -->
- For `khoj.el` from MELPA #### **With MELPA**
- Method 1 1. Run `M-x package-refresh-content`
- Run `M-x package-list-packages` to list all packages 2. Run `M-x package-reinstall khoj`
- Press `U` on `khoj` to mark it for upgrade
- Press `x` to execute the marked actions
- Method 2
- Run `M-x package-refresh-content`
- Run `M-x package-reinstall khoj`
- For `khoj.el` from Straight #### **With Straight.el**
- Run `M-x straight-pull-package khoj` - Run `M-x straight-pull-package khoj`
<!-- tabs:end -->

91
docs/obsidian.md
View file

@ -1,16 +1,15 @@
<h1><img src="./assets/khoj-logo-sideways-500.png" width="200" alt="Khoj Logo">Obsidian</h1> <h1><img src="./assets/khoj-logo-sideways-500.png" width="200" alt="Khoj Logo"> Obsidian</h1>
> An AI personal assistant for your Digital Brain in Obsidian > An AI copilot for your Second Brain in Obsidian
## Features ## Features
- **Chat**
- **Faster answers**: Find answers quickly, from your private notes or the public internet
- **Assisted creativity**: Smoothly weave across retrieving answers and generating content
- **Iterative discovery**: Iteratively explore and re-discover your notes
- **Search** - **Search**
- **Natural**: Advanced natural language understanding using Transformer based ML Models - **Natural**: Advanced natural language understanding using Transformer based ML Models
- **Local**: Your personal data stays local. All search and indexing is done on your machine. *Unlike chat which requires access to GPT.*
- **Incremental**: Incremental search for a fast, search-as-you-type experience - **Incremental**: Incremental search for a fast, search-as-you-type experience
- **Chat**
- **Faster answers**: Find answers faster and with less effort than search
- **Iterative discovery**: Iteratively explore and (re-)discover your notes
- **Assisted creativity**: Smoothly weave across answers retrieval and content generation
## Interface ## Interface
![](./assets/khoj_search_on_obsidian.png ':size=400px') ![](./assets/khoj_search_on_obsidian.png ':size=400px')
@ -18,93 +17,37 @@
## Setup ## Setup
- *Ensure you follow the ordering of the setup steps. Install the plugin after starting the khoj backend. This allows the plugin to configure the khoj backend*
### 1. Setup Backend
Follow the instructions in [the setup page](./setup.md?id=_1-install)
### 2. Setup Plugin
1. Open [Khoj](https://obsidian.md/plugins?id=khoj) from the *Community plugins* tab in Obsidian settings panel 1. Open [Khoj](https://obsidian.md/plugins?id=khoj) from the *Community plugins* tab in Obsidian settings panel
2. Click *Install*, then *Enable* on the Khoj plugin page in Obsidian 2. Click *Install*, then *Enable* on the Khoj plugin page in Obsidian
3. Get an API key from the Khoj web settings page. Use this key to configure the Khoj plugin in Obsidian. 3. Generate an API key on the [Khoj Web App](https://app.khoj.dev/config#clients)
3. [Optional] To enable Khoj Chat, set your [OpenAI API key](https://platform.openai.com/account/api-keys) in the Khoj plugin settings 4. Set your Khoj API Key in the Khoj plugin settings in Obsidian
See [official Obsidian plugin docs](https://help.obsidian.md/Extending+Obsidian/Community+plugins) for details See the official [Obsidian Plugin Docs](https://help.obsidian.md/Extending+Obsidian/Community+plugins) for more details on installing Obsidian plugins.
## Use ## Use
### Chat ### Chat
Run *Khoj: Chat* from the [Command Palette](https://help.obsidian.md/Plugins/Command+palette) and ask questions in a natural, conversational style.<br /> Run *Khoj: Chat* from the [Command Palette](https://help.obsidian.md/Plugins/Command+palette) and ask questions in a natural, conversational style.<br />
E.g "When did I file my taxes last year?" E.g *"When did I file my taxes last year?"*
Notes:
- *Using Khoj Chat will result in query relevant notes being shared with OpenAI for ChatGPT to respond.*
- *To use Khoj Chat, ensure you've set your [OpenAI API key](https://platform.openai.com/account/api-keys) in the Khoj plugin settings.*
See [Khoj Chat](/chat) for more details See [Khoj Chat](/chat) for more details
### Search
Click the *Khoj search* icon 🔎 on the [Ribbon](https://help.obsidian.md/User+interface/Workspace/Ribbon) or run *Khoj: Search* from the [Command Palette](https://help.obsidian.md/Plugins/Command+palette)
*Note: Ensure the khoj server is running in the background before searching. Execute `khoj` in your terminal if it is not already running*
[search_demo](https://user-images.githubusercontent.com/6413477/218801155-cd67e8b4-a770-404a-8179-d6b61caa0f93.mp4 ':include :type=mp4')
#### Query Filters
Use structured query syntax to filter the natural language search results
- **Word Filter**: Get entries that include/exclude a specified term
- Entries that contain term_to_include: `+"term_to_include"`
- Entries that contain term_to_exclude: `-"term_to_exclude"`
- **Date Filter**: Get entries containing dates in YYYY-MM-DD format from specified date (range)
- Entries from April 1st 1984: `dt:"1984-04-01"`
- Entries after March 31st 1984: `dt>="1984-04-01"`
- Entries before April 2nd 1984 : `dt<="1984-04-01"`
- **File Filter**: Get entries from a specified file
- Entries from incoming.org file: `file:"incoming.org"`
- Combined Example
- `what is the meaning of life? file:"1984.org" dt>="1984-01-01" dt<="1985-01-01" -"big" -"brother"`
- Adds all filters to the natural language query. It should return entries
- from the file *1984.org*
- containing dates from the year *1984*
- excluding words *"big"* and *"brother"*
- that best match the natural language query *"what is the meaning of life?"*
### Find Similar Notes ### Find Similar Notes
To see other notes similar to the current one, run *Khoj: Find Similar Notes* from the [Command Palette](https://help.obsidian.md/Plugins/Command+palette) To see other notes similar to the current one, run *Khoj: Find Similar Notes* from the [Command Palette](https://help.obsidian.md/Plugins/Command+palette)
### Search
Click the *Khoj search* icon 🔎 on the [Ribbon](https://help.obsidian.md/User+interface/Workspace/Ribbon) or run *Khoj: Search* from the [Command Palette](https://help.obsidian.md/Plugins/Command+palette)
See [Khoj Search](/search) for more details. Use [query filters](/advanced#query-filters) to limit entries to search
[search_demo](https://user-images.githubusercontent.com/6413477/218801155-cd67e8b4-a770-404a-8179-d6b61caa0f93.mp4 ':include :type=mp4')
## Upgrade ## Upgrade
### 1. Upgrade Backend
```shell
pip install --upgrade khoj-assistant
```
### 2. Upgrade Plugin
1. Open *Community plugins* tab in Obsidian settings 1. Open *Community plugins* tab in Obsidian settings
2. Click the *Check for updates* button 2. Click the *Check for updates* button
3. Click the *Update* button next to Khoj, if available 3. Click the *Update* button next to Khoj, if available
## Demo
### Search Demo
[demo](https://github-production-user-asset-6210df.s3.amazonaws.com/6413477/240061700-3e33d8ea-25bb-46c8-a3bf-c92f78d0f56b.mp4 ':include :type=mp4')
#### Description
1. Install Khoj via `pip` and start Khoj backend
```shell
python -m pip install khoj-assistant && khoj
```
2. Install Khoj plugin via Community Plugins settings pane on Obsidian app
- Check the new Khoj plugin settings
- Wait for Khoj backend to index markdown, PDF files in the current Vault
- Open Khoj plugin on Obsidian via Search button on Left Pane
- Search \"*Announce plugin to folks*\" in the [Obsidian Plugin docs](https://marcus.se.net/obsidian-plugin-docs/)
- Jump to the [search result](https://marcus.se.net/obsidian-plugin-docs/publishing/submit-your-plugin)
## Troubleshooting ## Troubleshooting
- Open the Khoj plugin settings pane, to configure Khoj - Open the Khoj plugin settings pane, to configure Khoj
- Toggle Enable/Disable Khoj, if setting changes have not applied - Toggle Enable/Disable Khoj, if setting changes have not applied
- Click *Update* button to force index to refresh, if results are failing or stale - Click *Update* button to force index to refresh, if results are failing or stale
## Current Limitations
- The plugin loads the index of only one vault at a time.<br/>
So notes across multiple vaults **cannot** be searched at the same time

15
docs/web.md
View file

@ -1,19 +1,18 @@
<h1><img src="./assets/khoj-logo-sideways-500.png" width="200" alt="Khoj Logo">Web</h1> <h1><img src="./assets/khoj-logo-sideways-500.png" width="200" alt="Khoj Logo"> Web</h1>
> An AI personal assistant for your Digital Brain > An AI copilot for your Second Brain
## Features ## Features
- **Chat**
- **Faster answers**: Find answers quickly, from your private notes or the public internet
- **Assisted creativity**: Smoothly weave across retrieving answers and generating content
- **Iterative discovery**: Iteratively explore and re-discover your notes
- **Search** - **Search**
- **Natural**: Advanced natural language understanding using Transformer based ML Models - **Natural**: Advanced natural language understanding using Transformer based ML Models
- **Local**: Your personal data stays local. All search and indexing is done on your machine. *Unlike chat which requires access to GPT.*
- **Incremental**: Incremental search for a fast, search-as-you-type experience - **Incremental**: Incremental search for a fast, search-as-you-type experience
- **Chat**
- **Faster answers**: Find answers faster and with less effort than search
- **Iterative discovery**: Iteratively explore and (re-)discover your notes
- **Assisted creativity**: Smoothly weave across answers retrieval and content generation
## Setup ## Setup
The Khoj web interface is the default interface. It comes packaged with the khoj server. No setup required. The Khoj web app is the default interface to Khoj. You can access it from any web browser. Try it on [Khoj Cloud](https://app.khoj.dev)
## Interface ## Interface
![](./assets/khoj_search_on_web.png ':size=400px') ![](./assets/khoj_search_on_web.png ':size=400px')