Add Docs to Search across Languages, Uninstall Khoj to Readme

Add details and fixes to Obsidian, Main readme
based on feedback, confusion from the Obsidian plugin announcement
This commit is contained in:
Debanjum Singh Solanky 2023-02-06 19:38:20 -03:00
parent cba9a6a703
commit 66dca6cf33
2 changed files with 27 additions and 2 deletions

View file

@ -29,11 +29,13 @@
- [Khoj Server](#upgrade-khoj-server) - [Khoj Server](#upgrade-khoj-server)
- [Khoj.el](#upgrade-khoj-on-emacs) - [Khoj.el](#upgrade-khoj-on-emacs)
- [Khoj Obsidian](#upgrade-khoj-on-obsidian) - [Khoj Obsidian](#upgrade-khoj-on-obsidian)
- [Uninstall Khoj](#uninstall-khoj)
- [Troubleshoot](#Troubleshoot) - [Troubleshoot](#Troubleshoot)
- [Advanced Usage](#advanced-usage) - [Advanced Usage](#advanced-usage)
- [Access Khoj on Mobile](#access-khoj-on-mobile) - [Access Khoj on Mobile](#access-khoj-on-mobile)
- [Chat with Notes](#chat-with-notes) - [Chat with Notes](#chat-with-notes)
- [Use OpenAI Models for Search](#use-openai-models-for-search) - [Use OpenAI Models for Search](#use-openai-models-for-search)
- [Search across Different Languages](#search-across-different-languages)
- [Miscellaneous](#Miscellaneous) - [Miscellaneous](#Miscellaneous)
- [Setup OpenAI API key in Khoj](#set-your-openai-api-key-in-khoj) - [Setup OpenAI API key in Khoj](#set-your-openai-api-key-in-khoj)
- [Beta API](#beta-api) - [Beta API](#beta-api)
@ -173,6 +175,12 @@ pip install --upgrade khoj-assistant
- Upgrade via the Community plugins tab on the settings pane in the Obsidian app - Upgrade via the Community plugins tab on the settings pane in the Obsidian app
- See the [khoj plugin readme](https://github.com/debanjum/khoj/tree/master/src/interface/obsidian#2-Setup-Plugin) for details - See the [khoj plugin readme](https://github.com/debanjum/khoj/tree/master/src/interface/obsidian#2-Setup-Plugin) for details
## Uninstall Khoj
1. (Optional) Hit `Ctrl-C` in the terminal running the khoj server to stop it
2. Delete the khoj directory in your home folder (i.e `~/.khoj` on Linux, Mac or `C:\Users\<your-username>\.khoj` on Windows)
3. Uninstall the khoj server with `pip uninstall khoj-assistant`
4. (Optional) Uninstall khoj.el or the khoj obsidian plugin in the standard way on Emacs, Obsidian
## Troubleshoot ## Troubleshoot
#### Install fails while building Tokenizer dependency #### Install fails while building Tokenizer dependency
@ -217,7 +225,7 @@ pip install --upgrade khoj-assistant
#### Use #### Use
1. [Setup your OpenAI API key in Khoj](#set-your-openai-api-key-in-khoj) 1. [Setup your OpenAI API key in Khoj](#set-your-openai-api-key-in-khoj)
2. Open [/chat?type=summarize](http://localhost:8000/chat?type=summarize)[^2] 2. Open [/chat?t=summarize](http://localhost:8000/chat?t=summarize)[^2]
3. Type your queries, see summarized response by Khoj from your notes 3. Type your queries, see summarized response by Khoj from your notes
#### Demo #### Demo
@ -246,6 +254,19 @@ pip install --upgrade khoj-assistant
- You will be **charged by OpenAI** based on the total tokens processed - You will be **charged by OpenAI** based on the total tokens processed
- It *requires an active internet connection* to search and index - It *requires an active internet connection* to search and index
### Search across Different Languages
To search for notes in multiple, different languages, you can use a [multi-lingual model](https://www.sbert.net/docs/pretrained_models.html#multi-lingual-models).<br />
For example, the [paraphrase-multilingual-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2) supports [50+ languages](https://www.sbert.net/docs/pretrained_models.html#:~:text=we%20used%20the%20following%2050%2B%20languages), has good search quality and speed. To use it:
1. Manually update `search-type > asymmetric > encoder` to `sentence-transformer/paraphrase-multilingual-MiniLM-L12-v2` in your `~/.khoj/khoj.yml` file for now. See diff of `khoj.yml` below for illustration:
```diff
asymmetric:
- encoder: "sentence-transformers/multi-qa-MiniLM-L6-cos-vi"
+ encoder: "sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2"
cross-encoder: "cross-encoder/ms-marco-MiniLM-L-6-v2"
model_directory: "~/.khoj/search/asymmetric/"
```
2. Regenerate your content index. For example, by opening [\<khoj-url\>/api/update?t=force](http://localhost:8000/api/update?t=force)
## Miscellaneous ## Miscellaneous
### Set your OpenAI API key in Khoj ### Set your OpenAI API key in Khoj

View file

@ -44,8 +44,10 @@ https://user-images.githubusercontent.com/6413477/210486007-36ee3407-e6aa-4185-8
![](https://github.com/debanjum/khoj/blob/master/src/interface/obsidian/docs/khoj_on_obsidian_0.2.5.png?) ![](https://github.com/debanjum/khoj/blob/master/src/interface/obsidian/docs/khoj_on_obsidian_0.2.5.png?)
## Setup ## Setup
### 1. Setup Backend *Note: Ensure you follow the ordering of the setup steps. Install the plugin after starting the khoj backend. This allows the plugin to configure the khoj backend.*
### 1. Setup Backend
Run below command in your terminal to install and start the khoj backend
```shell ```shell
pip install khoj-assistant && khoj --no-gui pip install khoj-assistant && khoj --no-gui
``` ```
@ -59,6 +61,8 @@ See [official docs](https://help.obsidian.md/Advanced+topics/Community+plugins#D
### Search ### Search
Click the *Khoj search* icon 🔎 on the [Ribbon](https://help.obsidian.md/User+interface/Workspace/Ribbon) or run *Khoj: Search* from the [Command Palette](https://help.obsidian.md/Plugins/Command+palette) Click the *Khoj search* icon 🔎 on the [Ribbon](https://help.obsidian.md/User+interface/Workspace/Ribbon) or run *Khoj: Search* from the [Command Palette](https://help.obsidian.md/Plugins/Command+palette)
*Note: Ensure the khoj server is running in the background before searching. Execute `khoj --no-gui` in your terminal if it is not already running*
<details><summary>Query Filters</summary> <details><summary>Query Filters</summary>
Use structured query syntax to filter the natural language search results Use structured query syntax to filter the natural language search results