From 66dca6cf33b6d4090ca2627ff361609ef055210f Mon Sep 17 00:00:00 2001 From: Debanjum Singh Solanky Date: Mon, 6 Feb 2023 19:38:20 -0300 Subject: [PATCH] Add Docs to Search across Languages, Uninstall Khoj to Readme Add details and fixes to Obsidian, Main readme based on feedback, confusion from the Obsidian plugin announcement --- README.md | 23 ++++++++++++++++++++++- src/interface/obsidian/README.md | 6 +++++- 2 files changed, 27 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index debbbb5d..0b879b51 100644 --- a/README.md +++ b/README.md @@ -29,11 +29,13 @@ - [Khoj Server](#upgrade-khoj-server) - [Khoj.el](#upgrade-khoj-on-emacs) - [Khoj Obsidian](#upgrade-khoj-on-obsidian) +- [Uninstall Khoj](#uninstall-khoj) - [Troubleshoot](#Troubleshoot) - [Advanced Usage](#advanced-usage) - [Access Khoj on Mobile](#access-khoj-on-mobile) - [Chat with Notes](#chat-with-notes) - [Use OpenAI Models for Search](#use-openai-models-for-search) + - [Search across Different Languages](#search-across-different-languages) - [Miscellaneous](#Miscellaneous) - [Setup OpenAI API key in Khoj](#set-your-openai-api-key-in-khoj) - [Beta API](#beta-api) @@ -173,6 +175,12 @@ pip install --upgrade khoj-assistant - Upgrade via the Community plugins tab on the settings pane in the Obsidian app - See the [khoj plugin readme](https://github.com/debanjum/khoj/tree/master/src/interface/obsidian#2-Setup-Plugin) for details +## Uninstall Khoj +1. (Optional) Hit `Ctrl-C` in the terminal running the khoj server to stop it +2. Delete the khoj directory in your home folder (i.e `~/.khoj` on Linux, Mac or `C:\Users\\.khoj` on Windows) +3. Uninstall the khoj server with `pip uninstall khoj-assistant` +4. (Optional) Uninstall khoj.el or the khoj obsidian plugin in the standard way on Emacs, Obsidian + ## Troubleshoot #### Install fails while building Tokenizer dependency @@ -217,7 +225,7 @@ pip install --upgrade khoj-assistant #### Use 1. [Setup your OpenAI API key in Khoj](#set-your-openai-api-key-in-khoj) -2. Open [/chat?type=summarize](http://localhost:8000/chat?type=summarize)[^2] +2. Open [/chat?t=summarize](http://localhost:8000/chat?t=summarize)[^2] 3. Type your queries, see summarized response by Khoj from your notes #### Demo @@ -246,6 +254,19 @@ pip install --upgrade khoj-assistant - You will be **charged by OpenAI** based on the total tokens processed - It *requires an active internet connection* to search and index +### Search across Different Languages + To search for notes in multiple, different languages, you can use a [multi-lingual model](https://www.sbert.net/docs/pretrained_models.html#multi-lingual-models).
+ For example, the [paraphrase-multilingual-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2) supports [50+ languages](https://www.sbert.net/docs/pretrained_models.html#:~:text=we%20used%20the%20following%2050%2B%20languages), has good search quality and speed. To use it: + 1. Manually update `search-type > asymmetric > encoder` to `sentence-transformer/paraphrase-multilingual-MiniLM-L12-v2` in your `~/.khoj/khoj.yml` file for now. See diff of `khoj.yml` below for illustration: + ```diff + asymmetric: +- encoder: "sentence-transformers/multi-qa-MiniLM-L6-cos-vi" ++ encoder: "sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2" + cross-encoder: "cross-encoder/ms-marco-MiniLM-L-6-v2" + model_directory: "~/.khoj/search/asymmetric/" + ``` + + 2. Regenerate your content index. For example, by opening [\/api/update?t=force](http://localhost:8000/api/update?t=force) ## Miscellaneous ### Set your OpenAI API key in Khoj diff --git a/src/interface/obsidian/README.md b/src/interface/obsidian/README.md index 60ecc93e..a90b0d5a 100644 --- a/src/interface/obsidian/README.md +++ b/src/interface/obsidian/README.md @@ -44,8 +44,10 @@ https://user-images.githubusercontent.com/6413477/210486007-36ee3407-e6aa-4185-8 ![](https://github.com/debanjum/khoj/blob/master/src/interface/obsidian/docs/khoj_on_obsidian_0.2.5.png?) ## Setup -### 1. Setup Backend +*Note: Ensure you follow the ordering of the setup steps. Install the plugin after starting the khoj backend. This allows the plugin to configure the khoj backend.* +### 1. Setup Backend +Run below command in your terminal to install and start the khoj backend ```shell pip install khoj-assistant && khoj --no-gui ``` @@ -59,6 +61,8 @@ See [official docs](https://help.obsidian.md/Advanced+topics/Community+plugins#D ### Search Click the *Khoj search* icon 🔎 on the [Ribbon](https://help.obsidian.md/User+interface/Workspace/Ribbon) or run *Khoj: Search* from the [Command Palette](https://help.obsidian.md/Plugins/Command+palette) +*Note: Ensure the khoj server is running in the background before searching. Execute `khoj --no-gui` in your terminal if it is not already running* +
Query Filters Use structured query syntax to filter the natural language search results