Update Readme to Install Khoj using Pip by Default

- Instructions to Install Khoj using Docker have been moved to the
  Development Section
- Simplify Section Headings
This commit is contained in:
Debanjum Singh Solanky 2022-08-04 23:32:32 +03:00
parent 1374065092
commit a3b35fbb6e

123
Readme.md
View file

@ -13,17 +13,17 @@
- [Analysis](#Analysis) - [Analysis](#Analysis)
- [Architecture](#Architecture) - [Architecture](#Architecture)
- [Setup](#Setup) - [Setup](#Setup)
- [Clone](#1.-Clone) - [Install](#1-Install)
- [Configure](#2.-Configure) - [Configure](#2-Configure)
- [Run](#3.-Run) - [Run](#3-Run)
- [Use](#Use) - [Use](#Use)
- [Upgrade](#Upgrade) - [Upgrade](#Upgrade)
- [Troubleshoot](#Troubleshoot) - [Troubleshoot](#Troubleshoot)
- [Miscellaneous](#Miscellaneous) - [Miscellaneous](#Miscellaneous)
- [Development Setup](#Development-setup) - [Development](#Development)
- [Setup on Local Machine](#Setup-on-local-machine) - [Setup](#Setup-1)
- [Upgrade on Local Machine](#Upgrade-on-local-machine) - [Upgrade](#Upgrade-1)
- [Run Unit Tests](#Run-unit-tests) - [Test](#Test)
- [Performance](#Performance) - [Performance](#Performance)
- [Query Performance](#Query-performance) - [Query Performance](#Query-performance)
- [Indexing Performance](#Indexing-performance) - [Indexing Performance](#Indexing-performance)
@ -61,52 +61,45 @@
![](https://github.com/debanjum/khoj/blob/master/docs/khoj_architecture.png) ![](https://github.com/debanjum/khoj/blob/master/docs/khoj_architecture.png)
## Setup ## Setup
### 1. Install
### 1. Clone ``` shell
pip install khoj-assistant
``` shell ```
git clone https://github.com/debanjum/khoj && cd khoj
```
### 2. Configure ### 2. Configure
- Set `input-files` or `input-filter` in each relevant `content-type` section of [khoj_sample.yml](./config/khoj_sample.yml)
- **Required**: Update [docker-compose.yml](./docker-compose.yml) to mount your images, (org-mode or markdown) notes and beancount directories - Set `input-directories` field in `content-type.image` section
- **Optional**: Edit application configuration in [khoj_sample.yml](./config/khoj_sample.yml) - Delete `content-type` sections irrelevant for your use-case
### 3. Run ### 3. Run
``` shell
``` shell khoj config/khoj_sample.yml -vv
docker-compose up -d ```
``` Loads ML model, generates embeddings and exposes API to search notes, images, transactions etc specified in config YAML
*Note: The first run will take time. Let it run, it\'s mostly not hung, just generating embeddings*
## Use ## Use
- **Khoj via Web** - **Khoj via Web**
- Go to <http://localhost:8000/> or open [index.html](./src/interface/web/index.html) in your browser - Open <http://localhost:8000/>
- **Khoj via Emacs** - **Khoj via Emacs**
- [Install](https://github.com/debanjum/khoj/tree/master/src/interface/emacs#installation) [khoj.el](./src/interface/emacs/khoj.el) - [Install](https://github.com/debanjum/khoj/tree/master/src/interface/emacs#installation) [khoj.el](./src/interface/emacs/khoj.el)
- Run `M-x khoj <user-query>` - Run `M-x khoj <user-query>`
- **Khoj via API** - **Khoj via API**
- See [Khoj FastAPI Docs](http://localhost:8000/docs) - See [Khoj FastAPI Docs](http://localhost:8000/docs), [Khoj FastAPI ReDocs](http://localhost:8000/redocs)
- [Query](http://localhost:8000/search?q=%22what%20is%20the%20meaning%20of%20life%22)
- [Regenerate Embeddings](http://localhost:8000/regenerate?t=ledger)
- [Configure Application](https://localhost:8000/ui)
## Upgrade ## Upgrade
``` shell ``` shell
docker-compose build --pull pip install --upgrade khoj-assistant
``` ```
## Troubleshoot ## Troubleshoot
- Symptom: Errors out with \"Killed\" in error message - Symptom: Errors out complaining about Tensors mismatch, null etc
- Mitigation: Delete `content-type` > `image` section from `khoj_sample.yml`
- Symptom: Errors out with \"Killed\" in error message in Docker
- Fix: Increase RAM available to Docker Containers in Docker Settings - Fix: Increase RAM available to Docker Containers in Docker Settings
- Refer: [StackOverflow Solution](https://stackoverflow.com/a/50770267), [Configure Resources on Docker for Mac](https://docs.docker.com/desktop/mac/#resources) - Refer: [StackOverflow Solution](https://stackoverflow.com/a/50770267), [Configure Resources on Docker for Mac](https://docs.docker.com/desktop/mac/#resources)
- Symptom: Errors out complaining about Tensors mismatch, null etc
- Mitigation: Delete content-type > image section from `khoj_sample.yml`
## Miscellaneous ## Miscellaneous
@ -114,45 +107,35 @@ docker-compose build --pull
- It is disabled by default - It is disabled by default
- To use it add your `openai-api-key` to config.yml - To use it add your `openai-api-key` to config.yml
## Development Setup ## Development
### Setup
#### Using Docker
1. Clone
### Setup on Local Machine ``` shell
git clone https://github.com/debanjum/khoj && cd khoj
```
#### Using Pip 2. Configure
1. Install Dependencies
1. Python3, Pip \[Required\]
2. Virualenv \[Optional\]
3. Install Exiftool \[Optional\]
``` shell
sudo apt-get -y install libimage-exiftool-perl
```
2. Install Khoj - **Required**: Update [docker-compose.yml](./docker-compose.yml) to mount your images, (org-mode or markdown) notes and beancount directories
``` shell - **Optional**: Edit application configuration in [khoj_docker.yml](./config/khoj_docker.yml)
virtualenv -m python3 .venv && source .venv/bin/activate # Optional
pip install khoj-assistant
```
3. Configure 3. Run
- Configure files/directories to search in `content-type` section of `khoj_sample.yml`
- To run application on test data, update file paths containing `/data/` to `tests/data/` in `khoj_sample.yml`
- Example replace `/data/notes/*.org` with `tests/data/notes/*.org`
4. Run ``` shell
Load ML model, generate embeddings and expose API to query notes, images, transactions etc specified in config YAML docker-compose up -d
```
``` shell *Note: The first run will take time. Let it run, it\'s mostly not hung, just generating embeddings*
khoj -c=config/khoj_sample.yml -vv
```
#### Using Conda #### Using Conda
1. Install Dependencies 1. Install Dependencies
1. Install Python3 \[Required\] - [Install Conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html) \[Required\]
2. [Install Conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html) \[Required\] - Install Exiftool \[Optional\]
3. Install Exiftool \[Optional\] ``` shell
``` shell sudo apt -y install libimage-exiftool-perl
sudo apt-get -y install libimage-exiftool-perl ```
```
2. Install Khoj 2. Install Khoj
``` shell ``` shell
@ -162,9 +145,9 @@ docker-compose build --pull
``` ```
3. Configure 3. Configure
- Configure files/directories to search in `content-type` section of `khoj_sample.yml` - Set `input-files` or `input-filter` in each relevant `content-type` section of `khoj_sample.yml`
- To run application on test data, update file paths containing `/data/` to `tests/data/` in `khoj_sample.yml` - Set `input-directories` field in `image` `content-type` section
- Example replace `/data/notes/*.org` with `tests/data/notes/*.org` - Delete `content-type` sections irrelevant for your use-case
4. Run 4. Run
Load ML model, generate embeddings and expose API to query notes, images, transactions etc specified in config YAML Load ML model, generate embeddings and expose API to query notes, images, transactions etc specified in config YAML
@ -173,10 +156,12 @@ docker-compose build --pull
python3 -m src.main -c=config/khoj_sample.yml -vv python3 -m src.main -c=config/khoj_sample.yml -vv
``` ```
### Upgrade On Local Machine ### Upgrade
#### Using Pip
#### Using Docker
``` shell ``` shell
pip install --upgrade khoj-assistant docker-compose build --pull
``` ```
#### Using Conda #### Using Conda
@ -188,7 +173,7 @@ conda env update -f config/environment.yml
conda activate khoj conda activate khoj
``` ```
### Run Unit Tests ### Test
``` shell ``` shell
pytest pytest