sij/khoj: Mirror of khoj from Github

sij/khoj

mirror of https://github.com/khoj-ai/khoj.git synced 2024-11-27 17:35:07 +01:00

Mirror of khoj from Github

Find a file

Debanjum Singh Solanky c1369233db Consistently use "entry", "score" in json response for all search types - Had already made some progress on this earlier by updating the image search responses. But needed to update the text search responses to use lowercase entry and score - Update khoj.el to consume the updated json response keys for text search		2022-07-20 20:33:27 +04:00
.github/workflows	Run build on PR	2022-07-04 18:09:47 -04:00
config	Allow indexing multiple image directories for image search	2022-07-20 02:56:01 +04:00
src	Consistently use "entry", "score" in json response for all search types	2022-07-20 20:33:27 +04:00
tests	Consistently use "entry", "score" in json response for all search types	2022-07-20 20:33:27 +04:00
views	Fix input text behavior for null/empty value fields	2021-12-04 10:45:48 -05:00
.dockerignore	Make Docker ignore unnecessary files	2022-06-29 22:29:34 +04:00
.gitignore	Create Basic Landing Page to Query Semantic Search and Render Results	2022-07-16 03:36:19 +04:00
demo.mp4	Add demo of semantic search to repository	2022-05-14 04:29:25 -04:00
docker-compose.yml	Correct syntax of memory limit in docker-compose.yml	2022-07-06 20:07:11 -04:00
Dockerfile	Give the project a short, less generic name. Rename it to Khoj	2022-07-19 18:26:16 +04:00
LICENSE	Add Readme, License. Update .gitignore	2021-08-15 22:52:37 -07:00
README.org	Give the project a short, less generic name. Rename it to Khoj	2022-07-19 18:26:16 +04:00

Khoj

Allow natural language search on user content like notes, images, transactions using transformer ML models

User can interface with Khoj via the API or Emacs. All search is done locally*

  git clone https://github.com/debanjum/khoj && cd khoj

[Required] Update docker-compose.yml to mount your images, org-mode notes and beancount directories
[Optional] Edit application configuration in sample_config.yml

docker-compose up -d

Note: The first run will take time. Let it run, it's mostly not hung, just generating embeddings

pytest

  docker-compose build --pull

Symptom: Errors out with "Killed" in error message
- Fix: Increase RAM available to Docker Containers in Docker Settings
- Refer: StackOverflow Solution, Configure Resources on Docker for Mac
Symptom: Errors out complaining about Tensors mismatch, null etc
- Mitigation: Delete content-type > image section from docker_sample_config.yml

The experimental chat API endpoint uses the OpenAI API
- It is disabled by default
- To use it add your openai-api-key to config.yml

Install Exiftool [Optional]

sudo apt-get -y install libimage-exiftool-perl

git clone https://github.com/debanjum/khoj && cd khoj
conda env create -f config/environment.yml
conda activate khoj

Configure files/directories to search in content-type section of sample_config.yml
To run application on test data, update file paths containing /data/ to tests/data/ in sample_config.yml
- Example replace /data/notes/*.org with tests/data/notes/*.org

Load ML model, generate embeddings and expose API to query notes, images, transactions etc specified in config YAML

python3 -m src.main -c=config/sample_config.yml -vv

  cd khoj
  git pull origin master
  conda deactivate khoj
  conda env update -f config/environment.yml
  conda activate khoj