sij/khoj
1
0
Fork 0
mirror of https://github.com/khoj-ai/khoj.git synced 2024-11-23 15:38:55 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 5
Mirror of khoj from Github
186 commits 8 branches 101 tags 130 MiB
Python 50.2%
TypeScript 32.9%
HTML 7.3%
Emacs Lisp 3.4%
JavaScript 3.1%
Other 3.1%
Find a file
Debanjum ed7c2901f5
Merge pull request #18 from debanjum/deb/save-models-to-disk-on-first-run 
Save Search Models to Disk on First Run

## Why
  - Improve application startup time
  - Startup application and perform semantic search even if user offline
  - Use search model config in YAML file for all search types (asymmetric, symmetric, image)

## Details
  - Load search models from disk when available
  - Use search model config specified in YAML file
  - Add search config for Symmetric Search used by Ledger/Beancount transaction search
2022-01-14 17:30:46 -08:00
.github/workflows Try cache conda build step using online docs 2021-12-15 11:48:14 +05:30
src Load model from HuggingFace if model_directory unset in config YAML 2022-01-14 17:36:59 -05:00
tests Setup Search with Search_Config to Fix Tests 2022-01-14 20:13:14 -05:00
views Fix input text behavior for null/empty value fields 2021-12-04 10:45:48 -05:00
.gitignore Set up basic ui page with no functionality 2021-11-26 14:51:11 -05:00
environment.yml Merge branch 'master' of github.com:debanjum/semantic-search into add-summarize-capability-to-chat-bot 2021-12-20 13:30:42 +05:30
LICENSE Add Readme, License. Update .gitignore 2021-08-15 22:52:37 -07:00
README.org Fix url error in README 2022-01-13 16:28:46 -08:00
sample_config.yml Save Image Search Model to Disk 2022-01-14 17:36:59 -05:00

README.org

https://github.com/debanjum/semantic-search/actions/workflows/build.yml/badge.svg

Semantic Search

Allow natural language search on user content like notes, images, transactions using transformer based models

All data is processed locally. User can interface with semantic-search app via Emacs, API or Commandline

Dependencies

Install 

git clone https://github.com/debanjum/semantic-search && cd semantic-search
conda env create -f environment.yml
conda activate semantic-search

Install Environmental Dependencies 

sudo apt-get -y install libimage-exiftool-perl

Configure

Configure application search types and their underlying data source/files in sample_config.yml Use the sample_config.yml as reference

Run

Load ML model, generate embeddings and expose API to query notes, images, transactions etc specified in config YAML

python3 -m src.main -c=sample_config.yml -vv

Use

Upgrade 

  cd semantic-search
  git pull origin master
  conda env update -f environment.yml
  conda activate semantic-search

Acknowledgments