mirror of
https://github.com/khoj-ai/khoj.git
synced 2024-11-30 19:03:01 +01:00
19f8f85333
- Use Markdown file to help upload demo to Github - Use generated link from upload into Readme org file
100 lines
4.4 KiB
Org Mode
100 lines
4.4 KiB
Org Mode
[[https://github.com/debanjum/semantic-search/actions/workflows/test.yml/badge.svg]] [[https://github.com/debanjum/semantic-search/actions/workflows/build.yml/badge.svg]]
|
|
|
|
* Semantic Search
|
|
/Allow natural language search on user content like notes, images, transactions using transformer ML models/
|
|
|
|
User can interface with semantic-search via the API or [[./src/interface/emacs/semantic-search.el][Emacs]]. All search is done locally[[https://github.com/debanjum/semantic-search#miscellaneous][*]]
|
|
|
|
** Demo
|
|
https://user-images.githubusercontent.com/6413477/168417719-8a8bc4e5-8404-42b2-89a7-4493e3d2582c.mp4
|
|
|
|
** Setup
|
|
|
|
*** 1. Clone
|
|
#+begin_src shell
|
|
git clone https://github.com/debanjum/semantic-search && cd semantic-search
|
|
#+end_src
|
|
|
|
*** 2. Configure
|
|
- [Required] Update [[./docker-compose.yml][docker-compose.yml]] to mount your images, org-mode notes and beancount directories
|
|
- [Optional] Edit application configuration in [[./config/sample_config.yml][sample_config.yml]]
|
|
|
|
*** 3. Run
|
|
#+begin_src shell
|
|
docker-compose up -d
|
|
#+end_src
|
|
|
|
/Note: The first run will take time. Let it run, it's mostly not hung, just generating embeddings/
|
|
|
|
** Use
|
|
- *Semantic Search via API*
|
|
- See [[http://localhost:8000/docs][Semantic Search API Docs]]
|
|
- [[http://localhost:8000/search?q=%22what%20is%20the%20meaning%20of%20life%22][Query]]
|
|
- [[http://localhost:8000/regenerate?t=ledger][Regenerate Embeddings]]
|
|
- [[https://localhost:8000/ui][Configure Application]]
|
|
|
|
- *Semantic Search via Emacs*
|
|
- [[https://github.com/debanjum/semantic-search/tree/master/src/interface/emacs#installation][Install]] [[./src/interface/emacs/semantic-search.el][semantic-search.el]]
|
|
- Run ~M-x semantic-search <user-query>~
|
|
|
|
** Upgrade
|
|
#+begin_src shell
|
|
docker-compose build
|
|
#+end_src
|
|
|
|
** Troubleshooting
|
|
- Symptom: Errors out with "Killed" in error message
|
|
- Fix: Increase RAM available to Docker Containers in Docker Settings
|
|
- Refer: [[https://stackoverflow.com/a/50770267][StackOverflow Solution]], [[https://docs.docker.com/desktop/mac/#resources][Configure Resources on Docker for Mac]]
|
|
- Symptom: Errors out complaining about Tensors mismatch, null etc
|
|
- Mitigation: Delete content-type > image section from docker_sample_config.yml
|
|
|
|
** Miscellaneous
|
|
- The experimental [[localhost:8000/chat][chat]] API endpoint uses the [[https://openai.com/api/][OpenAI API]]
|
|
- It is disabled by default
|
|
- To use it add your ~openai-api-key~ to config.yml
|
|
|
|
** Development Setup
|
|
*** Setup on Local Machine
|
|
|
|
**** 1. Install Dependencies
|
|
1. Install Python3 [Required]
|
|
2. [[https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html][Install Conda]] [Required]
|
|
3. Install Exiftool [Optional]
|
|
#+begin_src shell
|
|
sudo apt-get -y install libimage-exiftool-perl
|
|
#+end_src
|
|
|
|
**** 2. Install Semantic Search
|
|
#+begin_src shell
|
|
git clone https://github.com/debanjum/semantic-search && cd semantic-search
|
|
conda env create -f config/environment.yml
|
|
conda activate semantic-search
|
|
#+end_src
|
|
|
|
**** 3. Configure
|
|
- Configure files/directories to search in ~content-type~ section of ~sample_config.yml~
|
|
- To run application on test data, update file paths containing ~/data/~ to ~tests/data/~ in ~sample_config.yml~
|
|
- Example replace ~/data/notes/*.org~ with ~tests/data/notes/*.org~
|
|
|
|
**** 4. Run
|
|
Load ML model, generate embeddings and expose API to query notes, images, transactions etc specified in config YAML
|
|
|
|
#+begin_src shell
|
|
python3 -m src.main -c=config/sample_config.yml -vv
|
|
#+end_src
|
|
|
|
*** Upgrade On Local Machine
|
|
#+begin_src shell
|
|
cd semantic-search
|
|
git pull origin master
|
|
conda deactivate semantic-search
|
|
conda env update -f config/environment.yml
|
|
conda activate semantic-search
|
|
#+end_src
|
|
|
|
** Acknowledgments
|
|
- [[https://huggingface.co/sentence-transformers/msmarco-MiniLM-L-6-v3][MiniLM Model]] for Asymmetric Text Search. See [[https://www.sbert.net/examples/applications/retrieve_rerank/README.html][SBert Documentation]]
|
|
- [[https://github.com/openai/CLIP][OpenAI CLIP Model]] for Image Search. See [[https://www.sbert.net/examples/applications/image-search/README.html][SBert Documentation]]
|
|
- Charles Cave for [[http://members.optusnet.com.au/~charles57/GTD/orgnode.html][OrgNode Parser]]
|
|
- Sven Marnach for [[https://github.com/smarnach/pyexiftool/blob/master/exiftool.py][PyExifTool]]
|