mirror of
https://github.com/khoj-ai/khoj.git
synced 2024-12-04 04:43:01 +01:00
78a1f4ebb4
- YAML Config - Can specify all params[1] earlier being passed via cmd args in config YAML - Can now also configure sentence-transformer models to use etc for search - [1] Config params - org files - compressed entries file config path - embeddings file config path - Include sample_config.yaml - Include sample .org file from this repos readmes - CLI - Configuration Priority: Config via cmd > Config via YAML > Default Config - Test CLI, include test config.yml for the tests - Set default type to None unless set via query param to API Run notes search if search_enabled, also if type is None (default) Prepares for running queries on all search types unless type specified in API query param - Update Readme
2.1 KiB
2.1 KiB
Semantic Search
Allow natural language search on user content like notes, images using transformer based models
All data is processed locally. User can interface with semantic-search app via Emacs, API or Commandline
Dependencies
- Python3
- Miniconda
Install
git clone https://github.com/debanjum/semantic-search && cd semantic-search
conda env create -f environment.yml
conda activate semantic-search
Run
Load ML model, generate embeddings and expose API to query specified org-mode files
python3 src/main.py --org-files ~/Notes/Schedule.org ~/Notes/Incoming.org -c sample_config.yml --verbose
Use
-
Semantic Search via Emacs
- Install semantic-search.el
- Run
M-x semantic-search <user-query>
or CallC-c s
-
Semantic Search via API
- Query:
GET
http://localhost:8000/search?q="What is the meaning of life" - Regenerate Embeddings:
GET
http://localhost:8000/regenerate - Semantic Search API Docs
- Query:
Upgrade
cd semantic-search
git pull origin master
conda env update -f environment.yml
conda activate semantic-search
Acknowledgments
- MiniLM Model for Asymmetric Text Search. See SBert Documentation
- OpenAI CLIP Model for Image Search. See SBert Documentation
- Charles Cave for OrgNode Parser