sij/khoj
1
0
Fork 0
mirror of https://github.com/khoj-ai/khoj.git synced 2024-11-23 23:48:56 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 16

Acknowledge ML models used for search. Simplify path used in commands

Browse source
This commit is contained in:
Debanjum Singh Solanky 2021-08-15 23:56:18 -07:00
parent c58c1d96aa
commit 4839153086
1 changed files with 6 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
README.md
View file

@ -24,7 +24,7 @@ Setup
python3 processor/org-mode/org-to-jsonl.py \ python3 processor/org-mode/org-to-jsonl.py \
--org-files "Schedule.org" "Incoming.org" \ --org-files "Schedule.org" "Incoming.org" \
--org-directory "~/Notes" \ --org-directory "~/Notes" \
--jsonl-file ".data/notes.jsonl" \ --jsonl-file ".notes.jsonl" \
--compress \ --compress \
--verbose --verbose
``` ```
@ -33,7 +33,7 @@ Run
--- ---
Load ML model, generate embeddings and expose API interface to run user queries on above org-mode files Load ML model, generate embeddings and expose API interface to run user queries on above org-mode files
```sh ```sh
python3 main.py -j .data/notes.jsonl.gz -e .data/notes_embeddings.pt python3 main.py -j .notes.jsonl.gz -e .notes_embeddings.pt
``` ```
Use Use
@ -48,8 +48,8 @@ Use
- *Call Semantic Search via Python Script Directly* - *Call Semantic Search via Python Script Directly*
```sh ```sh
python3 search_types/asymmetric.py \ python3 search_types/asymmetric.py \
-j .data/notes.jsonl.gz \ -j .notes.jsonl.gz \
-e .data/notes_embeddings.pt \ -e .notes_embeddings.pt \
-n 5 \ -n 5 \
--verbose \ --verbose \
--interactive --interactive
@ -57,4 +57,6 @@ Use
Acknowledgments Acknowledgments
-- --
- [MiniLM Model](https://huggingface.co/sentence-transformers/msmarco-MiniLM-L-6-v3) for Asymmetric Text Search. See [SBert Documentation](https://www.sbert.net/examples/applications/retrieve_rerank/README.html)
- [OpenAI CLIP Model](https://github.com/openai/CLIP) for Image Search. See [SBert Documentation](https://www.sbert.net/examples/applications/image-search/README.html)
- Charles Cave for [OrgNode Parser](http://members.optusnet.com.au/~charles57/GTD/orgnode.html) - Charles Cave for [OrgNode Parser](http://members.optusnet.com.au/~charles57/GTD/orgnode.html)