sij/khoj
1
0
Fork 0
mirror of https://github.com/khoj-ai/khoj.git synced 2024-11-23 23:48:56 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 16
khoj/docs/advanced.md
sabaimran 3934633947 Update references to all documentation to reflect instructions for managed service 
- By default assume the audience of this website is people looking to understand the featuer offering of Khoj, and then people who are looking to self-host
2023-11-16 15:26:03 -08:00

2.1 KiB
Raw Permalink Blame History

Advanced Usage

Search across Different Languages (Self-Hosting)

To search for notes in multiple, different languages, you can use a multi-lingual model.
For example, the paraphrase-multilingual-MiniLM-L12-v2 supports 50+ languages, has good search quality and speed. To use it:

  1. Manually update the search config in server's admin settings page. Go to the search config. Either create a new one, if none exists, or update the existing one. Set the bi_encoder to sentence-transformers/multi-qa-MiniLM-L6-cos-v1 and the cross_encoder to cross-encoder/ms-marco-MiniLM-L-6-v2.
  2. Regenerate your content index from all the relevant clients. This step is very important, as you'll need to re-encode all your content with the new model.

Query Filters

Use structured query syntax to filter entries from your knowledge based used by search results or chat responses.

  • Word Filter: Get entries that include/exclude a specified term
    • Entries that contain term_to_include: +"term_to_include"
    • Entries that contain term_to_exclude: -"term_to_exclude"
  • Date Filter: Get entries containing dates in YYYY-MM-DD format from specified date (range)
    • Entries from April 1st 1984: dt:"1984-04-01"
    • Entries after March 31st 1984: dt>="1984-04-01"
    • Entries before April 2nd 1984 : dt<="1984-04-01"
  • File Filter: Get entries from a specified file
    • Entries from incoming.org file: file:"incoming.org"
  • Combined Example
    • what is the meaning of life? file:"1984.org" dt>="1984-01-01" dt<="1985-01-01" -"big" -"brother"
    • Adds all filters to the natural language query. It should return entries
      • from the file 1984.org
      • containing dates from the year 1984
      • excluding words "big" and "brother"
      • that best match the natural language query "what is the meaning of life?"