khoj/tests/data/config.yml

content-type:
  org:
    compressed-jsonl: .notes.json.gz
    embeddings-file: .note_embeddings.pt
    index-header-entries: true
    input-files:
    - ~/first_from_config.org
    - ~/second_from_config.org
    input-filter:
    - '*.org'
    - ~/notes/*.org
  plugins:
    content_plugin_1:
      compressed-jsonl: content_plugin_1.jsonl.gz
      embeddings-file: content_plugin_1_embeddings.pt
      input-files:
      - content_plugin_1_new.jsonl.gz
    content_plugin_2:
      compressed-jsonl: content_plugin_2.jsonl.gz
      embeddings-file: content_plugin_2_embeddings.pt
      input-filter:
      - '*2_new.jsonl.gz'
enable-offline-chat: false
search-type:
  asymmetric:
    cross-encoder: cross-encoder/ms-marco-MiniLM-L-6-v2
    encoder: sentence-transformers/msmarco-MiniLM-L-6-v3
version: 0.10.1
Use YAML file to allow user to configure application. Add tests - YAML Config - Can specify all params[1] earlier being passed via cmd args in config YAML - Can now also configure sentence-transformer models to use etc for search - [1] Config params - org files - compressed entries file config path - embeddings file config path - Include sample_config.yaml - Include sample .org file from this repos readmes - CLI - Configuration Priority: Config via cmd > Config via YAML > Default Config - Test CLI, include test config.yml for the tests - Set default type to None unless set via query param to API Run notes search if search_enabled, also if type is None (default) Prepares for running queries on all search types unless type specified in API query param - Update Readme 2021-08-22 03:47:55 +02:00			`content-type:`
			`org:`
Fix formatting of khoj test config and unused references in conftests 2023-07-22 09:28:14 +02:00			`compressed-jsonl: .notes.json.gz`
			`embeddings-file: .note_embeddings.pt`
Support multiple input-filters to configure content to index via khoj.yml - Update existings code, tests to process input-filters as list instead of str - Test `text_to_jsonl' get files methods to work with combination of `input-files' and `input-filters' Resolves #84 2022-09-12 09:39:39 +02:00			`index-header-entries: true`
Fix formatting of khoj test config and unused references in conftests 2023-07-22 09:28:14 +02:00			`input-files:`
			`- ~/first_from_config.org`
			`- ~/second_from_config.org`
			`input-filter:`
			`- '*.org'`
			`- ~/notes/*.org`
Read content plugin configs from Khoj config YAML Configure external text content plugins via the Khoj YAML Reuse existing TextContentConfig definition for external text content plugins 2023-02-22 21:47:22 +01:00			`plugins:`
			`content_plugin_1:`
Fix formatting of khoj test config and unused references in conftests 2023-07-22 09:28:14 +02:00			`compressed-jsonl: content_plugin_1.jsonl.gz`
			`embeddings-file: content_plugin_1_embeddings.pt`
			`input-files:`
			`- content_plugin_1_new.jsonl.gz`
Read content plugin configs from Khoj config YAML Configure external text content plugins via the Khoj YAML Reuse existing TextContentConfig definition for external text content plugins 2023-02-22 21:47:22 +01:00			`content_plugin_2:`
Fix formatting of khoj test config and unused references in conftests 2023-07-22 09:28:14 +02:00			`compressed-jsonl: content_plugin_2.jsonl.gz`
			`embeddings-file: content_plugin_2_embeddings.pt`
			`input-filter:`
			`- '*2_new.jsonl.gz'`
Add support for our first Local LLM 🤖🏠 (#330) * Add support for gpt4all's falcon model as an additional conversation processor - Update the UI pages to allow the user to point to the new endpoints for GPT - Update the internal schemas to support both GPT4 models and OpenAI - Add unit tests benchmarking some of the Falcon performance * Add exc_info to include stack trace in error logs for text processors * Pull shared functions into utils.py to be used across gpt4 and gpt * Add migration for new processor conversation schema * Skip GPT4All actor tests due to typing issues * Fix Obsidian processor configuration in auto-configure flow * Rename enable_local_llm to enable_offline_chat 2023-07-27 01:27:08 +02:00			`enable-offline-chat: false`
Use YAML file to allow user to configure application. Add tests - YAML Config - Can specify all params[1] earlier being passed via cmd args in config YAML - Can now also configure sentence-transformer models to use etc for search - [1] Config params - org files - compressed entries file config path - embeddings file config path - Include sample_config.yaml - Include sample .org file from this repos readmes - CLI - Configuration Priority: Config via cmd > Config via YAML > Default Config - Test CLI, include test config.yml for the tests - Set default type to None unless set via query param to API Run notes search if search_enabled, also if type is None (default) Prepares for running queries on all search types unless type specified in API query param - Update Readme 2021-08-22 03:47:55 +02:00			`search-type:`
			`asymmetric:`
Fix formatting of khoj test config and unused references in conftests 2023-07-22 09:28:14 +02:00			`cross-encoder: cross-encoder/ms-marco-MiniLM-L-6-v2`
			`encoder: sentence-transformers/msmarco-MiniLM-L-6-v3`
Add support for indexing plaintext files (#420) * Add support for indexing plaintext files - Adds backend support for parsing plaintext files generically (.html, .txt, .xml, .csv, .md) - Add equivalent frontend views for setting up plaintext file indexing - Update config, rawconfig, default config, search API, setup endpoints * Add a nifty plaintext file icon to configure plaintext files in the Web UI * Use generic glob path for plaintext files. Skip indexing files that aren't in whitelist 2023-08-10 00:44:40 +02:00			`version: 0.10.1`