sij/khoj
1
0
Fork 0
mirror of https://github.com/khoj-ai/khoj.git synced 2024-12-18 10:37:11 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 23
587 commits 9 branches 114 tags 135 MiB
Commit graph

587 commits

This branch
This branch
All branches
Author SHA1 Message Date
Debanjum Singh Solanky
179153dc5a Rename RawConfig Types for Consistency 
- Naming convention - [ContentType][ConfigType]Config
  - Where [ConfigType] ~ Content, Search, Processor
  - Where [ContentType] ~ Text, Image, Asymmetric, Symmetric, Conversation

- Current Configs:
  - Content:
    - Org Notes
    - Org Music
    - Image
    - Ledger/Beancount

  - Search:
     - Asymmetric
     - Symmetric
     - Image

  - Processor:
    - Conversation
 2022-01-14 20:54:38 -05:00
Debanjum
ed7c2901f5
Merge pull request #18 from debanjum/deb/save-models-to-disk-on-first-run 
Save Search Models to Disk on First Run

## Why
  - Improve application startup time
  - Startup application and perform semantic search even if user offline
  - Use search model config in YAML file for all search types (asymmetric, symmetric, image)

## Details
  - Load search models from disk when available
  - Use search model config specified in YAML file
  - Add search config for Symmetric Search used by Ledger/Beancount transaction search
 2022-01-14 17:30:46 -08:00
Debanjum Singh Solanky
ed144f7984 Setup Search with Search_Config to Fix Tests 
- Rename pytest fixture search_config to more appropriate
content_config
- Create search_config pytest fixture
- Use search_config where search being setup, used in tests
 2022-01-14 20:13:14 -05:00
Debanjum Singh Solanky
c64e0c2965 Load model from HuggingFace if model_directory unset in config YAML 
- Do not save/load the model to/from disk when model_directory unset
in config.yml
- Add symmetric search default config to cli.py
 2022-01-14 17:36:59 -05:00
Debanjum Singh Solanky
510faa1904 Save Image Search Model to Disk 2022-01-14 17:36:59 -05:00
Debanjum Singh Solanky
934ec233b0 Add Search Config for Symmetric Model. Save Model to Disk 2022-01-14 17:36:59 -05:00
Debanjum Singh Solanky
b63026d97c Save Asymmetric Search Model to Disk 
- Improve application load time
- Remove dependence on internet to startup application and perform semantic search
 2022-01-14 17:36:27 -05:00
Debanjum
e8146e8ebb
Merge pull request #17 from albd/patch-1 
Fix url error in README
 2022-01-13 18:09:10 -08:00
Albert Davies
2e2069f720
Fix url error in README 
Misplaced quotes
 2022-01-13 16:28:46 -08:00
Debanjum Singh Solanky
2e53fbc844 Fix the user intent extraction prompt for GPT. Clean up chatbot test 2022-01-12 10:36:01 -05:00
Debanjum Singh Solanky
ea28897cdd Remove deprecated conversation_history field from config 2022-01-12 10:35:52 -05:00
Debanjum Singh Solanky
5a686b7be9 Add logs for chat bot in verbose mode 2022-01-12 10:35:52 -05:00
Debanjum Singh Solanky
6dc2a99d35 Merge branch 'master' of github.com:debanjum/semantic-search into add-summarize-capability-to-chat-bot 
- Fix openai_api_key being set in ConfigProcessorConfig
- Merge addition of config UI and config instantiation updates
 2021-12-20 13:30:42 +05:30
Debanjum
29543d2dc3
Merge pull request #16 from debanjum/cache-conda-env-setup-to-improve-cloud-build 
Cache Conda Setup to Improve Cloud Build Time
 2021-12-18 06:58:42 -08:00
Saba
ee6aae3a40 Merge branch 'master' of github.com:debanjum/semantic-search 2021-12-16 20:36:36 -05:00
Saba
916a1ffc73 Fix formatting of REAMDE env dependencies 2021-12-16 20:36:31 -05:00
Debanjum Singh Solanky
d91f645198 Try cache conda build step using online docs 
Refer:
 - https://github.com/marketplace/actions/setup-miniconda
 2021-12-15 11:48:14 +05:30
Debanjum Singh Solanky
65da7daf1f Load, Save Conversation Session Summaries to Log. s/chat_log/chat_session 
Conversation logs structure now has session info too instead of just chat info
Session info will allow loading past conversation summaries as context for AI in new conversations

{
    "session": [
    {
        "summary": <chat_session_summary>,
        "session-start": <session_start_index_in_chat_log>,
        "session-end": <session_end_index_in_chat_log>
    }],
    "chat": [
    {
        "intent": <intent-object>
        "trigger-emotion": <emotion-triggered-by-message>
        "by": <AI|Human>
        "message": <chat_message>
        "created": <message_created_date>
    }]
}
 2021-12-15 10:17:07 +05:30
Debanjum
35b46849a2
Merge pull request #15 from debanjum/debanjum-skip-flaky-image-search-pytest 
Skip Flaky Image Search Test
 2021-12-12 02:28:15 -08:00
Debanjum
ef911aa6be
Skip Flaky Image Search Test 
Image search doesn't always return expected image path.
Should resolve remaining issues with failing cloud test. See #11
 2021-12-12 02:15:20 -08:00
Saba
d14f5d54e3 Merge branch 'master' of github.com:debanjum/semantic-search 2021-12-11 14:18:06 -05:00
Saba
4832a107d7 Merge branch 'saba/fix-unit-tests' 2021-12-11 14:17:52 -05:00
Saba
ba8dc9ed5f Update the search_config instantiated for tests in conftest 2021-12-11 14:14:31 -05:00
Saba
9ebf00e29b Add instructions for installing exiftool to README (for Ubuntu only) 2021-12-11 14:13:37 -05:00
Saba
97a6dfaa1e Use default value False for verbose parameter, and small changes 
Pass config as parameter to initialize_search, change name of API methods to handle config CRUD operations, and initalize config to FullConfig
 2021-12-11 14:13:14 -05:00
sabaimran
930bdb5403
Use ubuntu-latest as machine for Github Actions 2021-12-11 12:54:32 -05:00
sabaimran
386534c236
Remove apt-get update from build.yml 2021-12-11 12:52:13 -05:00
sabaimran
c7d88a7a63
Update build.yml to include exiftool package 2021-12-11 12:50:30 -05:00
Saba
9536358d34 Fix key error model_name issue by upgrade sentence-transformers version 
Refer to https://github.com/UKPLab/sentence-transformers/issues/1241
Also user verbose flag passed through function parameters in image_search
 2021-12-11 11:58:19 -05:00
Saba
ce7a751e6b Fix passing verbose flag down in symmetric_ledger.py 2021-12-11 11:36:32 -05:00
sabaimran
e5a19f5b0e
Merge pull request #10 from debanjum/saba/configui 
Create a UI from which a user can update their config and integrate BaseModel for the config input
 2021-12-10 22:10:49 -05:00
Saba
d65190c3ee Update unit tests, files with removing model suffix to config types 2021-12-09 08:50:38 -05:00
Debanjum Singh Solanky
0ac1e5f372 Summarize chat logs and notes returned by semantic search via /chat API 2021-12-08 02:34:07 +05:30
Saba
b3eac888fb Add jinja to conda environment config 2021-12-06 20:13:25 -05:00
Saba
76e9e9da2f Update unit tests to use the new BaseModel types 2021-12-05 09:31:39 -05:00
Saba
c4cd4b57f1 Update types used in conftest.py 2021-12-04 12:02:19 -05:00
Saba
9b16cdbb41 Use past tense for verbose log 2021-12-04 11:45:44 -05:00
Saba
10e4065e05 Consolidate the search config models and pass verbose as a top level flag 2021-12-04 11:43:48 -05:00
Saba
43e647835b Append Model Suffixed to config models 2021-12-04 10:51:21 -05:00
Saba
2490e72df5 Fix input text behavior for null/empty value fields 2021-12-04 10:45:48 -05:00
Saba
e068968b35 Update imports for raw config models in config.py 2021-12-04 10:44:55 -05:00
Saba
4d6284b0af Remove Test suffix from Config models 2021-12-04 10:44:13 -05:00
Saba
7fcc8d2cef Add null check for processor config 2021-12-04 10:11:00 -05:00
Saba
7ca4fc3453 Resolve mrege conflicts with updated processor conversation data model 2021-11-28 16:22:52 -05:00
Saba
6b8efb81b3 Document config.js 2021-11-28 14:41:57 -05:00
Saba
87a6c2d716 Use parse_obj instead of parse_raw as incoming data is in dict 2021-11-28 14:34:32 -05:00
Saba
faf1df47fe remove extra whitespace in environment.yml 2021-11-28 14:32:51 -05:00
Saba
04132a2f34 Remove config.yml and fix regenerate fetch response 2021-11-28 14:29:16 -05:00
Saba
112868962b Disable regenerate button while embeddings are being regenerated 2021-11-28 13:45:00 -05:00
Saba
5d50487d83 Linting 
New line at end of config.html
Remove debug print statement
 2021-11-28 13:32:56 -05:00