Commit graph

  • 0d9eba69ed Pin pytorch version to 2.0.1 in order to avoid exit code 139 in Docker container sabaimran 2023-10-20 13:19:01 -07:00
  • 4c824c1a8f Resolve conflicts post-merge sabaimran 2023-10-19 16:24:20 -07:00
  • 4952089dcf Merge branch 'master' of github.com:khoj-ai/khoj into features/chat-sessions-separation sabaimran 2023-10-19 16:12:39 -07:00
  • 485153b904 Fix named variable in indexer.py sabaimran 2023-10-19 16:00:13 -07:00
  • 6062237c1a Revert unit tests sabaimran 2023-10-19 15:59:56 -07:00
  • 32bdbc762f Merge branch 'features/use-config-with-multi-user' of github.com:khoj-ai/khoj into features/add-signup-login-pages sabaimran 2023-10-19 15:56:55 -07:00
  • 24ef04a81b Update unit test after merging sabaimran 2023-10-19 15:42:23 -07:00
  • 92005ade18 Resolve merge conflicts with master sabaimran 2023-10-19 15:22:34 -07:00
  • 39abefd6dc Resolve merge conflicts with master sabaimran 2023-10-19 15:15:56 -07:00
  • 963cd165eb Resolve merge conflicts sabaimran 2023-10-19 14:39:05 -07:00
  • 6deb22ca8c Convert bool to string when setting default value for enable_offline_chat sabaimran 2023-10-19 13:23:52 -07:00
  • 9da98f6959 Resolve all linting errors sabaimran 2023-10-19 13:15:37 -07:00
  • e3f8a95784
    Update emacs.md (#510) Simon Butler 2023-10-19 21:28:08 +02:00
  • b76a629869
    Update emacs.md Simon Butler 2023-10-19 21:02:14 +02:00
  • 16535aa11f Update searchresponse for image search and revert get org files import in unit test sabaimran 2023-10-19 11:37:59 -07:00
  • d2911b4e02 Resolve merge conflicts and update unit tests sabaimran 2023-10-18 21:58:23 -07:00
  • 1a3b19a7b5 Add configurtion for processor conversation-related objects and logs sabaimran 2023-10-18 17:49:21 -07:00
  • d93395ae48
    Set >=6Gb RAM required for offline chat Debanjum 2023-10-18 12:05:54 -07:00
  • 8346e1193c Release Khoj version 0.13.0 0.13.0 Debanjum Singh Solanky 2023-10-18 03:43:16 -07:00
  • 6631fc38db Delete plaintext config via API. Catch any offline model loading exception Debanjum Singh Solanky 2023-10-18 03:23:17 -07:00
  • 53abd1a506 Mark sync completed on desktop client, even when no files to send Debanjum Singh Solanky 2023-10-18 01:00:41 -07:00
  • 71b0012e8c Set offline chat config to default value if unset on server load Debanjum Singh Solanky 2023-10-18 00:59:43 -07:00
  • cf1cdc3fe1 Disambiguate input_filter variable names in fs_syncer functions Debanjum Singh Solanky 2023-10-17 23:30:45 -07:00
  • e3cd8b4150 Only index files returned by input-filter globs in fs_syncer Debanjum Singh Solanky 2023-10-17 22:59:10 -07:00
  • 51363d280d Do not configure khoj server for pull based indexing from khoj.el Debanjum Singh Solanky 2023-10-17 21:44:54 -07:00
  • d9d133dfb9 Read text files as utf-8, instead of default os locale Debanjum Singh Solanky 2023-10-17 21:31:15 -07:00
  • 3d4576ae38
    Fix encoding binary files for sync from the Desktop, Obsidian client (#506) Debanjum 2023-10-17 15:37:22 -07:00
  • c8293998d9 Fix encoding binary files like PDFs for sync from Obsidian client Debanjum Singh Solanky 2023-10-17 15:07:05 -07:00
  • ba60c869c9 Fix encoding binary files like PDFs for sync from Desktop client sabaimran 2023-10-17 13:05:50 -07:00
  • 178c291431 Resolve merge conflicts sabaimran 2023-10-17 15:05:18 -07:00
  • 3d7381446d
    Changed globbing. Now doesn't clobber a users glob if they want to a… (#496) Andrew Spott 2023-10-17 12:26:06 -06:00
  • ce1b3518ca Use older glob pattern for plaintext files sabaimran 2023-10-17 11:11:58 -07:00
  • 42bd218c87 Merge branch 'master' of github.com:khoj-ai/khoj into globbing_fix sabaimran 2023-10-17 10:52:26 -07:00
  • 2646c8554d Provide a default value to offline_chat configuration of the conversation processor sabaimran 2023-10-17 10:35:13 -07:00
  • b8976426eb Update offline chat model config schema used by Emacs, Obsidian clients Debanjum Singh Solanky 2023-10-17 06:30:20 -07:00
  • ecc6fbfeb2
    Push Files to Index from Emacs, Obsidian & Desktop Clients using Multi-Part Forms (#499) Debanjum 2023-10-17 06:05:15 -07:00
  • 7b1c62ba53 Mark test_get_configured_types_via_api unit test as flaky Debanjum Singh Solanky 2023-10-17 05:55:39 -07:00
  • 6a4f1b2188 Add more client, request details in logs by index/update API endpoint Debanjum Singh Solanky 2023-10-17 05:31:57 -07:00
  • 5efae1ad55 Update indexer API endpoint query params for force, content type Debanjum Singh Solanky 2023-10-17 04:42:04 -07:00
  • 84654ffc5d Update indexer API endpoint URL to index/update from indexer/batch Debanjum Singh Solanky 2023-10-17 04:30:27 -07:00
  • e347823ff4 Log telemetry for index updates via push to API endpoint Debanjum Singh Solanky 2023-10-17 04:09:33 -07:00
  • 05be6bd877 Clicking Update Index in Obsidian settings should push files to index Debanjum Singh Solanky 2023-10-17 03:27:41 -07:00
  • 13a3122bf3 Stop configuring server to pull files to index from Obsidian client Debanjum Singh Solanky 2023-10-17 03:23:25 -07:00
  • 99a2c934a3 Add CORS policy to allow requests from khoj apps, obsidian & localhost Debanjum Singh Solanky 2023-10-17 02:54:18 -07:00
  • 541cd59a49 Let fs_syncer pass PDF files directly as binary before indexing Debanjum Singh Solanky 2023-10-17 02:41:16 -07:00
  • d27dc71dfe Use encoding of each file set in indexer request to read file Debanjum Singh Solanky 2023-10-17 02:37:20 -07:00
  • 8e627a5809 Pass any files to be deleted to indexer API via Khoj Obsidian plugin Debanjum Singh Solanky 2023-10-17 02:51:54 -07:00
  • f2e293a149 Push Vault files to index to Khoj server using Khoj Obsidian plugin Debanjum Singh Solanky 2023-10-17 02:17:44 -07:00
  • 6baaaaf91a Test request body of multi-part form to update content index from khoj.el Debanjum Singh Solanky 2023-10-16 23:54:32 -07:00
  • 79b3f8273a Make khoj.el send files to be deleted from index to server Debanjum Singh Solanky 2023-10-16 23:53:02 -07:00
  • 05453ed729 Resolve linting errors with default args to conversationprocessorconfig and some routes with duplicate function names sabaimran 2023-10-16 22:08:52 -07:00
  • 5f7bd864e9 Fix unit tests with new authentication schema sabaimran 2023-10-16 21:34:21 -07:00
  • 8f36cf58ae Merge branch 'master' of github.com:khoj-ai/khoj into features/add-signup-login-pages sabaimran 2023-10-16 21:06:49 -07:00
  • 9fd9f93d77 Add sign in page and get access for most APIs behind login, unless in anonymous mode sabaimran 2023-10-16 20:20:45 -07:00
  • 5dc399b32e Document system requirements to run offline chat Debanjum Singh Solanky 2023-10-16 19:39:06 -07:00
  • f64fa06e22 Initialize the Khoj Transient menu on first run instead of load Debanjum Singh Solanky 2023-10-13 18:48:26 -07:00
  • b4949f7f0b
    Improve Offline Chat Model Experience (#494) Debanjum 2023-10-16 17:44:49 -07:00
  • 644c3b787f Scale no. of chat history messages to use as context with max_prompt_size Debanjum Singh Solanky 2023-10-16 11:15:38 -07:00
  • 343482a9ff Merge branch 'features/use-config-with-multi-user' of github.com:khoj-ai/khoj into features/add-signup-login-pages sabaimran 2023-10-16 15:06:33 -07:00
  • abf3bb2f46 Remove HNSW index for now -- to follow-up at a later time with more stress testing sabaimran 2023-10-16 14:56:49 -07:00
  • eeae28c9a1 Revert changes to PDF decoding and update settings pages to read from relevant data settings sabaimran 2023-10-16 14:51:26 -07:00
  • e4652bafd0 Simplify some of the PDF parsing code and remove unused imports sabaimran 2023-10-16 12:19:16 -07:00
  • 90e1d9e3d6 Pin gpt4all to 1.0.12 as next version will introduce breaking changes Debanjum Singh Solanky 2023-10-16 10:57:16 -07:00
  • 654425ce03 Merge branch 'master' of github.com:khoj-ai/khoj into features/use-config-with-multi-user sabaimran 2023-10-16 10:56:12 -07:00
  • d71cc1a3bf Update method to convert JSON config to DB objects sabaimran 2023-10-16 10:42:00 -07:00
  • 04ff879202 Init: add new routes and update methods for login/signup pages sabaimran 2023-10-16 10:16:07 -07:00
  • 5df9ce08ab Move main.py back under /khoj, add a migration for HNSW index sabaimran 2023-10-16 10:15:03 -07:00
  • 1a9023d396 Update Chat Actor test to not incept with prior world knowledge Debanjum Singh Solanky 2023-10-15 17:22:44 -07:00
  • df1d74a879 Use max_prompt_size, tokenizer from config for chat model context stuffing Debanjum Singh Solanky 2023-10-15 16:33:26 -07:00
  • 116595b351 Use chat_model specified in new offline_chat section of config Debanjum Singh Solanky 2023-10-15 14:24:28 -07:00
  • feb4f17e3d Update chat config schema. Make max_prompt, chat tokenizer configurable Debanjum Singh Solanky 2023-10-15 14:19:29 -07:00
  • ef18606916 Turn of telemtery if debug mode is enabled sabaimran 2023-10-15 12:12:09 -07:00
  • 9c9dbfa9c2 Resolve mypy linting issues for return type, parameters sabaimran 2023-10-15 10:51:26 -07:00
  • 4b37184923 Fix null check issues sabaimran 2023-10-15 10:35:49 -07:00
  • d4c2305085 Resolve merge conflicts after Part 0 sabaimran 2023-10-14 19:52:46 -07:00
  • c125995d94
    [Multi-User]: Part 0 - Add support for logging in with Google (#487) sabaimran 2023-10-14 19:39:13 -07:00
  • 497b843f0f Minor clean-up: Delete unused imports and files sabaimran 2023-10-14 19:36:33 -07:00
  • be2176f500 Add in cross encoder and rerank steps in the search path sabaimran 2023-10-14 19:26:11 -07:00
  • 247e75595c Use AutoTokenizer to support more tokenizers Debanjum Singh Solanky 2023-10-14 16:54:52 -07:00
  • ff2dbadc9d Use computed plaintext_content to set file content rather than calling f.read again Saba 2023-10-14 13:28:34 -07:00
  • 8dc0a5391c Fix typo in globformat variable Saba 2023-10-14 13:18:57 -07:00
  • 1ad8b150e8 Add default tokenizer, max_prompt as fallback for non-default offline chat models Debanjum Singh Solanky 2023-10-13 22:26:59 -07:00
  • 56bd69d5af Improve Llama v2 extract questions actor and associated prompt Debanjum Singh Solanky 2023-10-04 20:42:25 -07:00
  • 09bb3686cc
    Strip the incoming query from the slash conversation command (#500) sabaimran 2023-10-13 21:11:23 -07:00
  • b98742de90 Remove -n 4 from pytest ini configuration to isolate test failures sabaimran 2023-10-13 21:05:17 -07:00
  • cc4925fcee Return q when content index not loaded sabaimran 2023-10-13 20:57:17 -07:00
  • 96c0b21285 Sync desktop app package.json with other Khoj clients metadata Debanjum Singh Solanky 2023-10-13 20:22:33 -07:00
  • 5ccd2e41eb Strip the incoming query from the slash conversation command before passing it to the model or for search sabaimran 2023-10-13 20:23:17 -07:00
  • a6a4631ed2 Add basic instructions for using the new application setup sabaimran 2023-10-13 19:42:18 -07:00
  • 615a633b40 Set type for query cache sabaimran 2023-10-13 19:24:51 -07:00
  • 80fb56b8a5 Sync deksktop app package version with the other releases sabaimran 2023-10-13 19:23:00 -07:00
  • b669aa2395 Clean and fix the content indexing code in the Emacs client Debanjum Singh Solanky 2023-10-13 18:00:37 -07:00
  • 36c227ee43 Fix retrieval of KhojUser from request sabaimran 2023-10-13 17:53:30 -07:00
  • a7ce54c704 Update the docker setup to work with the new application design sabaimran 2023-10-13 17:18:29 -07:00
  • 394cbe69fc Clean up further tests and application code by removing dead code sabaimran 2023-10-13 16:38:01 -07:00
  • 7157ff44f9 Conditionally print date logline if new date associations were made sabaimran 2023-10-13 14:16:12 -07:00
  • 8e6d685a25 Add date filters to new search flow and update unit tests sabaimran 2023-10-13 13:33:41 -07:00
  • a3c386b45d Revert word_filter get_filter_terms change in function definition to maintain base class relationship sabaimran 2023-10-13 12:12:34 -07:00
  • 867d731ba2 Fix linting issues and update affected unit tests from new filtering logic sabaimran 2023-10-13 12:00:04 -07:00
  • 5c3995c128 Working explicit word filters and file filters based on regexp sabaimran 2023-10-13 11:41:06 -07:00