Commit graph

  • feb4f17e3d Update chat config schema. Make max_prompt, chat tokenizer configurable Debanjum Singh Solanky 2023-10-15 14:19:29 -0700
  • ef18606916 Turn of telemtery if debug mode is enabled sabaimran 2023-10-15 12:12:09 -0700
  • 9c9dbfa9c2 Resolve mypy linting issues for return type, parameters sabaimran 2023-10-15 10:51:26 -0700
  • 4b37184923 Fix null check issues sabaimran 2023-10-15 10:35:49 -0700
  • d4c2305085 Resolve merge conflicts after Part 0 sabaimran 2023-10-14 19:52:46 -0700
  • c125995d94
    [Multi-User]: Part 0 - Add support for logging in with Google (#487) sabaimran 2023-10-14 19:39:13 -0700
  • 497b843f0f Minor clean-up: Delete unused imports and files sabaimran 2023-10-14 19:36:33 -0700
  • be2176f500 Add in cross encoder and rerank steps in the search path sabaimran 2023-10-14 19:26:11 -0700
  • 247e75595c Use AutoTokenizer to support more tokenizers Debanjum Singh Solanky 2023-10-14 16:54:52 -0700
  • ff2dbadc9d Use computed plaintext_content to set file content rather than calling f.read again Saba 2023-10-14 13:28:34 -0700
  • 8dc0a5391c Fix typo in globformat variable Saba 2023-10-14 13:18:57 -0700
  • 1ad8b150e8 Add default tokenizer, max_prompt as fallback for non-default offline chat models Debanjum Singh Solanky 2023-10-13 22:26:59 -0700
  • 56bd69d5af Improve Llama v2 extract questions actor and associated prompt Debanjum Singh Solanky 2023-10-04 20:42:25 -0700
  • 09bb3686cc
    Strip the incoming query from the slash conversation command (#500) sabaimran 2023-10-13 21:11:23 -0700
  • b98742de90 Remove -n 4 from pytest ini configuration to isolate test failures sabaimran 2023-10-13 21:05:17 -0700
  • cc4925fcee Return q when content index not loaded sabaimran 2023-10-13 20:57:17 -0700
  • 96c0b21285 Sync desktop app package.json with other Khoj clients metadata Debanjum Singh Solanky 2023-10-13 20:22:33 -0700
  • 5ccd2e41eb Strip the incoming query from the slash conversation command before passing it to the model or for search sabaimran 2023-10-13 20:23:17 -0700
  • a6a4631ed2 Add basic instructions for using the new application setup sabaimran 2023-10-13 19:42:18 -0700
  • 615a633b40 Set type for query cache sabaimran 2023-10-13 19:24:51 -0700
  • 80fb56b8a5 Sync deksktop app package version with the other releases sabaimran 2023-10-13 19:23:00 -0700
  • b669aa2395 Clean and fix the content indexing code in the Emacs client Debanjum Singh Solanky 2023-10-13 18:00:37 -0700
  • 36c227ee43 Fix retrieval of KhojUser from request sabaimran 2023-10-13 17:53:30 -0700
  • a7ce54c704 Update the docker setup to work with the new application design sabaimran 2023-10-13 17:18:29 -0700
  • 394cbe69fc Clean up further tests and application code by removing dead code sabaimran 2023-10-13 16:38:01 -0700
  • 7157ff44f9 Conditionally print date logline if new date associations were made sabaimran 2023-10-13 14:16:12 -0700
  • 8e6d685a25 Add date filters to new search flow and update unit tests sabaimran 2023-10-13 13:33:41 -0700
  • a3c386b45d Revert word_filter get_filter_terms change in function definition to maintain base class relationship sabaimran 2023-10-13 12:12:34 -0700
  • 867d731ba2 Fix linting issues and update affected unit tests from new filtering logic sabaimran 2023-10-13 12:00:04 -0700
  • 5c3995c128 Working explicit word filters and file filters based on regexp sabaimran 2023-10-13 11:41:06 -0700
  • 8ce89b8c3e A new iteration of keyword search: just use contains sabaimran 2023-10-12 23:06:24 -0700
  • a7bfd30f8d Push version of postgres full text search sabaimran 2023-10-12 22:47:14 -0700
  • 2cca39c9cc Initial version of explicit keyword-based filtering sabaimran 2023-10-12 22:29:00 -0700
  • bea196aa30 Explicitly make GET request to /config/data from khoj.el:khoj-server-configure method Debanjum Singh Solanky 2023-10-12 20:40:39 -0700
  • 292f0420ad Send content for indexing on server at a regular interval from khoj.el Debanjum Singh Solanky 2023-10-12 20:32:41 -0700
  • bed3aff059 Update tests to test multi-part/form method of pushing files to index Debanjum Singh Solanky 2023-10-12 16:16:51 -0700
  • fc99431754 Send files to index on server from the khoj.el emacs client Debanjum Singh Solanky 2023-10-11 22:45:29 -0700
  • 68018ef397 Use multi-part form to send files to index on desktop client Debanjum Singh Solanky 2023-10-11 18:12:12 -0700
  • 7190b3811d Remove all filter terms in user query from defiltered_query Debanjum Singh Solanky 2023-10-12 20:45:22 -0700
  • 72f8fde7ef Run pytests in parallel on multiple CPU cores using pytest-xdist for speed Debanjum Singh Solanky 2023-10-12 16:19:48 -0700
  • 60e9a61647 Use multi-part form to receive files to index on server Debanjum Singh Solanky 2023-10-11 17:14:15 -0700
  • 7663e9d1b5 Fix issue relating to calling configure_content from an async context sabaimran 2023-10-11 19:26:32 -0700
  • ab2a4b36d4 Fix unit test with unused parameter and partially updated inputs for text_search.setup sabaimran 2023-10-11 18:48:34 -0700
  • 9ba173bc2d Improve emoji, message on content index updated via logger Debanjum Singh Solanky 2023-10-11 17:12:03 -0700
  • 39a8d63fe3 Move main.py back into the khoj directory for more intuitive design sabaimran 2023-10-11 16:32:44 -0700
  • c8a141cbdd Consolidate migrations and round up everything related to the settings configs into one big file sabaimran 2023-10-11 16:31:54 -0700
  • ab66441e7b Further deprecate plugins sabaimran 2023-10-11 13:09:54 -0700
  • 10526469af Remove additional plugin references and fix unit test with leaky state sabaimran 2023-10-11 13:08:09 -0700
  • e98066a0b4 Re-add step to install db dependencies sabaimran 2023-10-11 12:50:50 -0700
  • f16534883f Limit precommit runs to pushing to master (rather than any branch) sabaimran 2023-10-11 12:45:42 -0700
  • d46d11bc75 Remove some of the dependent steps to see whether they are necessary sabaimran 2023-10-11 12:44:10 -0700
  • a6d0f11735 Use ankane image for postgres installation to avoid installing pgvector extension manually. Remove manualy pgvecotr install steps and add DEBIAN_FRONTEND flag to postgres steps sabaimran 2023-10-11 12:31:29 -0700
  • 516ff3d245 Run PG Vector installation steps in a separate step sabaimran 2023-10-11 12:10:59 -0700
  • 0eb5b68db4 Set all db variables to postgres sabaimran 2023-10-11 12:00:23 -0700
  • 85fe869a8a Add explicit step to instruction installation of pip with python3 sabaimran 2023-10-11 11:55:17 -0700
  • 66247a903e Use name postgres for DB host sabaimran 2023-10-11 11:48:59 -0700
  • 071e120c78 Try using 127.0.0.1 for the host name to solve connection issues sabaimran 2023-10-11 11:42:20 -0700
  • 6ab71c32cb Remove git installation step in the precommit workflow sabaimran 2023-10-11 11:26:59 -0700
  • 0114f29ef7 Separte pre-commit step into a separate, dedicated workflow sabaimran 2023-10-11 11:20:37 -0700
  • 0b4b0a4db9 Try installing gcc to avert error with gcc not found sabaimran 2023-10-11 10:59:27 -0700
  • 46ac15bf19 Install make in the Docker container sabaimran 2023-10-11 10:55:14 -0700
  • 89068fa985 Fix typo in pgvector installation steps sabaimran 2023-10-11 10:49:18 -0700
  • d42fcceeca Install git dependency in the container sabaimran 2023-10-11 10:37:55 -0700
  • 90c9cc5206 Automatically accept installation for postgresql-server-dev-14 sabaimran 2023-10-11 10:23:03 -0700
  • 9ebe1c0d74 To avoid requirement of user input, set DEBIAN_FRONTEND to noninteractive and bypass any prompts sabaimran 2023-10-11 10:19:52 -0700
  • 5264b97baf Remove sudo from installation steps, as we are inside of a docker container now and root by default sabaimran 2023-10-11 09:55:09 -0700
  • 11b7e0d83c Import error for libsqlite3 is persistent. Try installing another package to see if that fixes it sabaimran 2023-10-11 09:48:35 -0700
  • 91aaac6ccb Add an additional step to install pip dependency sabaimran 2023-10-11 09:46:03 -0700
  • 5fce0330af Add the libsqlite3-0 package via apt to try to resolve the import error sabaimran 2023-10-11 09:40:34 -0700
  • be1553a40e Add missing installation for sqlite3, as it is missing in the host machine. Stack trace: sabaimran 2023-10-11 09:31:11 -0700
  • 388fa78436 Previous command is not valid without setuptool or wheel installed, so try this workaround we've integrated for our Docker image sabaimran 2023-10-11 09:23:44 -0700
  • 857f69b96e Had a LookupError caused by setuptools-scm being unable to detect the version of the Khoj package when running inside the container. Try using --no-use-pep517 flag to mitigate sabaimran 2023-10-11 09:18:25 -0700
  • cac2b0ff64 Add a service for the postgres db in the Github action workflow, update settings to read from env sabaimran 2023-10-11 09:06:10 -0700
  • 35afb5f37a add top level dir sabaimran 2023-10-10 22:47:26 -0700
  • c7b2afd994 add src dir to pytest search path sabaimran 2023-10-10 22:43:55 -0700
  • 346e3a4eea Address mypy warnings and errors sabaimran 2023-10-10 22:04:15 -0700
  • 7aff80df1d Try installing pgvector extension from source sabaimran 2023-10-10 19:04:11 -0700
  • 607429431e Run postgresql setup script before installing pgvector extension sabaimran 2023-10-10 18:34:36 -0700
  • dea0ea2f3d Install postgresql on ubuntu sabaimran 2023-10-10 18:30:10 -0700
  • 6377644176 Update pytest.ini to use app settings, and update code to work with existing unit tests sabaimran 2023-10-10 17:47:18 -0700
  • 917d92d836
    Changed globbing. Now doesn't clobber a users glob if they want to add it, but will (if just given a directory), add a recursive glob. Andrew Spott 2023-10-10 15:09:05 -0600
  • bc9b234e26 Update more unit tests, output number of deleted embeddings in text_search.setup sabaimran 2023-10-10 08:15:14 -0700
  • 65f97aa164 Run tests on every PR and install the pgvector extension using apt sabaimran 2023-10-10 08:14:16 -0700
  • 8a2a5dccee Work in progress to fix unit tests, and some code clean up for processors sabaimran 2023-10-09 21:57:11 -0700
  • 6aa69da3ef Put indexer API endpoint under /api path segment Debanjum Singh Solanky 2023-10-09 21:35:58 -0700
  • 148e8f468f Restrict openai package version below 1.0.0 to avoid breaking changes Debanjum Singh Solanky 2023-10-09 19:30:58 -0700
  • cde099c011 Add migrations for models, use database to store local configuration for indexable plaintext files sabaimran 2023-10-09 14:53:38 -0700
  • 65fe9fba55 WIP: complete integration from reading/writing to settings in db per user sabaimran 2023-10-08 10:11:06 -0700
  • da9a41dc78 Read/write settings from github and use the user credentials when indexing data sabaimran 2023-10-06 17:55:36 -0700
  • f6f7a62d80 Wait for user to stop typing to trigger search from khoj.el in Emacs Debanjum Singh Solanky 2023-10-06 12:39:19 -0700
  • 5c4f0d42b7 Return new default config in API endpoint sabaimran 2023-10-06 12:30:09 -0700
  • 052b25af0a Update default configuration passed to Khoj clients to circumvent valiation issues sabaimran 2023-10-06 12:29:15 -0700
  • e191db1f3e Start updating indexer, api code to be user aware sabaimran 2023-10-06 11:54:37 -0700
  • c813ae99db Simplify database migrations sabaimran 2023-10-06 11:54:08 -0700
  • 6d64506979 Add migrations for embeddings support and read data from the vector DB sabaimran 2023-10-05 17:10:25 -0700
  • a85ff941ca Make offline chat model user configurable Debanjum Singh Solanky 2023-10-04 20:39:31 -0700
  • d1ff812021 Run GPT4All Chat Model on GPU, when available Debanjum Singh Solanky 2023-10-04 18:42:12 -0700
  • 13b16a4364 Use default Llama 2 supported by GPT4All Debanjum Singh Solanky 2023-10-03 16:29:46 -0700
  • 52bcfe0c28 Merge branch 'master' of github.com:khoj-ai/khoj into features/use-config-with-multi-user sabaimran 2023-10-03 13:00:00 -0700
  • a7b9db9494 Merge branch 'master' of github.com:khoj-ai/khoj into features/multi-user sabaimran 2023-10-03 12:57:15 -0700