Commit graph

1694 commits

Author SHA1 Message Date
Debanjum Singh Solanky
28df08b907 Fix configure openai processor for khoj docker
Store khoj search models and embeddings in default location in docker
container under /root/.khoj
2023-07-30 02:07:33 -07:00
Debanjum Singh Solanky
dffbfee62b Fix sample khoj docker config to index test data using new schema 2023-07-30 01:48:18 -07:00
Debanjum Singh Solanky
53810a0ff7 Create khoj config dir if non-existant, before writing to khoj env file 2023-07-30 01:35:36 -07:00
Debanjum Singh Solanky
56394d2879 Update demo video to configure offline chat via the web interface 2023-07-29 19:17:40 -07:00
Debanjum Singh Solanky
b32673db8e Fix link to Docs website in Khoj readme on Github 2023-07-29 12:50:39 -07:00
Debanjum Singh Solanky
a3d1212e79 Align docs landing page with updated github readme
- Screenshots of khoj search, chat
- Put quickstart on landing page
- Put miscellaneous pages under separate section
- Move credits to separate page under miscellaneous
2023-07-29 12:42:36 -07:00
Debanjum Singh Solanky
d7205aed36 Update docs with setup instructions for Offline and Online Chat 2023-07-29 11:18:12 -07:00
Debanjum
0404e33437
Add screenshots, style content in README 2023-07-29 01:22:48 -07:00
sabaimran
f65d157244 Release Khoj version 0.10.0 2023-07-28 19:27:47 -07:00
Debanjum Singh Solanky
f76af869f1 Do not log the gpt4all chat response stream in khoj backend
Stream floods stdout and does not provide useful info to user
2023-07-28 19:14:04 -07:00
sabaimran
5ccb01343e
Add Offline chat to Obsidian (#359)
* Add support for configuring/using offline chat from within Obsidian
* Fix type checking for search type
* If Github is not configured, /update call should fail
* Fix regenerate tests same as the update ones
* Update help text for offline chat in obsidian
* Update relevant description for Khoj settings in Obsidian
* Simplify configuration logic and use smarter defaults
2023-07-28 18:47:56 -07:00
Debanjum
b3c1507708
Merge pull request #361 from khoj-ai/configure-offline-chat-from-emacs
- Configure using Offline Chat from Emacs: 
- Enable, Disable Offline Chat from Emacs

- Use: Enable offline chat with `(setq khoj-chat-offline t)' during khoj setup
- Benefits: Offline chat models are better for privacy but not great at answering questions
2023-07-28 18:06:58 -07:00
sabaimran
9f78db0579
Let Offline chat override OpenAI API settings (#362)
* Let Offline chat override OpenAI API settings
* Download the offline model whenever offline chat is enabled
* Add progressbar for download for llamav2 model to track progress
* Change ordering of n due to switch of default processor
* Flip ordering of offline/openai checks when extracting questions from query
2023-07-28 17:26:20 -07:00
Debanjum Singh Solanky
ebfbef1f68 Configure using offline chat from Emacs
Closes #358
2023-07-28 16:07:33 -07:00
Debanjum Singh Solanky
9b1048caf7 Remove asymmetric from name of remaining text search tests
Asymmetric search is the only search type used now in khoj.el. So
making distinction between between symmetric and asymmetric search
isn't necessary anymore
2023-07-28 15:33:22 -07:00
sabaimran
12cfb48f16
Fix gpt4all import error in Desktop builds (#356)
* Add gpt4all to imports via sysconfig path
2023-07-28 11:54:18 -07:00
Debanjum
4b0639cfbd
Merge pull request #354 from ducksblock/master
Fix #353: Remove references to localhost:8000 in docs
2023-07-28 11:00:12 -07:00
ducksblock
cbecd7b66f Fix #353: Remove references to localhost:8000 2023-07-28 13:57:00 +05:30
sabaimran
702486dab7 Add gpt4all for copying metadata 2023-07-27 22:22:24 -07:00
sabaimran
29081f4429 Adjust parameters for offline chat 2023-07-27 22:22:09 -07:00
sabaimran
124d97c26d
Replace Falcon 🦅 model with Llama V2 🦙 for offline chat (#352)
* Working example with LlamaV2 running locally on my machine

- Download from huggingface
- Plug in to GPT4All
- Update prompts to fit the llama format

* Add appropriate prompts for extracting questions based on a query based on llama format

* Rename Falcon to Llama and make some improvements to the extract_questions flow

* Do further tuning to extract question prompts and unit tests

* Disable extracting questions dynamically from Llama, as results are still unreliable
2023-07-27 20:51:20 -07:00
sabaimran
55965eea7d
Delete FUNDING.yml
Instead of this file, use an organization-level file: https://github.com/khoj-ai/.github
2023-07-27 15:28:47 -07:00
sabaimran
925177b150
Update FUNDING.yml
Change to use a single organization (remove list brackets)
2023-07-27 15:19:20 -07:00
sabaimran
78197bb5c3
Create FUNDING.yml
- Add github sponsor information directly to khoj project. Closes #302
2023-07-27 15:16:45 -07:00
Debanjum Singh Solanky
da3f4dc7e4 Fix test config to run OpenAI Chat Actor, Director tests
OpenAI conversation processor schema had updated but conftest hadn't
been updated to reflect the same.

Update conftest setup of conversation processor to fix this
2023-07-27 11:30:04 -07:00
Debanjum Singh Solanky
715d56d4f0 Use new schema to update khoj.yml config from khoj.el 2023-07-26 17:34:16 -07:00
sabaimran
8b2af0b5ef
Add support for our first Local LLM 🤖🏠 (#330)
* Add support for gpt4all's falcon model as an additional conversation processor
- Update the UI pages to allow the user to point to the new endpoints for GPT
- Update the internal schemas to support both GPT4 models and OpenAI
- Add unit tests benchmarking some of the Falcon performance
* Add exc_info to include stack trace in error logs for text processors
* Pull shared functions into utils.py to be used across gpt4 and gpt
* Add migration for new processor conversation schema
* Skip GPT4All actor tests due to typing issues
* Fix Obsidian processor configuration in auto-configure flow
* Rename enable_local_llm to enable_offline_chat
2023-07-26 16:27:08 -07:00
sabaimran
23d77ee338
Fix import issues in desktop image builds (#343) 2023-07-26 15:45:52 -07:00
Justin Bassett-Green
8dcc21052f
Add chat-model param in sample config yml and document (#341)
* add chat-model config param to docs

* add chat-model param to sample config yml
2023-07-22 16:53:08 -07:00
Debanjum Singh Solanky
5bb42e56a8 Fix formatting of khoj test config and unused references in conftests 2023-07-22 00:29:26 -07:00
Debanjum Singh Solanky
7722a9c347 Default to using the gpt-3.5-turbo model for chat from khoj.el 2023-07-22 00:29:26 -07:00
Saba
36d25c4f1d Center the title, add table headers 2023-07-21 23:36:38 -07:00
Saba
01b6a10cd1 Simplify readme 2023-07-21 23:30:44 -07:00
sabaimran
4ce072c4b3
Make the README on our Github minimal (#334)
* Make the README on our Github minimal
* Add a bit of formatting and more background
2023-07-21 23:29:04 -07:00
Debanjum Singh Solanky
4089e38283 Fix links to demos and screenshots in docs 2023-07-21 20:01:19 -07:00
Debanjum Singh Solanky
89ad362758 Update Screenshots and Demos in Docs 2023-07-21 15:22:35 -07:00
Debanjum Singh Solanky
f0d4a4cf9a Revert "Make configure_content functional. Do not pass content index state to it."
This reverts commit 2ddee7e745 as it
broke partial updates of the content index for just the specified
content types
2023-07-21 13:59:09 -07:00
sabaimran
82c725817e Merge branch 'master' of github.com:khoj-ai/khoj 2023-07-21 13:24:05 -07:00
sabaimran
596e11ec6d Use the same function for computing entries for IDs regardless of whether it has prev entries 2023-07-21 13:23:56 -07:00
Saba
634f0b4cc4 Fix docs indexing issue 2023-07-21 08:30:00 -07:00
Debanjum Singh Solanky
c28755ccd2 Fix diff blocks, links, remove footnotes & rearrange sections in docs
Extract performance into separate sectin into shoving it under search
Create page for web interface
2023-07-21 00:58:30 -07:00
Debanjum Singh Solanky
2ddee7e745 Make configure_content functional. Do not pass content index state to it. 2023-07-20 23:24:08 -07:00
Debanjum
e92bc0e2e6 Create CNAME to make Docs accessible at docs.khoj.dev 2023-07-20 23:24:08 -07:00
sabaimran
1610d2ebd9
📝 Add a documentation base for Khoj! (#333)
* Add docs for more organized, accessible information detailing Khoj setup
* Delete duplicated files
* Add a coverpage without enabling it. Add logo and theme
* Remove obsidian README.md
* Add plausible script to index.html via docsify
2023-07-20 22:34:25 -07:00
Debanjum Singh Solanky
3e59be7f1d Release Khoj version 0.9.0 2023-07-18 19:59:27 -07:00
Debanjum Singh Solanky
d078e7b1f6 Clean up search type usage in khoj server, tests and Readme 2023-07-18 19:57:55 -07:00
Debanjum Singh Solanky
4d910936b7 Fix triggering index update on khoj server from khoj.el 2023-07-18 19:57:54 -07:00
Debanjum Singh Solanky
5c7d7f558d Make AI model used for Khoj chat configurable from khoj.el
- Fix bug. Set the unused model-name to a standad default value
2023-07-18 19:57:54 -07:00
Debanjum
5f2be2a9bb
Merge pull request #298 from HyunggyuJang/patch-1
Encode config as utf-8 during setup in khoj.el. This will allow utf-8 encoded files etc to be passed in config
2023-07-18 17:54:11 -07:00
Debanjum
3a1c5a6dab
Merge pull request #329 from khoj-ai/create-schema-migration-func-and-reindex-to-fix-corruption
Create Schema Migrator and Reindex to Apply Index Corruption Fixes

- 83e1088 Manage `khoj.yml' config migrations on app start. Version the `khoj.yml' schema
- 429e1b4 Regenerate index to apply corruption fixes on first run of this khoj version
   Otherwise users would need to manually re-index their contents with khoj
2023-07-18 16:43:17 -07:00