sij/khoj
1
0
Fork 0
mirror of https://github.com/khoj-ai/khoj.git synced 2024-11-24 16:05:07 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 38
1466 commits 10 branches 107 tags 131 MiB
Commit graph

1466 commits

This branch
This branch
All branches
Author SHA1 Message Date
Debanjum Singh Solanky
41ac1e24c9 Add docs for a pre-emptive setup of Khoj for later offline usage 
Closes #151
 2023-07-05 20:48:51 -07:00
Debanjum
6c2a8a5bce
⚡️ Stream Responses by Khoj Chat on Web, Obsidian 
- What
   - Stream chat responses from OpenAI API to Web, Obsidian clients
      - Implement using a callback function which manages a queue where new tokens can be placed as they come on. As the thread is read from, tokens are removed.
      - When the final token has been processed, add the `compiled_references` to the queue to be rendered by the `chat` client
      - When the thread has been closed, save the accumulated conversation log in the user's history using a `partial func`
      - Incrementally decode tokens on the front end and add them as they appear from the streamed response

- Why
This significantly reduces perceived latency and OpenAI API request timeouts for Chat

Closes https://github.com/khoj-ai/khoj/issues/257
 2023-07-05 20:02:11 -07:00
Debanjum Singh Solanky
e111eda6ae Make client, app_config optional in telemetry logger for correct typing 2023-07-05 18:57:38 -07:00
Debanjum Singh Solanky
e562114f6b Improve comments, var names in js for chat streaming on web interface 2023-07-05 18:57:27 -07:00
Debanjum Singh Solanky
46269ddfd3 Fix chat logging messages to get context without flooding logs 2023-07-05 18:27:06 -07:00
Debanjum Singh Solanky
0ba838b53a Show temp status message in Khoj Obsidian chat while Khoj is thinking 
- Scroll to bottom after adding temporary status message and
references too
 2023-07-05 18:02:43 -07:00
Debanjum Singh Solanky
8271abe729 Use optional chaining operator to extract khojBannerSubmit from conditional 2023-07-05 18:02:43 -07:00
Debanjum Singh Solanky
c12ec1fd03 Show temp status message in Khoj web chat while Khoj is thinking 
- Scroll to bottom after adding temporary status message and
references too
 2023-07-05 18:02:30 -07:00
sabaimran
257a421e45 Bonus: add try-catch logic around telemetry upload in case of JSON serializability issues 2023-07-05 15:12:18 -07:00
sabaimran
4e6b66b139 Add support for streaming chat response from OpenAI to Obsidian 
- I needed to installed node-fetch to accomplish this, as the built-in request object from Obsidian doesn't seem to support streaming and the built-in fetch object is very sensitive to any and all cross origin requests
 2023-07-05 15:01:22 -07:00
sabaimran
3ff5074cf5 Log the end-to-end time of generating a streamed response from OpenAI 2023-07-05 14:59:44 -07:00
sabaimran
68e635cc32 Remove additional comments and debug statements 2023-07-05 11:33:56 -07:00
sabaimran
67a8795b1f Clean-up commented out code 2023-07-05 11:24:40 -07:00
sabaimran
79b1b1d350 Save streamed chat conversations via partial function passed to the ThreadGenerator 2023-07-04 17:33:52 -07:00
sabaimran
afd162de01 Add reference notes to result response from GPT when streaming is completed 
- NOTE: results are still not being saved to conversation history
 2023-07-04 12:47:50 -07:00
sabaimran
8f491d72de Initial code with chat streaming working (warning: messy code) 2023-07-04 10:14:39 -07:00
Debanjum Singh Solanky
5889eceba4 Make text selectable in Khoj chat modal on Obsidian 
Previously the text in the Khoj chat modal couldn't be copied as it
was not selectable

Resolves #206
 2023-07-03 23:24:04 -07:00
sabaimran
89354def9b Update request timeout window to 20 seconds 2023-07-03 22:28:18 -07:00
sabaimran
b1940519c3 Log error if unable to decode chunk from Github 2023-07-03 16:29:32 -07:00
Debanjum Singh Solanky
ecf9730cd7 Disable Chat, Search on Web if Khoj not configured & show next steps 2023-07-03 16:04:32 -07:00
sabaimran
017e8c1aef
Skip indexing a PDF that has an indexing error (#274) 2023-07-03 15:55:11 -07:00
sabaimran
a6f313589e Release Khoj version 0.7.1 2023-07-03 12:26:41 -07:00
Debanjum Singh Solanky
70f6b8266c Upgrade minimum supported pydantic version 2023-07-03 12:22:56 -07:00
sabaimran
8bfd5828e6 Remove deprecation notice since we're opening the web UI by default 2023-07-03 12:01:09 -07:00
sabaimran
92d81d3b16
Initialize the search.model field to SearchModels() and fix Reinitialize API call (#273) 2023-07-03 11:32:44 -07:00
sabaimran
61403138d5
Merge pull request #269 from khoj-ai/features/simplify-configuration-steps 
Simplify some common configuration steps
 2023-07-03 00:16:51 -07:00
sabaimran
ea3dc2cfa3 Simplify rendering of content type pages and logic of selecting config 2023-07-03 00:15:29 -07:00
sabaimran
260272dca2 Check if state.config is populated before configuring via the update method 2023-07-03 00:10:56 -07:00
sabaimran
bf8914d0c8 Fix default config initialization for for chat.html 2023-07-03 00:00:47 -07:00
Debanjum
faad1297f4
Drop Support for Org Music, Ledger Content Types 
Removing unused content types will reduce khoj code to manage

- 0f993b3 Drop support for Ledger as a separate content type
   Khoj will soon get a generic text indexing content type in Index plain text files #237.
   This along with a file filter should suffice for searching through Ledger transactions

- c9db532 Remove unused org-music as an indexable content type from Khoj
   Org-music was just a custom content type that worked with org-music.
   It was mostly only useful for me.
 2023-07-02 17:48:29 -07:00
Debanjum Singh Solanky
0f993b332e Drop support for Ledger as a separate content type 
Khoj will soon get a generic text indexing content type. This along
with a file filter should suffice for searching through Ledger
transactions, if required.

Having a specific content type for niche use-case like ledger isn't
useful. Removing unused content types will reduce khoj code to manage.
 2023-07-02 16:57:49 -07:00
sabaimran
fa218ff5aa Fix call to update for Reinitialize button 2023-07-02 16:31:30 -07:00
sabaimran
a8b83da872 Merge branch 'master' of github.com:debanjum/khoj into features/simplify-configuration-steps 2023-07-02 16:21:54 -07:00
Debanjum Singh Solanky
c9db5321e7 Remove unused org-music as an indexable content type from Khoj 
Org-music was just a custom content type that worked with org-music.
It was mostly only useful for me.

Cleaning up that code will reduce number of content types for khoj to
manage.
 2023-07-02 16:21:21 -07:00
sabaimran
77a45f4215
Merge pull request #265 from khoj-ai/fix/obsidian-setup-issues 
Fix configuration setup logic in Obsidian
 2023-07-02 16:21:18 -07:00
sabaimran
b86a3bb0c5 Merge branch 'master' of github.com:debanjum/khoj into fix/obsidian-setup-issues 2023-07-02 16:21:05 -07:00
sabaimran
a52c1c8380 Use built-in app.vault to determine whether there are any PDF files within 2023-07-02 16:20:43 -07:00
sabaimran
eff1436857 Overwrite existing PDFs in Obsidian as well, make if-block more legible 2023-07-02 16:17:25 -07:00
Debanjum Singh Solanky
30459ee4ba Fix Khoj subtitle in desktop entry, pyproject, cli and Obsidian Readme 2023-07-02 16:09:07 -07:00
sabaimran
feac71ce1e
Merge pull request #268 from khoj-ai/fix/threading-issue-in-update-api 
Add try-except-finally blocks around configure calls in /update
 2023-07-02 16:08:29 -07:00
sabaimran
1a1b044d12 Simplify settings pages for configuration 
- Add one-click disablement
- Remove fields that probably don't need to be edited (our implementation details)
- Add a green tick if a given field is configured
 2023-07-02 16:04:05 -07:00
sabaimran
e4c445f805 Add try-except-finally blocks around configure calls in /update 2023-07-02 13:35:02 -07:00
sabaimran
4b02a8c788 Fix PDF setup in Obsidian plugin and force Obsidian configuration for markdown 2023-07-02 12:37:24 -07:00
sabaimran
b6772d8fc3
Merge pull request #264 from khoj-ai/fix/remove-guidance-for-desktop-gui 
Escape special characters in the URL when adding a link to the remote file
 2023-07-02 09:14:08 -07:00
sabaimran
2a7e4f2b71 Escape special characters in the URL when adding a link to the remote file 2023-07-02 09:13:28 -07:00
sabaimran
4915b7214d
Merge pull request #263 from khoj-ai/fix/remove-guidance-for-desktop-gui 
[Fix] Remove the default behavior of using GUI for Khoj
 2023-07-01 21:37:11 -07:00
sabaimran
c747562897 Update the GUI to just be a simple box with a button for the web UI 2023-07-01 20:37:21 -07:00
sabaimran
bab7f39d47 Move logic to open the web browser into the GUI section 2023-07-01 20:11:27 -07:00
sabaimran
36537606da Update unit test and preserve prior operational ordering in main.py 2023-07-01 20:02:35 -07:00
sabaimran
ea9ae4ae28 Configure Khoj to automatically open the browser to their web home page when Khoj is up 2023-07-01 19:46:31 -07:00