Commit graph

336 commits

Author SHA1 Message Date
Debanjum Singh Solanky
de24ffcf0d Upgrade Axios, a desktop app dependency, to version 1.7.4 2024-08-20 12:32:36 -07:00
sabaimran
1ac8de6c3a Release Khoj version 1.21.1 2024-08-20 11:55:34 -07:00
sabaimran
029775420c Release Khoj version 1.21.0 2024-08-20 10:01:56 -07:00
Debanjum Singh Solanky
7815e02dd4 Release Khoj version 1.20.4 2024-08-11 16:00:13 +05:30
sabaimran
acf1c14122 Release Khoj version 1.20.3 2024-08-09 18:11:11 +05:30
sabaimran
00ee4c2697 Release Khoj version 1.20.2 2024-08-06 16:16:33 +05:30
sabaimran
7607abc726 Release Khoj version 1.20.1 2024-08-06 10:05:41 +05:30
Debanjum Singh Solanky
e412ed3bcb Release Khoj version 1.20.0 2024-08-05 16:25:21 +05:30
sabaimran
91047d1619 Use a png for the windows desktop icon 2024-08-05 15:29:30 +05:30
sabaimran
1151d14466 Add a separate windows object in the todesktop configuration 2024-08-05 14:27:56 +05:30
sabaimran
c56072aa7b Update todesktop runtime and use the icns file for the todesktop configuration 2024-08-05 14:19:38 +05:30
sabaimran
484b0aa96b Use the newer, simpler favicon across desktop and documentation. Update the macos icon set 2024-08-05 14:06:04 +05:30
Debanjum Singh Solanky
1cdfa8087c Update Khoj tagline to "Your Second Brain" 2024-08-05 02:27:05 +05:30
Debanjum Singh Solanky
46f928165c Fix deep linking to settings page cards from docs 2024-08-05 02:27:05 +05:30
sabaimran
1a1d9c7257 Merge branch 'master' of github.com:khoj-ai/khoj into features/big-upgrade-chat-ux 2024-07-27 14:18:05 +05:30
sabaimran
eb5af38f33 Release Khoj version 1.17.0 2024-07-26 20:14:45 +05:30
Debanjum Singh Solanky
37b8fc5577 Extract events even when http chunk contains partial or mutiple events
Previous logic was more brittle to break with simple unbalanced
'{' or '}' string present in the event data. This method of trying to
identify valid json obj was fairly brittle. It only allowed json
objects or processed event as raw strings.

Now we buffer chunk until we see our unicode magic delimiter and only
then process it.

This is much less likely to break based on event data and the
delimiter is more tunable if we want to reduce rendering breakage
likelihood further
2024-07-24 17:17:39 +05:30
Debanjum Singh Solanky
eb4e12d3c5 s/online_context/onlineContext chat API response field for consistency
This will align the name of the online context field returned by
current chat message and chat history
2024-07-23 19:50:43 +05:30
Debanjum Singh Solanky
0277d16daf Share desktop chat streaming utility funcs across chat, shortcut views
Null check menu, menuContainer to avoid errors on Khoj mini
2024-07-23 19:16:33 +05:30
Debanjum Singh Solanky
fc33162ec6 Use new chat streaming API to show Khoj train of thought in Desktop app
Show loading spinner at end of current message
2024-07-23 18:17:47 +05:30
Debanjum Singh Solanky
469a1cb6a2 Move API endpoints under /api/configure/content/ to /api/content/
Pull out /api/configure/content API endpoints into /api/content to
allow for more logical organization of API path hierarchy

This should make the url more succinct and API request intent more
understandable by using existing HTTP method semantics along with the
path.

The /configure URL path segment was either
- redundant (e.g POST /configure/notion) or
- incorrect (e.g GET /configure/files)

Some example of naming improvements:
- GET /configure/types -> GET /content/types
- GET /configure/files -> GET /content/files
- DELETE /configure/files -> DELETE /content/files

This should also align, merge better the the content indexing API
triggered via PUT, PATCH /content

Refactor Flow
1. Rename /api/configure/types -> /api/content/types
2. Rename /api/configure -> /api
3. Move /api/content to api_content from under api_config
2024-07-19 05:40:34 +05:30
Debanjum Singh Solanky
5923b6d89e Split /api/v1/index/update into /api/content PUT, PATCH API endpoints
- This utilizes PUT, PATCH HTTP method semantics to remove need for
  the "regenerate" query param and "/update" url suffix
- This should make the url more succinct and API request intent more
  understandable by using existing HTTP method semantics
2024-07-19 01:45:53 +05:30
Debanjum Singh Solanky
a1c362a4f7 Expose web, desktop settings page under /settings, not /configure
- Update references to the settings page to use new url across docs
  and code
- Rename desktop and web settings page to settigns.html instead of
  config[ure].html
2024-07-17 13:17:29 +05:30
Debanjum Singh Solanky
59000a47cb Move Desktop config page to /configure from /config url path
Update references to point to page at /configure instead of /config
2024-07-16 16:13:27 +05:30
Debanjum Singh Solanky
a5c16ad600 Move Web client config page to /configure from /config url path
Update docs, clients and error messages to point to /configure
instead of /config
2024-07-16 16:13:27 +05:30
Debanjum Singh Solanky
de15a7a3fc Rename API path /api/config to /api/configure
- Update clients calling /api/config to call /api/configure instead
2024-07-16 16:13:27 +05:30
Debanjum Singh Solanky
3a75838196 Add Keyboard shortcuts to navigate in Khoj Desktop 2024-07-11 16:29:53 +05:30
Debanjum Singh Solanky
9e31ebff93 Release Khoj version 1.16.0 2024-07-07 18:26:10 +05:30
Raghav Tirumale
8eccd8a5e4
Support Indexing Images via OCR (#823)
- Added support for uploading .jpeg, .jpg, and .png files to Khoj from Web, Desktop app
- Updating indexer to generate raw text and entries using RapidOCR
- Details
  * added support for indexing images via ocr
  * fixed pyproject.toml
  * Update src/khoj/processor/content/images/image_to_entries.py
     Co-authored-by: Debanjum <debanjum@gmail.com>
  * Update src/khoj/processor/content/images/image_to_entries.py
     Co-authored-by: Debanjum <debanjum@gmail.com>
  * removed redudant try except blocks
  * updated desktop js file to support image formats
  * added tests for jpg and png
  * Fix processing for image to entries files
  * Update unit tests with working image indexer
  * Change png test from version verificaition to open-cv verification

---------

Co-authored-by: Debanjum <debanjum@gmail.com>
Co-authored-by: sabaimran <narmiabas@gmail.com>
2024-07-01 06:00:00 -07:00
Raghav Tirumale
24a0d8b073
Add OS Level Shortcut Window for Quick Access to Khoj Desktop (#815)
* rough sketch of desktop shortcuts. many bugs to fix still

* working MVP of desktop shortcut khoj

* UI fixes

* UI improvements for editable shortcut message

* major rendering fix to prevent clipboard text from getting lost

* UI improvements and bug fixes

* UI upgrades: custom top bar, edit sent message and color matching

* removed debug javascript file

* font reverted to Noto Sans

* cleaning up the code and removing diffs

* UX fixes

* cleaning up unused methods from html

* front end for button to send user back to main window to continue conversation

* UX fix for window and continue conversation support added

* migrated common js functions into chatutils.js

* Fix window closing issue in macos by

1. Use a helper function to determine if the window is open by seeing if there's a browser window with shortcut.html loaded
2. Use the  event listener on the window to handle teardown

* removed extra comment and renamed continue convo button

---------

Co-authored-by: sabaimran <narmiabas@gmail.com>
2024-06-27 07:20:13 -07:00
Debanjum Singh Solanky
4f89319b40 Release Khoj version 1.15.0 2024-06-26 10:38:16 +05:30
Debanjum Singh Solanky
abd6f58aee Upgrade Desktop app package dependencies 2024-06-22 17:38:52 +05:30
Debanjum Singh Solanky
5e5fe4b7af Improve font size, spacing of conversation session on desktop app 2024-06-21 12:25:35 +05:30
Debanjum Singh Solanky
677d49d438 Release Khoj version 1.14.0 2024-06-18 17:13:46 +05:30
Debanjum Singh Solanky
6814dadd21 Fix opening Web, Desktop setup links on first run from Desktop app
Previous version failed to open the setup links
2024-06-18 17:04:25 +05:30
sabaimran
06a47ee457
Add language-specific syntax highlighting via highlight.js (#802)
* Add language-specific syntax highlighting via highlight.js

- Add highlight.js to our assets CDN for fast load and compliance with the CSP
- See other stylesheets options here: https://cdnjs.com/libraries/highlight.js

* Bonus: set min-height to prevent increasing length of the sessions pane

* Fix references rendering and add highlight.js in public conversation
2024-06-08 15:17:09 +05:30
sabaimran
58a02f06ea
Fix multilingual font rendering (#797)
* Fix multilingual font rendering; fallback to an Arabic language font which contains more Asian characters. Close #756

* Tune font-sizes and styling to accomodate new fonts with old sizing

- Move connection-status styling out from inline html into css block
- Remove start typing chat-input height jitter
- align new-conversation button, text
- use relative font sizes instead of absolute font sizes in most places

---------

Co-authored-by: Debanjum Singh Solanky <debanjum@gmail.com>
2024-06-07 11:53:47 +05:30
Raghav Tirumale
ba16afd3c2
New Feature: Adding File Filtering to Conversations (#788)
* UI update for file filtered conversations
* Interactive file menu #UI to add/remove files on each conversation as references.
* Backend changes implemented to load selected file filters from a conversation into the querying process.
---------

Co-authored-by: sabaimran <narmiabas@gmail.com>
2024-06-07 10:53:37 +05:30
Debanjum Singh Solanky
85a343363b Release Khoj version 1.13.0 2024-06-04 11:57:44 +05:30
Debanjum Singh Solanky
b757ba664f Sanitize chat messages to render in Obsidian, Desktop, Web apps
Use DOMPurify to escape any unsafe HTML in chat message before adding
it to DOM via innerHTML updates to a HTML element
2024-06-04 10:53:30 +05:30
Debanjum Singh Solanky
9f80c2ab76 Enforce Content-Security-Policy (CSP) in Obsidian, Desktop, Web apps
Prevent XSS attacks by enforcing Content-Security-Policy (CSP) in apps.
Do not allow loading images, other assets from untrusted domains.

- Only allow loading assets from trusted domains
  like 'self', khoj.dev, ipapi for geolocation, google (fonts, img)
  - images from khoj domain, google (for profile pic)
  - assets from khoj domain
  - Do not allow iframe src
  - Allow unsafe-inline script and styles for now as markdown-it escapes html
    in user, khoj chat

- Add hostURL to CSP of the Desktop, Obsidian apps
  Given web client is served by khoj server, it doesn't need to
  explicitly allow for khoj.dev domain. So if user self-hosting, it'll
  automatically allow the domain in the CSP (via 'self')

  Whereas the Obsidian, Desktop clients allow configure the server URL.
  Note *switching server URL breaks CSP until app is reloaded*
2024-06-04 10:53:30 +05:30
Debanjum Singh Solanky
8cdab5f31a Update slash command UX in chat UI of desktop app to match web app
Make commands in popup menu on typing slash in chat input selectable
2024-06-02 17:27:37 +05:30
Debanjum
cf8c9c2a3d
Serve image assets from Khoj domain, not directly from S3 bucket (#734)
- Serve generated images from Khoj domain instead of directly from AWS S3
- Rename assets URL from Khoj S3 bucket to assets.khoj.dev
2024-06-02 17:24:35 +05:30
Debanjum Singh Solanky
7a93599fe8 Merge branch 'master' into upgrade-khoj-on-obsidian
- Conflicts:
  - src/khoj/interface/web/chat.html
    Use our changes with feedback button changes from master
2024-06-01 10:07:43 +05:30
Debanjum Singh Solanky
7b18919564 Tag external links to open in a separate window on the Desktop app
Previously clicking inline links would open the URL directly in the
Desktop app. This was strange and it didn't provide any way to go back
to Khoj desktop app UI from the opened link
2024-05-29 10:12:50 +05:30
Debanjum Singh Solanky
7594401461 Fix expand chat reference animation in web, desktop, obsidian clients 2024-05-28 20:56:26 +05:30
sabaimran
607534021b Add a link to github in the settings menu, improve styling 2024-05-27 11:39:30 +05:30
Debanjum Singh Solanky
e24ca9ec28 Pass file path of each doc reference in references returned by API
- Pass file path of reference along with the compiled reference in
  list of references returned by chat API converts
- Update the structure of references from list of strings to list of
  dictionary (containing 'compiled' and 'file' keys)
- Pull out the compiled reference from the new references data struct
  wherever it was is being used
2024-05-26 18:02:11 +05:30
Debanjum Singh Solanky
f495d338eb Modularize render message with references func in web based clients
Simplify, reuse, standardize code to render messages with references
in the obsidian, web and desktop clients. Specifically:

- Reuse function to create reference section, dedupe code
- Create reusable function to generate image markdown
- Simplify logic to render message with references
2024-05-26 13:55:22 +05:30
sabaimran
e23c803cee Release Khoj version 1.12.1 2024-05-24 21:42:03 +05:30
sabaimran
14c9bea663
Make conversations optionally shareable (#712)
* Make conversations optionally shareable

- Shared conversations are viewable by anyone, without a login wall
- Can share a conversation from the three dot menu
- Add a new model for Public Conversation
- The rationale for a separate model is that public and private conversations have different assumptions. Separating them reduces some of the code specificity on our server-side code and allows us for easier interpretation and stricter security. Separating the data model makes it harder to accidentally view something that was meant to be private
- Add a new, read-only view for public conversations
2024-05-05 23:16:04 +05:30
Debanjum Singh Solanky
425496844b Rename assets URL from Khoj S3 bucket to assets.khoj.dev
Server khoj assets from khoj domain
2024-05-04 20:07:10 +05:30
sabaimran
bc8b92a77d Release Khoj version 1.12.0 2024-05-01 16:30:48 +05:30
sabaimran
7fd57d737e Adjustments to improve overall styling of config page, email template 2024-05-01 14:19:47 +05:30
Debanjum Singh Solanky
c17dbbeb92 Render next run time in user timezone in config, chat UIs
- Pass timezone string from ipapi to khoj via clients
  - Pass this data from web, desktop and obsidian clients to server
- Use user tz to render next run time of scheduled task in user tz
2024-05-01 08:30:10 +05:30
Debanjum Singh Solanky
648f1a5c71 Suffix chat response element vars with "El" in chat.html of web, desktop apps 2024-05-01 08:28:59 +05:30
sabaimran
37879a7850 Release Khoj version 1.11.2 2024-04-30 13:31:06 +05:30
sabaimran
12258f02d7 Release Khoj version 1.11.1 2024-04-27 18:42:24 +05:30
sabaimran
49834e3b00 Add a hero image for the og:image meta tag 2024-04-27 17:07:21 +05:30
sabaimran
138f12f957 Fix indentation and revert first run message link styling to all links 2024-04-27 09:56:58 +05:30
sabaimran
d8f2eac6e0 Release Khoj version 1.11.0 2024-04-25 17:24:59 +05:30
Debanjum Singh Solanky
1842017393 Skip trying to index deleted files, folders from Desktop app
Previously app would crash on startup if desktop app was told to
index a file that had been deleted afterwards
2024-04-25 15:23:05 +05:30
Debanjum Singh Solanky
8196ab62f9 Make valid file extension checking case insensitive on Desktop app 2024-04-24 09:35:20 +05:30
Debanjum Singh Solanky
5def14e3bb Skip indexing non-existent folders on Desktop app 2024-04-24 09:35:20 +05:30
Debanjum Singh Solanky
cd05f262a6 Pass auth headers to fix lazy load of chat messages on Desktop app 2024-04-24 09:35:20 +05:30
Debanjum Singh Solanky
4d5d3e6433 Set chat-message height to height of content in web, desktop
In some cases, especially with image generation requests, this was
causing the chat messages to overlap in the chat UI
2024-04-24 09:35:20 +05:30
Olatoyan George
ad59180fb8
Added indication in the desktop UI for back-end connectivity (#711)
* Changed the styling of the link that takes a user to the settings page into a button
* added an indicator that shows if a user is connected to the server or not
* made a class name more descriptive and also made the text in first run message more intuitive
* changed the command to install dependencies in the README.md
* changed the class name of the first run message text to be more descriptive
* added icons in the desktop UI that shows if a file is synced successfully or not
* made the link class name in the homepage more descriptive
* fixed the hover issue on status box in the chat header pane
* fixed hovering issue on status box on macOS
2024-04-23 16:43:48 +05:30
Debanjum Singh Solanky
1f2ffce85b Copy chat message with it's markdown formatting in Web, Desktop apps 2024-04-16 22:10:34 +05:30
Debanjum Singh Solanky
128829c477 Show latest msgs on chat session load. Fetch rest as they near viewport
- Reduces time to first render when loading long chat sessions
- Limits size of first page load, when loading long chat sessions

These performance improvements are maximally felt for large chat
sessions with lots of images generated by Khoj

Updated web and desktop app to support these changes for now
2024-04-15 16:10:56 +05:30
Debanjum Singh Solanky
e5ff85f6fb Start fetching khoj css before icons to reduce time with no styling
This should reduce frequency of page load jitter when icons are loaded
before style is applied
2024-04-15 15:32:32 +05:30
Debanjum Singh Solanky
c6e8443631 Update clients to support rendering webp images inline
This is for self-hosted scenarios where AWS S3 uploads is not enabled
2024-04-13 13:11:18 +05:30
Debanjum Singh Solanky
b8bc6bee83 Always remove loading animation on Desktop app if can't login to server 2024-04-13 11:02:44 +05:30
Debanjum Singh Solanky
89b8ec3546 Release Khoj version 1.10.2 2024-04-12 11:53:32 +05:30
Debanjum Singh Solanky
50b4788a91 Remove chat loading animation in login required state on Desktop app 2024-04-12 11:50:54 +05:30
Debanjum Singh Solanky
b3f4794d91 Remove the unnecessary async/await func chains on Desktop app 2024-04-12 11:49:25 +05:30
Debanjum Singh Solanky
1e30a072d4 Just use file ext to identify indexable files to fix Desktop app install
- Magika on Desktop app was too bloated (100Mb to 250Mb) and broke
  install for some reason. Not sure why it was causing the app install
  to fail but do not have time to currently investigate

- Just use file extensions whitelist it's good enough for now. Let
  server handle the deeper identification of file type
2024-04-12 11:16:07 +05:30
Debanjum Singh Solanky
07f8fb5c5b Release Khoj version 1.10.1 2024-04-12 02:18:07 +05:30
Debanjum Singh Solanky
60337086f9 Release Khoj version 1.10.0 2024-04-12 01:01:02 +05:30
Debanjum Singh Solanky
34c3f70203 Index only files with valid text extension in folders synced by Desktop app
This maintains consistent set of indexable files from Desktop app,
whether indexing via file or folder filters
2024-04-12 00:59:54 +05:30
Debanjum
9a48f72041
Index more text file types from Desktop, Github (#692)
### Index more text file types 
- Index all text, code files in Github repos. Not just md, org files
- Send more text file types from Desktop app and improve indexing them
- Identify file type by content & allow server to index all text files

### Deprecate Github Indexing Features
- Stop indexing commits, issues and issue comments in a Github repo
- Skip indexing Github repo on hitting Github API rate limit

### Fixes and Improvements
- **Fix indexing files in sub-folders from Desktop app**
- Standardize structure of text to entries to match other entry processors
2024-04-12 00:08:29 +05:30
Debanjum Singh Solanky
f2dc9709b7 Use Magika to more robustly identify text files to send for indexing
- `file-type' doesn't handle mis-labelled files or files without
   extensions well

- Only show supported file types in file selector dialog on Desktop app
  Use Magika to get list of text file extensions. Combine with other
  supported extensions to get complete list of supported file extensions.
  Use it to limit selectable files in the File Open dialog.

  Note: Folder selector will index text files with no extensions as well
2024-04-10 22:44:24 +05:30
Debanjum Singh Solanky
48d249db9e Center the nav item text and user profile initial icons 2024-04-10 19:38:43 +05:30
Debanjum Singh Solanky
60f6a1c6f1 Use svg icons in nav pane to standardize styling on Web, Desktop apps
Emojis varied based on device. svg icons standardize icon styles of
the web, desktop apps
2024-04-10 19:38:43 +05:30
Debanjum Singh Solanky
804c04f7b9 Do not render copy message button on every Khoj thinking step
Only render copy chat message button once, after message text is rendered
2024-04-10 14:48:36 +05:30
Debanjum Singh Solanky
cadeaac769 Align conversation sessions side panel on Desktop app with Web app
- Move new conversation button to right of "Conversation" title
- Reduce size of chat message loading ellipsis animation
- Add loading animation for chat session
2024-04-10 10:34:36 +05:30
Debanjum Singh Solanky
1c3d129e08 Add button to copy chat messages on Desktop client 2024-04-10 10:34:36 +05:30
Debanjum Singh Solanky
0a5a91619e Improve copy button's icon, hover color & click animation in Desktop UI 2024-04-10 10:34:36 +05:30
Debanjum Singh Solanky
8ff3890ba8 Dynamically generate navigation menu based on user info from server 2024-04-10 10:34:36 +05:30
Debanjum Singh Solanky
377e979800 Make current chat expand to full width when session panel collapsed
This behavior also matches web client behavior on chat session panel
collapse
2024-04-09 21:04:44 +05:30
Debanjum Singh Solanky
913dcdfbcd Only render first run setup message once if error or server not running 2024-04-09 21:04:44 +05:30
Debanjum Singh Solanky
76cb543347 Show title bar in Khoj desktop app on Windows 2024-04-09 21:04:44 +05:30
Debanjum Singh Solanky
f040418cf1 Fix indexing files in sub-folders on the Desktop app
- `fs.readdir' func in node version 18.18.2 has buggy `recursive' option
  See nodejs/node#48640, effect-ts/effect#1801 for details

- We were recursing down a folder in two ways on the Desktop app.
  Remove `recursive: True' option to the `fs.readdirSync' method call
  to recurse down via app code only
2024-04-09 20:19:40 +05:30
Debanjum Singh Solanky
7ff1bd9f8b Send more text file types from Desktop app and improve indexing them
- Allow syncing more file types from desktop app to index on server
  - Use `file-type' package to identify valid text file types on Desktop app

- Split plaintext entries into smaller logical units than a whole file
  Since the text splitting upgrades in #645, compiled chunks have more
  logical splits like paragraph, sentence.
  Show those (potentially) smaller snippets to the user as references

- Tangential Fix:
  Initialize unbound currentTime variable for error log timestamp
2024-04-09 20:19:40 +05:30
Debanjum Singh Solanky
4be4c53222 Release Khoj version 1.9.0 2024-04-05 17:13:58 +05:30
Debanjum Singh Solanky
4f65dde201 Release Khoj version 1.8.0 2024-03-31 00:06:15 +05:30
Debanjum
586654e2af
Allow directly reading web pages, even when SERP not enabled (#676)
### Overview
Khoj can now read website directly without needing to go through the search step first

### Details
- Parallelize simple webpage read and extractor
- Rename extract_content online results field to web pages
- Tweak prompts to extract information from webpages, online results
- Test select webpage as data source and extract web urls chat actors

- Render webpage read in chat response references on Web, Desktop apps
- Pass multiple webpages with their urls in online results context

- Support webpage command in chat API
- Add webpage chat command for read web pages requested by user
- Create chat actor for directly reading webpages based on user message
2024-03-24 16:25:25 +05:30
Debanjum Singh Solanky
dabf71bc3c Render webpage read in chat response references on Web, Desktop apps 2024-03-24 15:47:38 +05:30
Debanjum Singh Solanky
ec0c35b7ed Improve delete, rename chat session UX in Desktop, Web app
- Ask for Confirmation before deleting chat session in Desktop, Web app
- Save chat session rename on hitting enter in title edit input box
- No need to flash previous conversation cleared status message
- Move chat session delete button after rename button in Desktop app
2024-03-15 15:58:19 +05:30
Debanjum Singh Solanky
c792fa819f Fix setting chat session title from Desktop app
Pass auth headers to not have the chat session title update request fail
2024-03-15 15:19:20 +05:30
Debanjum Singh Solanky
fba0338787 Release Khoj version 1.7.0 2024-03-15 00:08:32 +05:30