Commit graph

2834 commits

Author SHA1 Message Date
Debanjum Singh Solanky
e15dc23bbe Improve logic to create vs reuse window for khoj side pane logic
Khoj side pane occupies a vertically split bottom right side pane.
If the bottom right window is not a vertical split, create a new
vertical split pane for khoj, otherwise reuse the existing window
2024-06-15 16:37:41 +05:30
Debanjum Singh Solanky
055e5e8d26 Create new conversation from the chat menu in Khoj Emacs 2024-06-15 16:37:41 +05:30
Debanjum Singh Solanky
c33954cd93 Fix loading an empty chat session in Emacs 2024-06-15 16:37:41 +05:30
Debanjum Singh Solanky
e21c0648ae Create, use reusable function to call Khoj API from elisp 2024-06-15 16:37:41 +05:30
Debanjum Singh Solanky
7bcb49b6e7 Support conversation sessions in the Khoj Emacs client
Add option in khoj main transient menu option to open menu to
- switch between existing conversations
2024-06-15 13:13:20 +05:30
Debanjum Singh Solanky
df9c5ff263 Show online references used for chat response as footnotes in Emacs
Previously online references used weren't being shown
2024-06-15 13:13:19 +05:30
Debanjum Singh Solanky
385057f09e Make khoj.el chat API call async to not block user interactions 2024-06-12 21:04:48 +05:30
Debanjum Singh Solanky
906ebee075 Open Khoj chat, search in right pane to allow for ambient engagement
See the currently active window in context while doing chat, search
or find similar operations in a side pane.

This is similar to how we've moved Khoj on Obsidian into the side pane
as well
2024-06-09 23:32:34 +05:30
Debanjum Singh Solanky
cd4baa3fa5 Fix loading chat history, references in khoj.el chat buffer 2024-06-09 18:34:00 +05:30
Debanjum
6afbd8032e
Improve Intermediate Steps in Formulating Chat Response (#799)
# Major
- Disambiguate Text output mode to disambiguate from Default data source lookup
- Fix showing headings in intermediate step in generating chat response
- Remove "Path" prefix from org ancestor heading in compiled entry

# Minor
- Fix OpenAI chat actor, director unit tests
2024-06-09 07:55:01 +05:30
Debanjum Singh Solanky
f440ddbe1d Fix openai chat actor, director tests
- Update test ChatModelOptions setup since update to it's schema
- Fix stale function calls using their updated signatures
2024-06-09 07:24:47 +05:30
sabaimran
2e209ab28b Handle case where conversation does not (yet) exist 2024-06-08 16:22:12 +05:30
sabaimran
849c38c0a4 Add support for managing audiences for new users 2024-06-08 15:51:17 +05:30
sabaimran
06a47ee457
Add language-specific syntax highlighting via highlight.js (#802)
* Add language-specific syntax highlighting via highlight.js

- Add highlight.js to our assets CDN for fast load and compliance with the CSP
- See other stylesheets options here: https://cdnjs.com/libraries/highlight.js

* Bonus: set min-height to prevent increasing length of the sessions pane

* Fix references rendering and add highlight.js in public conversation
2024-06-08 15:17:09 +05:30
Debanjum Singh Solanky
5f2442450c Update truncation test to reduce flakyness in cloud tests
Removed dependency on faker, factory for the truncation tests as that
seems to be the point of flakiness
2024-06-07 19:42:48 +05:30
sabaimran
dbb06466bf Minor fit/finish updates to the file filter experience 2024-06-07 15:05:00 +05:30
sabaimran
58a02f06ea
Fix multilingual font rendering (#797)
* Fix multilingual font rendering; fallback to an Arabic language font which contains more Asian characters. Close #756

* Tune font-sizes and styling to accomodate new fonts with old sizing

- Move connection-status styling out from inline html into css block
- Remove start typing chat-input height jitter
- align new-conversation button, text
- use relative font sizes instead of absolute font sizes in most places

---------

Co-authored-by: Debanjum Singh Solanky <debanjum@gmail.com>
2024-06-07 11:53:47 +05:30
Raghav Tirumale
ba16afd3c2
New Feature: Adding File Filtering to Conversations (#788)
* UI update for file filtered conversations
* Interactive file menu #UI to add/remove files on each conversation as references.
* Backend changes implemented to load selected file filters from a conversation into the querying process.
---------

Co-authored-by: sabaimran <narmiabas@gmail.com>
2024-06-07 10:53:37 +05:30
Debanjum Singh Solanky
f91cdf8e18 Fix showing headings in intermediate step in generating chat response 2024-06-06 16:52:23 +05:30
Debanjum Singh Solanky
18f7e6e7ed Remove "Path" prefix from org ancestor heading in compiled entry 2024-06-06 16:51:26 +05:30
sabaimran
8d701ebe22
Add fedCM to accommodate google migration (#798)
- See migration guidelines here: https://developers.google.com/identity/gsi/web/guides/fedcm-migration#fedcm_flag
2024-06-06 14:23:16 +05:30
Debanjum Singh Solanky
dd2225b1aa Use Text output mode to disambiguate from Default data source lookup
Previously if default output was selected by Khoj, we'd end up doing
an documents search as well, even when Khoj selected internet or
general data source to lookup.

This update disambiguates the default information mode from the text
output mode. To avoid doing documents search when not deemed necessary
by Khoj
2024-06-06 11:56:48 +05:30
Debanjum Singh Solanky
a1e4f4bde7 Gracefully skip indexing when empty list of docs provided
Improve error message when fail to index content
2024-06-05 19:39:15 +05:30
Debanjum Singh Solanky
21987f60c7 Use `-difference' to get files to delete. Make batch size defcustom
Improve docstrings to align with `checkdoc' requirement for all args
being mentioned
2024-06-05 19:39:15 +05:30
Debanjum
bfacd65971
Batch upload files for indexing from the Emacs client (#735) from yuzhou721/master
Encode filenames and batch file uploads to improve sending content to index from the Emacs client
2024-06-05 19:31:06 +05:30
sabaimran
a9c383e62c Use an ASGI application, rather than WSGI
- ASGI should be the preferred application, as our codebase runs a lot of async code
2024-06-05 09:25:08 +05:30
sabaimran
0816cec4bc Manually close old db connections periodically 2024-06-04 22:19:47 +05:30
sabaimran
acfdc8da77 Explicitly set the connection age to 0 in the django settings. Seems to be some strange behavior with async gunicorn + django db 2024-06-04 20:31:51 +05:30
Debanjum Singh Solanky
85a343363b Release Khoj version 1.13.0 2024-06-04 11:57:44 +05:30
Debanjum
1dfd6d7391
Merge pull request from GHSA-h2q2-vch3-72qm
Add CSP and sanitize chat messages in Obsidian, Desktop, Web apps
2024-06-04 11:29:21 +05:30
Debanjum Singh Solanky
b757ba664f Sanitize chat messages to render in Obsidian, Desktop, Web apps
Use DOMPurify to escape any unsafe HTML in chat message before adding
it to DOM via innerHTML updates to a HTML element
2024-06-04 10:53:30 +05:30
Debanjum Singh Solanky
9f80c2ab76 Enforce Content-Security-Policy (CSP) in Obsidian, Desktop, Web apps
Prevent XSS attacks by enforcing Content-Security-Policy (CSP) in apps.
Do not allow loading images, other assets from untrusted domains.

- Only allow loading assets from trusted domains
  like 'self', khoj.dev, ipapi for geolocation, google (fonts, img)
  - images from khoj domain, google (for profile pic)
  - assets from khoj domain
  - Do not allow iframe src
  - Allow unsafe-inline script and styles for now as markdown-it escapes html
    in user, khoj chat

- Add hostURL to CSP of the Desktop, Obsidian apps
  Given web client is served by khoj server, it doesn't need to
  explicitly allow for khoj.dev domain. So if user self-hosting, it'll
  automatically allow the domain in the CSP (via 'self')

  Whereas the Obsidian, Desktop clients allow configure the server URL.
  Note *switching server URL breaks CSP until app is reloaded*
2024-06-04 10:53:30 +05:30
Debanjum Singh Solanky
179c70dba8 Upgrade Khoj llama-cpp, django and jinja dependencies 2024-06-04 09:05:53 +05:30
Debanjum Singh Solanky
bbcdb8413d Add null checks, fix build errors in Khoj plugin on newer Obsidian 2024-06-03 18:03:11 +05:30
Debanjum Singh Solanky
d8ace4d34c Highlight the agents, automation tab when active on the web app 2024-06-03 16:57:03 +05:30
sabaimran
4679f07336 Clean up some of the design of agents, inspired by dicussion #792 2024-06-03 12:52:07 +05:30
Debanjum Singh Solanky
8cdab5f31a Update slash command UX in chat UI of desktop app to match web app
Make commands in popup menu on typing slash in chat input selectable
2024-06-02 17:27:37 +05:30
Debanjum Singh Solanky
7828bd6f2e Hide command popup & focus on chatInput on selecting command in web app
Style command popup cursor and add highlight to indicate using slash
command
2024-06-02 17:27:37 +05:30
Debanjum
cf8c9c2a3d
Serve image assets from Khoj domain, not directly from S3 bucket (#734)
- Serve generated images from Khoj domain instead of directly from AWS S3
- Rename assets URL from Khoj S3 bucket to assets.khoj.dev
2024-06-02 17:24:35 +05:30
sabaimran
5bb3689562 Do not stream responses in the scheduled_chat response 2024-06-02 11:31:15 +05:30
sabaimran
5132b01ab1 Remove intent_type from telemetry update in api_chat 2024-06-02 10:21:38 +05:30
Raghav Tirumale
a3934b3aaa
Improved Command Menu and Help Command (#774)
* The command menu (triggered by "/") now has a clickable list of possible commands, that automatically fill into the chat when pressed.
* The `/help` command now searches `khoj.dev` pages to provide useful assistance to the user.

---------

Co-authored-by: raghavt3 <raghavt3@illinois.edu>
Co-authored-by: sabaimran <65192171+sabaimran@users.noreply.github.com>
2024-06-01 22:33:31 +05:30
sabaimran
6d10f98498 Add additional lines for KHOJ_NO_HTTPS and KHOJ_DOMAIN in the docker-compose 2024-06-01 21:48:43 +05:30
sabaimran
841cbff249 Add documentation for setting up google auth in self-hosted khoj. Closes #771 2024-06-01 21:38:21 +05:30
sabaimran
89178bcebd Fix formatting issues for task email in mobile 2024-06-01 14:19:12 +05:30
Debanjum
b499b3fe2a
Upgrade Khoj Obsidian: Chat from Side Pane, Stream Intermediate Steps, Copy Message to Clipboard (#736)
### Details
- **Chat with Khoj from right pane on Obsidian**
  - Modal was too ephemeral, couldn't have it open for reference, quick jump to Khoj chat
- **Stream intermediate steps taken by Khoj** for generating response to the chat pane
  Gives more transparency into Khoj 'thinking' process, e.g internet, notes searches performed, documents read etc. 
  The feedback allows us to tune our messages to elicit better responses by Khoj
- Add ability to **copy message to clipboard, paste chat messages directly into current file**
- Jump to **Search**, **Find Similar** functions from navigation bar on the Khoj Obsidian side pane
- Improve spacing, use consistent colors in chat message references and buttons

Resolves #789, #754
2024-06-01 13:29:21 +05:30
sabaimran
8b9c26c468 Remove unused method 2024-06-01 12:54:43 +05:30
sabaimran
5ec641837a
Allow automations to be shareable (#790)
* Updating the API / UI to support sharing of automations
* Allow people to see the automations even when not logged in, and add an overlay effect
* Handle unauthenticated users taking actions
* Support showing pre-filled automation details on the config automations page
* Redirect user to login if they try to add an automation while unauthenticated
2024-06-01 12:44:49 +05:30
Debanjum Singh Solanky
7d7d4cf5c3 Make new chat message text selectable in Obsidian side pane
Resolves #789
2024-06-01 11:01:39 +05:30
Debanjum Singh Solanky
7fb7f200b3 Fix rendering text in chat messages with bulleted lists
Improves #789
2024-06-01 10:51:22 +05:30