Commit graph

3914 commits

Author SHA1 Message Date
sabaimran
1af9dbb083 Switch node/yarn install steps to use more native installation patterns 2024-07-19 17:10:08 +05:30
sabaimran
6d5ca5a3e1 yarn clean cache before build 2024-07-19 16:06:38 +05:30
sabaimran
7f0d1bd414 Add verbose logs when outputing yarn install steps 2024-07-19 15:48:43 +05:30
sabaimran
7426a4f819 Prefetch related agent when retrieving the conversation for performance improvements 2024-07-19 14:43:30 +05:30
Debanjum Singh Solanky
07f36fa95a Update new web interface with update calls to /content, /model APIs 2024-07-19 12:23:22 +05:30
Debanjum Singh Solanky
f03525f431 Add back /api/configure as /api/settings API endpoint
It had been removed during the /api/configure/content to /api/content
API migration before
2024-07-19 05:40:34 +05:30
Debanjum Singh Solanky
3832ef0236 Move API endpoints under /api/configure/phone/ to /api/phone/
Pull out /api/configure/phone API endpoints into /api/phone for
more concise and sufficiently explanatory API path

Refactor Flow
1. Rename /api/configure/phone -> /api/phone
2024-07-19 05:40:34 +05:30
Debanjum Singh Solanky
1197266912 Move API endpoints under /configure/<type>/model to /api/model/<type>
Now the API to configure all the AI models is under /api/models.
This provides better organization and API hierarchy. The /configure
url segment was redundant.

- Rename POST /api/phone to PATCH /api/phone
- Rename GET /api/configure to GET /api/settings

Refactor Flow
1. Move out POST /user/name to main api.py
2. Rename /api/configure/<type>/model -> /api/model/<type>
3. Rename @api_configure to @api_mode
4. Rename file api_config.py to api_model.py
2024-07-19 05:40:34 +05:30
Debanjum Singh Solanky
469a1cb6a2 Move API endpoints under /api/configure/content/ to /api/content/
Pull out /api/configure/content API endpoints into /api/content to
allow for more logical organization of API path hierarchy

This should make the url more succinct and API request intent more
understandable by using existing HTTP method semantics along with the
path.

The /configure URL path segment was either
- redundant (e.g POST /configure/notion) or
- incorrect (e.g GET /configure/files)

Some example of naming improvements:
- GET /configure/types -> GET /content/types
- GET /configure/files -> GET /content/files
- DELETE /configure/files -> DELETE /content/files

This should also align, merge better the the content indexing API
triggered via PUT, PATCH /content

Refactor Flow
1. Rename /api/configure/types -> /api/content/types
2. Rename /api/configure -> /api
3. Move /api/content to api_content from under api_config
2024-07-19 05:40:34 +05:30
Debanjum Singh Solanky
bba4e0b529 Accept file deletion requests by clients during sync
- Remove unused full_corpus boolean. The full_corpus=False code path
  wasn't being used (accept for in a test)
- The full_corpus=True code path used was ignoring file deletion
  requests sent by clients during sync. Unclear why this was done
- Added unit test to prevent regression and show file deletion by
  clients during sync not ignored now
2024-07-19 04:53:01 +05:30
Debanjum Singh Solanky
5923b6d89e Split /api/v1/index/update into /api/content PUT, PATCH API endpoints
- This utilizes PUT, PATCH HTTP method semantics to remove need for
  the "regenerate" query param and "/update" url suffix
- This should make the url more succinct and API request intent more
  understandable by using existing HTTP method semantics
2024-07-19 01:45:53 +05:30
Debanjum Singh Solanky
e9f86e320b Fix and improve offline chat actor, director tests
- Use updated references schema with compiled key
- Enable director tests that are now expected to pass and that do pass
  (with Gemma 2 at least)
2024-07-18 03:43:09 +05:30
Debanjum Singh Solanky
b0ee78586c Improve offline chat truncation to consider message separator tokens 2024-07-18 03:43:09 +05:30
Debanjum Singh Solanky
6f46e6afc6 Improve and fix chat model prompts for better, consistent context
- Add day of week to system prompt of openai, anthropic, offline chat models
- Pass more context to offline chat system prompt to
  - ask follow-up questions
  - know where to find information about khoj (itself)
- Fix output mode selection prompt. Log error if model does not select
  valid option from list of valid output modes provided
- Use consistent names for question, answers passed to
  extract_questions_offline prompt

- Log which model extracts question, what the offline chat model sees
  as context. Similar to debug log shown for openai models
2024-07-18 03:43:09 +05:30
Debanjum Singh Solanky
53eabe0c06 Support Gemma 2 for Offline Chat
- Pass system message as the first user chat message as Gemma 2
  doesn't support system messages
- Use gemma-2 chat format
- Pass chat model name to generic, extract questions chat actors
  Used to figure out chat template to use for model
  For generic chat actor argument was anyway available but not being
  passed, which is confusing
2024-07-18 03:09:38 +05:30
Debanjum Singh Solanky
65dade4838 Create API endpoints to get user content configurations
This is to be used by the new Next.js web client
2024-07-17 13:41:14 +05:30
Debanjum
2ab8fb78b1
Migrate the PyPI package to use project name: khoj (#853)
### Changes
- Deprecate [khoj-assistant](https://pypi.org/project/khoj-assistant) pypi package. Use more accurate and succinct pypi project name, [khoj](https://pypi.org/project/khoj)
- Update references to use `khoj` pypi package in docs and code
- Update pypi workflow to publish to both khoj, khoj-assistant for now
- Update stale python 3.9 support mentioned in our pyproject
   Can't support python 3.9 as depend on [Django 5.0.7](https://pypi.org/project/Django/5.0.7/) which needs python >=3.10

### Verify
- Updated `pypi.yml` github workflow publishes to both (new) [khoj](https://pypi.org/project/khoj/1.16.1.dev16/), (old) [khoj-assistant](https://pypi.org/project/khoj-assistant/1.16.1.dev16/) pypi projects
- Can install Khoj python package with `pip install khoj`
2024-07-17 01:05:51 -07:00
Debanjum
bf815e4463
Refactor Config API and Settings pages for Reuse and Consistency (#852)
### Major
- Reuse get config data logic across config pages on web client
- Make config api endpoint urls and response fields consistent
- Rename API path /api/config to /api/configure
- Move Web, Desktop client settings page to be under `/settings` from the previous `/config` url path

### Minor
- Pass isMobileWidth prop to SidePanel via chat share interface
- Turn prettier off instead of throwing error for now
- Do no explicitly add line-clamp plugin as it's in Tailwind by default
2024-07-17 01:03:06 -07:00
Debanjum Singh Solanky
a1c362a4f7 Expose web, desktop settings page under /settings, not /configure
- Update references to the settings page to use new url across docs
  and code
- Rename desktop and web settings page to settigns.html instead of
  config[ure].html
2024-07-17 13:17:29 +05:30
Debanjum Singh Solanky
b015b0e83d Arrange config API detailed response fields to improve readability
There are a lot of fields being returned. Group returned fields and
add comment header to each Group for readability
2024-07-17 13:17:28 +05:30
Debanjum Singh Solanky
71ebf31a54 Make config API detailed response fields more intuitive, consistent
- Use name, id for every [search|chat|voice|pain]_model_option
- Rename current_model_state field to more intuitive enabled_content_source
- Update references to the update fields in config.html
2024-07-17 12:41:01 +05:30
Debanjum Singh Solanky
30d60aaae9 Add, fix Khoj Docker container labels 2024-07-17 10:41:17 +05:30
Debanjum Singh Solanky
583fa3c188 Migrate the pypi package to khoj project name. Update references
- Deprecate khoj-assistant pypi package. Use more accurate and
  succinct pypi project name, khoj
- Update references to sye khoj pypi package in docs and code instead
  of the legacy khoj-assistant pypi package
- Update pypi workflow to publish to both khoj, khoj-assistant for now
- Update stale python 3.9 support mentioned in our pyproject. Can't
  support python 3.9 as depend on latest django which support >=3.10
2024-07-17 10:41:16 +05:30
Debanjum Singh Solanky
7316e6b9d3 Pass isMobileWidth prop to SidePanel via chat share interface 2024-07-16 16:13:27 +05:30
Debanjum Singh Solanky
4759c4ac96 Turn prettier off instead of throwing error for now
Until web interface code is reformatted with prettier
2024-07-16 16:13:27 +05:30
Debanjum Singh Solanky
466ef3f8f1 Do no explicitly add line-clamp plugin as it's in Tailwind by default 2024-07-16 16:13:27 +05:30
Debanjum Singh Solanky
59000a47cb Move Desktop config page to /configure from /config url path
Update references to point to page at /configure instead of /config
2024-07-16 16:13:27 +05:30
Debanjum Singh Solanky
a5c16ad600 Move Web client config page to /configure from /config url path
Update docs, clients and error messages to point to /configure
instead of /config
2024-07-16 16:13:27 +05:30
Debanjum Singh Solanky
de15a7a3fc Rename API path /api/config to /api/configure
- Update clients calling /api/config to call /api/configure instead
2024-07-16 16:13:27 +05:30
Debanjum Singh Solanky
dd31936746 Make config api endpoint urls consistent
- Consistently use /content/ for data. Remove content-source from path
- Remove unnecessary /data/ prefix for API endpoints under /config
2024-07-16 16:13:27 +05:30
Debanjum Singh Solanky
e8176b41ef Reuse get config data logic across config pages on web client
- Put logic to get config data, detailed or basic into router helpers module
- Use the get config func across the config pages on web clients

- Put configure content and get_notion_auth_url funcs in router helper
  module to avoid circular import
2024-07-16 16:13:27 +05:30
sabaimran
1a5405e24c Fix interpretation of day of week in automation form 2024-07-16 10:12:30 +05:30
sabaimran
c837f3779e
Update the agents page with new UX (#850)
- Use icons/colors for setting the styling of agents
- Update automations page to use the shadcn cards: https://github.com/shadcn-ui/ui
2024-07-16 10:10:55 +05:30
sabaimran
1c6ed9bc6d
Migrate the existing automations page to use React (#849)
Migrates the Automations page to React, mostly keeping the overall design consistent with organization. Use component library, with some changes in color. Add easier management with straightforward form and editing experience.
Use system preference for determining dark mode if not explicitly set.
2024-07-15 21:42:33 +05:30
Debanjum
c7764c7470
Fix, Improve Behavior, Styling of Chat View on Web (#851)
### Behavior
- Close chat sessions side panel on click open a chat session
- Show agent profile card with description when hover on agent in chat view
- Show action bar on last chat message without hover
- Show chat message action buttons without hover on mobile interfaces
- Show chat message timestamp on hover in chat view
- Show text descriptions of chat message action buttons on hover
- Render inline png, webp images generated by Khoj in chat view

### Fixes
- Do not render references with broken links in chat view
- Fix closing side panel on mobile when click open a chat session
- Only open side panel as drawer in mobile view
- Constrain chat messages to stay within view port across screen sizes

### Styling: Spacing, Sizing, Mobile Friendly
- Make Khoj icon appropriately sized and side panel arrow bold
- Conversations list should resize to take max space on side panel
- Make loading message, styling configurable. Do not show agent when no data
- Improve Train of Thought icons spacing and loading circle
- Improve mobile friendly styling of chat session side panel
- Improve styling of chat input, references UI across screen sizes
- Center cursor in chat input. See upto 2 lines for multi-line context

### Miscellaneous
- Add code formatter for web interface with prettier
2024-07-15 08:39:14 -07:00
Debanjum Singh Solanky
6c630bc6c3 Constrain chat messages to stay in view port across screen sizes
- Constrain chat messages max width to view port across screen sizes
- Wrap references on smaller screens, use tailwind, not js to apply styling
2024-07-15 21:00:50 +05:30
sabaimran
9a5bf4c701 Fix rendering of teaser reference panel in mobile width 2024-07-15 19:40:55 +05:30
sabaimran
2e9275c0f3 Remove side panel padding in desktop view. Fix width in mobile view 2024-07-15 19:33:12 +05:30
Debanjum Singh Solanky
ba0ba6b59f Merge branch 'features/big-upgrade-chat-ux' of github.com:khoj-ai/khoj into document-styling-on-chat-ux 2024-07-15 10:42:56 +05:30
Debanjum
23f61d49e0
Support syncing, searching images from Obsidian plugin (#847)
- Sync images from Obsidian vault with Khoj server now that Khoj can OCR images
- Support rendering images returned by Khoj search modal
2024-07-14 20:41:39 -07:00
Debanjum Singh Solanky
6f8f846086 Standardize code format for web interface with prettier
Use husky, lint-staged to run prettier pre-commit
2024-07-15 00:34:54 +05:30
sabaimran
06dce4729b
Make most major changes for an updated chat UI (#843)
- Updated references panel
- Use subtle coloring for chat cards
- Chat streaming with train of thought
- Side panel with limited sessions, expandable
- Manage conversation file filters easily from the side panel
- Updated nav menu, easily go to agents/automations/profile
- Upload data from the chat UI (on click attachment icon)
- Slash command pop-up menu, scrollable and selectable
- Dark mode-enabled
- Mostly mobile friendly
2024-07-14 23:18:06 +05:30
Debanjum Singh Solanky
6dd90931e8 Fix closing side panel on mobile when click open a chat session 2024-07-14 22:54:49 +05:30
Debanjum Singh Solanky
47b754c07b Only open side panel as drawer in mobile view 2024-07-14 14:08:41 +05:30
Debanjum Singh Solanky
b47f30ad77 Make Khoj icon appropriately sized and side panel arrow bold 2024-07-14 14:06:36 +05:30
Debanjum Singh Solanky
e6b21144e2 Conversations list should resize to take max space on side panel 2024-07-14 13:49:36 +05:30
Debanjum Singh Solanky
c2bf405489 Make loading message, styling configurable. Do not show agent when no data
- Pass Loading message, class name via props to both inline and normal
  loading spinners
- Pass loading conversation message to loading spinner when chat
  history is being fetched
2024-07-14 13:00:36 +05:30
Debanjum Singh Solanky
63719747cb Show agent profile card with description when hover on agent in chat view
- Create profile card componennt. Use it for agent profile card
- Pass agent persona from khoj server via API
- Put link to agent profile page in the hover card to make it 2 clicks
  away. Othewise inadvertent clicks on agent in chat view lead away to
  agent page
- Use tailwind line-clamp extension to clamp card to first two lines
2024-07-14 12:20:11 +05:30
Debanjum Singh Solanky
dbbd4b9777 Show action bar on last chat message without hover 2024-07-14 10:32:31 +05:30
Debanjum Singh Solanky
a0f38e079f Improve Train of Thought icons spacing and loading circle 2024-07-14 09:35:15 +05:30