Update the documentation with swanky new demo videos

documentation/docs/clients/web.md

@@ -8,6 +8,10 @@ sidebar_position: 4
 
 Without any desktop clients, you can start chatting with Khoj on the web. Bear in mind you do need one of the desktop clients in order to share and sync your data with Khoj.
 
+Go see it here --> [Khoj Cloud](https://app.khoj.dev).
+
+![](/img/khoj_web_app_home.png)
+
 ## Features
 - **Chat**
   - **Faster answers**: Find answers quickly, from your private notes or the public internet
@@ -25,7 +29,7 @@ You can upload documents to Khoj from the web interface, one at a time. This is
 1. You can drag and drop the document into the chat window.
 2. Or click the paperclip icon in the chat window and select the document from your file system.
 
-![demo of dragging and dropping a file](https://assets.khoj.dev/drag_drop_file.gif)
+![demo of dragging and dropping a file](https://assets.khoj.dev/home_page_data_upload.gif)
 
 ### Install on Phone
 You can optionally install Khoj as a [Progressive Web App (PWA)](https://web.dev/learn/pwa/installation). This makes it quick and easy to access Khoj on your phone.
@@ -37,9 +41,3 @@ You can optionally install Khoj as a [Progressive Web App (PWA)](https://web.dev
 | Step 1 | Step 2 | Step 3|
 |:---:|:---:|:---:|
 | ![](/img/pwa_install_1.png) | ![](/img/pwa_install_2.png) | ![](/img/pwa_install_3.png) |
-
-## Interface
-
-| Search | Chat |
-|:------:|:----:|
-| ![](/img/khoj_search_on_web.png) | ![](/img/khoj_chat_on_web.png) |

documentation/docs/contributing/development.mdx

@@ -224,7 +224,7 @@ The core code for the Obsidian plugin is under `src/interface/obsidian`. The fil
 4. Open the `khoj` folder in the file explorer that opens. You'll see a file called `main.js` in this folder. To test your changes, replace this file with the `main.js` file that was generated by the development server in the previous section.
 
 ## Create Khoj Release (Only for Maintainers)
-Follow the steps below to [release](https://github.com/debanjum/khoj/releases/) Khoj. This will create a stable release of Khoj on [Pypi](https://pypi.org/project/khoj/), [Melpa](https://stable.melpa.org/#%252Fkhoj) and [Obsidian](https://obsidian.md/plugins?id%253Dkhoj). It will also create desktop apps of Khoj and attach them to the latest release.
+Follow the steps below to [release](https://github.com/khoj-ai/khoj/releases/) Khoj. This will create a stable release of Khoj on [Pypi](https://pypi.org/project/khoj/), [Melpa](https://stable.melpa.org/#%252Fkhoj) and [Obsidian](https://obsidian.md/plugins?id%253Dkhoj). It will also create desktop apps of Khoj and attach them to the latest release.
 
 1. Create and tag release commit by running the bump_version script. The release commit sets version number in required metadata files.
   ```shell
@@ -243,7 +243,7 @@ Follow the steps below to [release](https://github.com/debanjum/khoj/releases/)
 
 ## Visualize Codebase
 
-*[Interactive Visualization](https://mango-dune-07a8b7110.1.azurestaticapps.net/?repo=debanjum%2Fkhoj)*
+*[Interactive Visualization](https://mango-dune-07a8b7110.1.azurestaticapps.net/?repo=khoj-ai%2Fkhoj)*
 
 ![](/img/khoj_codebase_visualization_0.2.1.png)
 

documentation/docs/data-sources/share_your_data.md

@@ -13,4 +13,4 @@ There are several ways you can get started with sharing your data with the Khoj
 - Setup the sync options for either [Obsidian](/clients/obsidian) or [Emacs](/clients/emacs) to automatically sync your documents with Khoj. This is best if you are already using these tools and want to leverage Khoj's AI capabilities.
 - Configure your [Notion](/data-sources/notion_integration) or [Github](/data-sources/github_integration) to sync with Khoj. By providing your credentials, you can keep the data synced in the background.
 
-![demo of dragging and dropping a file](https://assets.khoj.dev/drag_drop_file.gif)
+![demo of dragging and dropping a file](https://assets.khoj.dev/upload_pdf_doc.gif)

documentation/docs/features/agents.md

@@ -6,7 +6,7 @@ sidebar_position: 4
 
 You can use agents to setup custom system prompts with Khoj. The server host can setup their own agents, which are accessible to all users. You can see ours at https://app.khoj.dev/agents.
 
-![Demo](https://assets.khoj.dev/agents_demo.gif)
+![Demo](/img/agents_page_full.png)
 
 ## Creating an Agent (Self-Hosted)
 

documentation/docs/features/chat.md

@@ -6,7 +6,7 @@ sidebar_position: 2
 
 You can configure Khoj to chat with you about anything. When relevant, it'll use any notes or documents you shared with it to respond. It acts as an excellent research assistant, search engine, or personal tutor.
 
-<img src="/img/khoj_chat_on_web.png" alt="Chat on Web" style={{width: '400px'}}/>
+<img src="https://assets.khoj.dev/vision_chat_example.png" alt="Chat on Web" style={{width: '400px'}}/>
 
 ### Overview
 - Creates a personal assistant for you to inquire and engage with your notes or online information as needed
@@ -15,33 +15,7 @@ You can configure Khoj to chat with you about anything. When relevant, it'll use
 - Shows reference notes used to generate a response
 
 ### Setup (Self-Hosting)
-#### Offline Chat
+See [the setup guide](/get-started/setup.mdx) to configure your chat models.
-Offline chat stays completely private and can work without internet using open-source models.
-
-> **System Requirements**:
->  - Minimum 8 GB RAM. Recommend **16Gb VRAM**
->  - Minimum **5 GB of Disk** available
->  - A CPU supporting [AVX or AVX2 instructions](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions) is required
->  - An Nvidia, AMD GPU or a Mac M1+ machine would significantly speed up chat response times
-
-1. Open your [Khoj offline settings](http://localhost:42110/server/admin/database/offlinechatprocessorconversationconfig/) and click *Enable* on the Offline Chat configuration.
-2. Open your [Chat model options settings](http://localhost:42110/server/admin/database/chatmodeloptions/) and add any [GGUF chat model](https://huggingface.co/models?library=gguf) to use for offline chat. Make sure to use `Offline` as its type. For a balanced chat model that runs well on standard consumer hardware we recommend using [Llama 3.1 by Meta](https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF) by default. For machines with no or small GPU we recommend using [Gemma 2 2B](https://huggingface.co/bartowski/gemma-2-2b-it-GGUF) or [Phi 3.5 mini](https://huggingface.co/bartowski/Phi-3.5-mini-instruct-GGUF)
-
-
-:::tip[Note]
-Offline chat is not supported for a multi-user scenario. The host machine will encounter segmentation faults if multiple users try to use offline chat at the same time.
-:::
-
-#### Online Chat
-Online chat requires internet to use ChatGPT but is faster, higher quality and less compute intensive.
-
-:::danger[Warning]
-This will enable Khoj to send your chat queries and query relevant notes to OpenAI for processing.
-:::
-
-1. Get your [OpenAI API Key](https://platform.openai.com/account/api-keys)
-2. Open your [Khoj Online Chat settings](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/). Add a new setting with your OpenAI API key, and click *Save*. Only one configuration will be used, so make sure that's the only one you have.
-3. Open your [Chat model options](http://localhost:42110/server/admin/database/chatmodeloptions/) and add a new option for the OpenAI chat model you want to use. Make sure to use `OpenAI` as its type.
 
 ### Use
 1. Open Khoj Chat
@@ -52,13 +26,13 @@ This will enable Khoj to send your chat queries and query relevant notes to Open
 
 
 #### Details
-1. Your query is used to retrieve the most relevant notes, if any, using Khoj search
+1. Your query is used to retrieve the most relevant notes, if any, using Khoj search using RAG.
 2. These notes, the last few messages and associated metadata is passed to the enabled chat model along with your query to generate a response
 
 #### Conversation File Filters
 You can use conversation file filters to limit the notes used in the chat response. To do so, use the left panel in the web UI. Alternatively, you can also use [query filters](/miscellaneous/advanced#query-filters) to limit the notes used in the chat response.
 
-<img src="/img/select_file_filter.png" alt="Conversation File Filter" style={{width: '400px'}}/>
+<img src="/img/file_filters_conversation.png" alt="Conversation File Filter" style={{width: '400px'}}/>
 
 #### Commands
 Slash commands allows you to change what Khoj uses to respond to your query

documentation/docs/features/keyboard_shortcuts.md

@@ -2,7 +2,7 @@
 
 Oftentimes, having to leave your keyboard to move the mouse can break your flow. We want to make it as easy as possible for AI to flow with you as you are, so we've added some keyboard shortcuts to facilitate that.
 
-## Web App
+## Obsidian
 
 ### Up/Down Arrow Keys (Chat Input)
 

documentation/docs/features/search.md

@@ -14,4 +14,4 @@ Take advantage of super fast search to find relevant notes and documents from yo
 2. Query using natural language to find relevant entries from your knowledge base. Use [query filters](/miscellaneous/advanced#query-filters) to limit entries to search
 
 ### Demo
-![](/img/khoj_search_on_web.png ':size=400px')
+![](/img/search_agents_markdown.png ':size=400px')

documentation/docs/features/share.md

@@ -4,4 +4,4 @@ You can share any of your conversations by going to the three dot menu on the co
 
 This means you can easily share a conversation with someone to show them how you solved a problem, or to get help with something you're working on.
 
-![demo of sharing a conversation](https://assets.khoj.dev/shareable_conversations.gif)
+![demo of sharing a conversation](https://assets.khoj.dev/share_side_panel_conversation.gif)

documentation/docs/features/voice-chat.md

@@ -2,16 +2,16 @@
 
 You can talk to Khoj using your voice. Khoj will respond to your queries using the same models as the chat feature. You can use voice chat on the web, Desktop, and Obsidian apps.
 
-![Voice Chat](/img/mic_chat_icon.png)
+![Voice Chat](https://assets.khoj.dev/speech_to_text_demo.gif)
 
-Click on the little mic icon to send your voice message to Khoj. It will send back what it heard via text. You'll have some time to edit it before sending it, if required. Try it at https://app.khoj.dev/.
+Click on the little mic icon to send your voice message to Khoj. It will send back what it heard via text. You can edit the message before sending it, if required. Try it at https://app.khoj.dev/.
 
 ## Voice Response
 
 If you send a voice message, Khoj will automatically respond back with a voice message.
 You can also click on the speaker icon next to any message to hear it out loud. The voice response feature is available only on the web view right now.
 
-![Speaker Icon](/img/speaker_icon.png)
+![Speaker Icon](/img/text_to_speech.png)
 
 ## Setup (Self-Hosting)
 

documentation/docs/get-started/overview.md

@@ -31,11 +31,10 @@ Welcome to the Khoj Docs! This is the best place to get setup and explore Khoj's
 - Quickly [find](/features/search) relevant notes and documents using natural language
 - It understands pdf, plaintext, markdown, org-mode files, [notion pages](/data-sources/notion_integration) and [github repositories](/data-sources/github_integration)
 - Access it from your [Emacs](/clients/emacs), [Obsidian](/clients/obsidian), the [Khoj desktop app](/clients/desktop), or [any web browser](/clients/web)
-- Use [cloud](https://app.khoj.dev/login) to access your Khoj anytime from anywhere, [self-host](/get-started/setup) on consumer hardware for privacy
+- Use our [cloud](https://app.khoj.dev/login) instance to access your Khoj anytime from anywhere, [self-host](/get-started/setup) on consumer hardware for privacy
+
+![demo_chat](https://assets.khoj.dev/quadratic_equation_khoj_web.gif)
 
 ## Quickstart
 - [Try Khoj Cloud](https://app.khoj.dev) to get started quickly
 - [Read these instructions](/get-started/setup) to self-host a private instance of Khoj
-
-## At a Glance
-![demo_chat](https://assets.khoj.dev/using_khoj_for_studying.gif)

documentation/docs/get-started/setup.mdx

@@ -206,11 +206,32 @@ Using Ollama? See the [Ollama Integration](/advanced/ollama) section for more cu
 :::
 
 1. Go to the [OpenAI settings](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/) in the server admin settings to add an OpenAI processor conversation config. This is where you set your API key and server API base URL. The API base URL is optional - it's only relevant if you're using another OpenAI-compatible proxy server.
+
+![example configuration for openai processor](/img/example_openai_processor_config.png)
+
 2. Go over to configure your [chat model options](http://localhost:42110/server/admin/database/chatmodeloptions/). Set the `chat-model` field to a supported chat model[^1] of your choice. For example, you can specify `gpt-4o` if you're using OpenAI.
     - Make sure to set the `model-type` field to `OpenAI`.
     - The `tokenizer` and `max-prompt-size` fields are optional. Set them only if you're sure of the tokenizer or token limit for the model you're using. Contact us if you're unsure what to do here.
+    - If your model supports vision, set the `vision enabled` field to `true`. This is currently only supported for OpenAI models with vision capabilities.
+
+![example configuration for chat model options](/img/example_chatmodel_option.png)
+
+##### Configure Anthropic Chat
+1. Go to the [OpenAI settings](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/) in the server admin settings to add an OpenAI processor conversation config. This is kind of a misnomer, we know. Do not configure the API base url. Just add your API key and give the configuration a friendly name.
+2. Go over to configure your [chat model options](http://localhost:42110/server/admin/database/chatmodeloptions/). Set the `chat-model` field to a supported chat model by Anthropic of your choice. For example, you can specify `claude-3-5-sonnet-20240620`.
+    - Make sure to set the `model-type` field to `Anthropic`.
+    - The `tokenizer` and `max-prompt-size` fields are optional. Set them only if you're sure of the tokenizer or token limit for the model you're using. Contact us if you're unsure what to do here.
 
 ##### Configure Offline Chat
+
+Offline chat stays completely private and can work without internet using open-source models.
+
+**System Requirements**:
+- Minimum 8 GB RAM. Recommend **16Gb VRAM**
+- Minimum **5 GB of Disk** available
+- A CPU supporting [AVX or AVX2 instructions](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions) is required
+- An Nvidia, AMD GPU or a Mac M1+ machine would significantly speed up chat response times
+
 Any chat model on Huggingface in GGUF format can be used for local chat. Here's how you can set it up:
 
 1. No need to setup a conversation processor config!