diff --git a/documentation/docs/online-data-sources/_category_.json b/documentation/docs/data-sources/_category_.json similarity index 100% rename from documentation/docs/online-data-sources/_category_.json rename to documentation/docs/data-sources/_category_.json diff --git a/documentation/docs/online-data-sources/github_integration.md b/documentation/docs/data-sources/github_integration.md similarity index 100% rename from documentation/docs/online-data-sources/github_integration.md rename to documentation/docs/data-sources/github_integration.md diff --git a/documentation/docs/online-data-sources/notion_integration.md b/documentation/docs/data-sources/notion_integration.md similarity index 100% rename from documentation/docs/online-data-sources/notion_integration.md rename to documentation/docs/data-sources/notion_integration.md diff --git a/documentation/docs/data-sources/share_your_data.md b/documentation/docs/data-sources/share_your_data.md new file mode 100644 index 00000000..2299b7d0 --- /dev/null +++ b/documentation/docs/data-sources/share_your_data.md @@ -0,0 +1,16 @@ +--- +sidebar_position: 0 +keywords: ["upload data", "upload files", "share data", "share files", "pdf ai", "ai for pdf", "ai for documents", "ai for files", "local ai pdf", "local ai documents", "local ai files"] + +--- + +# Upload your data + +There are several ways you can get started with sharing your data with the Khoj AI. + +- Drag and drop your documents via [the web UI](/clients/web/#upload-documents). This is best if you have a one-off document you need to interact with. +- Use the desktop app to [upload and sync your documents](/clients/desktop). This is best if you have a lot of documents on your computer or you need the docs to stay in sync. +- Setup the sync options for either [Obsidian](/clients/obsidian) or [Emacs](/clients/emacs) to automatically sync your documents with Khoj. This is best if you are already using these tools and want to leverage Khoj's AI capabilities. +- Configure your [Notion](/data-sources/notion_integration) or [Github](/data-sources/github_integration) to sync with Khoj. By providing your credentials, you can keep the data synced in the background. + +![demo of dragging and dropping a file](https://khoj-web-bucket.s3.amazonaws.com/drag_drop_file.gif) diff --git a/documentation/docs/features/all_features.md b/documentation/docs/features/all_features.md index 3d3b8941..28c49f9e 100644 --- a/documentation/docs/features/all_features.md +++ b/documentation/docs/features/all_features.md @@ -29,6 +29,6 @@ Khoj is available as a [Desktop app](/clients/desktop), [Emacs package](/clients ![](/img/khoj_clients.svg ':size=400px') ### Supported Data Sources -Khoj can understand your org-mode, markdown, PDF, plaintext files, [Github projects](/online-data-sources/github_integration) and [Notion pages](/online-data-sources/notion_integration). +Khoj can understand your org-mode, markdown, PDF, plaintext files, [Github projects](/data-sources/github_integration) and [Notion pages](/data-sources/notion_integration). ![](/img/khoj_datasources.svg ':size=200px') diff --git a/documentation/docs/get-started/overview.md b/documentation/docs/get-started/overview.md index b0d2a51c..57f93804 100644 --- a/documentation/docs/get-started/overview.md +++ b/documentation/docs/get-started/overview.md @@ -1,6 +1,7 @@ --- sidebar_position: 0 slug: / +keywords: ["khoj", "khoj ai", "khoj docs", "khoj documentation", "khoj features", "khoj overview", "khoj quickstart", "khoj chat", "khoj search", "khoj cloud", "khoj self-host", "khoj setup", "open source ai", "local llm", "ai copilot", "second brain ai", "ai search engine"] --- # Overview @@ -28,7 +29,7 @@ Welcome to the Khoj Docs! This is the best place to get setup and explore Khoj's - Khoj is an open source, personal AI - You can [chat](/features/chat) with it about anything. It'll use files you shared with it to respond, when relevant - Quickly [find](/features/search) relevant notes and documents using natural language -- It understands pdf, plaintext, markdown, org-mode files, [notion pages](/online-data-sources/notion_integration) and [github repositories](/online-data-sources/github_integration) +- It understands pdf, plaintext, markdown, org-mode files, [notion pages](/data-sources/notion_integration) and [github repositories](/data-sources/github_integration) - Access it from your [Emacs](/clients/emacs), [Obsidian](/clients/obsidian), [Web browser](/clients/web) or the [Khoj Desktop app](/clients/desktop) - Use [cloud](https://app.khoj.dev/login) to access your Khoj anytime from anywhere, [self-host](/get-started/setup) on consumer hardware for privacy diff --git a/documentation/docs/get-started/setup.mdx b/documentation/docs/get-started/setup.mdx index 36aefd08..b9f32048 100644 --- a/documentation/docs/get-started/setup.mdx +++ b/documentation/docs/get-started/setup.mdx @@ -199,15 +199,24 @@ To disable HTTPS, set the `KHOJ_NO_HTTPS` environment variable to `True`. This c ### 2. Configure 1. Go to http://localhost:42110/server/admin and login with your admin credentials. - 1. Go to [OpenAI settings](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/) in the server admin settings to add an OpenAI processor conversation config. This is where you set your API key. Alternatively, you can go to the [offline chat settings](http://localhost:42110/server/admin/database/offlinechatprocessorconversationconfig/) and simply create a new setting with `Enabled` set to `True`. - 2. Go to the ChatModelOptions if you want to add additional models for chat. - - Set the `chat-model` field to a supported chat model[^1] of your choice. For example, you can specify `gpt-4-turbo-preview` if you're using OpenAI or `NousResearch/Hermes-2-Pro-Mistral-7B-GGUF` if you're using offline chat. - - Make sure to set the `model-type` field to `OpenAI` or `Offline` respectively. - - The `tokenizer` and `max-prompt-size` fields are optional. Set them only when using a non-standard model (i.e not mistral, gpt or llama2 model). +#### Configure Chat Model +##### Configure OpenAI or a custom OpenAI-compatible proxy server +1. Go to the [OpenAI settings](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/) in the server admin settings to add an OpenAI processor conversation config. This is where you set your API key and server API base URL. The API base URL is optional - it's only relevant if you're using another OpenAI-compatible proxy server. +2. Go over to configure your [chat model options](http://localhost:42110/server/admin/database/chatmodeloptions/). Set the `chat-model` field to a supported chat model[^1] of your choice. For example, you can specify `gpt-4-turbo-preview` if you're using OpenAI. + - Make sure to set the `model-type` field to `OpenAI`. + - The `tokenizer` and `max-prompt-size` fields are optional. Set them only if you're sure of the tokenizer or token limit for the model you're using. Contact us if you're unsure what to do here. + +##### Configure Offline Chat +1. No need to setup a conversation processor config! +2. Go over to configure your [chat model options](http://localhost:42110/server/admin/database/chatmodeloptions/). Set the `chat-model` field to a supported chat model[^1] of your choice. For example, we recommend `NousResearch/Hermes-2-Pro-Mistral-7B-GGUF`, but [any gguf model on huggingface](https://huggingface.co/models?library=gguf) should work. + - Make sure to set the `model-type` to `Offline`. Do not set `openai config`. + - The `tokenizer` and `max-prompt-size` fields are optional. Set them only when using a non-standard model (i.e not mistral, gpt or llama2 model) when you know the token limit. + +#### Share your data 1. Select files and folders to index [using the desktop client](/get-started/setup#2-download-the-desktop-client). When you click 'Save', the files will be sent to your server for indexing. - Select Notion workspaces and Github repositories to index using the web interface. -[^1]: Khoj, by default, can use [OpenAI GPT3.5+ chat models](https://platform.openai.com/docs/models/overview) or [GGUF chat models](https://huggingface.co/models?library=gguf). See [this section](/miscellaneous/advanced#use-openai-compatible-llm-api-server-self-hosting) to use non-standard chat models +[^1]: Khoj, by default, can use [OpenAI GPT3.5+ chat models](https://platform.openai.com/docs/models/overview) or [GGUF chat models](https://huggingface.co/models?library=gguf). See [this section](/miscellaneous/advanced#use-openai-compatible-llm-api-server-self-hosting) on how to locally use OpenAI-format compatible proxy servers. :::tip[Note] Using Safari on Mac? You might not be able to login to the admin panel. Try using Chrome or Firefox instead.