sij/khoj
1
0
Fork 0
mirror of https://github.com/khoj-ai/khoj.git synced 2024-12-18 02:27:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 23
khoj/documentation/docs/get-started/setup.mdx
Debanjum Singh Solanky 591c582eeb Fix list references in use openai proxy docs
2024-09-30 23:21:33 -07:00

446 lines
21 KiB
Text
Raw Blame History

---
sidebar_position: 1
---
# Self-Host
Learn about how to self-host Khoj on your own machine.
Benefits to self-hosting:
1. **Privacy**: Your data will never have to leave your private network. You can even use Khoj without an internet connection if deployed on your personal computer.
2. **Customization**: You can customize Khoj to your liking, from models, to host URL, to feature enablement.
```mdx-code-block
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
```
## Setup Khoj
These are the general setup instructions for self-hosted Khoj.
You can install the Khoj server using either [Docker](?server=docker) or [Pip](?server=pip).
:::info[Offline Model + GPU]
If you want to use the offline chat model and you have a GPU, you should use Installation Option 2 - local setup via the Python package directly. Our Docker image doesn't currently support running the offline chat model on GPU, making inference times really slow.
:::
<Tabs groupId="server" queryString>
<TabItem value="docker" label="Docker">
<Tabs groupId="operating-systems" queryString="os">
<TabItem value="macos" label="MacOS">
<h3>Prerequisites</h3>
Install [Docker Desktop](https://docs.docker.com/desktop/install/mac-install/)
</TabItem>
<TabItem value="windows" label="Windows">
<h3>Prerequisites</h3>
1. Install [WSL2](https://learn.microsoft.com/en-us/windows/wsl/install) and restart your machine
```shell
# Run in PowerShell
wsl --install
```
2. Install [Docker Desktop](https://docs.docker.com/desktop/install/windows-install/) with **[WSL2 backend](https://docs.docker.com/desktop/wsl/#turn-on-docker-desktop-wsl-2)** (default)
</TabItem>
<TabItem value="linux" label="Linux">
<h3>Prerequisites</h3>
Install [Docker Desktop](https://docs.docker.com/desktop/install/windows-install/).
You can also use your package manager to install Docker Engine & Docker Compose.
</TabItem>
</Tabs>
<h3>Setup</h3>
1. Download the Khoj docker-compose.yml file [from Github](https://github.com/khoj-ai/khoj/blob/master/docker-compose.yml)
```shell
# Windows users should use their WSL2 terminal to run these commands
mkdir ~/.khoj && cd ~/.khoj
wget https://raw.githubusercontent.com/khoj-ai/khoj/master/docker-compose.yml
```
2. Configure the environment variables in the docker-compose.yml
- Set `KHOJ_ADMIN_PASSWORD`, `KHOJ_DJANGO_SECRET_KEY` (and optionally the `KHOJ_ADMIN_EMAIL`) to something secure. This allows you to customize Khoj later via the admin panel.
- Set `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, or `GEMINI_API_KEY` to your API key if you want to use OpenAI, Anthropic or Gemini chat models respectively.
3. Start Khoj by running the following command in the same directory as your docker-compose.yml file.
```shell
# Windows users should use their WSL2 terminal to run these commands
cd ~/.khoj
docker-compose up
```
:::info[Remote Access]
By default Khoj is only accessible on the machine it is running. To access Khoj from a remote machine see [Remote Access Docs](/advanced/remote).
:::
Your setup is complete once you see `🌖 Khoj is ready to use` in the server logs on your terminal.
</TabItem>
<TabItem value="pip" label="Pip">
<h3>1. Install Postgres (with PgVector)</h3>
Khoj uses Postgres DB for all server configuration and to scale to multi-user setups. It uses the pgvector package in Postgres to manage your document embeddings. Both Postgres and pgvector need to be installed for Khoj to work.
<Tabs groupId="operating-systems" queryString="os">
<TabItem value="macos" label="MacOS">
Install [Postgres.app](https://postgresapp.com/). This comes pre-installed with `pgvector` and relevant dependencies.
</TabItem>
<TabItem value="windows" label="Windows">
For detailed instructions and troubleshooting, see [this section](/contributing/development#2-postgres-installation--setup).
1. Use the [recommended installer](https://www.postgresql.org/download/windows/).
2. Follow instructions to [Install PgVector](https://github.com/pgvector/pgvector#windows) in case you need to manually install it. Windows support is experimental for pgvector currently, so we recommend using Docker.
</TabItem>
<TabItem value="linux" label="Linux">
From [official instructions](https://wiki.postgresql.org/wiki/Apt)
</TabItem>
<TabItem value="source" label="From Source">
1. Follow instructions to [Install Postgres](https://www.postgresql.org/download/)
2. Follow instructions to [Install PgVector](https://github.com/pgvector/pgvector#installation) in case you need to manually install it.
</TabItem>
</Tabs>
<h3>2. Create the Khoj database</h3>
<Tabs groupId="operating-systems" queryString="os">
<TabItem value="macos" label="MacOS">
```shell
createdb khoj -U postgres --password
```
</TabItem>
<TabItem value="windows" label="Windows">
```shell
createdb -U postgres khoj --password
```
</TabItem>
<TabItem value="linux" label="Linux">
```shell
sudo -u postgres createdb khoj --password
```
</TabItem>
</Tabs>
:::info[Postgres Env Config]
Make sure to update the `POSTGRES_HOST`, `POSTGRES_PORT`, `POSTGRES_USER`, `POSTGRES_DB` or `POSTGRES_PASSWORD` environment variables to match any customizations in your Postgres configuration.
:::
<h3>3. Install Khoj Server</h3>
- Make sure [python](https://realpython.com/installing-python/) and [pip](https://pip.pypa.io/en/stable/installation/) are installed on your machine
- Check [llama-cpp-python setup](https://python.langchain.com/docs/integrations/llms/llamacpp#installation) if you hit any llama-cpp issues with the installation
Run the following command in your terminal to install the Khoj server.
<Tabs groupId="operating-systems" queryString="os">
<TabItem value="macos" label="MacOS">
<Tabs groupId="gpu" queryString="gpu">
<TabItem value="arm" label="ARM/M1+">
```shell
CMAKE_ARGS="-DGGML_METAL=on" python -m pip install khoj
```
</TabItem>
<TabItem value="intel" label="Intel">
```shell
python -m pip install khoj
```
</TabItem>
</Tabs>
</TabItem>
<TabItem value="windows" label="Windows">
Run the following command in PowerShell on Windows
<Tabs groupId="gpu" queryString="gpu">
<TabItem value="cpu" label="CPU">
```shell
# Install Khoj
py -m pip install khoj
```
</TabItem>
<TabItem value="nvidia" label="NVIDIA (CUDA) GPU">
```shell
# 1. To use NVIDIA (CUDA) GPU
$env:CMAKE_ARGS = "-DGGML_CUDA=on"
# 2. Install Khoj
py -m pip install khoj
```
</TabItem>
<TabItem value="amd" label="AMD (ROCm) GPU">
```shell
# 1. To use AMD (ROCm) GPU
$env:CMAKE_ARGS = "-DGGML_HIPBLAS=on"
# 2. Install Khoj
py -m pip install khoj
```
</TabItem>
<TabItem value="vulkan" label="VULKAN GPU">
```shell
# 1. To use VULCAN GPU
$env:CMAKE_ARGS = "-DGGML_VULKAN=on"
# 2. Install Khoj
py -m pip install khoj
```
</TabItem>
</Tabs>
</TabItem>
<TabItem value="linux" label="Linux">
<Tabs groupId="gpu" queryString="gpu">
<TabItem value="cpu" label="CPU">
```shell
python -m pip install khoj
```
</TabItem>
<TabItem value="nvidia" label="NVIDIA (CUDA) GPU">
```shell
CMAKE_ARGS="-DGGML_CUDA=on" FORCE_CMAKE=1 python -m pip install khoj
```
</TabItem>
<TabItem value="amd" label="AMD (ROCm) GPU">
```shell
CMAKE_ARGS="-DGGML_HIPBLAS=on" FORCE_CMAKE=1 python -m pip install khoj
```
</TabItem>
<TabItem value="vulkan" label="VULKAN GPU">
```shell
CMAKE_ARGS="-DGGML_VULKAN=on" FORCE_CMAKE=1 python -m pip install khoj
```
</TabItem>
</Tabs>
</TabItem>
</Tabs>
<h3>4. Start Khoj Server</h3>
Run the following command from your terminal to start the Khoj service.
```shell
khoj --anonymous-mode
```
`--anonymous-mode` allows access to Khoj without requiring login. This is usually fine for local only, single user setups. If you need authentication follow the [authentication setup docs](/advanced/authentication).
<h4>First Run</h4>
On the first run of the above command, you will be prompted to:
1. Create an admin account with a email and secure password
2. Customize the chat models to enable
- Keep your [OpenAI](https://platform.openai.com/api-keys), [Anthropic](https://console.anthropic.com/account/keys), [Gemini](https://aistudio.google.com/app/apikey) API keys and [OpenAI](https://platform.openai.com/docs/models), [Anthropic](https://docs.anthropic.com/en/docs/about-claude/models#model-names), [Gemini](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models#gemini-models), [Offline](https://huggingface.co/models?pipeline_tag=text-generation&library=gguf) chat model names handy to set any of them up during first run.
3. Your setup is complete once you see `🌖 Khoj is ready to use` in the server logs on your terminal!
:::tip[Auto Start]
To start Khoj automatically in the background use [Task scheduler](https://www.windowscentral.com/how-create-automated-task-using-task-scheduler-windows-10) on Windows or [Cron](https://en.wikipedia.org/wiki/Cron) on Mac, Linux (e.g. with `@reboot khoj`)
:::
</TabItem>
</Tabs>
## Use Khoj
You can now open the web app at http://localhost:42110 and start interacting!<br />
Nothing else is necessary, but you can customize your setup further by following the steps below.
:::info[First Message to Offline Chat Model]
The offline chat model gets downloaded when you first send a message to it. The download can take a few minutes! Subsequent messages should be faster.
:::
### Add Chat Models
<h4>Login to the Khoj Admin Panel</h4>
Go to http://localhost:42110/server/admin and login with the admin credentials you setup during installation.
:::info[CSRF Error]
Ensure you are using **localhost, not 127.0.0.1**, to access the admin panel to avoid the CSRF error.
:::
:::info[DISALLOWED HOST or Bad Request (400) Error]
You may hit this if you try access Khoj exposed on a custom domain (e.g. 192.168.12.3 or example.com) or over HTTP.
Set the environment variables KHOJ_DOMAIN=your-domain and KHOJ_NO_HTTPS=false if required to avoid this error.
:::
:::tip[Note]
Using Safari on Mac? You might not be able to login to the admin panel. Try using Chrome or Firefox instead.
:::
<h4>Configure Chat Model</h4>
Setup which chat model you'd want to use. Khoj supports local and online chat models.
<Tabs groupId="chatmodel" queryString>
<TabItem value="openai" label="OpenAI">
:::info[Ollama Integration]
Using Ollama? See the [Ollama Integration](/advanced/ollama) section for more custom setup instructions.
:::
1. Create a new [OpenAI processor conversation config](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/add) in the server admin settings. This is kind of a misnomer, we know.
- Add your [OpenAI API key](https://platform.openai.com/api-keys)
- Give the configuration a friendly name like `OpenAI`
- (Optional) Set the API base URL. It is only relevant if you're using another OpenAI-compatible proxy server like [Ollama](/advanced/ollama) or [LMStudio](/advanced/lmstudio).<br />
![example configuration for openai processor](/img/example_openai_processor_config.png)
2. Create a new [chat model options](http://localhost:42110/server/admin/database/chatmodeloptions/add)
- Set the `chat-model` field to an [OpenAI chat model](https://platform.openai.com/docs/models). Example: `gpt-4o`.
- Make sure to set the `model-type` field to `OpenAI`.
- If your model supports vision, set the `vision enabled` field to `true`. This is currently only supported for OpenAI models with vision capabilities.
- The `tokenizer` and `max-prompt-size` fields are optional. Set them only if you're sure of the tokenizer or token limit for the model you're using. Contact us if you're unsure what to do here.<br />
![example configuration for chat model options](/img/example_chatmodel_option.png)
</TabItem>
<TabItem value="anthropic" label="Anthropic">
1. Create a new [OpenAI processor conversation config](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/add) in the server admin settings. This is kind of a misnomer, we know.
- Add your [Anthropic API key](https://console.anthropic.com/account/keys)
- Give the configuration a friendly name like `Anthropic`. Do not configure the API base url.
2. Create a new [chat model options](http://localhost:42110/server/admin/database/chatmodeloptions/add)
- Set the `chat-model` field to an [Anthropic chat model](https://docs.anthropic.com/en/docs/about-claude/models#model-names). Example: `claude-3-5-sonnet-20240620`.
- Set the `model-type` field to `Anthropic`.
- Set the `Openai config` field to the OpenAI processor conversation config for Anthropic you created in step 1.
</TabItem>
<TabItem value="gemini" label="Gemini">
1. Create a new [OpenAI processor conversation config](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/add) in the server admin settings. This is kind of a misnomer, we know.
- Add your [Gemini API key](https://aistudio.google.com/app/apikey)
- Give the configuration a friendly name like `Gemini`. Do not configure the API base url.
2. Create a new [chat model options](http://localhost:42110/server/admin/database/chatmodeloptions/add)
- Set the `chat-model` field to a [Google Gemini chat model](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models#gemini-models). Example: `gemini-1.5-flash`.
- Set the `model-type` field to `Gemini`.
- Set the `Openai config` field to the OpenAI processor conversation config for Gemini you created in step 1.
</TabItem>
<TabItem value="offline" label="Offline">
Offline chat stays completely private and can work without internet using any open-weights model.
:::tip[System Requirements]
- Minimum 8 GB RAM. Recommend **16Gb VRAM**
- Minimum **5 GB of Disk** available
- A Nvidia, AMD GPU or a Mac M1+ machine would significantly speed up chat responses
:::
1. Get the name of your preferred chat model from [HuggingFace](https://huggingface.co/models?pipeline_tag=text-generation&library=gguf). *Most GGUF format chat models are supported*.
2. Open the [create chat model page](http://localhost:42110/server/admin/database/chatmodeloptions/add/) on the admin panel
3. Set the `chat-model` field to the name of your preferred chat model
- Make sure the `model-type` is set to `Offline`
4. Set the newly added chat model as your preferred model in your [User chat settings](http://localhost:42110/settings) and [Server chat settings](http://localhost:42110/server/admin/database/serverchatsettings/).
5. Restart the Khoj server and [start chatting](http://localhost:42110) with your new offline model!
</TabItem>
</Tabs>
:::tip[Multiple Chat Models]
Set your preferred default chat model in the `Default`, `Advanced` fields of your [ServerChatSettings](http://localhost:42110/server/admin/database/serverchatsettings/).
Khoj uses these chat model for all intermediate steps like intent detection, web search etc.
:::
:::info[Chat Model Fields]
- The `tokenizer` and `max-prompt-size` fields are optional. Set them only if you're sure of the tokenizer or token limit for the model you're using. This improves context stuffing. Contact us if you're unsure what to do here.
- Only tick the `vision enabled` field for OpenAI models with vision capabilities like gpt-4o. Vision capabilities in other chat models is not currently utilized.
:::
### Sync your Knowledge
- You can chat with your notes and documents using Khoj.
- Khoj can keep your files and folders synced using the Khoj [Desktop](/clients/desktop#setup), [Obsidian](/clients/obsidian#setup) or [Emacs](/clients/emacs#setup) clients.
- Your [Notion workspace](/data-sources/notion_integration) can be directly synced from the web app.
- You can also just drag and drop specific files you want to chat with on the [Web app](/clients/web#upload-documents).
### Setup Khoj Clients
The Khoj web app is available by default to chat, search and configure Khoj.<br />
You can also install a Khoj client to easily access it from Obsidian, Emacs, Whatsapp or your OS and keep your documents synced with Khoj.
:::info[Note]
Set the host URL on your clients settings page to your Khoj server URL. By default, use `http://127.0.0.1:42110` or `http://localhost:42110`. Note that `localhost` may not work in all cases.
:::
<Tabs groupId="client" queryString>
<TabItem value="desktop" label="Desktop">
- Read the Khoj Desktop app [setup docs](/clients/desktop#setup).
</TabItem>
<TabItem value="emacs" label="Emacs">
- Read the Khoj Emacs package [setup docs](/clients/emacs#setup).
</TabItem>
<TabItem value="obsidian" label="Obsidian">
- Read the Khoj Obsidian plugin [setup docs](/clients/obsidian#setup).
</TabItem>
<TabItem value="whatsapp" label="Whatsapp">
- Read the Khoj Whatsapp app [setup docs](/clients/whatsapp).
</TabItem>
</Tabs>
## Upgrade
### Upgrade Server
<Tabs groupId="server" queryString>
<TabItem value="pip" label="Pip">
```shell
pip install --upgrade khoj
```
*Note: To upgrade to the latest pre-release version of the khoj server run below command*
</TabItem>
<TabItem value="docker" label="Docker">
Run the commands below from the same directory where you have your `docker-compose.yml` file.
This will fetch the latest build and upgrade your server.
```shell
# Windows users should use their WSL2 terminal to run these commands
cd ~/.khoj # assuming your khoj docker-compose.yml file is here
docker-compose up --build
```
</TabItem>
</Tabs>
### Upgrade Clients
<Tabs groupId="client" queryString>
<TabItem value="desktop" label="Desktop">
- The Desktop app automatically updates to the latest released version on restart.
- You can manually download the latest version from the [Khoj Website](https://khoj.dev/downloads).
</TabItem>
<TabItem value="emacs" label="Emacs">
- Use your Emacs Package Manager to Upgrade
- See [khoj.el package setup](/clients/emacs#setup) for details
</TabItem>
<TabItem value="obsidian" label="Obsidian">
- Upgrade via the Community plugins tab on the settings pane in the Obsidian app
- See the [khoj plugin setup](/clients/obsidian#setup) for details
</TabItem>
</Tabs>
## Uninstall
### Uninstall Server
<Tabs groupId="server" queryString>
<TabItem value="pip" label="Pip">
```shell
# uninstall khoj server
pip uninstall khoj
# delete khoj postgres db
dropdb khoj -U postgres
```
</TabItem>
<TabItem value="docker" label="Docker">
Run the command below from the same directory where you have your `docker-compose` file.
This will remove the server containers, networks, images and volumes.
```shell
docker-compose down --volumes
```
</TabItem>
</Tabs>
### Uninstall Clients
<Tabs groupId="client" queryString>
<TabItem value="desktop" label="Desktop">
Uninstall the Khoj Desktop client in the standard way from your OS.
</TabItem>
<TabItem value="emacs" label="Emacs">
Uninstall the Khoj Emacs package in the standard way from Emacs.
</TabItem>
<TabItem value="obsidian" label="Obsidian">
Uninstall via the Community plugins tab on the settings pane in the Obsidian app
</TabItem>
</Tabs>
## Troubleshoot
#### Dependency conflict when trying to install Khoj python package with pip
- **Reason**: When conflicting dependency versions are required by Khoj vs other python packages installed on your system
- **Fix**: Install Khoj in a python virtual environment using [venv](https://docs.python.org/3/library/venv.html) or [pipx](https://pypa.github.io/pipx) to avoid this dependency conflicts
- **Process**:
1. Install [pipx](https://pypa.github.io/pipx/#install-pipx)
2. Use `pipx` to install Khoj to avoid dependency conflicts with other python packages.
```shell
pipx install khoj
```
3. Now start `khoj` using the standard steps described earlier
#### Install fails while building Tokenizer dependency
- **Details**: `pip install khoj` fails while building the `tokenizers` dependency. Complains about Rust.
- **Fix**: Install Rust to build the tokenizers package. For example on Mac run:
```shell
brew install rustup
rustup-init
source ~/.cargo/env
```
- **Refer**: [Issue with Fix](https://github.com/khoj-ai/khoj/issues/82#issuecomment-1241890946) for more details
#### Khoj in Docker errors out with \"Killed\" in error message
- **Fix**: Increase RAM available to Docker Containers in Docker Settings
- **Refer**: [StackOverflow Solution](https://stackoverflow.com/a/50770267), [Configure Resources on Docker for Mac](https://docs.docker.com/desktop/mac/#resources)
Reference in a new issue View git blame Copy permalink