mirror of
https://github.com/khoj-ai/khoj.git
synced 2025-02-17 08:04:21 +00:00
Refactor Config API and Settings pages for Reuse and Consistency (#852)
### Major - Reuse get config data logic across config pages on web client - Make config api endpoint urls and response fields consistent - Rename API path /api/config to /api/configure - Move Web, Desktop client settings page to be under `/settings` from the previous `/config` url path ### Minor - Pass isMobileWidth prop to SidePanel via chat share interface - Turn prettier off instead of throwing error for now - Do no explicitly add line-clamp plugin as it's in Tailwind by default
This commit is contained in:
Debanjum
GitHub
commit
bf815e4463
42 changed files with 453 additions and 557 deletions
|
|
@ -34,4 +34,4 @@ Using LiteLLM with Khoj makes it possible to turn any LLM behind an API into you
|
|||
5. Create a new [Server Chat Setting](http://localhost:42110/server/admin/database/serverchatsettings/add/) on your Khoj admin panel
|
||||
- Default model: `<name of chat model option you created in step 4>`
|
||||
- Summarizer model: `<name of chat model option you created in step 4>`
|
||||
6. Go to [your config](http://localhost:42110/config) and select the model you just created in the chat model dropdown.
|
||||
6. Go to [your config](http://localhost:42110/settings) and select the model you just created in the chat model dropdown.
|
||||
|
|
|
|||
|
|
@ -27,4 +27,4 @@ LM Studio can expose an [OpenAI API compatible server](https://lmstudio.ai/docs/
|
|||
5. Create a new [Server Chat Setting](http://localhost:42110/server/admin/database/serverchatsettings/add/) on your Khoj admin panel
|
||||
- Default model: `<name of chat model option you created in step 4>`
|
||||
- Summarizer model: `<name of chat model option you created in step 4>`
|
||||
6. Go to [your config](http://localhost:42110/config) and select the model you just created in the chat model dropdown.
|
||||
6. Go to [your config](http://localhost:42110/settings) and select the model you just created in the chat model dropdown.
|
||||
|
|
|
|||
|
|
@ -31,6 +31,6 @@ Ollama exposes a local [OpenAI API compatible server](https://github.com/ollama/
|
|||
5. Create a new [Server Chat Setting](http://localhost:42110/server/admin/database/serverchatsettings/add/) on your Khoj admin panel
|
||||
- Default model: `<name of chat model option you created in step 4>`
|
||||
- Summarizer model: `<name of chat model option you created in step 4>`
|
||||
6. Go to [your config](http://localhost:42110/config) and select the model you just created in the chat model dropdown.
|
||||
6. Go to [your config](http://localhost:42110/settings) and select the model you just created in the chat model dropdown.
|
||||
|
||||
That's it! You should now be able to chat with your Ollama model from Khoj. If you want to add additional models running on Ollama, repeat step 6 for each model.
|
||||
|
|
|
|||
|
|
@ -34,4 +34,4 @@ For specific integrations, see our [Ollama](/advanced/ollama), [LMStudio](/advan
|
|||
5. Create a new [Server Chat Setting](http://localhost:42110/server/admin/database/serverchatsettings/add/) on your Khoj admin panel
|
||||
- Default model: `<name of chat model option you created in step 4>`
|
||||
- Summarizer model: `<name of chat model option you created in step 4>`
|
||||
6. Go to [your config](http://localhost:42110/config) and select the model you just created in the chat model dropdown.
|
||||
6. Go to [your config](http://localhost:42110/settings) and select the model you just created in the chat model dropdown.
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@ Khoj will keep these files in sync to provide contextual responses when you sear
|
|||
## Setup
|
||||
|
||||
1. Install the [Khoj Desktop app](https://khoj.dev/downloads) for your OS
|
||||
2. Generate an API key on the [Khoj Web App](https://app.khoj.dev/config#clients)
|
||||
2. Generate an API key on the [Khoj Web App](https://app.khoj.dev/settings#clients)
|
||||
3. Set your Khoj API Key on the *Settings* page of the Khoj Desktop app
|
||||
4. [Optional] Add any files, folders you'd like Khoj to be aware of on the *Settings* page and Click *Save*
|
||||
These files and folders will be automatically kept in sync for you
|
||||
|
|
|
|||
|
|
@ -30,7 +30,7 @@ sidebar_position: 2
|
|||
|  |  |
|
||||
|
||||
## Setup
|
||||
1. Generate an API key on the [Khoj Web App](https://app.khoj.dev/config#clients)
|
||||
1. Generate an API key on the [Khoj Web App](https://app.khoj.dev/settings#clients)
|
||||
2. Add below snippet to your Emacs config file, usually at `~/.emacs.d/init.el`
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@ sidebar_position: 3
|
|||
|
||||
1. Open [Khoj](https://obsidian.md/plugins?id=khoj) from the *Community plugins* tab in Obsidian settings panel
|
||||
2. Click *Install*, then *Enable* on the Khoj plugin page in Obsidian
|
||||
3. Generate an API key on the [Khoj Web App](https://app.khoj.dev/config#clients)
|
||||
3. Generate an API key on the [Khoj Web App](https://app.khoj.dev/settings#clients)
|
||||
4. Set your Khoj API Key in the Khoj plugin settings in Obsidian
|
||||
|
||||
See the official [Obsidian Plugin Docs](https://help.obsidian.md/Extending+Obsidian/Community+plugins) for more details on installing Obsidian plugins.
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ Text [+1 (848) 800 4242](https://wa.me/18488004242) or scan [this QR code](https
|
|||
|
||||
Without any desktop clients, you can start chatting with Khoj on WhatsApp. Bear in mind you do need one of the desktop clients in order to share and sync your data with Khoj. The WhatsApp AI bot will work right away for answering generic queries and using Khoj in default mode.
|
||||
|
||||
In order to use Khoj on WhatsApp with your own data, you need to setup a Khoj Cloud account and connect your WhatsApp account to it. This is a one time setup and you can do it from the [Khoj Cloud config page](https://app.khoj.dev/config).
|
||||
In order to use Khoj on WhatsApp with your own data, you need to setup a Khoj Cloud account and connect your WhatsApp account to it. This is a one time setup and you can do it from the [Khoj Cloud config page](https://app.khoj.dev/settings).
|
||||
|
||||
If you hit usage limits for the WhatsApp bot, upgrade to [a paid plan](https://khoj.dev/pricing) on Khoj Cloud.
|
||||
|
||||
|
|
|
|||
|
|
@ -4,11 +4,11 @@ The Github integration allows you to index as many repositories as you want. It'
|
|||
|
||||
# Configure your settings
|
||||
|
||||
1. Go to [https://app.khoj.dev/config](https://app.khoj.dev/config) and enter in settings for the data sources you want to index. You'll have to specify the file paths.
|
||||
1. Go to [https://app.khoj.dev/settings](https://app.khoj.dev/settings) and enter in settings for the data sources you want to index. You'll have to specify the file paths.
|
||||
|
||||
## Use the Github plugin
|
||||
|
||||
1. Generate a [classic PAT (personal access token)](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) from [Github](https://github.com/settings/tokens) with `repo` and `admin:org` scopes at least.
|
||||
2. Navigate to [https://app.khoj.dev/config/content-source/github](https://app.khoj.dev/config/content-source/github) to configure your Github settings. Enter in your PAT, along with details for each repository you want to index.
|
||||
2. Navigate to [https://app.khoj.dev/settings/content/github](https://app.khoj.dev/settings/content/github) to configure your Github settings. Enter in your PAT, along with details for each repository you want to index.
|
||||
3. Click `Save`. Go back to the settings page and click `Configure`.
|
||||
4. Go to [https://app.khoj.dev/](https://app.khoj.dev/) and start searching!
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
The Notion integration allows you to search/chat with your Notion workspaces. [Notion](https://notion.so/) is a platform people use for taking notes, especially for collaboration.
|
||||
|
||||
Go to https://app.khoj.dev/config to connect your Notion workspace(s) to Khoj.
|
||||
Go to https://app.khoj.dev/settings to connect your Notion workspace(s) to Khoj.
|
||||
|
||||
|
||||
|
||||
|
|
@ -13,7 +13,7 @@ Go to https://app.khoj.dev/config to connect your Notion workspace(s) to Khoj.
|
|||
|
||||
3. Share all the workspaces that you want to integrate with the Khoj integration you just made in the previous step
|
||||
|
||||
4. In the first step, you generated an API key. Use the newly generated API Key in your Khoj settings, by default at http://localhost:42110/config/content-source/notion. Click `Save`.
|
||||
5. Click `Configure` in http://localhost:42110/config to index your Notion workspace(s).
|
||||
4. In the first step, you generated an API key. Use the newly generated API Key in your Khoj settings, by default at http://localhost:42110/settings/content/notion. Click `Save`.
|
||||
5. Click `Configure` in http://localhost:42110/settings to index your Notion workspace(s).
|
||||
|
||||
That's it! You should be ready to start searching and chatting. Make sure you've configured your [chat settings](/get-started/setup#2-configure).
|
||||
|
|
|
|||
|
|
@ -253,7 +253,7 @@ function pushDataToKhoj (regenerate = false) {
|
|||
console.error(error);
|
||||
state["completed"] = false;
|
||||
if (error?.response?.status === 429 && (BrowserWindow.getAllWindows().find(win => win.webContents.getURL().includes('config')))) {
|
||||
state["error"] = `Looks like you're out of space to sync your files. <a href="https://app.khoj.dev/config">Upgrade your plan</a> to unlock more space.`;
|
||||
state["error"] = `Looks like you're out of space to sync your files. <a href="https://app.khoj.dev/settings">Upgrade your plan</a> to unlock more space.`;
|
||||
const win = BrowserWindow.getAllWindows().find(win => win.webContents.getURL().includes('config'));
|
||||
if (win) win.webContents.send('needsSubscription', true);
|
||||
} else if (error?.code === 'ECONNREFUSED') {
|
||||
|
|
|
|||
|
|
@ -182,7 +182,7 @@ window.updateStateAPI.onUpdateState((event, state) => {
|
|||
window.needsSubscriptionAPI.onNeedsSubscription((event, needsSubscription) => {
|
||||
console.log("needs subscription", needsSubscription);
|
||||
if (needsSubscription) {
|
||||
window.alert("Looks like you're out of space to sync your files. Upgrade your plan to unlock more space here: https://app.khoj.dev/config");
|
||||
window.alert("Looks like you're out of space to sync your files. Upgrade your plan to unlock more space here: https://app.khoj.dev/settings");
|
||||
needsSubscriptionElement.style.display = 'block';
|
||||
}
|
||||
});
|
||||
|
|
|
|||
|
|
@ -212,12 +212,12 @@
|
|||
const headers = { 'Authorization': `Bearer ${khojToken}` };
|
||||
|
||||
// Populate type dropdown field with enabled content types only
|
||||
fetch(`${hostURL}/api/config/types`, { headers })
|
||||
fetch(`${hostURL}/api/configure/types`, { headers })
|
||||
.then(response => response.json())
|
||||
.then(enabled_types => {
|
||||
// Show warning if no content types are enabled
|
||||
if (enabled_types.detail) {
|
||||
document.getElementById("results").innerHTML = "<div id='results-error'>To use Khoj search, setup your content plugins on the Khoj <a class='inline-chat-link' href='/config'>settings page</a>.</div>";
|
||||
document.getElementById("results").innerHTML = "<div id='results-error'>To use Khoj search, setup your content plugins on the Khoj <a class='inline-chat-link' href='/settings'>settings page</a>.</div>";
|
||||
document.getElementById("query").setAttribute("disabled", "disabled");
|
||||
document.getElementById("query").setAttribute("placeholder", "Configure Khoj to enable search");
|
||||
return [];
|
||||
|
|
|
|||
|
|
@ -85,7 +85,7 @@ async function populateHeaderPane() {
|
|||
<div id="khoj-nav-menu" class="khoj-nav-dropdown-content">
|
||||
<div class="khoj-nav-username"> ${username} </div>
|
||||
<a id="github-nav" class="khoj-nav" href="https://github.com/khoj-ai/khoj">GitHub</a>
|
||||
<a id="settings-nav" class="khoj-nav" href="./config.html">⚙️ Settings</a>
|
||||
<a id="settings-nav" class="khoj-nav" href="./settings.html">⚙️ Settings</a>
|
||||
</div>
|
||||
</div>
|
||||
` : ''}
|
||||
|
|
|
|||
|
|
@ -99,7 +99,7 @@
|
|||
:type 'boolean)
|
||||
|
||||
(defcustom khoj-api-key nil
|
||||
"API Key to your Khoj. Default at https://app.khoj.dev/config#clients."
|
||||
"API Key to your Khoj. Default at https://app.khoj.dev/settings#clients."
|
||||
:group 'khoj
|
||||
:type 'string)
|
||||
|
||||
|
|
@ -697,7 +697,7 @@ Optionally apply CALLBACK with JSON parsed response and CBARGS."
|
|||
|
||||
(defun khoj--get-enabled-content-types ()
|
||||
"Get content types enabled for search from API."
|
||||
(khoj--call-api "/api/config/types" "GET" nil `(lambda (item) (mapcar #'intern item))))
|
||||
(khoj--call-api "/api/configure/types" "GET" nil `(lambda (item) (mapcar #'intern item))))
|
||||
|
||||
(defun khoj--query-search-api-and-render-results (query content-type buffer-name &optional rerank is-find-similar)
|
||||
"Query Khoj Search API with QUERY, CONTENT-TYPE and RERANK as query params.
|
||||
|
|
|
|||
|
|
@ -201,12 +201,12 @@ export function getBackendStatusMessage(
|
|||
): string {
|
||||
// Welcome message with default settings. Khoj cloud always expects an API key.
|
||||
if (!khojApiKey && khojUrl === 'https://app.khoj.dev')
|
||||
return `🌈 Welcome to Khoj! Get your API key from ${khojUrl}/config#clients and set it in the Khoj plugin settings on Obsidian`;
|
||||
return `🌈 Welcome to Khoj! Get your API key from ${khojUrl}/settings#clients and set it in the Khoj plugin settings on Obsidian`;
|
||||
|
||||
if (!connectedToServer)
|
||||
return `❗️Could not connect to Khoj at ${khojUrl}. Ensure your can access it`;
|
||||
else if (!userEmail)
|
||||
return `✅ Connected to Khoj. ❗️Get a valid API key from ${khojUrl}/config#clients to log in`;
|
||||
return `✅ Connected to Khoj. ❗️Get a valid API key from ${khojUrl}/settings#clients to log in`;
|
||||
else if (userEmail === 'default@example.com')
|
||||
// Logged in as default user in anonymous mode
|
||||
return `✅ Signed in to Khoj`;
|
||||
|
|
|
|||
|
|
@ -6,6 +6,6 @@
|
|||
],
|
||||
"plugins": ["prettier"],
|
||||
"rules": {
|
||||
"prettier/prettier": "error"
|
||||
"prettier/prettier": "off"
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -68,8 +68,8 @@ interface ModelPickerProps {
|
|||
}
|
||||
|
||||
export const ModelPicker: React.FC<any> = (props: ModelPickerProps) => {
|
||||
const { data: models } = useOptionsRequest('/api/config/data/conversation/model/options');
|
||||
const { data: selectedModel } = useSelectedModel('/api/config/data/conversation/model');
|
||||
const { data: models } = useOptionsRequest('/api/configure/chat/model/options');
|
||||
const { data: selectedModel } = useSelectedModel('/api/configure/chat/model');
|
||||
const [openLoginDialog, setOpenLoginDialog] = React.useState(false);
|
||||
|
||||
let userData = useAuthenticatedData();
|
||||
|
|
@ -94,7 +94,7 @@ export const ModelPicker: React.FC<any> = (props: ModelPickerProps) => {
|
|||
props.setModelUsed(model);
|
||||
}
|
||||
|
||||
fetch('/api/config/data/conversation/model' + '?id=' + String(model.id), { method: 'POST', body: JSON.stringify(model) })
|
||||
fetch('/api/configure/chat/model' + '?id=' + String(model.id), { method: 'POST', body: JSON.stringify(model) })
|
||||
.then((response) => {
|
||||
if (!response.ok) {
|
||||
throw new Error('Failed to select model');
|
||||
|
|
|
|||
|
|
@ -128,7 +128,7 @@ export default function NavMenu(props: NavMenuProps) {
|
|||
<DropdownMenuSeparator />
|
||||
<DropdownMenuLabel>Profile</DropdownMenuLabel>
|
||||
<DropdownMenuItem>
|
||||
<Link href="/config">Settings</Link>
|
||||
<Link href="/settings">Settings</Link>
|
||||
</DropdownMenuItem>
|
||||
<DropdownMenuItem>
|
||||
<Link href="https://docs.khoj.dev">Help</Link>
|
||||
|
|
@ -172,7 +172,7 @@ export default function NavMenu(props: NavMenuProps) {
|
|||
{userData &&
|
||||
<>
|
||||
<MenubarItem>
|
||||
<Link href="/config">
|
||||
<Link href="/settings">
|
||||
Settings
|
||||
</Link>
|
||||
</MenubarItem>
|
||||
|
|
|
|||
|
|
@ -148,7 +148,7 @@ interface FilesMenuProps {
|
|||
|
||||
function FilesMenu(props: FilesMenuProps) {
|
||||
// Use SWR to fetch files
|
||||
const { data: files, error } = useSWR<string[]>(props.conversationId ? '/api/config/data/computer' : null, fetcher);
|
||||
const { data: files, error } = useSWR<string[]>(props.conversationId ? '/api/configure/content/computer' : null, fetcher);
|
||||
const { data: selectedFiles, error: selectedFilesError } = useSWR(props.conversationId ? `/api/chat/conversation/file-filters/${props.conversationId}` : null, fetcher);
|
||||
const [isOpen, setIsOpen] = useState(false);
|
||||
const [unfilteredFiles, setUnfilteredFiles] = useState<string[]>([]);
|
||||
|
|
@ -604,7 +604,7 @@ function UserProfileComponent(props: UserProfileProps) {
|
|||
|
||||
return (
|
||||
<div className={styles.profile}>
|
||||
<Link href="/config" target="_blank" rel="noopener noreferrer">
|
||||
<Link href="/settings">
|
||||
<Avatar>
|
||||
<AvatarImage src={props.userProfile.photo} alt="user profile" />
|
||||
<AvatarFallback>
|
||||
|
|
|
|||
|
|
@ -533,7 +533,7 @@ export default function FactChecker() {
|
|||
<Button disabled={clickedVerify} onClick={() => onClickVerify()}>Verify</Button>
|
||||
</div>
|
||||
<h3 className={`mt-4 mb-4`}>
|
||||
Try with a particular model. You must be <a href="/config" className="font-medium text-blue-600 dark:text-blue-500 hover:underline">subscribed</a> to configure the model.
|
||||
Try with a particular model. You must be <a href="/settings" className="font-medium text-blue-600 dark:text-blue-500 hover:underline">subscribed</a> to configure the model.
|
||||
</h3>
|
||||
</div>
|
||||
}
|
||||
|
|
|
|||
|
|
@ -299,7 +299,9 @@ export default function SharedChat() {
|
|||
<SidePanel
|
||||
webSocketConnected={!!conversationId ? (chatWS != null) : true}
|
||||
conversationId={conversationId ?? null}
|
||||
uploadedFiles={uploadedFiles} />
|
||||
uploadedFiles={uploadedFiles}
|
||||
isMobileWidth={isMobileWidth}
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div className={styles.chatBox}>
|
||||
|
|
|
|||
|
|
@ -35,7 +35,6 @@
|
|||
"@radix-ui/react-slot": "^1.1.0",
|
||||
"@radix-ui/react-toast": "^1.2.1",
|
||||
"@radix-ui/react-toggle": "^1.1.0",
|
||||
"@tailwindcss/line-clamp": "^0.4.4",
|
||||
"@types/dompurify": "^3.0.5",
|
||||
"@types/katex": "^0.16.7",
|
||||
"@types/markdown-it": "^14.1.1",
|
||||
|
|
|
|||
|
|
@ -76,7 +76,6 @@ const config = {
|
|||
},
|
||||
plugins: [
|
||||
require("tailwindcss-animate"),
|
||||
require('@tailwindcss/line-clamp'),
|
||||
],
|
||||
} satisfies Config
|
||||
|
||||
|
|
|
|||
|
|
@ -1090,11 +1090,6 @@
|
|||
"@swc/counter" "^0.1.3"
|
||||
tslib "^2.4.0"
|
||||
|
||||
"@tailwindcss/line-clamp@^0.4.4":
|
||||
version "0.4.4"
|
||||
resolved "https://registry.yarnpkg.com/@tailwindcss/line-clamp/-/line-clamp-0.4.4.tgz#767cf8e5d528a5d90c9740ca66eb079f5e87d423"
|
||||
integrity sha512-5U6SY5z8N42VtrCrKlsTAA35gy2VSyYtHWCsg1H87NU1SXnEfekTVlrga9fzUDrrHcGi2Lb5KenUWb4lRQT5/g==
|
||||
|
||||
"@ts-morph/common@~0.19.0":
|
||||
version "0.19.0"
|
||||
resolved "https://registry.yarnpkg.com/@ts-morph/common/-/common-0.19.0.tgz#927fcd81d1bbc09c89c4a310a84577fb55f3694e"
|
||||
|
|
|
|||
|
|
@ -316,7 +316,7 @@ def configure_routes(app):
|
|||
app.include_router(api, prefix="/api")
|
||||
app.include_router(api_chat, prefix="/api/chat")
|
||||
app.include_router(api_agents, prefix="/api/agents")
|
||||
app.include_router(api_config, prefix="/api/config")
|
||||
app.include_router(api_config, prefix="/api/configure")
|
||||
app.include_router(indexer, prefix="/api/v1/index")
|
||||
app.include_router(notion_router, prefix="/api/notion")
|
||||
app.include_router(web_client)
|
||||
|
|
@ -336,7 +336,7 @@ def configure_routes(app):
|
|||
if is_twilio_enabled():
|
||||
from khoj.routers.api_phone import api_phone
|
||||
|
||||
app.include_router(api_phone, prefix="/api/config/phone")
|
||||
app.include_router(api_phone, prefix="/api/configure/phone")
|
||||
logger.info("📞 Enabled Twilio")
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -44,7 +44,7 @@ Hi, I am Khoj, your open, personal AI 👋🏽. I can:
|
|||
- 📚 Understand files you drag & drop here
|
||||
- 👩🏾🚀 Be tuned to your conversation needs via [agents](./agents)
|
||||
|
||||
Get the Khoj [Desktop](https://khoj.dev/downloads), [Obsidian](https://docs.khoj.dev/clients/obsidian#setup), [Emacs](https://docs.khoj.dev/clients/emacs#setup) apps to search, chat with your 🖥️ computer docs. You can manage all the files you've shared with me at any time by going to [your settings](/config/content-source/computer/).
|
||||
Get the Khoj [Desktop](https://khoj.dev/downloads), [Obsidian](https://docs.khoj.dev/clients/obsidian#setup), [Emacs](https://docs.khoj.dev/clients/emacs#setup) apps to search, chat with your 🖥️ computer docs. You can manage all the files you've shared with me at any time by going to [your settings](/settings/content/computer/).
|
||||
|
||||
To get started, just start typing below. You can also type / to see a list of commands.
|
||||
`.trim()
|
||||
|
|
@ -1333,7 +1333,7 @@ To get started, just start typing below. You can also type / to see a list of co
|
|||
- 📚 Understand files you drag & drop here
|
||||
- 👩🏾🚀 Be tuned to your conversation needs via [agents](./agents)
|
||||
|
||||
Get the Khoj [Desktop](https://khoj.dev/downloads), [Obsidian](https://docs.khoj.dev/clients/obsidian#setup), or [Emacs](https://docs.khoj.dev/clients/emacs#setup) app to keep your files in sync. You can manage all the files you've shared with me at any time by going to [your settings](/config/content-source/computer/).
|
||||
Get the Khoj [Desktop](https://khoj.dev/downloads), [Obsidian](https://docs.khoj.dev/clients/obsidian#setup), or [Emacs](https://docs.khoj.dev/clients/emacs#setup) app to keep your files in sync. You can manage all the files you've shared with me at any time by going to [your settings](/settings/content/computer/).
|
||||
|
||||
To get started, just start typing below. You can also type / to see a list of commands.
|
||||
|
||||
|
|
@ -1954,7 +1954,7 @@ To get started, just start typing below. You can also type / to see a list of co
|
|||
}
|
||||
var allFiles;
|
||||
function renderAllFiles() {
|
||||
fetch('/api/config/data/computer')
|
||||
fetch('/api/configure/content/computer')
|
||||
.then(response => response.json())
|
||||
.then(data => {
|
||||
var indexedFiles = document.getElementsByClassName("indexed-files")[0];
|
||||
|
|
|
|||
|
|
@ -32,7 +32,7 @@
|
|||
</style>
|
||||
<script>
|
||||
function removeFile(path) {
|
||||
fetch('/api/config/data/file?filename=' + path, {
|
||||
fetch('/api/configure/content/file?filename=' + path, {
|
||||
method: 'DELETE',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
|
|
@ -48,7 +48,7 @@
|
|||
|
||||
// Get all currently indexed files
|
||||
function getAllComputerFilenames() {
|
||||
fetch('/api/config/data/computer')
|
||||
fetch('/api/configure/content/computer')
|
||||
.then(response => response.json())
|
||||
.then(data => {
|
||||
var indexedFiles = document.getElementsByClassName("indexed-files")[0];
|
||||
|
|
@ -122,7 +122,7 @@
|
|||
deleteAllComputerFilesButton.textContent = "🗑️ Deleting...";
|
||||
deleteAllComputerFilesButton.disabled = true;
|
||||
|
||||
fetch('/api/config/data/content-source/computer', {
|
||||
fetch('/api/configure/content/computer', {
|
||||
method: 'DELETE',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
|
|
|
|||
|
|
@ -165,7 +165,7 @@
|
|||
|
||||
// Save Github config on server
|
||||
const csrfToken = document.cookie.split('; ').find(row => row.startsWith('csrftoken'))?.split('=')[1];
|
||||
fetch('/api/config/data/content-source/github', {
|
||||
fetch('/api/configure/content/github', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
|
|
|
|||
|
|
@ -45,7 +45,7 @@
|
|||
|
||||
// Save Notion config on server
|
||||
const csrfToken = document.cookie.split('; ').find(row => row.startsWith('csrftoken'))?.split('=')[1];
|
||||
fetch('/api/config/data/content-source/notion', {
|
||||
fetch('/api/configure/content/notion', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
|
|
|
|||
|
|
@ -34,7 +34,7 @@ Hi, I am Khoj, your open, personal AI 👋🏽. I can:
|
|||
- 📚 Understand files you drag & drop here
|
||||
- 👩🏾🚀 Be tuned to your conversation needs via [agents](./agents)
|
||||
|
||||
Get the Khoj [Desktop](https://khoj.dev/downloads), [Obsidian](https://docs.khoj.dev/clients/obsidian#setup), [Emacs](https://docs.khoj.dev/clients/emacs#setup) apps to search, chat with your 🖥️ computer docs. You can manage all the files you've shared with me at any time by going to [your settings](/config/content-source/computer/).
|
||||
Get the Khoj [Desktop](https://khoj.dev/downloads), [Obsidian](https://docs.khoj.dev/clients/obsidian#setup), [Emacs](https://docs.khoj.dev/clients/emacs#setup) apps to search, chat with your 🖥️ computer docs. You can manage all the files you've shared with me at any time by going to [your settings](/settings/content/computer/).
|
||||
|
||||
To get started, just start typing below. You can also type / to see a list of commands.
|
||||
`.trim()
|
||||
|
|
|
|||
|
|
@ -209,12 +209,12 @@
|
|||
|
||||
function populate_type_dropdown() {
|
||||
// Populate type dropdown field with enabled content types only
|
||||
fetch("/api/config/types")
|
||||
fetch("/api/configure/types")
|
||||
.then(response => response.json())
|
||||
.then(enabled_types => {
|
||||
// Show warning if no content types are enabled, or just one ("all")
|
||||
if (enabled_types[0] === "all" && enabled_types.length === 1) {
|
||||
document.getElementById("results").innerHTML = "<div id='results-error'>To use Khoj search, setup your content plugins on the Khoj <a class='inline-chat-link' href='/config'>settings page</a>.</div>";
|
||||
document.getElementById("results").innerHTML = "<div id='results-error'>To use Khoj search, setup your content plugins on the Khoj <a class='inline-chat-link' href='/settings'>settings page</a>.</div>";
|
||||
document.getElementById("query").setAttribute("disabled", "disabled");
|
||||
document.getElementById("query").setAttribute("placeholder", "Configure Khoj to enable search");
|
||||
return [];
|
||||
|
|
|
|||
|
|
@ -34,7 +34,7 @@
|
|||
<h3 id="card-title-computer" class="card-title">
|
||||
<span>Files</span>
|
||||
<img id="configured-icon-computer"
|
||||
style="display: {% if not current_model_state.computer %}none{% endif %}"
|
||||
style="display: {% if not enabled_content_source.computer %}none{% endif %}"
|
||||
class="configured-icon"
|
||||
src="/static/assets/icons/confirm-icon.svg"
|
||||
alt="Configured">
|
||||
|
|
@ -44,8 +44,8 @@
|
|||
<p class="card-description">Manage files from your computer</p>
|
||||
</div>
|
||||
<div class="card-action-row">
|
||||
<a class="card-button" href="/config/content-source/computer">
|
||||
{% if current_model_state.computer %}
|
||||
<a class="card-button" href="/settings/content/computer">
|
||||
{% if enabled_content_source.computer %}
|
||||
Update
|
||||
{% else %}
|
||||
Setup
|
||||
|
|
@ -53,7 +53,7 @@
|
|||
<svg xmlns="http://www.w3.org/2000/svg" width="1em" height="1em" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M5 12h14M12 5l7 7-7 7"></path></svg>
|
||||
</a>
|
||||
<div id="clear-computer" class="card-action-row"
|
||||
style="display: {% if not current_model_state.computer %}none{% endif %}">
|
||||
style="display: {% if not enabled_content_source.computer %}none{% endif %}">
|
||||
<button class="card-button" onclick="clearContentType('computer')">
|
||||
Disable
|
||||
</button>
|
||||
|
|
@ -69,15 +69,15 @@
|
|||
class="configured-icon"
|
||||
src="/static/assets/icons/confirm-icon.svg"
|
||||
alt="Configured"
|
||||
style="display: {% if not current_model_state.github %}none{% endif %}">
|
||||
style="display: {% if not enabled_content_source.github %}none{% endif %}">
|
||||
</h3>
|
||||
</div>
|
||||
<div class="card-description-row">
|
||||
<p class="card-description">Set repositories to index</p>
|
||||
</div>
|
||||
<div class="card-action-row">
|
||||
<a class="card-button" href="/config/content-source/github">
|
||||
{% if current_model_state.github %}
|
||||
<a class="card-button" href="/settings/content/github">
|
||||
{% if enabled_content_source.github %}
|
||||
Update
|
||||
{% else %}
|
||||
Setup
|
||||
|
|
@ -86,7 +86,7 @@
|
|||
</a>
|
||||
<div id="clear-github"
|
||||
class="card-action-row"
|
||||
style="display: {% if not current_model_state.github %}none{% endif %}">
|
||||
style="display: {% if not enabled_content_source.github %}none{% endif %}">
|
||||
<button class="card-button" onclick="clearContentType('github')">
|
||||
Disable
|
||||
</button>
|
||||
|
|
@ -102,15 +102,15 @@
|
|||
class="configured-icon"
|
||||
src="/static/assets/icons/confirm-icon.svg"
|
||||
alt="Configured"
|
||||
style="display: {% if not current_model_state.notion %}none{% endif %}">
|
||||
style="display: {% if not enabled_content_source.notion %}none{% endif %}">
|
||||
</h3>
|
||||
</div>
|
||||
<div class="card-description-row">
|
||||
<p class="card-description">Sync your Notion pages</p>
|
||||
</div>
|
||||
<div class="card-action-row">
|
||||
{% if current_model_state.notion %}
|
||||
<a class="card-button" href="/config/content-source/notion">
|
||||
{% if enabled_content_source.notion %}
|
||||
<a class="card-button" href="/settings/content/notion">
|
||||
Update
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="1em" height="1em" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M5 12h14M12 5l7 7-7 7"></path></svg>
|
||||
</a>
|
||||
|
|
@ -120,7 +120,7 @@
|
|||
<svg xmlns="http://www.w3.org/2000/svg" width="1em" height="1em" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M5 12h14M12 5l7 7-7 7"></path></svg>
|
||||
</a>
|
||||
{% else %}
|
||||
<a class="card-button" href="/config/content-source/notion">
|
||||
<a class="card-button" href="/settings/content/notion">
|
||||
Setup
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="1em" height="1em" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M5 12h14M12 5l7 7-7 7"></path></svg>
|
||||
</a>
|
||||
|
|
@ -128,7 +128,7 @@
|
|||
|
||||
<div id="clear-notion"
|
||||
class="card-action-row"
|
||||
style="display: {% if not current_model_state.notion %}none{% endif %}">
|
||||
style="display: {% if not enabled_content_source.notion %}none{% endif %}">
|
||||
<button class="card-button" onclick="clearContentType('notion')">
|
||||
Disable
|
||||
</button>
|
||||
|
|
@ -181,8 +181,8 @@
|
|||
</div>
|
||||
<div class="card-description-row">
|
||||
<select id="chat-models">
|
||||
{% for option in conversation_options %}
|
||||
<option value="{{ option.id }}" {% if option.id == selected_conversation_config %}selected{% endif %}>{{ option.chat_model }}</option>
|
||||
{% for option in chat_model_options %}
|
||||
<option value="{{ option.id }}" {% if option.id == selected_chat_model_config %}selected{% endif %}>{{ option.name }}</option>
|
||||
{% endfor %}
|
||||
</select>
|
||||
</div>
|
||||
|
|
@ -208,7 +208,7 @@
|
|||
<div class="card-description-row">
|
||||
<select id="paint-models">
|
||||
{% for option in paint_model_options %}
|
||||
<option value="{{ option.id }}" {% if option.id == selected_paint_model_config %}selected{% endif %}>{{ option.model_name }}</option>
|
||||
<option value="{{ option.id }}" {% if option.id == selected_paint_model_config %}selected{% endif %}>{{ option.name }}</option>
|
||||
{% endfor %}
|
||||
</select>
|
||||
</div>
|
||||
|
|
@ -394,7 +394,7 @@
|
|||
|
||||
function saveProfileGivenName() {
|
||||
const givenName = document.getElementById("profile_given_name").value;
|
||||
fetch('/api/config/user/name?name=' + givenName, {
|
||||
fetch('/api/configure/user/name?name=' + givenName, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
|
|
@ -421,7 +421,7 @@
|
|||
saveVoiceModelButton.disabled = true;
|
||||
saveVoiceModelButton.textContent = "Saving...";
|
||||
|
||||
fetch('/api/config/data/voice/model?id=' + voiceModel, {
|
||||
fetch('/api/configure/voice/model?id=' + voiceModel, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
|
|
@ -455,7 +455,7 @@
|
|||
saveModelButton.innerHTML = "";
|
||||
saveModelButton.textContent = "Saving...";
|
||||
|
||||
fetch('/api/config/data/conversation/model?id=' + chatModel, {
|
||||
fetch('/api/configure/chat/model?id=' + chatModel, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
|
|
@ -494,7 +494,7 @@
|
|||
saveSearchModelButton.disabled = true;
|
||||
saveSearchModelButton.textContent = "Saving...";
|
||||
|
||||
fetch('/api/config/data/search/model?id=' + searchModel, {
|
||||
fetch('/api/configure/search/model?id=' + searchModel, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
|
|
@ -526,7 +526,7 @@
|
|||
saveModelButton.disabled = true;
|
||||
saveModelButton.innerHTML = "Saving...";
|
||||
|
||||
fetch('/api/config/data/paint/model?id=' + paintModel, {
|
||||
fetch('/api/configure/paint/model?id=' + paintModel, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
|
|
@ -553,7 +553,7 @@
|
|||
};
|
||||
|
||||
function clearContentType(content_source) {
|
||||
fetch('/api/config/data/content-source/' + content_source, {
|
||||
fetch('/api/configure/content/' + content_source, {
|
||||
method: 'DELETE',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
|
|
@ -676,7 +676,7 @@
|
|||
|
||||
content_sources = ["computer", "github", "notion"];
|
||||
content_sources.forEach(content_source => {
|
||||
fetch(`/api/config/data/${content_source}`, {
|
||||
fetch(`/api/configure/content/${content_source}`, {
|
||||
method: 'GET',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
|
|
@ -807,7 +807,7 @@
|
|||
|
||||
function getIndexedDataSize() {
|
||||
document.getElementById("indexed-data-size").textContent = "Calculating...";
|
||||
fetch('/api/config/index/size')
|
||||
fetch('/api/configure/content/size')
|
||||
.then(response => response.json())
|
||||
.then(data => {
|
||||
document.getElementById("indexed-data-size").textContent = data.indexed_data_size_in_mb + " MB used";
|
||||
|
|
@ -815,7 +815,7 @@
|
|||
}
|
||||
|
||||
function removeFile(path) {
|
||||
fetch('/api/config/data/file?filename=' + path, {
|
||||
fetch('/api/configure/content/file?filename=' + path, {
|
||||
method: 'DELETE',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
|
|
@ -890,7 +890,7 @@
|
|||
})
|
||||
|
||||
phonenumberRemoveButton.addEventListener("click", () => {
|
||||
fetch('/api/config/phone', {
|
||||
fetch('/api/configure/phone', {
|
||||
method: 'DELETE',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
|
|
@ -917,7 +917,7 @@
|
|||
}, 5000);
|
||||
} else {
|
||||
const mobileNumber = iti.getNumber();
|
||||
fetch('/api/config/phone?phone_number=' + mobileNumber, {
|
||||
fetch('/api/configure/phone?phone_number=' + mobileNumber, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
|
|
@ -970,7 +970,7 @@
|
|||
return;
|
||||
}
|
||||
|
||||
fetch('/api/config/phone/verify?code=' + otp, {
|
||||
fetch('/api/configure/phone/verify?code=' + otp, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
|
|
@ -36,7 +36,7 @@
|
|||
{% endif %}
|
||||
<div id="khoj-nav-menu" class="khoj-nav-dropdown-content">
|
||||
<div class="khoj-nav-username"> {{ username }} </div>
|
||||
<a id="settings-nav" class="khoj-nav" href="/config">Settings</a>
|
||||
<a id="settings-nav" class="khoj-nav" href="/settings">Settings</a>
|
||||
<a id="github-nav" class="khoj-nav" href="https://github.com/khoj-ai/khoj">GitHub</a>
|
||||
<a id="help-nav" class="khoj-nav" href="https://docs.khoj.dev" target="_blank">Help</a>
|
||||
<a class="khoj-nav" href="/auth/logout">Logout</a>
|
||||
|
|
|
|||
|
|
@ -6,7 +6,6 @@ import os
|
|||
import threading
|
||||
import time
|
||||
import uuid
|
||||
from random import random
|
||||
from typing import Any, Callable, List, Optional, Union
|
||||
|
||||
import cron_descriptor
|
||||
|
|
@ -190,7 +189,7 @@ def update(
|
|||
):
|
||||
user = request.user.object
|
||||
if not state.config:
|
||||
error_msg = f"🚨 Khoj is not configured.\nConfigure it via http://localhost:42110/config, plugins or by editing {state.config_file}."
|
||||
error_msg = f"🚨 Khoj is not configured.\nConfigure it via http://localhost:42110/settings, plugins or by editing {state.config_file}."
|
||||
logger.warning(error_msg)
|
||||
raise HTTPException(status_code=500, detail=error_msg)
|
||||
try:
|
||||
|
|
|
|||
|
|
@ -98,9 +98,9 @@ def _initialize_config():
|
|||
state.config.search_type = SearchConfig.model_validate(constants.default_config["search-type"])
|
||||
|
||||
|
||||
@api_config.post("/data/content-source/github", status_code=200)
|
||||
@api_config.post("/content/github", status_code=200)
|
||||
@requires(["authenticated"])
|
||||
async def set_content_config_github_data(
|
||||
async def set_content_github(
|
||||
request: Request,
|
||||
updated_config: Union[GithubContentConfig, None],
|
||||
client: Optional[str] = None,
|
||||
|
|
@ -130,9 +130,9 @@ async def set_content_config_github_data(
|
|||
return {"status": "ok"}
|
||||
|
||||
|
||||
@api_config.post("/data/content-source/notion", status_code=200)
|
||||
@api_config.post("/content/notion", status_code=200)
|
||||
@requires(["authenticated"])
|
||||
async def set_content_config_notion_data(
|
||||
async def set_content_notion(
|
||||
request: Request,
|
||||
updated_config: Union[NotionContentConfig, None],
|
||||
client: Optional[str] = None,
|
||||
|
|
@ -161,9 +161,9 @@ async def set_content_config_notion_data(
|
|||
return {"status": "ok"}
|
||||
|
||||
|
||||
@api_config.delete("/data/content-source/{content_source}", status_code=200)
|
||||
@api_config.delete("/content/{content_source}", status_code=200)
|
||||
@requires(["authenticated"])
|
||||
async def remove_content_source_data(
|
||||
async def delete_content_source(
|
||||
request: Request,
|
||||
content_source: str,
|
||||
client: Optional[str] = None,
|
||||
|
|
@ -189,9 +189,9 @@ async def remove_content_source_data(
|
|||
return {"status": "ok"}
|
||||
|
||||
|
||||
@api_config.delete("/data/file", status_code=200)
|
||||
@api_config.delete("/content/file", status_code=201)
|
||||
@requires(["authenticated"])
|
||||
async def remove_file_data(
|
||||
async def delete_content_file(
|
||||
request: Request,
|
||||
filename: str,
|
||||
client: Optional[str] = None,
|
||||
|
|
@ -210,9 +210,9 @@ async def remove_file_data(
|
|||
return {"status": "ok"}
|
||||
|
||||
|
||||
@api_config.get("/data/{content_source}", response_model=List[str])
|
||||
@api_config.get("/content/{content_source}", response_model=List[str])
|
||||
@requires(["authenticated"])
|
||||
async def get_all_filenames(
|
||||
async def get_content_source(
|
||||
request: Request,
|
||||
content_source: str,
|
||||
client: Optional[str] = None,
|
||||
|
|
@ -229,7 +229,7 @@ async def get_all_filenames(
|
|||
return await sync_to_async(list)(EntryAdapters.get_all_filenames_by_source(user, content_source)) # type: ignore[call-arg]
|
||||
|
||||
|
||||
@api_config.get("/data/conversation/model/options", response_model=Dict[str, Union[str, int]])
|
||||
@api_config.get("/chat/model/options", response_model=Dict[str, Union[str, int]])
|
||||
def get_chat_model_options(
|
||||
request: Request,
|
||||
client: Optional[str] = None,
|
||||
|
|
@ -243,7 +243,7 @@ def get_chat_model_options(
|
|||
return Response(content=json.dumps(all_conversation_options), media_type="application/json", status_code=200)
|
||||
|
||||
|
||||
@api_config.get("/data/conversation/model")
|
||||
@api_config.get("/chat/model")
|
||||
@requires(["authenticated"])
|
||||
def get_user_chat_model(
|
||||
request: Request,
|
||||
|
|
@ -259,7 +259,7 @@ def get_user_chat_model(
|
|||
return Response(status_code=200, content=json.dumps({"id": chat_model.id, "chat_model": chat_model.chat_model}))
|
||||
|
||||
|
||||
@api_config.post("/data/conversation/model", status_code=200)
|
||||
@api_config.post("/chat/model", status_code=200)
|
||||
@requires(["authenticated", "premium"])
|
||||
async def update_chat_model(
|
||||
request: Request,
|
||||
|
|
@ -284,7 +284,7 @@ async def update_chat_model(
|
|||
return {"status": "ok"}
|
||||
|
||||
|
||||
@api_config.post("/data/voice/model", status_code=200)
|
||||
@api_config.post("/voice/model", status_code=200)
|
||||
@requires(["authenticated", "premium"])
|
||||
async def update_voice_model(
|
||||
request: Request,
|
||||
|
|
@ -308,7 +308,7 @@ async def update_voice_model(
|
|||
return Response(status_code=202, content=json.dumps({"status": "ok"}))
|
||||
|
||||
|
||||
@api_config.post("/data/search/model", status_code=200)
|
||||
@api_config.post("/search/model", status_code=200)
|
||||
@requires(["authenticated"])
|
||||
async def update_search_model(
|
||||
request: Request,
|
||||
|
|
@ -341,7 +341,7 @@ async def update_search_model(
|
|||
return {"status": "ok"}
|
||||
|
||||
|
||||
@api_config.post("/data/paint/model", status_code=200)
|
||||
@api_config.post("/paint/model", status_code=200)
|
||||
@requires(["authenticated"])
|
||||
async def update_paint_model(
|
||||
request: Request,
|
||||
|
|
@ -370,9 +370,9 @@ async def update_paint_model(
|
|||
return {"status": "ok"}
|
||||
|
||||
|
||||
@api_config.get("/index/size", response_model=Dict[str, int])
|
||||
@api_config.get("/content/size", response_model=Dict[str, int])
|
||||
@requires(["authenticated"])
|
||||
async def get_indexed_data_size(request: Request, common: CommonQueryParams):
|
||||
async def get_content_size(request: Request, common: CommonQueryParams):
|
||||
user = request.user.object
|
||||
indexed_data_size_in_mb = await sync_to_async(EntryAdapters.get_size_of_indexed_data_in_mb)(user)
|
||||
return Response(
|
||||
|
|
|
|||
|
|
@ -5,6 +5,7 @@ import io
|
|||
import json
|
||||
import logging
|
||||
import math
|
||||
import os
|
||||
import re
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
|
@ -35,6 +36,7 @@ from PIL import Image
|
|||
from starlette.authentication import has_required_scope
|
||||
from starlette.requests import URL
|
||||
|
||||
from khoj.database import adapters
|
||||
from khoj.database.adapters import (
|
||||
AgentAdapters,
|
||||
AutomationAdapters,
|
||||
|
|
@ -42,18 +44,30 @@ from khoj.database.adapters import (
|
|||
EntryAdapters,
|
||||
create_khoj_token,
|
||||
get_khoj_tokens,
|
||||
get_user_name,
|
||||
get_user_subscription_state,
|
||||
run_with_process_lock,
|
||||
)
|
||||
from khoj.database.models import (
|
||||
ChatModelOptions,
|
||||
ClientApplication,
|
||||
Conversation,
|
||||
GithubConfig,
|
||||
KhojUser,
|
||||
NotionConfig,
|
||||
ProcessLock,
|
||||
Subscription,
|
||||
TextToImageModelConfig,
|
||||
UserRequests,
|
||||
)
|
||||
from khoj.processor.content.docx.docx_to_entries import DocxToEntries
|
||||
from khoj.processor.content.github.github_to_entries import GithubToEntries
|
||||
from khoj.processor.content.images.image_to_entries import ImageToEntries
|
||||
from khoj.processor.content.markdown.markdown_to_entries import MarkdownToEntries
|
||||
from khoj.processor.content.notion.notion_to_entries import NotionToEntries
|
||||
from khoj.processor.content.org_mode.org_to_entries import OrgToEntries
|
||||
from khoj.processor.content.pdf.pdf_to_entries import PdfToEntries
|
||||
from khoj.processor.content.plaintext.plaintext_to_entries import PlaintextToEntries
|
||||
from khoj.processor.conversation import prompts
|
||||
from khoj.processor.conversation.anthropic.anthropic_chat import (
|
||||
anthropic_send_message_to_model,
|
||||
|
|
@ -69,11 +83,15 @@ from khoj.processor.conversation.utils import (
|
|||
generate_chatml_messages_with_context,
|
||||
save_to_conversation_log,
|
||||
)
|
||||
from khoj.processor.speech.text_to_speech import is_eleven_labs_enabled
|
||||
from khoj.routers.email import is_resend_enabled, send_task_email
|
||||
from khoj.routers.storage import upload_image
|
||||
from khoj.routers.twilio import is_twilio_enabled
|
||||
from khoj.search_type import text_search
|
||||
from khoj.utils import state
|
||||
from khoj.utils.config import OfflineChatProcessorModel
|
||||
from khoj.utils.helpers import (
|
||||
LRU,
|
||||
ConversationCommand,
|
||||
ImageIntentType,
|
||||
is_none_or_empty,
|
||||
|
|
@ -90,6 +108,11 @@ logger = logging.getLogger(__name__)
|
|||
executor = ThreadPoolExecutor(max_workers=1)
|
||||
|
||||
|
||||
NOTION_OAUTH_CLIENT_ID = os.getenv("NOTION_OAUTH_CLIENT_ID")
|
||||
NOTION_OAUTH_CLIENT_SECRET = os.getenv("NOTION_OAUTH_CLIENT_SECRET")
|
||||
NOTION_REDIRECT_URI = os.getenv("NOTION_REDIRECT_URI")
|
||||
|
||||
|
||||
def is_query_empty(query: str) -> bool:
|
||||
return is_none_or_empty(query.strip())
|
||||
|
||||
|
|
@ -902,7 +925,7 @@ class ApiUserRateLimiter:
|
|||
)
|
||||
raise HTTPException(
|
||||
status_code=429,
|
||||
detail="We're glad you're enjoying Khoj! You've exceeded your usage limit for today. Come back tomorrow or subscribe to increase your usage limit via [your settings](https://app.khoj.dev/config).",
|
||||
detail="We're glad you're enjoying Khoj! You've exceeded your usage limit for today. Come back tomorrow or subscribe to increase your usage limit via [your settings](https://app.khoj.dev/settings).",
|
||||
)
|
||||
|
||||
# Add the current request to the cache
|
||||
|
|
@ -941,7 +964,7 @@ class ConversationCommandRateLimiter:
|
|||
if not subscribed and count_requests >= self.trial_rate_limit:
|
||||
raise HTTPException(
|
||||
status_code=429,
|
||||
detail=f"We're glad you're enjoying Khoj! You've exceeded your `/{conversation_command.value}` command usage limit for today. Subscribe to increase your usage limit via [your settings](https://app.khoj.dev/config).",
|
||||
detail=f"We're glad you're enjoying Khoj! You've exceeded your `/{conversation_command.value}` command usage limit for today. Subscribe to increase your usage limit via [your settings](https://app.khoj.dev/settings).",
|
||||
)
|
||||
await UserRequests.objects.acreate(user=user, slug=command_slug)
|
||||
return
|
||||
|
|
@ -1186,3 +1209,284 @@ def construct_automation_created_message(automation: Job, crontime: str, query_t
|
|||
|
||||
Manage your automations [here](/automations).
|
||||
""".strip()
|
||||
|
||||
|
||||
def get_user_config(user: KhojUser, request: Request, is_detailed: bool = False):
|
||||
user_picture = request.session.get("user", {}).get("picture")
|
||||
is_active = has_required_scope(request, ["premium"])
|
||||
has_documents = EntryAdapters.user_has_entries(user=user)
|
||||
|
||||
if not is_detailed:
|
||||
return {
|
||||
"request": request,
|
||||
"username": user.username if user else None,
|
||||
"user_photo": user_picture,
|
||||
"is_active": is_active,
|
||||
"has_documents": has_documents,
|
||||
"khoj_version": state.khoj_version,
|
||||
}
|
||||
|
||||
user_subscription_state = get_user_subscription_state(user.email)
|
||||
user_subscription = adapters.get_user_subscription(user.email)
|
||||
subscription_renewal_date = (
|
||||
user_subscription.renewal_date.strftime("%d %b %Y")
|
||||
if user_subscription and user_subscription.renewal_date
|
||||
else (user_subscription.created_at + timedelta(days=7)).strftime("%d %b %Y")
|
||||
)
|
||||
given_name = get_user_name(user)
|
||||
|
||||
enabled_content_sources_set = set(EntryAdapters.get_unique_file_sources(user))
|
||||
enabled_content_sources = {
|
||||
"computer": ("computer" in enabled_content_sources_set),
|
||||
"github": ("github" in enabled_content_sources_set),
|
||||
"notion": ("notion" in enabled_content_sources_set),
|
||||
}
|
||||
|
||||
selected_chat_model_config = ConversationAdapters.get_conversation_config(user)
|
||||
chat_models = ConversationAdapters.get_conversation_processor_options().all()
|
||||
chat_model_options = list()
|
||||
for chat_model in chat_models:
|
||||
chat_model_options.append({"name": chat_model.chat_model, "id": chat_model.id})
|
||||
|
||||
search_model_options = adapters.get_or_create_search_models().all()
|
||||
all_search_model_options = list()
|
||||
for search_model_option in search_model_options:
|
||||
all_search_model_options.append({"name": search_model_option.name, "id": search_model_option.id})
|
||||
|
||||
current_search_model_option = adapters.get_user_search_model_or_default(user)
|
||||
|
||||
selected_paint_model_config = ConversationAdapters.get_user_text_to_image_model_config(user)
|
||||
paint_model_options = ConversationAdapters.get_text_to_image_model_options().all()
|
||||
all_paint_model_options = list()
|
||||
for paint_model in paint_model_options:
|
||||
all_paint_model_options.append({"name": paint_model.model_name, "id": paint_model.id})
|
||||
|
||||
notion_oauth_url = get_notion_auth_url(user)
|
||||
|
||||
eleven_labs_enabled = is_eleven_labs_enabled()
|
||||
|
||||
voice_models = ConversationAdapters.get_voice_model_options()
|
||||
voice_model_options = list()
|
||||
for voice_model in voice_models:
|
||||
voice_model_options.append({"name": voice_model.name, "id": voice_model.model_id})
|
||||
|
||||
if len(voice_model_options) == 0:
|
||||
eleven_labs_enabled = False
|
||||
|
||||
selected_voice_config = ConversationAdapters.get_voice_model_config(user)
|
||||
|
||||
return {
|
||||
"request": request,
|
||||
# user info
|
||||
"username": user.username if user else None,
|
||||
"user_photo": user_picture,
|
||||
"is_active": is_active,
|
||||
"given_name": given_name,
|
||||
"phone_number": user.phone_number,
|
||||
"is_phone_number_verified": user.verified_phone_number,
|
||||
# user content, model settings
|
||||
"enabled_content_source": enabled_content_sources,
|
||||
"has_documents": has_documents,
|
||||
"search_model_options": all_search_model_options,
|
||||
"selected_search_model_config": current_search_model_option.id,
|
||||
"chat_model_options": chat_model_options,
|
||||
"selected_chat_model_config": selected_chat_model_config.id if selected_chat_model_config else None,
|
||||
"paint_model_options": all_paint_model_options,
|
||||
"selected_paint_model_config": selected_paint_model_config.id if selected_paint_model_config else None,
|
||||
"voice_model_options": voice_model_options,
|
||||
"selected_voice_config": selected_voice_config.model_id if selected_voice_config else None,
|
||||
# user billing info
|
||||
"subscription_state": user_subscription_state,
|
||||
"subscription_renewal_date": subscription_renewal_date,
|
||||
# server settings
|
||||
"khoj_cloud_subscription_url": os.getenv("KHOJ_CLOUD_SUBSCRIPTION_URL"),
|
||||
"billing_enabled": state.billing_enabled,
|
||||
"is_eleven_labs_enabled": eleven_labs_enabled,
|
||||
"is_twilio_enabled": is_twilio_enabled(),
|
||||
"khoj_version": state.khoj_version,
|
||||
"anonymous_mode": state.anonymous_mode,
|
||||
"notion_oauth_url": notion_oauth_url,
|
||||
}
|
||||
|
||||
|
||||
def configure_content(
|
||||
files: Optional[dict[str, dict[str, str]]],
|
||||
regenerate: bool = False,
|
||||
t: Optional[state.SearchType] = state.SearchType.All,
|
||||
full_corpus: bool = True,
|
||||
user: KhojUser = None,
|
||||
) -> bool:
|
||||
success = True
|
||||
if t == None:
|
||||
t = state.SearchType.All
|
||||
|
||||
if t is not None and t in [type.value for type in state.SearchType]:
|
||||
t = state.SearchType(t)
|
||||
|
||||
if t is not None and not t.value in [type.value for type in state.SearchType]:
|
||||
logger.warning(f"🚨 Invalid search type: {t}")
|
||||
return False
|
||||
|
||||
search_type = t.value if t else None
|
||||
|
||||
no_documents = all([not files.get(file_type) for file_type in files])
|
||||
|
||||
if files is None:
|
||||
logger.warning(f"🚨 No files to process for {search_type} search.")
|
||||
return True
|
||||
|
||||
try:
|
||||
# Initialize Org Notes Search
|
||||
if (search_type == state.SearchType.All.value or search_type == state.SearchType.Org.value) and files["org"]:
|
||||
logger.info("🦄 Setting up search for orgmode notes")
|
||||
# Extract Entries, Generate Notes Embeddings
|
||||
text_search.setup(
|
||||
OrgToEntries,
|
||||
files.get("org"),
|
||||
regenerate=regenerate,
|
||||
full_corpus=full_corpus,
|
||||
user=user,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"🚨 Failed to setup org: {e}", exc_info=True)
|
||||
success = False
|
||||
|
||||
try:
|
||||
# Initialize Markdown Search
|
||||
if (search_type == state.SearchType.All.value or search_type == state.SearchType.Markdown.value) and files[
|
||||
"markdown"
|
||||
]:
|
||||
logger.info("💎 Setting up search for markdown notes")
|
||||
# Extract Entries, Generate Markdown Embeddings
|
||||
text_search.setup(
|
||||
MarkdownToEntries,
|
||||
files.get("markdown"),
|
||||
regenerate=regenerate,
|
||||
full_corpus=full_corpus,
|
||||
user=user,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"🚨 Failed to setup markdown: {e}", exc_info=True)
|
||||
success = False
|
||||
|
||||
try:
|
||||
# Initialize PDF Search
|
||||
if (search_type == state.SearchType.All.value or search_type == state.SearchType.Pdf.value) and files["pdf"]:
|
||||
logger.info("🖨️ Setting up search for pdf")
|
||||
# Extract Entries, Generate PDF Embeddings
|
||||
text_search.setup(
|
||||
PdfToEntries,
|
||||
files.get("pdf"),
|
||||
regenerate=regenerate,
|
||||
full_corpus=full_corpus,
|
||||
user=user,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"🚨 Failed to setup PDF: {e}", exc_info=True)
|
||||
success = False
|
||||
|
||||
try:
|
||||
# Initialize Plaintext Search
|
||||
if (search_type == state.SearchType.All.value or search_type == state.SearchType.Plaintext.value) and files[
|
||||
"plaintext"
|
||||
]:
|
||||
logger.info("📄 Setting up search for plaintext")
|
||||
# Extract Entries, Generate Plaintext Embeddings
|
||||
text_search.setup(
|
||||
PlaintextToEntries,
|
||||
files.get("plaintext"),
|
||||
regenerate=regenerate,
|
||||
full_corpus=full_corpus,
|
||||
user=user,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"🚨 Failed to setup plaintext: {e}", exc_info=True)
|
||||
success = False
|
||||
|
||||
try:
|
||||
if no_documents:
|
||||
github_config = GithubConfig.objects.filter(user=user).prefetch_related("githubrepoconfig").first()
|
||||
if (
|
||||
search_type == state.SearchType.All.value or search_type == state.SearchType.Github.value
|
||||
) and github_config is not None:
|
||||
logger.info("🐙 Setting up search for github")
|
||||
# Extract Entries, Generate Github Embeddings
|
||||
text_search.setup(
|
||||
GithubToEntries,
|
||||
None,
|
||||
regenerate=regenerate,
|
||||
full_corpus=full_corpus,
|
||||
user=user,
|
||||
config=github_config,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"🚨 Failed to setup GitHub: {e}", exc_info=True)
|
||||
success = False
|
||||
|
||||
try:
|
||||
if no_documents:
|
||||
# Initialize Notion Search
|
||||
notion_config = NotionConfig.objects.filter(user=user).first()
|
||||
if (
|
||||
search_type == state.SearchType.All.value or search_type == state.SearchType.Notion.value
|
||||
) and notion_config:
|
||||
logger.info("🔌 Setting up search for notion")
|
||||
text_search.setup(
|
||||
NotionToEntries,
|
||||
None,
|
||||
regenerate=regenerate,
|
||||
full_corpus=full_corpus,
|
||||
user=user,
|
||||
config=notion_config,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"🚨 Failed to setup Notion: {e}", exc_info=True)
|
||||
success = False
|
||||
|
||||
try:
|
||||
# Initialize Image Search
|
||||
if (search_type == state.SearchType.All.value or search_type == state.SearchType.Image.value) and files[
|
||||
"image"
|
||||
]:
|
||||
logger.info("🖼️ Setting up search for images")
|
||||
# Extract Entries, Generate Image Embeddings
|
||||
text_search.setup(
|
||||
ImageToEntries,
|
||||
files.get("image"),
|
||||
regenerate=regenerate,
|
||||
full_corpus=full_corpus,
|
||||
user=user,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"🚨 Failed to setup images: {e}", exc_info=True)
|
||||
success = False
|
||||
try:
|
||||
if (search_type == state.SearchType.All.value or search_type == state.SearchType.Docx.value) and files["docx"]:
|
||||
logger.info("📄 Setting up search for docx")
|
||||
text_search.setup(
|
||||
DocxToEntries,
|
||||
files.get("docx"),
|
||||
regenerate=regenerate,
|
||||
full_corpus=full_corpus,
|
||||
user=user,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"🚨 Failed to setup docx: {e}", exc_info=True)
|
||||
success = False
|
||||
|
||||
# Invalidate Query Cache
|
||||
if user:
|
||||
state.query_cache[user.uuid] = LRU()
|
||||
|
||||
return success
|
||||
|
||||
|
||||
def get_notion_auth_url(user: KhojUser):
|
||||
if not NOTION_OAUTH_CLIENT_ID or not NOTION_OAUTH_CLIENT_SECRET or not NOTION_REDIRECT_URI:
|
||||
return None
|
||||
return f"https://api.notion.com/v1/oauth/authorize?client_id={NOTION_OAUTH_CLIENT_ID}&redirect_uri={NOTION_REDIRECT_URI}&response_type=code&state={user.uuid}"
|
||||
|
|
|
|||
|
|
@ -6,20 +6,14 @@ from fastapi import APIRouter, Depends, Header, Request, Response, UploadFile
|
|||
from pydantic import BaseModel
|
||||
from starlette.authentication import requires
|
||||
|
||||
from khoj.database.models import GithubConfig, KhojUser, NotionConfig
|
||||
from khoj.processor.content.docx.docx_to_entries import DocxToEntries
|
||||
from khoj.processor.content.github.github_to_entries import GithubToEntries
|
||||
from khoj.processor.content.images.image_to_entries import ImageToEntries
|
||||
from khoj.processor.content.markdown.markdown_to_entries import MarkdownToEntries
|
||||
from khoj.processor.content.notion.notion_to_entries import NotionToEntries
|
||||
from khoj.processor.content.org_mode.org_to_entries import OrgToEntries
|
||||
from khoj.processor.content.pdf.pdf_to_entries import PdfToEntries
|
||||
from khoj.processor.content.plaintext.plaintext_to_entries import PlaintextToEntries
|
||||
from khoj.routers.helpers import ApiIndexedDataLimiter, update_telemetry_state
|
||||
from khoj.search_type import text_search
|
||||
from khoj.routers.helpers import (
|
||||
ApiIndexedDataLimiter,
|
||||
configure_content,
|
||||
update_telemetry_state,
|
||||
)
|
||||
from khoj.utils import constants, state
|
||||
from khoj.utils.config import SearchModels
|
||||
from khoj.utils.helpers import LRU, get_file_type
|
||||
from khoj.utils.helpers import get_file_type
|
||||
from khoj.utils.rawconfig import ContentConfig, FullConfig, SearchConfig
|
||||
from khoj.utils.yaml import save_config_to_file_updated_state
|
||||
|
||||
|
|
@ -170,180 +164,3 @@ def configure_search(search_models: SearchModels, search_config: Optional[Search
|
|||
search_models = SearchModels()
|
||||
|
||||
return search_models
|
||||
|
||||
|
||||
def configure_content(
|
||||
files: Optional[dict[str, dict[str, str]]],
|
||||
regenerate: bool = False,
|
||||
t: Optional[state.SearchType] = state.SearchType.All,
|
||||
full_corpus: bool = True,
|
||||
user: KhojUser = None,
|
||||
) -> bool:
|
||||
success = True
|
||||
if t == None:
|
||||
t = state.SearchType.All
|
||||
|
||||
if t is not None and t in [type.value for type in state.SearchType]:
|
||||
t = state.SearchType(t)
|
||||
|
||||
if t is not None and not t.value in [type.value for type in state.SearchType]:
|
||||
logger.warning(f"🚨 Invalid search type: {t}")
|
||||
return False
|
||||
|
||||
search_type = t.value if t else None
|
||||
|
||||
no_documents = all([not files.get(file_type) for file_type in files])
|
||||
|
||||
if files is None:
|
||||
logger.warning(f"🚨 No files to process for {search_type} search.")
|
||||
return True
|
||||
|
||||
try:
|
||||
# Initialize Org Notes Search
|
||||
if (search_type == state.SearchType.All.value or search_type == state.SearchType.Org.value) and files["org"]:
|
||||
logger.info("🦄 Setting up search for orgmode notes")
|
||||
# Extract Entries, Generate Notes Embeddings
|
||||
text_search.setup(
|
||||
OrgToEntries,
|
||||
files.get("org"),
|
||||
regenerate=regenerate,
|
||||
full_corpus=full_corpus,
|
||||
user=user,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"🚨 Failed to setup org: {e}", exc_info=True)
|
||||
success = False
|
||||
|
||||
try:
|
||||
# Initialize Markdown Search
|
||||
if (search_type == state.SearchType.All.value or search_type == state.SearchType.Markdown.value) and files[
|
||||
"markdown"
|
||||
]:
|
||||
logger.info("💎 Setting up search for markdown notes")
|
||||
# Extract Entries, Generate Markdown Embeddings
|
||||
text_search.setup(
|
||||
MarkdownToEntries,
|
||||
files.get("markdown"),
|
||||
regenerate=regenerate,
|
||||
full_corpus=full_corpus,
|
||||
user=user,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"🚨 Failed to setup markdown: {e}", exc_info=True)
|
||||
success = False
|
||||
|
||||
try:
|
||||
# Initialize PDF Search
|
||||
if (search_type == state.SearchType.All.value or search_type == state.SearchType.Pdf.value) and files["pdf"]:
|
||||
logger.info("🖨️ Setting up search for pdf")
|
||||
# Extract Entries, Generate PDF Embeddings
|
||||
text_search.setup(
|
||||
PdfToEntries,
|
||||
files.get("pdf"),
|
||||
regenerate=regenerate,
|
||||
full_corpus=full_corpus,
|
||||
user=user,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"🚨 Failed to setup PDF: {e}", exc_info=True)
|
||||
success = False
|
||||
|
||||
try:
|
||||
# Initialize Plaintext Search
|
||||
if (search_type == state.SearchType.All.value or search_type == state.SearchType.Plaintext.value) and files[
|
||||
"plaintext"
|
||||
]:
|
||||
logger.info("📄 Setting up search for plaintext")
|
||||
# Extract Entries, Generate Plaintext Embeddings
|
||||
text_search.setup(
|
||||
PlaintextToEntries,
|
||||
files.get("plaintext"),
|
||||
regenerate=regenerate,
|
||||
full_corpus=full_corpus,
|
||||
user=user,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"🚨 Failed to setup plaintext: {e}", exc_info=True)
|
||||
success = False
|
||||
|
||||
try:
|
||||
if no_documents:
|
||||
github_config = GithubConfig.objects.filter(user=user).prefetch_related("githubrepoconfig").first()
|
||||
if (
|
||||
search_type == state.SearchType.All.value or search_type == state.SearchType.Github.value
|
||||
) and github_config is not None:
|
||||
logger.info("🐙 Setting up search for github")
|
||||
# Extract Entries, Generate Github Embeddings
|
||||
text_search.setup(
|
||||
GithubToEntries,
|
||||
None,
|
||||
regenerate=regenerate,
|
||||
full_corpus=full_corpus,
|
||||
user=user,
|
||||
config=github_config,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"🚨 Failed to setup GitHub: {e}", exc_info=True)
|
||||
success = False
|
||||
|
||||
try:
|
||||
if no_documents:
|
||||
# Initialize Notion Search
|
||||
notion_config = NotionConfig.objects.filter(user=user).first()
|
||||
if (
|
||||
search_type == state.SearchType.All.value or search_type == state.SearchType.Notion.value
|
||||
) and notion_config:
|
||||
logger.info("🔌 Setting up search for notion")
|
||||
text_search.setup(
|
||||
NotionToEntries,
|
||||
None,
|
||||
regenerate=regenerate,
|
||||
full_corpus=full_corpus,
|
||||
user=user,
|
||||
config=notion_config,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"🚨 Failed to setup Notion: {e}", exc_info=True)
|
||||
success = False
|
||||
|
||||
try:
|
||||
# Initialize Image Search
|
||||
if (search_type == state.SearchType.All.value or search_type == state.SearchType.Image.value) and files[
|
||||
"image"
|
||||
]:
|
||||
logger.info("🖼️ Setting up search for images")
|
||||
# Extract Entries, Generate Image Embeddings
|
||||
text_search.setup(
|
||||
ImageToEntries,
|
||||
files.get("image"),
|
||||
regenerate=regenerate,
|
||||
full_corpus=full_corpus,
|
||||
user=user,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"🚨 Failed to setup images: {e}", exc_info=True)
|
||||
success = False
|
||||
try:
|
||||
if (search_type == state.SearchType.All.value or search_type == state.SearchType.Docx.value) and files["docx"]:
|
||||
logger.info("📄 Setting up search for docx")
|
||||
text_search.setup(
|
||||
DocxToEntries,
|
||||
files.get("docx"),
|
||||
regenerate=regenerate,
|
||||
full_corpus=full_corpus,
|
||||
user=user,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"🚨 Failed to setup docx: {e}", exc_info=True)
|
||||
success = False
|
||||
|
||||
# Invalidate Query Cache
|
||||
if user:
|
||||
state.query_cache[user.uuid] = LRU()
|
||||
|
||||
return success
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ from starlette.responses import RedirectResponse
|
|||
|
||||
from khoj.database.adapters import aget_user_by_uuid
|
||||
from khoj.database.models import KhojUser, NotionConfig
|
||||
from khoj.routers.indexer import configure_content
|
||||
from khoj.routers.helpers import configure_content
|
||||
from khoj.utils.state import SearchType
|
||||
|
||||
NOTION_OAUTH_CLIENT_ID = os.getenv("NOTION_OAUTH_CLIENT_ID")
|
||||
|
|
@ -25,12 +25,6 @@ executor = ThreadPoolExecutor()
|
|||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def get_notion_auth_url(user: KhojUser):
|
||||
if not NOTION_OAUTH_CLIENT_ID or not NOTION_OAUTH_CLIENT_SECRET or not NOTION_REDIRECT_URI:
|
||||
return None
|
||||
return f"https://api.notion.com/v1/oauth/authorize?client_id={NOTION_OAUTH_CLIENT_ID}&redirect_uri={NOTION_REDIRECT_URI}&response_type=code&state={user.uuid}"
|
||||
|
||||
|
||||
async def run_in_executor(func, *args):
|
||||
loop = asyncio.get_event_loop()
|
||||
return await loop.run_in_executor(executor, func, *args)
|
||||
|
|
|
|||
|
|
@ -1,30 +1,21 @@
|
|||
# System Packages
|
||||
import json
|
||||
import os
|
||||
from datetime import timedelta
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.responses import FileResponse, HTMLResponse, RedirectResponse
|
||||
from fastapi.templating import Jinja2Templates
|
||||
from starlette.authentication import has_required_scope, requires
|
||||
from starlette.authentication import requires
|
||||
|
||||
from khoj.database import adapters
|
||||
from khoj.database.adapters import (
|
||||
AgentAdapters,
|
||||
ConversationAdapters,
|
||||
EntryAdapters,
|
||||
PublicConversationAdapters,
|
||||
get_user_github_config,
|
||||
get_user_name,
|
||||
get_user_notion_config,
|
||||
get_user_subscription_state,
|
||||
)
|
||||
from khoj.database.models import KhojUser
|
||||
from khoj.processor.speech.text_to_speech import is_eleven_labs_enabled
|
||||
from khoj.routers.helpers import get_next_url
|
||||
from khoj.routers.notion import get_notion_auth_url
|
||||
from khoj.routers.twilio import is_twilio_enabled
|
||||
from khoj.routers.helpers import get_next_url, get_user_config
|
||||
from khoj.utils import constants, state
|
||||
from khoj.utils.rawconfig import (
|
||||
GithubContentConfig,
|
||||
|
|
@ -42,80 +33,36 @@ templates = Jinja2Templates([constants.web_directory, constants.next_js_director
|
|||
@requires(["authenticated"], redirect="login_page")
|
||||
def index(request: Request):
|
||||
user = request.user.object
|
||||
user_picture = request.session.get("user", {}).get("picture")
|
||||
has_documents = EntryAdapters.user_has_entries(user=user)
|
||||
user_config = get_user_config(user, request)
|
||||
|
||||
return templates.TemplateResponse(
|
||||
"chat.html",
|
||||
context={
|
||||
"request": request,
|
||||
"username": user.username,
|
||||
"user_photo": user_picture,
|
||||
"is_active": has_required_scope(request, ["premium"]),
|
||||
"has_documents": has_documents,
|
||||
"khoj_version": state.khoj_version,
|
||||
},
|
||||
)
|
||||
return templates.TemplateResponse("chat.html", context=user_config)
|
||||
|
||||
|
||||
@web_client.post("/", response_class=FileResponse)
|
||||
@requires(["authenticated"], redirect="login_page")
|
||||
def index_post(request: Request):
|
||||
user = request.user.object
|
||||
user_picture = request.session.get("user", {}).get("picture")
|
||||
has_documents = EntryAdapters.user_has_entries(user=user)
|
||||
user_config = get_user_config(user, request)
|
||||
|
||||
return templates.TemplateResponse(
|
||||
"chat.html",
|
||||
context={
|
||||
"request": request,
|
||||
"username": user.username,
|
||||
"user_photo": user_picture,
|
||||
"is_active": has_required_scope(request, ["premium"]),
|
||||
"has_documents": has_documents,
|
||||
"khoj_version": state.khoj_version,
|
||||
},
|
||||
)
|
||||
return templates.TemplateResponse("chat.html", context=user_config)
|
||||
|
||||
|
||||
@web_client.get("/search", response_class=FileResponse)
|
||||
@requires(["authenticated"], redirect="login_page")
|
||||
def search_page(request: Request):
|
||||
user = request.user.object
|
||||
user_picture = request.session.get("user", {}).get("picture")
|
||||
has_documents = EntryAdapters.user_has_entries(user=user)
|
||||
user_config = get_user_config(user, request)
|
||||
|
||||
return templates.TemplateResponse(
|
||||
"search.html",
|
||||
context={
|
||||
"request": request,
|
||||
"username": user.username,
|
||||
"user_photo": user_picture,
|
||||
"is_active": has_required_scope(request, ["premium"]),
|
||||
"has_documents": has_documents,
|
||||
"khoj_version": state.khoj_version,
|
||||
},
|
||||
)
|
||||
return templates.TemplateResponse("search.html", context=user_config)
|
||||
|
||||
|
||||
@web_client.get("/chat", response_class=FileResponse)
|
||||
@requires(["authenticated"], redirect="login_page")
|
||||
def chat_page(request: Request):
|
||||
user = request.user.object
|
||||
user_picture = request.session.get("user", {}).get("picture")
|
||||
has_documents = EntryAdapters.user_has_entries(user=user)
|
||||
user_config = get_user_config(user, request)
|
||||
|
||||
return templates.TemplateResponse(
|
||||
"chat.html",
|
||||
context={
|
||||
"request": request,
|
||||
"username": user.username,
|
||||
"user_photo": user_picture,
|
||||
"is_active": has_required_scope(request, ["premium"]),
|
||||
"has_documents": has_documents,
|
||||
"khoj_version": state.khoj_version,
|
||||
},
|
||||
)
|
||||
return templates.TemplateResponse("chat.html", context=user_config)
|
||||
|
||||
|
||||
@web_client.get("/experimental", response_class=FileResponse)
|
||||
|
|
@ -169,25 +116,14 @@ def agents_page(request: Request):
|
|||
@web_client.get("/agent/{agent_slug}", response_class=HTMLResponse)
|
||||
def agent_page(request: Request, agent_slug: str):
|
||||
user: KhojUser = request.user.object if request.user.is_authenticated else None
|
||||
user_picture = request.session.get("user", {}).get("picture") if user else None
|
||||
|
||||
user_config = get_user_config(user, request)
|
||||
agent = AgentAdapters.get_agent_by_slug(agent_slug)
|
||||
has_documents = EntryAdapters.user_has_entries(user=user)
|
||||
|
||||
if agent == None:
|
||||
return templates.TemplateResponse(
|
||||
"404.html",
|
||||
context={
|
||||
"request": request,
|
||||
"khoj_version": state.khoj_version,
|
||||
"username": user.username if user else None,
|
||||
"has_documents": False,
|
||||
"is_active": has_required_scope(request, ["premium"]),
|
||||
"user_photo": user_picture,
|
||||
},
|
||||
)
|
||||
user_config["has_documents"] = False
|
||||
return templates.TemplateResponse("404.html", context=user_config)
|
||||
|
||||
agent_metadata = {
|
||||
user_config["agent"] = {
|
||||
"slug": agent.slug,
|
||||
"avatar": agent.avatar,
|
||||
"name": agent.name,
|
||||
|
|
@ -199,115 +135,23 @@ def agent_page(request: Request, agent_slug: str):
|
|||
"creator_not_self": agent.creator != user,
|
||||
}
|
||||
|
||||
return templates.TemplateResponse(
|
||||
"agent.html",
|
||||
context={
|
||||
"request": request,
|
||||
"agent": agent_metadata,
|
||||
"khoj_version": state.khoj_version,
|
||||
"username": user.username if user else None,
|
||||
"has_documents": has_documents,
|
||||
"is_active": has_required_scope(request, ["premium"]),
|
||||
"user_photo": user_picture,
|
||||
},
|
||||
)
|
||||
return templates.TemplateResponse("agent.html", context=user_config)
|
||||
|
||||
|
||||
@web_client.get("/config", response_class=HTMLResponse)
|
||||
@web_client.get("/settings", response_class=HTMLResponse)
|
||||
@requires(["authenticated"], redirect="login_page")
|
||||
def config_page(request: Request):
|
||||
user: KhojUser = request.user.object
|
||||
user_picture = request.session.get("user", {}).get("picture")
|
||||
has_documents = EntryAdapters.user_has_entries(user=user)
|
||||
user_config = get_user_config(user, request, is_detailed=True)
|
||||
|
||||
user_subscription_state = get_user_subscription_state(user.email)
|
||||
user_subscription = adapters.get_user_subscription(user.email)
|
||||
subscription_renewal_date = (
|
||||
user_subscription.renewal_date.strftime("%d %b %Y")
|
||||
if user_subscription and user_subscription.renewal_date
|
||||
else (user_subscription.created_at + timedelta(days=7)).strftime("%d %b %Y")
|
||||
)
|
||||
given_name = get_user_name(user)
|
||||
|
||||
enabled_content_source = set(EntryAdapters.get_unique_file_sources(user))
|
||||
successfully_configured = {
|
||||
"computer": ("computer" in enabled_content_source),
|
||||
"github": ("github" in enabled_content_source),
|
||||
"notion": ("notion" in enabled_content_source),
|
||||
}
|
||||
|
||||
selected_conversation_config = ConversationAdapters.get_conversation_config(user)
|
||||
conversation_options = ConversationAdapters.get_conversation_processor_options().all()
|
||||
all_conversation_options = list()
|
||||
for conversation_option in conversation_options:
|
||||
all_conversation_options.append({"chat_model": conversation_option.chat_model, "id": conversation_option.id})
|
||||
|
||||
search_model_options = adapters.get_or_create_search_models().all()
|
||||
all_search_model_options = list()
|
||||
for search_model_option in search_model_options:
|
||||
all_search_model_options.append({"name": search_model_option.name, "id": search_model_option.id})
|
||||
|
||||
current_search_model_option = adapters.get_user_search_model_or_default(user)
|
||||
|
||||
selected_paint_model_config = ConversationAdapters.get_user_text_to_image_model_config(user)
|
||||
paint_model_options = ConversationAdapters.get_text_to_image_model_options().all()
|
||||
all_paint_model_options = list()
|
||||
for paint_model in paint_model_options:
|
||||
all_paint_model_options.append({"model_name": paint_model.model_name, "id": paint_model.id})
|
||||
|
||||
notion_oauth_url = get_notion_auth_url(user)
|
||||
|
||||
eleven_labs_enabled = is_eleven_labs_enabled()
|
||||
|
||||
voice_models = ConversationAdapters.get_voice_model_options()
|
||||
voice_model_options = list()
|
||||
for voice_model in voice_models:
|
||||
voice_model_options.append({"name": voice_model.name, "id": voice_model.model_id})
|
||||
|
||||
if len(voice_model_options) == 0:
|
||||
eleven_labs_enabled = False
|
||||
|
||||
selected_voice_config = ConversationAdapters.get_voice_model_config(user)
|
||||
|
||||
return templates.TemplateResponse(
|
||||
"config.html",
|
||||
context={
|
||||
"request": request,
|
||||
"current_model_state": successfully_configured,
|
||||
"anonymous_mode": state.anonymous_mode,
|
||||
"username": user.username,
|
||||
"given_name": given_name,
|
||||
"search_model_options": all_search_model_options,
|
||||
"selected_search_model_config": current_search_model_option.id,
|
||||
"conversation_options": all_conversation_options,
|
||||
"selected_conversation_config": selected_conversation_config.id if selected_conversation_config else None,
|
||||
"paint_model_options": all_paint_model_options,
|
||||
"selected_paint_model_config": selected_paint_model_config.id if selected_paint_model_config else None,
|
||||
"user_photo": user_picture,
|
||||
"billing_enabled": state.billing_enabled,
|
||||
"subscription_state": user_subscription_state,
|
||||
"subscription_renewal_date": subscription_renewal_date,
|
||||
"khoj_cloud_subscription_url": os.getenv("KHOJ_CLOUD_SUBSCRIPTION_URL"),
|
||||
"is_active": has_required_scope(request, ["premium"]),
|
||||
"has_documents": has_documents,
|
||||
"is_twilio_enabled": is_twilio_enabled(),
|
||||
"is_eleven_labs_enabled": eleven_labs_enabled,
|
||||
"voice_model_options": voice_model_options,
|
||||
"selected_voice_config": selected_voice_config.model_id if selected_voice_config else None,
|
||||
"phone_number": user.phone_number,
|
||||
"is_phone_number_verified": user.verified_phone_number,
|
||||
"khoj_version": state.khoj_version,
|
||||
"notion_oauth_url": notion_oauth_url,
|
||||
},
|
||||
)
|
||||
return templates.TemplateResponse("settings.html", context=user_config)
|
||||
|
||||
|
||||
@web_client.get("/config/content-source/github", response_class=HTMLResponse)
|
||||
@web_client.get("/settings/content/github", response_class=HTMLResponse)
|
||||
@requires(["authenticated"], redirect="login_page")
|
||||
def github_config_page(request: Request):
|
||||
user = request.user.object
|
||||
user_picture = request.session.get("user", {}).get("picture")
|
||||
has_documents = EntryAdapters.user_has_entries(user=user)
|
||||
user_config = get_user_config(user, request)
|
||||
current_github_config = get_user_github_config(user)
|
||||
|
||||
if current_github_config:
|
||||
|
|
@ -329,66 +173,32 @@ def github_config_page(request: Request):
|
|||
else:
|
||||
current_config = {} # type: ignore
|
||||
|
||||
return templates.TemplateResponse(
|
||||
"content_source_github_input.html",
|
||||
context={
|
||||
"request": request,
|
||||
"current_config": current_config,
|
||||
"username": user.username,
|
||||
"user_photo": user_picture,
|
||||
"is_active": has_required_scope(request, ["premium"]),
|
||||
"has_documents": has_documents,
|
||||
"khoj_version": state.khoj_version,
|
||||
},
|
||||
)
|
||||
user_config["current_config"] = current_config
|
||||
return templates.TemplateResponse("content_source_github_input.html", context=user_config)
|
||||
|
||||
|
||||
@web_client.get("/config/content-source/notion", response_class=HTMLResponse)
|
||||
@web_client.get("/settings/content/notion", response_class=HTMLResponse)
|
||||
@requires(["authenticated"], redirect="login_page")
|
||||
def notion_config_page(request: Request):
|
||||
user = request.user.object
|
||||
user_picture = request.session.get("user", {}).get("picture")
|
||||
has_documents = EntryAdapters.user_has_entries(user=user)
|
||||
user_config = get_user_config(user, request)
|
||||
|
||||
current_notion_config = get_user_notion_config(user)
|
||||
|
||||
current_config = NotionContentConfig(
|
||||
token=current_notion_config.token if current_notion_config else "",
|
||||
)
|
||||
|
||||
token = current_notion_config.token if current_notion_config else ""
|
||||
current_config = NotionContentConfig(token=token)
|
||||
current_config = json.loads(current_config.model_dump_json())
|
||||
|
||||
return templates.TemplateResponse(
|
||||
"content_source_notion_input.html",
|
||||
context={
|
||||
"request": request,
|
||||
"current_config": current_config,
|
||||
"username": user.username,
|
||||
"user_photo": user_picture,
|
||||
"is_active": has_required_scope(request, ["premium"]),
|
||||
"has_documents": has_documents,
|
||||
"khoj_version": state.khoj_version,
|
||||
},
|
||||
)
|
||||
user_config["current_config"] = current_config
|
||||
return templates.TemplateResponse("content_source_notion_input.html", context=user_config)
|
||||
|
||||
|
||||
@web_client.get("/config/content-source/computer", response_class=HTMLResponse)
|
||||
@web_client.get("/settings/content/computer", response_class=HTMLResponse)
|
||||
@requires(["authenticated"], redirect="login_page")
|
||||
def computer_config_page(request: Request):
|
||||
user = request.user.object if request.user.is_authenticated else None
|
||||
user_picture = request.session.get("user", {}).get("picture") if user else None
|
||||
has_documents = EntryAdapters.user_has_entries(user=user) if user else False
|
||||
user_config = get_user_config(user, request)
|
||||
|
||||
return templates.TemplateResponse(
|
||||
"content_source_computer_input.html",
|
||||
context={
|
||||
"request": request,
|
||||
"username": user.username,
|
||||
"user_photo": user_picture,
|
||||
"is_active": has_required_scope(request, ["premium"]),
|
||||
"has_documents": has_documents,
|
||||
"khoj_version": state.khoj_version,
|
||||
},
|
||||
)
|
||||
return templates.TemplateResponse("content_source_computer_input.html", context=user_config)
|
||||
|
||||
|
||||
@web_client.get("/share/chat/{public_conversation_slug}", response_class=HTMLResponse)
|
||||
|
|
@ -404,8 +214,9 @@ def view_public_conversation(request: Request):
|
|||
},
|
||||
)
|
||||
user = request.user.object if request.user.is_authenticated else None
|
||||
user_picture = request.session.get("user", {}).get("picture") if user else None
|
||||
has_documents = EntryAdapters.user_has_entries(user=user) if user else False
|
||||
user_config = get_user_config(user, request)
|
||||
user_config["public_conversation_slug"] = public_conversation_slug
|
||||
user_config["google_client_id"] = os.environ.get("GOOGLE_CLIENT_ID")
|
||||
|
||||
all_agents = AgentAdapters.get_all_accessible_agents(request.user.object if request.user.is_authenticated else None)
|
||||
|
||||
|
|
@ -420,28 +231,15 @@ def view_public_conversation(request: Request):
|
|||
"name": agent.name,
|
||||
}
|
||||
)
|
||||
user_config["agents"] = agents_packet
|
||||
|
||||
google_client_id = os.environ.get("GOOGLE_CLIENT_ID")
|
||||
redirect_uri = str(request.app.url_path_for("auth"))
|
||||
next_url = str(
|
||||
request.app.url_path_for("view_public_conversation", public_conversation_slug=public_conversation_slug)
|
||||
)
|
||||
user_config["redirect_uri"] = f"{redirect_uri}?next={next_url}"
|
||||
|
||||
return templates.TemplateResponse(
|
||||
"public_conversation.html",
|
||||
context={
|
||||
"request": request,
|
||||
"username": user.username if user else None,
|
||||
"user_photo": user_picture,
|
||||
"is_active": has_required_scope(request, ["premium"]),
|
||||
"has_documents": has_documents,
|
||||
"khoj_version": state.khoj_version,
|
||||
"public_conversation_slug": public_conversation_slug,
|
||||
"agents": agents_packet,
|
||||
"google_client_id": google_client_id,
|
||||
"redirect_uri": f"{redirect_uri}?next={next_url}",
|
||||
},
|
||||
)
|
||||
return templates.TemplateResponse("public_conversation.html", context=user_config)
|
||||
|
||||
|
||||
@web_client.get("/automations", response_class=HTMLResponse)
|
||||
|
|
@ -452,20 +250,9 @@ def automations_config_page(
|
|||
queryToRun: Optional[str] = None,
|
||||
):
|
||||
user = request.user.object if request.user.is_authenticated else None
|
||||
user_picture = request.session.get("user", {}).get("picture")
|
||||
has_documents = EntryAdapters.user_has_entries(user=user) if user else False
|
||||
user_config = get_user_config(user, request)
|
||||
user_config["subject"] = subject if subject else ""
|
||||
user_config["crontime"] = crontime if crontime else ""
|
||||
user_config["queryToRun"] = queryToRun if queryToRun else ""
|
||||
|
||||
return templates.TemplateResponse(
|
||||
"config_automation.html",
|
||||
context={
|
||||
"request": request,
|
||||
"username": user.username if user else None,
|
||||
"user_photo": user_picture,
|
||||
"is_active": has_required_scope(request, ["premium"]),
|
||||
"has_documents": has_documents,
|
||||
"khoj_version": state.khoj_version,
|
||||
"subject": subject if subject else "",
|
||||
"crontime": crontime if crontime else "",
|
||||
"queryToRun": queryToRun if queryToRun else "",
|
||||
},
|
||||
)
|
||||
return templates.TemplateResponse("config_automation.html", context=user_config)
|
||||
|
|
|
|||
|
|
@ -269,7 +269,7 @@ def test_get_api_config_types(client, sample_org_data, default_user: KhojUser):
|
|||
text_search.setup(OrgToEntries, sample_org_data, regenerate=False, user=default_user)
|
||||
|
||||
# Act
|
||||
response = client.get(f"/api/config/types", headers=headers)
|
||||
response = client.get(f"/api/configure/types", headers=headers)
|
||||
|
||||
# Assert
|
||||
assert response.status_code == 200
|
||||
|
|
@ -289,7 +289,7 @@ def test_get_configured_types_with_no_content_config(fastapi_app: FastAPI):
|
|||
client = TestClient(fastapi_app)
|
||||
|
||||
# Act
|
||||
response = client.get(f"/api/config/types")
|
||||
response = client.get(f"/api/configure/types")
|
||||
|
||||
# Assert
|
||||
assert response.status_code == 200
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue