Commit graph

2998 commits

Author SHA1 Message Date
sabaimran
a1263951e9 Use mail to in email contact link 2024-05-03 12:16:56 +05:30
sabaimran
7c9847fe48 Increase jitter to 60 2024-05-03 11:38:22 +05:30
sabaimran
737ebfd521 Make improvements to online search prompts and use a custom domain for automations emails 2024-05-03 10:47:42 +05:30
sabaimran
42e9504ba8 Use a different function for getting last run time, avoid async/sync issues 2024-05-02 12:13:45 +05:30
sabaimran
9e8491b814 Add experimental disclaimers to the automations 2024-05-02 11:40:37 +05:30
sabaimran
c418449311 Add additional robustness in verifying job execution parameters at run time 2024-05-02 11:13:04 +05:30
sabaimran
690e9d8ed3 Collapse the reminders after they're successfully scheduled 2024-05-02 09:55:04 +05:30
sabaimran
6b648ee3ad Add experimental disclaimer in the automation page 2024-05-02 09:21:27 +05:30
sabaimran
f4fbc91515 Remove the exclamation point from the email 2024-05-01 19:01:51 +05:30
sabaimran
bddd1d0fcb Quip, smart reminders 2024-05-01 16:39:07 +05:30
sabaimran
bc8b92a77d Release Khoj version 1.12.0 2024-05-01 16:30:48 +05:30
sabaimran
9d02c354dd
Merge pull request #732 from khoj-ai/fit-and-finish/schedule-tasks
Fixes and improves for scheduled tasks
2024-05-01 03:16:09 -07:00
sabaimran
b499851097 Use the cleaned query as the reference query in the email notification 2024-05-01 15:33:11 +05:30
sabaimran
f24495e0e6 Fix time zone used in query history. Closes #694 2024-05-01 15:31:48 +05:30
sabaimran
7fd57d737e Adjustments to improve overall styling of config page, email template 2024-05-01 14:19:47 +05:30
sabaimran
28578310d1 Add log line when sending a task-related email 2024-05-01 13:56:02 +05:30
sabaimran
a86f95117e Add the subject generation prompt and helper method 2024-05-01 13:55:32 +05:30
sabaimran
c30ba2e551 Set subject dynamically when creating new tasks, and make some minor improvments to the automations UI 2024-05-01 13:54:59 +05:30
sabaimran
d1b2037676 Shutdown the scheduler when the application is exiting 2024-05-01 13:53:34 +05:30
Debanjum
10f623154e
Enable Creating Automations from Khoj (#731)
## Support Scheduling Automations (#695) 
   1. Detect when user intends to schedule a task, aka reminder
      - Support new `reminder` output mode to the response type chat actor
      - Show examples of selecting the reminder output mode to the response type chat actor
   2. Extract schedule time (as cron timestring) and inferred query to run from user message
   3. Use APScheduler to call chat with inferred query at scheduled time

## Make Automations Persistent (#714) 
  - Make scheduled jobs persistent and work in multiple worker setups
  - Add new operation Scheduled Job to Operation enum of ProcessLock

## Add UX to Configure Scheduled Tasks (#715)
  - Add section in settings page to view, delete your scheduled tasks
  - Add API endpoints to get and delete user scheduled tasks

## Make Automations more Robust. Improve UX (#718)
  - Decouple Task Run from User Notification
  - Make Scheduling more Robust
    - Use JSON mode to get parse-able output from chat model
    - Make timezone calculation programmatic on server instead of asking chat model
    - Use django-apscheduler to handle apscheduler and django interfacing
  - Improve automation UX. Move it out into separate top level page
    - Allow creating, modifying automations from the automations page
    - Infer cron from natural language client side to avoid roundtrip
2024-05-01 11:08:19 +05:30
Debanjum Singh Solanky
89a8dbb81a Fix edit job API. Use user timezone, pass all reqd. params to automation
- Pass user and calling_url to the scheduled chat too when modifying
  params of automation
- Update to use user timezone even when update job via API
- Move timezone string to timezone object calculation into the
  schedule automation method
2024-05-01 10:29:49 +05:30
Debanjum Singh Solanky
19c5af3ebc Handle natural language to cron translation error on web client 2024-05-01 09:10:18 +05:30
Debanjum Singh Solanky
70ee9ddf91 Merge migrations from main with feature branch 2024-05-01 09:10:18 +05:30
Debanjum Singh Solanky
8f28f6cc1e Remove now unused location data from being passed to automation funcs 2024-05-01 08:48:16 +05:30
Debanjum Singh Solanky
815966cb25 Unify, modularize DB adapters to get automation metadata by user further 2024-05-01 08:47:50 +05:30
Debanjum Singh Solanky
21bdf45d6f Add link to Automate page in nav pane of the web app 2024-05-01 08:47:50 +05:30
Debanjum Singh Solanky
bd5008136a Move automations into independent page. Allow direct automation
- Previously it was a section in the settings page. Move it to
  independent, top-level page to improve visibility of feature

- Calculate crontime from natural language on web client before
  sending it to to server for saving new/updated schedule to disk.
  - Avoids round-trip of call to chat model

- Convert POST /api/automation API endpoint into a direct request for
  automation with query_to_run, subject and schedule provided via the
  automation page. This allows more granular control to create automation
  - Make the POST automations endpoint more robust; runs validation
    checks, normalizes parameters
2024-05-01 08:47:48 +05:30
Debanjum Singh Solanky
cbc8a02179 Make, use func for constructing the automation created response
- Dedupe logic across http, ws chat API endpoints
- Reduces size of already too long http, ws chat API endpoint funcs
2024-05-01 08:30:10 +05:30
Debanjum Singh Solanky
c52ed333fa Make content, cards on config pages occupy the whole middle column
- Make the config page content use  the same top level 3-column layout
  as the khoj-header-wrapper
  This ensures the content is aligned with heading pane width
- Let cards and other settings sections scale to the width of their
  grid element. This utilizes more of the screen space and does it
  consistently across the different settings pages
2024-05-01 08:30:10 +05:30
sabaimran
ad4145e48c Fix unique has for job id 2024-05-01 08:30:10 +05:30
sabaimran
311d58e1ed Ensure the automated_task command is removed from the prepended query 2024-05-01 08:30:10 +05:30
sabaimran
eb65532386 Use Django ap scheduler in place of the sqlalchemy one 2024-05-01 08:30:10 +05:30
sabaimran
06213ea814 Fix token retrieval when executing the job and name async job approriately 2024-05-01 08:30:10 +05:30
sabaimran
ca8a7d8368 Revert sync -> aync in send welcome email method 2024-05-01 08:30:10 +05:30
Debanjum Singh Solanky
6936875a82 Use DB adapter to unify logic to get, delete automation by auth user
To use place with logic to get, view, delete (and edit soon) automations
by (authenticated) user, instead of scattered across code
2024-05-01 08:30:10 +05:30
Debanjum Singh Solanky
1238cadd31 Allow editting query-to-run from the automation config section 2024-05-01 08:30:10 +05:30
Debanjum Singh Solanky
cb2b1dccc5 Add icon for Automation feature. Replace old icons for delete, new 2024-05-01 08:30:10 +05:30
Debanjum Singh Solanky
23f2057868 Allow creating automations from automation settings section in web ui
- Create new POST API endpoint to create automations
- Use it in the settings page on the web interface to create
  new automations

This simplified managing automations from the setting page by allowing
both delete and create from the same page
2024-05-01 08:30:10 +05:30
Debanjum Singh Solanky
2f9241b5a3 Rename scheduled task to automations across code and UX
- Fix query, subject parameters passed to email template
- Show 12 hour scheduled time in automation created chat message
2024-05-01 08:30:10 +05:30
Debanjum Singh Solanky
230d160602 Improve rendering task scheduled settings view and message
- Render crontime string in natural language in message & settings UI
- Show more fields in tasks web config UI
- Add link to the tasks settings page in task scheduled chat response
- Improve task variables names
  Rename executing_query to query_to_run. scheduling_query to
  scheduling_request
2024-05-01 08:30:10 +05:30
Debanjum Singh Solanky
d341b1efe8 Store, retrieve task metadata from the job name field 2024-05-01 08:30:10 +05:30
Debanjum Singh Solanky
ae10ff4a5f Create create_scheduled_task func to dedupe logic across ws, http APIs
Previously, both the websocket and http endpoint were implementing
the same logic. This was becoming too unwieldy
2024-05-01 08:30:10 +05:30
Debanjum Singh Solanky
8dfa0bf047 Simplify task scheduler prompt. No timezone conversion. Infer subject
- Make timezone aware scheduling programmatic, instead of asking the
  chat model to do the conversion. This removes the need for
  scratchpad and may let smaller models handle the task as well
- Make chat model infer subject for email. This should make the
  notification email more readable
- Improve email by using subject in email subject, task heading. Move
  query to email final paragraph, which is where task metadata  should
  go
2024-05-01 08:30:10 +05:30
Debanjum Singh Solanky
2c563ad280 Use hash of query in process lock id to standardize id format
- Using inferred_query directly was brittle (like previous job id)
- And process lock id had a limited size, so wouldn't work for larger
  inferred query strings
2024-05-01 08:30:10 +05:30
Debanjum Singh Solanky
3ce06a938c Render scheduled task response as html to improve readability in email 2024-05-01 08:30:10 +05:30
Debanjum Singh Solanky
c17dbbeb92 Render next run time in user timezone in config, chat UIs
- Pass timezone string from ipapi to khoj via clients
  - Pass this data from web, desktop and obsidian clients to server
- Use user tz to render next run time of scheduled task in user tz
2024-05-01 08:30:10 +05:30
Debanjum Singh Solanky
6736551ba3 Improve scheduled task text rendered in UI 2024-05-01 08:30:10 +05:30
Debanjum Singh Solanky
0e01362469 Merge DB migrations from master with those from scheduled task feature 2024-05-01 08:30:10 +05:30
Debanjum Singh Solanky
a5ed4f2af2 Send email to share results of scheduled task 2024-05-01 08:30:10 +05:30
Debanjum Singh Solanky
69775b6d6e Add /task command. Use it to disable scheduling tasks from tasks
This takes the load of the task scheduling chat actor / prompt from
having to artifically differentiate query to create scheduled task
from a scheduled task run.
2024-05-01 08:30:10 +05:30