Commit graph

170 commits

Author SHA1 Message Date
Debanjum Singh Solanky
5c2327ee4f Configure org directories to index from khoj.el
Converts paths to glob style regexes that will index all org files
recursively under the specified list of path

Should help setup for org-roam users from khoj.el
2023-03-27 18:30:53 +07:00
Debanjum Singh Solanky
6e8a40906d Allow disabling automatic server setup. Fix server start vs ready logic
- khoj-auto-setup controls whether to automatically check for and
  setup khoj server from within Emacs
- extract install, start, configure sequence into public, interactive
  method. Allows calling khoj-setup during package load via init.el

- Fix: Do not attempt to configure or wait for server ready if
  user has said no to auto-setup request
- Fix logic to mark server started vs ready
  - Previously the started/running vs ready variables defs were getting
    intertwined
  - Server started indicates server bootup has been triggered
  - Server ready indicates server API ready to accept requests
2023-03-27 17:53:08 +07:00
Debanjum Singh Solanky
526a927bce Fix org entry extraction test, variable prefixed with khoj in khoj.el
Discovered via failing build and test workflows on Github
2023-03-27 16:44:50 +07:00
Debanjum Singh Solanky
7243059507 Track index update asynchronously via moon phase progressbar in khoj.el 2023-03-27 06:01:04 +07:00
Debanjum Singh Solanky
8a9055f918 Restrict server messages show in echo area to main server files 2023-03-27 04:59:55 +07:00
Debanjum Singh Solanky
ae535a06eb Configure Khoj chat using khoj.el by setting OpenAI API key in Emacs 2023-03-27 04:59:54 +07:00
Debanjum Singh Solanky
36b17d4ae0 Generalize the directory from config extraction elisp method 2023-03-27 03:44:03 +07:00
Debanjum Singh Solanky
924424c754 Throw actionable exceptions when content types or chat not configured 2023-03-27 02:47:44 +07:00
Debanjum Singh Solanky
359a2cacef Fix khoj--server-running to work with unconfigured or external server
- If khoj server started outside emacs, khoj--server-ready should be set
to true by khoj--server-running method (instead of waiting for proc msg)

- If khoj server is unconfigured the /config/types endpoint wouldn't
return anything. Using config/data/default allows checking khoj server
running status without requiring it to be configured as well
2023-03-27 02:45:59 +07:00
Debanjum Singh Solanky
d7fb9a596e Auto configure server before loading khoj-menu
If the config hasn't changed there'll be no update. If config has
changed indexing will get triggered asynchronously. But user cannot
make query till indexing done

As easier to know when server ready to configure
2023-03-27 02:44:02 +07:00
Debanjum Singh Solanky
8a21aff438 Make khoj.el server start, stop, restart, setup methods interactive
No need to erase temporary buffers before working on them
2023-03-27 01:53:15 +07:00
Debanjum Singh Solanky
cb40a96c85 Index configured org files from khoj.el
- Set `khoj-org-files-index' to list of files to index
- Defaults to indexing org-agenda-files
- Uses khoj server api to configure org files to index
2023-03-27 01:05:26 +07:00
Debanjum Singh Solanky
50760acc37 Wait for Khoj server to get ready before opening khoj.el transient menu
- Use process filter, sentinel to mark when khoj server is ready or not
- Display server messages for visibility into server boot-up process
- Wait until server ready to open khoj transient menu in Emacs
  Until then khoj features wouldn't work anyway, so avoids confusion
2023-03-26 13:00:01 +07:00
Debanjum Singh Solanky
82eb4bfd0d Setup Khoj server on opening khoj from with Emacs
- Create helper methods to check, stop, restart, setup khoj server
- (Ask to) setup khoj server on calling khoj main entrypoint function
2023-03-26 10:12:06 +07:00
Debanjum Singh Solanky
99d19dcf43 Start Khoj server from Emacs using khoj.el 2023-03-26 09:38:46 +07:00
Debanjum Singh Solanky
c92d79118a Install Khoj server from Emacs using khoj.el 2023-03-26 08:50:03 +07:00
Debanjum Singh Solanky
e281a498b4 Style Khoj search org buffer via elisp instead of in-buffer settings 2023-03-26 06:34:18 +07:00
Debanjum Singh Solanky
4f655d20ae Style Khoj chat directly via elisp instead of via in-buffer settings 2023-03-26 06:03:30 +07:00
Debanjum Singh Solanky
f6ff7b1beb Render foonote reference links as superscript for Khoj Chat on Emacs 2023-03-26 05:33:08 +07:00
Debanjum Singh Solanky
ff846f05c5 Clean-up khoj.el based on linting helpers and manual review 2023-03-25 05:47:49 +07:00
Debanjum Singh Solanky
4725416fbd Use shortcut keybindings in buffer to ease sending messages to Khoj 2023-03-25 05:06:01 +07:00
Debanjum Singh Solanky
508b2176b7 Update Chat API, Logs, Interfaces to store, use references as list
- Remove the need to split by magic string in emacs and chat interfaces
- Move compiling references into string as context for GPT to GPT layer
- Update setup in tests to use new style of setting references
- Name first argument to converse as more appropriate "references"
2023-03-24 22:10:11 +07:00
Debanjum Singh Solanky
b08745b541 Keep chat messages at 1 empty line visible distance in khoj.el
- Clean redundant concat, format string
- Improve variable name to emojified sender
2023-03-24 22:10:11 +07:00
Debanjum Singh Solanky
5e9558d39d Stylize references shown as footnote links in chat messages
- Render references as superscript
- Show reference definitions on hover over reference links to ease access
- Truncate reference def shown on hover to 70 char
  - Add continuation suffix, ..., when reference definition truncated
2023-03-24 20:38:05 +07:00
Debanjum Singh Solanky
cf28f104c7 Register separate timestamps for user query and response by Khoj Chat 2023-03-24 18:31:58 +07:00
Debanjum Singh Solanky
93e2aff786 Add references as org footnotes instead of links 2023-03-24 18:31:42 +07:00
Debanjum Singh Solanky
d78454d4ad Load Khoj Chat buffer before asking for query to provide context 2023-03-24 13:43:46 +07:00
Debanjum Singh Solanky
863933daaa Resolve build issues found by melpazoid 2023-03-23 02:25:34 +04:00
Debanjum Singh Solanky
06df394d6c Style chat messages as org-mode entries in Emacs
- Style Message as Org Entries instead of List
- Put khoj response as child of user query entry
  - Improves color coding for readability
  - Allows folding each back-n-forth
- Put timestamp of message received into property drawer
- Use standardized time format for new and old chat messages
2023-03-22 12:00:43 -06:00
Debanjum Singh Solanky
364e6c11af Render chat history from API in chat buffer on first run
- Generalize the render-chat-response method to handle rendering
  history or chat response from chat API reponse

- Trigger rendering of khoj chat history if Khoj chat buffer not
  created for this session yet
2023-03-22 12:00:35 -06:00
Debanjum Singh Solanky
36b52fdd0a Properly escape reference links before rendering
- Use org-insert-link method to improve link rendering robustness
  Previous simple mechanism to crete org-links would result in links
  escaping out of formating. Use a user-facing org-mode method to
  remove/reduce probability of this

- Replace newlines with space to render reference notes as links
2023-03-22 11:05:38 -06:00
Debanjum Singh Solanky
72f63a6ef7 Add basic chat interface for Khoj on Emacs
- Query khoj chat API to get Khoj Chat response to user message
- Render chat messages as a org-mode list in format:
  - [sender-name]: *[message]*
    - /[receive-date]/
- Add references as org links with context visible on hover,
  but no jump to note
- Require dash library for khoj.el to simplify list manipulation.
  Use `-map-indexed' method from dash
2023-03-22 10:47:55 -06:00
Debanjum Singh Solanky
e4d67694e1 Add search to method, variable names meant for khoj search in khoj.el
In preparation to introduce Khoj chat in Emacs
2023-03-21 21:44:11 -06:00
Debanjum Singh Solanky
8bb8824d0c Bump khoj versions in obsidian, emacs files 2023-03-10 15:23:17 -06:00
Debanjum Singh Solanky
b177adf3a7 Return value of search_type in /config/type API endpoint
- Remove need for interfaces to downcase content types returned by API
  before using the type in search and other API endpoint
- Fix to check for search_type.name in plugin keys instead of value
2023-02-28 21:49:26 -06:00
Debanjum Singh Solanky
c2814fce58 Improve rendering search results of plugin content types in khoj.el
Render only the entry from plugin search response instead of raw json
2023-02-28 20:25:51 -06:00
Debanjum Singh Solanky
1e43f1a12e Use new config/types API to set enabled content types in khoj.el menu 2023-02-28 20:25:51 -06:00
Debanjum Singh Solanky
61b6ee2857 Use helper script to bump khoj pre-release versions 2023-02-17 20:31:51 -06:00
Debanjum Singh Solanky
3ec41c4d64 Wrap lines for org, markdown results in khoj search results buffer 2023-02-12 07:33:50 -06:00
Debanjum Singh Solanky
215235efd2 Bump khoj pre-release version 2023-02-08 20:24:36 -03:00
Debanjum Singh Solanky
2e13e15625 Ensure markdown entries in khoj.el results separated by empty line
- Update khoj.el test to reflect updated rendering logic
- Move ledger render function before image rendered to group functions
  with similar logic closer
2023-01-26 19:13:02 -03:00
Debanjum Singh Solanky
85ae46f429 Use thread_last to make results rendering funcs more readable in khoj.el 2023-01-26 18:59:44 -03:00
Debanjum Singh Solanky
0d0bf3b5aa Simplify get-current-text functions for Find Similar in khoj.el
Use existing functions like `string-trim', `thing-at-point' and
remove unneeded code from the two functions
2023-01-23 19:15:52 -03:00
Debanjum Singh Solanky
07e9e4ecc3 Get current paragraph text when point at start of paragraph in khoj.el
Previously if cursor was at start of current paragraph, it would get
text for the current and next paragraph, instead of just the current one
2023-01-23 18:05:54 -03:00
Debanjum Singh Solanky
a0b03c8bb1 Get current entry text when point at heading for Find Similar in khoj.el
Previously if cursor was at heading of current entry, it would find entries
similar to the previous outline heading, instead of the current one
2023-01-23 10:01:25 -03:00
Debanjum Singh Solanky
013c7c10a4 Bump khoj pre-release version 2023-01-22 18:45:56 -03:00
Debanjum Singh Solanky
ad3c9b5f44 Bump khoj version to 0.2.5 in preparation for release 2023-01-22 18:18:21 -03:00
Debanjum Singh Solanky
b7aa22a059 Change order of arg passed to query-api-and-render-results by importance 2023-01-20 22:13:24 -03:00
Debanjum Singh Solanky
936a88fa7e Find items of specified type similar to current text item at point
- Support querying with text surrounding point in any text buffer
  Previously could only find items similar to org entry at point

- Find similar items of specified content type indexed on khoj
  Previously only looked for similar org entries indexed on khoj

  Now uses the content-type configured in khoj transient menu to find
  items of the specified content type

- Details
  - Generalize the get-current-org-entry-text func to get text for any
    outline section
  - Replace leading whitespaces from query text as well
  - Create method to get current paragraph text from non-outline mode
    buffers
  - Update transient, find-similar funcs to pass, use content-type
    configured in khoj transient menu
  - Generalize query title creation logic to remove markdown headings
    prefix (#) apart from org heading prefix (*) as well
  - Update last used khoj content-type and results from the
    find-similar and update funcs for later reuse
  - Jump to top of results buffer after results rendered
2023-01-20 22:12:54 -03:00
Debanjum Singh Solanky
17aaadea1f Find notes similar to current org entry at point 2023-01-20 05:14:54 -03:00
Debanjum Singh Solanky
44bbc0a417 Add section separators to khoj.el for easier code traversal 2023-01-19 23:36:54 -03:00
Debanjum Singh Solanky
48ad3c535e Use default content types if fail to call backend on khoj.el load
Do not want khoj.el to fail on init/load if khoj backend not running
2023-01-19 20:13:49 -03:00
Debanjum Singh Solanky
0dd1cba272 Rename configuration sections in khoj.el transient menu 2023-01-19 03:03:08 -03:00
Debanjum Singh Solanky
5d0f369186 Add ability to quit khoj transient with standard q keybinding 2023-01-19 02:47:07 -03:00
Debanjum Singh Solanky
87c7cf4272 Use single khoj func as entrypoint. Group khoj.el code into sections
- Give more relevant, specific name to khoj suffix commands
- Remove `khoj-simple'. Have single `khoj' function for entrypoint
2023-01-19 02:38:19 -03:00
Debanjum Singh Solanky
9d64a009fd Allow updating khoj content index from within khoj.el
- Split transient config menu by type
2023-01-18 23:07:59 -03:00
Debanjum Singh Solanky
a8d0c7d905 Rename search type to more apt content type in khoj.el 2023-01-18 22:13:49 -03:00
Debanjum Singh Solanky
00daea16df Allow setting default-search-type to image. Make docstrings compact 2023-01-18 22:01:17 -03:00
Debanjum Singh Solanky
216b17cfd0 Dynamically populate content type choices when khoj transient invoked 2023-01-18 22:00:56 -03:00
Debanjum Singh Solanky
5f446b1440 Convert main khoj.el entrypoint into transient menu for richer configuration 2023-01-18 21:50:07 -03:00
Debanjum
abd035e2fa
Merge PR #112 to fix quote usage in khoj.el docstring from suliveevil/master
Fix usage warning for unescaped single quote in `khoj.el' docstring. 
Converts usage of '<text>' into `<text>' to use the correct quote forms in generated docs
2023-01-05 13:24:11 -03:00
Debanjum Singh Solanky
e792523849 Bump version in metadata packages for khoj, khoj.el and obsidian plugin 2023-01-05 12:50:27 -03:00
suliveevil
b2812b409f
fix docstring usage warning
 Warning (comp): khoj.el:119:2: Warning: docstring has wrong usage of unescaped single quotes (use \= or different quoting)
 Warning (comp): khoj.el:120:2: Warning: docstring has wrong usage of unescaped single quotes (use \= or different quoting)
 Warning (comp): khoj.el:121:2: Warning: docstring has wrong usage of unescaped single quotes (use \= or different quoting)
 Warning (comp): khoj.el:168:2: Warning: docstring has wrong usage of unescaped single quotes (use \= or different quoting)
2023-01-05 16:47:38 +08:00
Debanjum Singh Solanky
da17ff6ac8 Add Upgrade instructions for Khoj.el Readme. Fix version of khoj.el 2023-01-04 20:06:39 -03:00
Debanjum Singh Solanky
08dc5e3324 Update instructions in khoj.el to install it from MELPA stable
- The instructions suggest installing khoj-assistant via pip install.
  This installs the latest tagged/release version of khoj
- To match that version user should install khoj.el from MELPA stable
  instead of MELPA
2022-12-23 19:08:38 -03:00
Debanjum Singh Solanky
23ca5a2d43 Improve (un-)quoting of funcs used in `khoj--get-enabled-content-types'
- Based on melpa package feedback for khoj.el
- Verified these changes don't affect behavior of the function
2022-12-19 18:02:23 -03:00
Debanjum Singh Solanky
5db3a67df5 Fix Khoj Emacs package URL in khoj.el 2022-12-14 22:49:19 -03:00
Debanjum Singh Solanky
abad6d5f44 Declare external khoj.el funcs. Remove undefined func warnings on install 2022-12-14 22:36:04 -03:00
Debanjum Singh Solanky
1990d09032 Bump khoj version in setup.py, khoj.el to 0.2.0 2022-12-02 14:58:54 -03:00
Debanjum Singh Solanky
1c40f97114 Merge branch 'master' of github.com:debanjum/khoj into modularize-api-and-increase-typing
- Conflicts:
  - src/interface/emacs/khoj.el
    Use our update to `config-url', use their `url-request-method'
2022-10-19 16:46:53 +05:30
Debanjum Singh Solanky
d292bdcc11 Do not version API. Premature given current state of the codebase
- Reason
  - All clients that currently consume the API are part of Khoj
  - Any breaking API changes will be fixed in clients immediately
  - So decoupling client from API is not required
  - This removes the burden of maintaining muliple versions of the API
2022-10-08 16:32:46 +03:00
Debanjum Singh Solanky
0521ea10d6 Put image score breakdown under `additional' field in search response
- Update web, emacs interfaces to consume the scores from new schema
2022-10-08 12:06:01 +03:00
Debanjum Singh Solanky
e42a38e825 Version Khoj API, Update frontends, tests and docs to reflect it
- Split router.py into v1.0, beta and frontend (no-prefix) api modules
  under new router package. Version tag in main.py via prefix
- Update frontends to use the versioned api endpoints
- Update tests to work with versioned api endpoints
- Update docs to mentioned, reference only versioned api endpoints
2022-09-28 20:08:38 +03:00
Robert Irelan
d25e1d8e86
fix: explicitly set url-request-method
In my installation, it appears that `url-request-method` is sometimes set
globally to POST.  Need to explicitly set it to ensure that GET is always
used as intended.
2022-09-19 15:46:46 -04:00
Debanjum Singh Solanky
4d776d9c7a Bump khoj version to 0.1.9 2022-09-09 07:50:15 +03:00
Debanjum Singh Solanky
dc8dcc94a6 Bump Khoj.el package version to 0.1.6 2022-08-19 20:48:42 +03:00
Debanjum Singh Solanky
2647e6bab4 Display re-ranked results triggered via keybinding in khoj.el
- Prevent immediate overwrite of re-ranked results by
  incremental-search without rerank triggered via post-command-hook.

- This triggers right after the reranking results are rendered, so
  user never ends up seeing them
2022-08-15 18:41:12 +03:00
Debanjum Singh Solanky
a91d2df300 Simplify Emacs interface to only rerank results on explicit command 2022-08-15 06:20:13 +03:00
Debanjum Singh Solanky
e846829a2e Reset Khoj.el version to align with Khoj package version 2022-08-15 06:20:13 +03:00
Debanjum Singh Solanky
541e03da3d Make khoj.el pass checkdoc, package-lint, flycheck checks
- Add docstrings, mention args in them. Make docstring crisper
- prefix funcs, variables with khoj--
- Require emacs >27.1 for json-parse-buffer
- Use lexical binding
- Add quickstart docs to elisp file itself
- Bump version of khoj.el
2022-08-13 21:37:41 +03:00
Debanjum Singh Solanky
eddf88f818 Org buffer customization settings to tail of khoj.el results buffer
- Results get priority screen real estate
- Allows quick speed key based traversal of results as cursor
  on switching to buffer is at top level heading
  - E.g C-x o n n o 2 jumps to entry in actual file of second result
  - Unlike before when it is at the #+STARTUP org buffer customization
    settings
2022-08-10 12:57:37 +03:00
Debanjum Singh Solanky
57026b802c Set size of rendered images using user customizable vars 2022-08-09 21:06:37 +03:00
Debanjum Singh Solanky
0a758c9f0f By default, wait for 2 seconds before initiating rerank in khoj.el
- Subjectively, previous default seems to aggressive based on usage
  Doesn't give time for user to think and type their query
2022-08-09 21:06:30 +03:00
Debanjum Singh Solanky
f01fb16ebb Use single hyphen in name of user configurable variables in khoj.el
- Follow convention, two hyphens indicate variable private to library
- Defcustom are user configurable variables. So they should have single -
- Use khoj-results-count variable directly in code
2022-08-09 20:49:34 +03:00
Debanjum Singh Solanky
dddc57e132 Rename get-enabled-search-types to get-enabled-content-types as more appropriate 2022-08-07 18:53:14 +03:00
Debanjum Singh Solanky
127c6e78df Only show keybindings for enabled search types in simple info menu too
Convert the khoj--keybindings-info-message into a func
Dynamically generate info menu
Show keybindings for enabled search types only
2022-08-07 18:40:35 +03:00
Debanjum Singh Solanky
d08c25b62b Make default search type used in the Emacs interface configurable 2022-08-07 18:24:53 +03:00
Debanjum Singh Solanky
5a10c47499 Allow setting music as search type in khoj.el. Had forgotten to include it earlier 2022-08-07 18:24:53 +03:00
Debanjum Singh Solanky
ebee716026 Only show keybindings reference for enabled search types in khoj.el 2022-08-07 18:24:53 +03:00
Debanjum Singh Solanky
6dc9801f45 Get Khoj search-types enabled by user in Emacs 2022-08-07 18:24:53 +03:00
Debanjum Singh Solanky
f3c1512c38 Fix to let user to start enter query right after initiating khoj on emacs
- Fix regression since moving to use `which-key-show-full-keymap~
- The above function reads user keypress, so eats up 1 keypress
  before starting to enter query
- No way to pass no-paging config via the external function to the
  internally used which-key--show-keymap function that does allow
  setting no-paging to not read user keypress
- So use the internal function instead and set no-paging arg to t
2022-08-07 15:57:08 +03:00
Debanjum Singh Solanky
e95686c89c Show complete Khoj keybindings when initiate search in Emacs
- The keybindings to select search types was previously confusing as
  it only highlighted the final symbol to press (the C-x was shown but
  it wasn't made apparent that it had to be pressed before)

- Previously some keybindings unrelated to khoj were also being shown
  in the which-key popup. Now only the khoj keybindings are visible
2022-08-06 16:36:57 +03:00
Debanjum Singh Solanky
4696eadc02 Fix definition of khoj--search-<content-type> functions in khoj.el 2022-08-06 15:19:01 +03:00
Debanjum Singh Solanky
9fa3345000 Show available Khoj keybindings to customize search using which-key
Fallback to showing simple khoj keybindings info message in echo area
when which-key not available
2022-08-05 20:24:29 +03:00
Debanjum Singh Solanky
6a8b2a6936 Do not run incremental search when query is empty 2022-08-05 19:35:42 +03:00
Debanjum Singh Solanky
609cd6e8bb Show keybindings to set khoj search type in echo area to assist user 2022-08-05 19:35:42 +03:00
Debanjum Singh Solanky
48e4a983c5 Allow switching search type in the middle of querying Khoj on Emacs
- More generally, this allows configuring the khoj search anytime
  while in khoj minibuffer window
- Earlier could only configure search type at the start of the search
2022-08-05 19:35:42 +03:00
Debanjum Singh Solanky
48c33b93cc Generalize khoj keymap to func that can update existing keybdings 2022-08-05 19:35:42 +03:00
Debanjum Singh Solanky
19c4701f3f Default to ledger search from files with .beancount extensions 2022-08-05 19:35:42 +03:00
Debanjum Singh Solanky
cc9a395e0a Keep name of buffer for Khoj results in a variable 2022-08-05 19:35:42 +03:00