Commit graph

1180 commits

Author SHA1 Message Date
Debanjum Singh Solanky
e846829a2e Reset Khoj.el version to align with Khoj package version 2022-08-15 06:20:13 +03:00
Debanjum Singh Solanky
fed0b591af Package Khoj as Debian app in Github Release Workflow 2022-08-14 05:07:58 +03:00
Debanjum Singh Solanky
541e03da3d Make khoj.el pass checkdoc, package-lint, flycheck checks
- Add docstrings, mention args in them. Make docstring crisper
- prefix funcs, variables with khoj--
- Require emacs >27.1 for json-parse-buffer
- Use lexical binding
- Add quickstart docs to elisp file itself
- Bump version of khoj.el
2022-08-13 21:37:41 +03:00
Debanjum Singh Solanky
3300378804 Minimal formatting to render beancount results legibly on web interface 2022-08-13 05:03:45 +03:00
Debanjum Singh Solanky
a0759dd923 Convert Configure Screen into the Main Application Window
- What
  - Convert the config screen into the main application window
    with configuration as just one of the functionality it provides
  - Rename config screen to main window to match new designation

- Why
  - System Tray isn't available everywhere (e.g Linux)
  - This requires moving functionality into a normal window for cross-compat
2022-08-13 02:05:52 +03:00
Debanjum Singh Solanky
c2815c5d09 Open Search from Khoj Configure Screen
- Start evolving configure screen away from just being a configure screen
  - Update Window Title to just say Khoj
- Allow Opening Web Interface to Search from Khoj configure screen
- Rename "Start" Button to more accurate "Configure"
- Disable Search button on first run and while configuring app
2022-08-13 00:43:49 +03:00
Debanjum Singh Solanky
28a91ad1fd Deep copy the default_config constant to prevent it being overwritten
- Issue
  - In the previous form, updates to self.current_config would update
    default_config as python does a shallow copy
  - So self.current_config is just referencing the values of default_config
  - Hence updates to current_config updates the default_config values too
  - This is not what we want

- Fix
  - Deep copy the default_config values. Now updates to
    self.current_config wouldn't affect the default_config
2022-08-12 23:54:16 +03:00
Debanjum Singh Solanky
62ac41ce3b Reload settings in a separate thread to not freeze Config Screen
- Generating embeddings takes time
- If user enables a content type and clicks start.
  The app starts to generate embeddings when loading the new settings
- Run this function in a separate thread to keep config screen responsive
- But disable start button to prevent re-entrant threads
- Also show a minimal visual indication that the app is saving state
2022-08-12 23:34:00 +03:00
Debanjum Singh Solanky
927547d0af Update Title of Configure Screen to follow "<Screen> - App" pattern 2022-08-12 22:53:10 +03:00
Debanjum Singh Solanky
43301d488a Increase Width of Configure Screen 2022-08-12 18:34:47 +03:00
Debanjum Singh Solanky
9baea9c9fd Let Input Fields Wrap. Adjust Height based on Text in Field
- Convert Input Fields into PlainTextEdit
- Display Each Selected File on a Separate Line in Field
- Set Height of FileBrowser Input Field based on Number of Lines/Files
2022-08-12 18:33:56 +03:00
Debanjum Singh Solanky
b7b96110e9 Rename FileBrowser Button Text to "Select" instead of "Add" 2022-08-12 17:08:40 +03:00
Debanjum Singh Solanky
a1c58a9470 Create, Use a Labelled Text Field for the Conversation Input Field
- This fixes the field expanding when configure screen is expanded
- Allows for reusability of the labelled text field
- Simplifies the logic to save settings for conversation processor
2022-08-12 16:59:15 +03:00
Debanjum Singh Solanky
fa7e36cada Rename external *.js files to *.min.js to mark them as vendored
- Excludes from Github language stats.
  See linguists/vendor.yml for exclusion rules
- Signifies them as external for Khoj developers too
2022-08-12 04:08:50 +03:00
Debanjum Singh Solanky
110e3df0b7 Set default config in the constant module. Use from there to configure app
- Avoid having to pass the khoj_sample.yml data file into pip, native apps
- Packaging data files into python packages is annoying.
  - There's `MANIFEST.in`, `data_files` and `package_data` in setup.py
  - Bdist, wheel, generated source tarball use different set of these fields
    and put the data files in different locations
  - Rather just code the default config into a constant. Avoid
    pointless file reads as well this way
2022-08-12 02:18:46 +03:00
Debanjum Singh Solanky
44fe70513a Handle situation where default config directory or file does not exist
- Include khoj_sample.yml in pip package to load default config from
- Create khoj config directory if it doesn't exist
- Load config from khoj_sample.yml if khoj.yml config doesn't exist
2022-08-12 01:17:34 +03:00
Debanjum Singh Solanky
41520e1608 Improve Docstring for Configure Screen and System Tray class, funcs 2022-08-11 23:36:02 +03:00
Debanjum Singh Solanky
a748acfeeb Merge branch 'master' of github.com:debanjum/khoj into create-native-gui
Conflicts:
- src/main.py
  - router functions have moved to router
  - move logic to handle null query perf timer variables into
    router.py
  - set main.py to current branch, not master
2022-08-11 21:09:42 +03:00
Debanjum Singh Solanky
b74ca1def6 Wrap error message instead of expanding screen to show message 2022-08-11 20:51:56 +03:00
Debanjum Singh Solanky
2646fa825b Get Files from File input line to match user expectation
- If a user manually edits the input file lines, clicking start should
  use that. Currently it just looks at the files selected last via file
  browser
- We want to allow users to manually enter file paths in field. Which
  is why the field hasn't been set to read-only
2022-08-11 20:48:45 +03:00
Debanjum Singh Solanky
dad9133598 Split save_settings method into smaller methods for modularization 2022-08-11 20:00:52 +03:00
Debanjum Singh Solanky
56ba91fec8 Remove unused methods in file browser widget. Improve name of existing 2022-08-11 19:46:09 +03:00
Debanjum Singh Solanky
fd4e41495c Use appropriate label for directory input types to minimize confusion 2022-08-11 19:45:19 +03:00
Debanjum Singh Solanky
c1e1466fb1 Validate new config before write. Show error if new config invalid 2022-08-11 19:18:22 +03:00
Debanjum Singh Solanky
1ff049599f Show current config on config screen. Load default config if config unset
- Track current (saved/loaded) config separate from the new config (to
  be written) when user clicks Start

- Fallback to using default config when no config for the specific
  content type or processor is specified in khoj.yml
  - Earlier were only loading default config on first run, not after

- Create Child CheckBox, LineEdit classes for Processor Widgets
  - Create ProcessorType, similar to SearchType
  - Track ProcessorType the widgets are associated with
  - Simplify update, save, load of config based on type
2022-08-11 19:11:25 +03:00
Debanjum Singh Solanky
678fb6a3c7 Add Settings Panel for Conversation Settings to Config Screen 2022-08-11 04:52:40 +03:00
Debanjum Singh Solanky
c1fcf44405 Initialize Settings on Config Screen with Existing Settings from File 2022-08-11 04:51:33 +03:00
Debanjum Singh Solanky
3cec6229ad Hot swap backend config via config screen start button click
- Update configuration to use by the backend, while app is running
- Trigger after user hits start button with their config.
  The config gets written to khoj.yml file first, then the updated
  config is loaded onto memory
2022-08-11 00:32:11 +03:00
Debanjum Singh Solanky
cc6ef0f450 Save configure screen settings to app config yaml on clicking Start 2022-08-10 23:10:39 +03:00
Debanjum Singh Solanky
dae65c5b6b Create child class of Qt CheckBox to track search type it enables/disables 2022-08-10 22:44:37 +03:00
Debanjum Singh Solanky
f42f54019b Type parent_layout passed as arguments to ConfigureScreen methods 2022-08-10 22:43:20 +03:00
Debanjum Singh Solanky
f63f11186f Pass config file for app to configure screen 2022-08-10 22:42:32 +03:00
Debanjum Singh Solanky
eddf88f818 Org buffer customization settings to tail of khoj.el results buffer
- Results get priority screen real estate
- Allows quick speed key based traversal of results as cursor
  on switching to buffer is at top level heading
  - E.g C-x o n n o 2 jumps to entry in actual file of second result
  - Unlike before when it is at the #+STARTUP org buffer customization
    settings
2022-08-10 12:57:37 +03:00
Debanjum Singh Solanky
daef276fd1 Add files for each search type. Extract config on clicking start
- Only allow adding files with appropriate file extension for each search type
  - e.g .org for org-mode search, directory for image search

- Extract file paths added to config and enablement state of each search type
  - This extracted state will be used to populate the khoj.yml config file
2022-08-10 03:27:22 +03:00
Debanjum Singh Solanky
d74134e6cc Reuse Single Method to Create Setting Panels for each Search Type 2022-08-09 23:50:43 +03:00
Debanjum Singh Solanky
509d52e2cd Toggle Editability instead of Visibility of Per Search Type Settings
- Simplifies the configure screen layout and allows it to be of constant width

- It was buggy, the configure screen would dynamically expand but not
  restore back to original size on disabling search type after enable
2022-08-09 23:34:54 +03:00
Debanjum Singh Solanky
3c788f1d29 Rename configure window to more generic configure screen 2022-08-09 22:44:05 +03:00
Debanjum Singh Solanky
c50ab7c3ad Split config settings GUI into functions. Convert Config Window to Dialog 2022-08-09 22:36:41 +03:00
Debanjum Singh Solanky
664713b24e Extract Qt GUI code from main.py into separate interface/desktop dir 2022-08-09 22:12:29 +03:00
Debanjum Singh Solanky
57026b802c Set size of rendered images using user customizable vars 2022-08-09 21:06:37 +03:00
Debanjum Singh Solanky
0a758c9f0f By default, wait for 2 seconds before initiating rerank in khoj.el
- Subjectively, previous default seems to aggressive based on usage
  Doesn't give time for user to think and type their query
2022-08-09 21:06:30 +03:00
Debanjum Singh Solanky
f01fb16ebb Use single hyphen in name of user configurable variables in khoj.el
- Follow convention, two hyphens indicate variable private to library
- Defcustom are user configurable variables. So they should have single -
- Use khoj-results-count variable directly in code
2022-08-09 20:49:34 +03:00
Debanjum Singh Solanky
e5691f9d1d PyInstaller Spec to Wrap Khoj into a Basic Native App
- Verified functionality on MacOS

- Add ICNS Icon to use as MacOS App Icon
- Spec generated by PyInstaller:
  ```sh
  pyinstaller \
       src/main.py \
       --windowed \
       --onefile \
       --name "Khoj" \
       --target-arch arm64 \
       -i src/interface/web/assets/icons/favicon.icns \
       --add-data "src/interface/web:src/interface/web" \
       --copy-metadata tqdm \
       --copy-metadata regex \
       --copy-metadata requests \
       --copy-metadata packaging \
       --copy-metadata filelock \
       --copy-metadata numpy \
       --copy-metadata tokenizers
  ```
2022-08-08 23:23:02 +03:00
Debanjum Singh Solanky
dddc57e132 Rename get-enabled-search-types to get-enabled-content-types as more appropriate 2022-08-07 18:53:14 +03:00
Debanjum Singh Solanky
127c6e78df Only show keybindings for enabled search types in simple info menu too
Convert the khoj--keybindings-info-message into a func
Dynamically generate info menu
Show keybindings for enabled search types only
2022-08-07 18:40:35 +03:00
Debanjum Singh Solanky
d08c25b62b Make default search type used in the Emacs interface configurable 2022-08-07 18:24:53 +03:00
Debanjum Singh Solanky
5a10c47499 Allow setting music as search type in khoj.el. Had forgotten to include it earlier 2022-08-07 18:24:53 +03:00
Debanjum Singh Solanky
ebee716026 Only show keybindings reference for enabled search types in khoj.el 2022-08-07 18:24:53 +03:00
Debanjum Singh Solanky
6dc9801f45 Get Khoj search-types enabled by user in Emacs 2022-08-07 18:24:53 +03:00
Debanjum Singh Solanky
f3c1512c38 Fix to let user to start enter query right after initiating khoj on emacs
- Fix regression since moving to use `which-key-show-full-keymap~
- The above function reads user keypress, so eats up 1 keypress
  before starting to enter query
- No way to pass no-paging config via the external function to the
  internally used which-key--show-keymap function that does allow
  setting no-paging to not read user keypress
- So use the internal function instead and set no-paging arg to t
2022-08-07 15:57:08 +03:00
Debanjum Singh Solanky
e95686c89c Show complete Khoj keybindings when initiate search in Emacs
- The keybindings to select search types was previously confusing as
  it only highlighted the final symbol to press (the C-x was shown but
  it wasn't made apparent that it had to be pressed before)

- Previously some keybindings unrelated to khoj were also being shown
  in the which-key popup. Now only the khoj keybindings are visible
2022-08-06 16:36:57 +03:00
Debanjum Singh Solanky
4696eadc02 Fix definition of khoj--search-<content-type> functions in khoj.el 2022-08-06 15:19:01 +03:00
Debanjum Singh Solanky
9fa3345000 Show available Khoj keybindings to customize search using which-key
Fallback to showing simple khoj keybindings info message in echo area
when which-key not available
2022-08-05 20:24:29 +03:00
Debanjum Singh Solanky
6a8b2a6936 Do not run incremental search when query is empty 2022-08-05 19:35:42 +03:00
Debanjum Singh Solanky
609cd6e8bb Show keybindings to set khoj search type in echo area to assist user 2022-08-05 19:35:42 +03:00
Debanjum Singh Solanky
48e4a983c5 Allow switching search type in the middle of querying Khoj on Emacs
- More generally, this allows configuring the khoj search anytime
  while in khoj minibuffer window
- Earlier could only configure search type at the start of the search
2022-08-05 19:35:42 +03:00
Debanjum Singh Solanky
48c33b93cc Generalize khoj keymap to func that can update existing keybdings 2022-08-05 19:35:42 +03:00
Debanjum Singh Solanky
19c4701f3f Default to ledger search from files with .beancount extensions 2022-08-05 19:35:42 +03:00
Debanjum Singh Solanky
cc9a395e0a Keep name of buffer for Khoj results in a variable 2022-08-05 19:35:42 +03:00
Debanjum Singh Solanky
0a5c6d067a Do not prompt user to set search type before querying Khoj via Emacs
- What
  - Default to last used search type, when no search type specified
  - Allow user to change search type before they enter query (and
    after they've called khoj), if they want

- Why
  - Reduce time from intent to results by using reasonable defaults
  - Make interactions smoother, more intuitive
2022-08-05 19:35:38 +03:00
Debanjum Singh Solanky
24ccba74d4 Put type dropdown, regenerate button on same row. Regain screen space 2022-08-05 06:17:43 +03:00
Debanjum Singh Solanky
017e287b8a Remove redundant query as title in results section
- Regain screen real-estate
- Remove unused parameters, html being returned by org.js
2022-08-05 06:17:25 +03:00
Debanjum Singh Solanky
06afeec7e2 Hide stars of org entry results on Emacs to reduce visual clutter
They've all been normlized to the same level and hence don't hold much
data. So good opportunity to reduce, non-useful visual clutter
2022-08-05 05:27:57 +03:00
Debanjum Singh Solanky
4d4d2ff921 Ensure all org entries are unfolded in results buffer on Emacs 2022-08-05 04:54:29 +03:00
Debanjum Singh Solanky
49ef741d4b Prevent Zoom on Input in Web Interface. Document Pip upgrade in Readme
- Name /Reload API Controller Reload
2022-08-05 03:51:34 +03:00
Debanjum Singh Solanky
61c26ba611 Only show large Khoj favicon on web interface
- Do not want browsers to use the small, grainy favicons
- Firefox for Android does use the bigger icon, when it's the only one available
- Update svg to match the 144x144 ratio just for consistency
2022-08-04 14:33:29 +03:00
Debanjum Singh Solanky
1649fa644c Autofocus on Query field in Web Interface. Improve time to query 2022-08-04 05:23:19 +03:00
Debanjum Singh Solanky
71fcb1087f Add icons for web interface to render on more browsers and as PWA
Safari, Firefox for Android etc don't support SVG Favicons yet
2022-08-03 18:52:41 +03:00
Debanjum Singh Solanky
5b6b7ec123 Delete khoj network connections on incremental search teardown on Emacs interface
Currently only get into this state when debug breakpoints on backend
are keeping the connection open and user exits khoj search from Emacs
Results in a number of open connections that slow khoj down.
2022-08-03 18:52:41 +03:00
Debanjum Singh Solanky
f642450ed9 Disable Incremental Search for Images on Web
Bug introduced in commit da118b3fed
2022-08-03 11:52:51 +03:00
Debanjum Singh Solanky
0ebfbb43ce Nest org, md results at level 2 on Emacs interface. Improve readability
- Makes it easier to fold/unfold, traverse and read results
- This 2 level nesting is already being used on the web interface

- Previously we were using the original nesting depth of the entry.
  This was aimed at providing more of the orginal context of the
  results. But currently this additional information does not provide
  as much, for the decreased legibility of the results
2022-08-01 04:01:18 +03:00
Debanjum Singh Solanky
1201bfddf3 Simplify name of config css from config-style.css to config.css 2022-08-01 01:34:00 +03:00
Debanjum Singh Solanky
075dba5d64 Use Khoj Title, Favicon in Config Page for Consistency 2022-08-01 01:27:14 +03:00
Debanjum Singh Solanky
56a4429f01 Move web interface to configure application into src/interface/web directory
- Improve code layout by ensuring all web interface specific code
  under the src/interface/web directory
- Rename config API to more specifi /config instead of /ui
- Rename config data GET, POST api to /config/data instead of /config
2022-08-01 00:53:42 +03:00
Debanjum
bb2ccec1ca
Populate type dropdown on the web interface with only enabled search types
- Previously we were statically populating types dropdown field in the web interface with all available search types
- This change populates the type dropdown field with only search types that are enabled/configured
- It queries the `/config` backend API to see which of the available search types are configured
2022-08-01 00:20:45 +03:00
Debanjum Singh Solanky
8b6058c879 Fix instantiating type field with value from URL query parameter
- Populate via `.then` after enabled search types in dropdown are
  populated
- Call to `/config` API is async and will usually complete after the value of type field is set from url
- So value of type field would earlier be overridden when search types
  dropdown is populated after the call to `/config` API completes
2022-08-01 00:04:50 +03:00
Debanjum Singh Solanky
be253bab39 Populate type dropdown with only enabled search types in web interface
- Get /config API and check config for which available search types is
  populated. This gives us the list of enabled search types
- Dynamically populate search type field with enabled search types only
2022-07-31 23:42:00 +03:00
Debanjum Singh Solanky
0abd40aeb7 Only set query field when appropriate query param passed via URL
- Setting query value to default option when query param wasn't
  passed via URL was overriding placeholder text in query field

- We wanted placeholder text in field, not the query field to actually
  be populated by placeholder text

- This clears field when user starts typing query into the query field,
  instead of them having to manually delete the  default text populated
2022-07-31 22:29:23 +03:00
Saba
b55159f5bd Fix URL for khoj.el quelpa setup instructions 2022-07-29 23:01:04 -04:00
Debanjum Singh Solanky
da118b3fed Simplify incremental search function used in web interface
Re-rank isn't passed to image search API in search function.
So don't need to check type in incremental_search function too
2022-07-29 23:18:01 +04:00
Debanjum Singh Solanky
3079614981 Allow set up of search form via query params in web interface
- Default search type to org, instead of images
2022-07-29 23:13:26 +04:00
Debanjum Singh Solanky
02ca2c05a1 Add Eagle Icon for Khoj to Web, Emacs Interfaces and Readme 2022-07-29 17:50:29 +04:00
Debanjum Singh Solanky
78314263a0 Add Table of Contents, Features, Performance Details to Readme 2022-07-29 17:08:17 +04:00
Debanjum Singh Solanky
ed181f47c9 Prettify rendering of org music results on Khoj web interface 2022-07-29 04:28:22 +04:00
Debanjum Singh Solanky
7e5291a38e Make org result headings at same level. Improve spacing of results
Having org-mode result headings change size based on their depth in
the source document makes is a confusing UI experience.

Improve font-size, line-spacing and margins of results to make
delineation between entries, and differntiating between entry heading
and it's body easier to visually infer.

Do not white-space: pre-line. Improves rendering of Markdown results
2022-07-29 01:55:46 +04:00
Debanjum Singh Solanky
a9bc17a6b0 Prettify Render of Markdown Results in Web Interface 2022-07-28 20:56:37 +04:00
Debanjum Singh Solanky
a6ae74f52e Move JS files like org.js into a separate assets/ directory 2022-07-28 20:46:48 +04:00
Debanjum
a71253e137
Support Incremental Search on Web Interface
## Support Incremental Search on Khoj Web Interface
- Use default, fast path to query /search API while user is typing
- Upgrade to cross-encoder re-ranked results once user hits enter on search box

## Improve Render of Org Results on Web Interface
- We were previously just wrapping results from /search API into a pre formatted div field. This was not easy to read
- Use [org.js](https://mooz.github.io/org-js/) to render results from Khoj `/search` API as proper HTML
- Improve org.js to render all task states, stylize task tags and make org-mode results look more like original content

Closes #42 #41
2022-07-28 09:31:57 -07:00
Debanjum Singh Solanky
e8029bf415 Extract and Highlight org-mode tags in HTML render of search results 2022-07-28 19:55:15 +04:00
Debanjum Singh Solanky
c6c248df26 Improve styling of org-mode results to original alignment, line breaks 2022-07-28 19:55:15 +04:00
Debanjum Singh Solanky
9f59897eeb Highlight all org-mode task states in HTML. Not just TODO, DONE.
- Make logic to extract, mark todo state in org.js more generic
- Add default todo state styling to html
2022-07-28 19:55:15 +04:00
Debanjum Singh Solanky
f040b3f65c Stylize TODO/DONE states with CSS 2022-07-28 19:55:15 +04:00
Debanjum Singh Solanky
581b6097c7 Clean Results. Remove TOC, Heading Number and Property Drawers 2022-07-28 19:55:15 +04:00
Debanjum Singh Solanky
965a93a2f2 Add Basic HTML Rendering of Org-Mode Results 2022-07-28 19:55:15 +04:00
Debanjum Singh Solanky
1da44d4dfe Add Incremental Search to Khoj Web Interface 2022-07-28 19:55:15 +04:00
Debanjum Singh Solanky
80ac10835c Rerank results on normal minibuffer exit
In current state:
 - Rerank results:
   - If user idles while entering query OR
   - exits normally

 - Do not rerank results:
   - If user exits abnormally, e.g via C-g from query
2022-07-28 03:37:16 +04:00
Debanjum Singh Solanky
1b759597df Make incremental search more robust. Follow standard user expectations
- Rename functions to more standard, descriptive names
- Keep known, required code for incremental search
  - E.g Do not set buffer local flag in hooks on minibuffer setup

- Only query when user in khoj minibuffer
  - Use active-minibuffer-window and track khoj minibuffer
  - (minibuffer-prompt) is not useful for our use-case here

- (For now) Run re-rank only if user idle while querying
  - Do not run rerank on teardown/completion
    - The reranking lag (~2s) is annoying; hit enter,
      wait to see results
    - Also triggered when user exits abnormally,
      so C-g also results in rerank which is even more annoying
  - Emacs will still hang if re-ranking gets triggered on idle but
    that's better than always getting triggered. And better than not
    having mechanism to get results re-ranked via cross-encoder at all
2022-07-28 02:52:27 +04:00
Debanjum Singh Solanky
9a6eee31be Make number of results to get from Khoj API customizable in khoj.el 2022-07-27 18:55:18 +04:00
Debanjum Singh Solanky
9302b45fe0 Use khoj-incremental as the main khoj func. Rename khoj to khoj-simple
- Update khoj-simple to work cross-encoder re-ranked results like before
- Increment major version as incremental search considered a breaking
  change and a major update to search capability
2022-07-27 18:18:17 +04:00
Debanjum Singh Solanky
9ab3edf6d6 Re-rank incremental search results using cross-encoder if user idle
This provides a relatively smooth mechanism
- to improve relevance of results on idle
- while providing the rapid, incremental results while typing
2022-07-27 07:25:42 +04:00
Debanjum Singh Solanky
ad242cafa7 Support querying all text search types in incremental search
- Before incremental search was hard-coded to only query org
2022-07-27 07:25:42 +04:00
Debanjum Singh Solanky
bfcb962cbe Use post-command-hook to only query on user input
- Hooking into after-change-functions results in system logs triggering query
2022-07-27 07:25:42 +04:00
Debanjum Singh Solanky
0d49398954 Reuse code to query api, render results. Formalize method, arg names 2022-07-27 07:25:42 +04:00
Debanjum Singh Solanky
fd1963d781 Implement Basic Incremental Search Interface in Emacs for Org Mode Notes 2022-07-27 03:05:00 +04:00
Debanjum Singh Solanky
3fa7d8f03a Skeleton to allow incremental search on Khoj via Emacs 2022-07-27 02:48:27 +04:00
Debanjum Singh Solanky
65fea7681a Rename notes search type to org search, now that markdown notes supported 2022-07-21 22:09:44 +04:00
Debanjum Singh Solanky
4c24202e42 Update documentation. Simplify, reflect current capabilities 2022-07-21 22:09:44 +04:00
Debanjum Singh Solanky
d4d7dbaca6 Support Natural Search on Markdown Files
- Reason:
  Allow natural search on markdown based notes, documentation,
  websites etc

- Details:
  - Create markdown processor to extract Markdown entries (identified by
    Heading) into standard jsonl format required by text_search
  - Update API, Configs to support interfacing with new markdown type
  - Update Emacs, Web clients to support interfacing with new markdown
    type via API
  - Update Readme to mentiond markdown is also supported

Closes #35
2022-07-21 22:07:05 +04:00
Debanjum Singh Solanky
c1369233db Consistently use "entry", "score" in json response for all search types
- Had already made some progress on this earlier by updating the image
  search responses. But needed to update the text search responses to
  use lowercase entry and score

- Update khoj.el to consume the updated json response keys for text
  search
2022-07-20 20:33:27 +04:00
Debanjum Singh Solanky
732b2d287f Give the project a short, less generic name. Rename it to Khoj
- Semantic Search was just a placeholder used to test the idea out
  Didn't want to get into naming at that point of time
2022-07-19 18:26:16 +04:00
Debanjum Singh Solanky
5e302dbcda Fix using 1 column layout on small screens 2022-07-18 02:40:16 +04:00
Debanjum Singh Solanky
7d16b673b1 Use Single Column Layout for Small Screens on Web Interface 2022-07-18 02:08:52 +04:00
Debanjum Singh Solanky
31a221a76b Auto focus cursor on query input box to simplify, speed interactions
- Avoids having to click the query input box
- Just open page, type whatever and hit enter to do image search
  - For other search types select appropriate type from dropdown
2022-07-16 19:39:15 +04:00
Debanjum Singh Solanky
06b0c720d6 Improve Rendering of Image Search Results in Emacs
- Use shr to render image response from html in result buffer
  Earlier was using org-mode. But rendering HTML with shr seems cleaner
- Use Headings to Add highlights
- Use Random to Force fetch of Image. Similar to what was done for Web interface
- Remove trailing elisp brackets from response
- Show query match scores by image model for each image in results
2022-07-16 19:31:49 +04:00
Debanjum Singh Solanky
28ec9af589 Extract image URL location from response in elisp after API update 2022-07-16 18:43:55 +04:00
Debanjum Singh Solanky
47613cba1f Improve Landing Page Look in General and Layout for Mobile
- Ask for 6 Images to Fill Grid into 3x2 Layout
- Submit Form on Hitting Enter
2022-07-16 16:55:13 +04:00
Debanjum Singh Solanky
cf207d6ebe Add title, heading to the semantic search web interface 2022-07-16 03:44:29 +04:00
Debanjum Singh Solanky
ca3f93e641 Add button on web interface to regenerate embeddings of specified type 2022-07-16 03:36:19 +04:00
Debanjum Singh Solanky
231cc91e14 Force reload of images every time user clicks search button
Adding a random, unused url param at the end of the img.src string
fixes the issue. As the browser thinks it's a new image and doesn't
use the image data that's already cached because of which it wasn't
even making the fetch call for the image
2022-07-16 03:36:19 +04:00
Debanjum Singh Solanky
a6aef62a99 Create Basic Landing Page to Query Semantic Search and Render Results
- Allow viewing image results returned by Semantic Search.
  Until now there wasn't any interface within the app to view image
  search results. For text results, we at least had the emacs interface

- This should help with debugging issues with image search too
  For text the Swagger interface was good enough
2022-07-16 03:36:19 +04:00
Debanjum Singh Solanky
bb5d0d8908 Improve Semantic Search Buffer Names in Emacs
- Allow multiple semantic searches buffers to exist simultaneously
  - Uniquify semantic search buffer namew
- Add query and search-type to semantic search buffer name for easier
  disambiguration, search and find appropriate
2022-02-26 18:30:14 -05:00
Debanjum Singh Solanky
b3ac2dd730 Improve Results Rendered on Emacs from Semantic Search on Ledger
- Add search query to top of buffer as Beancount comment
- Remove trailing ) from response
- Separate entries by empty line
- Load beancount-mode in semantic search on ledger buffer
2022-02-26 17:48:45 -05:00
Debanjum Singh Solanky
5db08c5293 Set query as heading of notes search results in Emacs Org buffer 2021-09-29 13:30:15 -07:00
Debanjum Singh Solanky
fdb60a8dcf Set Query as Heading of Image Search Results Emacs Buffer 2021-09-16 12:30:06 -07:00
Debanjum Singh Solanky
0e34c8f493 Allow semantic search on images from Emacs
Images are rendered inline a temporary org-mode buffer
2021-09-10 01:14:34 -07:00
Debanjum Singh Solanky
7d5514ecaa Allow user to override inferred search type with other valid options 2021-09-10 00:58:24 -07:00
Debanjum Singh Solanky
3bdeeb1e19 Autoload main semantic-search function 2021-09-09 22:10:37 -07:00
Debanjum Singh Solanky
f4bde75249 Decouple results shown to user and text the model is trained on
- Previously:
  The text the model was trained on was being used to
  re-create a semblance of the original org-mode entry.

- Now:
  - Store raw entry as another key:value in each entry json too
    Only return actual raw org entries in results
    But create embeddings like before
  - Also add link to entry in file:<filename>::<line_number> form
    in property drawer of returned results
    This can be used to jump to actual entry in it's original file
2021-08-29 06:06:54 -07:00
Debanjum Singh Solanky
0263d4d068 Enable semantic search for songs in org-music
Org-Music: https://github.com/debanjum/org-music
2021-08-29 06:06:28 -07:00
Debanjum Singh Solanky
af9660f28e Move application files under src directory. Update Readmes
- Remove callign asymmetric search script directly command.
  It doesn't work anymore on calling directly due to internal package
  import issues
2021-08-17 04:11:03 -07:00