Commit graph

3255 commits

Author SHA1 Message Date
Debanjum Singh Solanky
e95686c89c Show complete Khoj keybindings when initiate search in Emacs
- The keybindings to select search types was previously confusing as
  it only highlighted the final symbol to press (the C-x was shown but
  it wasn't made apparent that it had to be pressed before)

- Previously some keybindings unrelated to khoj were also being shown
  in the which-key popup. Now only the khoj keybindings are visible
2022-08-06 16:36:57 +03:00
Debanjum Singh Solanky
4696eadc02 Fix definition of khoj--search-<content-type> functions in khoj.el 2022-08-06 15:19:01 +03:00
Debanjum Singh Solanky
c5bf051a29 Rename initialize_{search,processor,server} to configure_{search,procesor,server}
- Search is being reconfigured multiple times in /regenerate and
  n/reload. More appropriate name is configure_ rather than initialize_
  for it
- Standardize name of methods under configure.py
2022-08-06 03:23:02 +03:00
Debanjum Singh Solanky
7b04978f52 Put global state variables into separate state module
- Variables storing app, device state aren't constants.
  Do not mix with actual constants like empty_escape_sequence, web_directory
2022-08-06 03:13:18 +03:00
Debanjum Singh Solanky
bc423d8f76 Disable image search in tests. Import global state from constants module
- Upstream issues causing load of image search model to fail.
  Disable tests related to image search for now
2022-08-06 02:47:52 +03:00
Debanjum Singh Solanky
b04c84721b Extract configure and routers from main.py into separate modules
- Main.py was becoming too big to manage. It had both
  controllers/routers and component configurations (search, processors)
  in it

- Now that the native app GUI code is also getting added to the main
  path, good time to split/modularize/clean main.py

- Put global state into a separate file to share across modules
2022-08-06 02:39:18 +03:00
Debanjum Singh Solanky
083fefdd07 Create Native Menu Bar with PyQt to open Search, Config webpages
- Run FastAPI server in a separate thread.
  - This allows starting both the server and gui in parallel

- Create System Tray for Khoj
  - Contains menu items that open search or config pages in browser

- Rearrange code to have only the code required to start Backend and
  GUI in the run() method
  - Move the backend setup code into a separate method
2022-08-06 01:00:25 +03:00
Debanjum
3a4e5de7fa
Do Not Prompt User to Set Search Type before Querying Khoj via Emacs
### Why
- Reduce time from intent to results by using reasonable defaults
- Make interactions smoother, more intuitive
- Users expect to start querying immediately. The prompt to enter search type creates unnecessary friction

### How
- By default, search using last searched content type
- Allow user to change search type, while querying, by using keyboard shortcuts
- For reference, display keyboard shortcuts to set search-type when user initiates (incremental) search
2022-08-05 20:51:00 +03:00
Debanjum Singh Solanky
9fa3345000 Show available Khoj keybindings to customize search using which-key
Fallback to showing simple khoj keybindings info message in echo area
when which-key not available
2022-08-05 20:24:29 +03:00
Debanjum Singh Solanky
6a8b2a6936 Do not run incremental search when query is empty 2022-08-05 19:35:42 +03:00
Debanjum Singh Solanky
609cd6e8bb Show keybindings to set khoj search type in echo area to assist user 2022-08-05 19:35:42 +03:00
Debanjum Singh Solanky
48e4a983c5 Allow switching search type in the middle of querying Khoj on Emacs
- More generally, this allows configuring the khoj search anytime
  while in khoj minibuffer window
- Earlier could only configure search type at the start of the search
2022-08-05 19:35:42 +03:00
Debanjum Singh Solanky
48c33b93cc Generalize khoj keymap to func that can update existing keybdings 2022-08-05 19:35:42 +03:00
Debanjum Singh Solanky
19c4701f3f Default to ledger search from files with .beancount extensions 2022-08-05 19:35:42 +03:00
Debanjum Singh Solanky
cc9a395e0a Keep name of buffer for Khoj results in a variable 2022-08-05 19:35:42 +03:00
Debanjum Singh Solanky
0a5c6d067a Do not prompt user to set search type before querying Khoj via Emacs
- What
  - Default to last used search type, when no search type specified
  - Allow user to change search type before they enter query (and
    after they've called khoj), if they want

- Why
  - Reduce time from intent to results by using reasonable defaults
  - Make interactions smoother, more intuitive
2022-08-05 19:35:38 +03:00
Debanjum Singh Solanky
24ccba74d4 Put type dropdown, regenerate button on same row. Regain screen space 2022-08-05 06:17:43 +03:00
Debanjum Singh Solanky
017e287b8a Remove redundant query as title in results section
- Regain screen real-estate
- Remove unused parameters, html being returned by org.js
2022-08-05 06:17:25 +03:00
Debanjum Singh Solanky
06afeec7e2 Hide stars of org entry results on Emacs to reduce visual clutter
They've all been normlized to the same level and hence don't hold much
data. So good opportunity to reduce, non-useful visual clutter
2022-08-05 05:27:57 +03:00
Debanjum Singh Solanky
31eabee10a Rearrange Development and Performance Sections in Readme 2022-08-05 05:27:09 +03:00
Debanjum Singh Solanky
0098f27f0c Add Section for Development with Pip 2022-08-05 04:59:52 +03:00
Saba
d1fe6353b5 Check whether processor_config exists during shutdown event 2022-08-04 21:57:36 -04:00
Debanjum Singh Solanky
4d4d2ff921 Ensure all org entries are unfolded in results buffer on Emacs 2022-08-05 04:54:29 +03:00
Debanjum Singh Solanky
49ef741d4b Prevent Zoom on Input in Web Interface. Document Pip upgrade in Readme
- Name /Reload API Controller Reload
2022-08-05 03:51:34 +03:00
Debanjum Singh Solanky
675e821d95 Make embeddings, jsonl paths absolute. Create directories if non-existent 2022-08-05 02:57:59 +03:00
Debanjum Singh Solanky
d5b43eb836 Use input filter in image search setup. Input filter wasn't used earlier 2022-08-05 02:40:03 +03:00
Debanjum Singh Solanky
ca5a8bd113 Make config file a positional argument, as it is required
- Test invalid config file path throws. Remove redundant cli test

- Simplify cli parser code
  - Do not need to explicitly check if args.config_file set.
    argparser checks for positional arguments automatically

- Use standard semantics for cli args
  - All positional args are required. Non positional args are optional

- Improve command line --help description
2022-08-05 01:09:40 +03:00
Debanjum Singh Solanky
a3b35fbb6e Update Readme to Install Khoj using Pip by Default
- Instructions to Install Khoj using Docker have been moved to the
  Development Section
- Simplify Section Headings
2022-08-05 01:09:40 +03:00
Debanjum Singh Solanky
1374065092 Mark all required fields for config. Throw if no input_* field specified
- Add custom validator to throw if neither input_filter or
  input_<files|directories> are specified

- Set field expecting paths to type Path

- Now that default_config isn't used in code. We can update
  fields in rawconfig to specify whether they're required or not.
  This lets pydantic validate config file and throw appropriate error
2022-08-05 01:08:48 +03:00
Debanjum Singh Solanky
f78d6ae754 Create khoj_sample file with all configurable fields in one place
- Reason
  - Simplifies code. No merge_dict required
  - 1 place for user to see all configurables, defaults and required values

- Details
  - Remove default_config from code. Set defaults in khoj_sample.yml itself
  - Keep fields required to be set by user as empty in khoj_sample to YAML
  - Set defaults for fields not requiring configuration by user
2022-08-05 01:08:33 +03:00
Debanjum Singh Solanky
4788143aa6 Set clip model name in conftest to sentence-tranformers/clip as well 2022-08-04 22:54:39 +03:00
Debanjum Singh Solanky
3abf3e5ee0 Update merge_dicts to recursively merge the dictionaries
Previously it was only merging dictionary at the first/top level
2022-08-04 22:46:20 +03:00
Debanjum Singh Solanky
a876b652d8 Rename khoj_sample.yml to more specific khoj_docker.yml
- Update docker-compose.yml to start container using khoj_docker.yml
- Use /data/org in input-filter for content-type > org
2022-08-04 22:42:05 +03:00
Debanjum Singh Solanky
f50f343f73 Rename org-mode test data directory to more specific org/ from notes/ 2022-08-04 22:29:57 +03:00
Debanjum Singh Solanky
61c26ba611 Only show large Khoj favicon on web interface
- Do not want browsers to use the small, grainy favicons
- Firefox for Android does use the bigger icon, when it's the only one available
- Update svg to match the 144x144 ratio just for consistency
2022-08-04 14:33:29 +03:00
Debanjum Singh Solanky
1649fa644c Autofocus on Query field in Web Interface. Improve time to query 2022-08-04 05:23:19 +03:00
Debanjum Singh Solanky
8646dc0de1 Add publish badge to Readme. Link badges to their workflow pages 2022-08-04 05:23:19 +03:00
Debanjum
496806452f
Use Pip to setup Khoj in Docker, Github Workflows. Publish to PyPi Automatically
### Details
#### Github Workflows
- Make Github Workflows use `pip` instead of `conda`
- Publish Khoj as Python Package Automatically
   - Version based on [PEP440](https://peps.python.org/pep-0440/) and [Semantic Versioning](https://semver.org/) specs
   - On Push to Master: Publish a pre-release (alpha) to PyPI
   - On Push to PR: Publish a development-release on TestPyPI
   - On creating a (release) tag: Publish the latest version (Major.Minor.Patch) to PyPI
#### Docker 
   - Make `Dockerfile` use `pip` to build image for Khoj. Faster, more standard flow than `conda`
   - Only Build Docker Image on Push to Master
      - Availability of pip package negates the need for docker image for PR testing 
   - Make Docker-Compose use Docker Image published to Github Container Registry
- Default Image search to disabled. [Fix loading CLIP model](https://github.com/UKPLab/sentence-transformers/issues/1659) before re-enabling
2022-08-04 05:09:26 +03:00
Debanjum Singh Solanky
ebaf5524d1 Bump khoj patch version in setup.py
- PRs and Pushes to Master show up as newer than previous release. But
they're marked as development release or pre-release (alpha)
- Once create tag. The published pip package than becomes a final release
  in PEP440 terminology
2022-08-04 04:45:20 +03:00
Debanjum Singh Solanky
50575b283c Reorder publish actions by order of importance in publish workflow 2022-08-04 04:43:05 +03:00
Debanjum Singh Solanky
c3ae3cb021 Publish to (Test) PyPi with PEP440 Compliant Version Numbers
- Use .devN for publish to testpypi on push to PR
- Use aN to publish to pypi on push to master
2022-08-04 04:29:15 +03:00
Debanjum Singh Solanky
89044c70bd Publish to Test PyPI on Push to PR or Master Branch 2022-08-04 03:59:38 +03:00
Debanjum Singh Solanky
4bc5ac2420 Add pre-release version to package and upload to TestPyPI
Update version to form {current-version}-alpha.{unix-timestamp} in setup.py
2022-08-04 03:21:33 +03:00
Debanjum Singh Solanky
b3ebb01beb Disable image search for now as unable to load CLIP model
Loading CLIP Model from Sentence-Transformer is failing
See https://github.com/UKPLab/sentence-transformers/issues/1659 for details
2022-08-04 02:20:45 +03:00
Debanjum Singh Solanky
b2beff6a0e Only run build workflow on publish to master. Not on pull request 2022-08-04 02:20:05 +03:00
Debanjum Singh Solanky
84adf0c568 Use published docker image to run khoj service using docker-compose 2022-08-04 02:17:01 +03:00
Debanjum Singh Solanky
ee65809dc6 Update Dockerfile to use Pip instead of Conda to install application
- Reduce size of app copied to container by adding unneeded
  directories to .dockerignore. E.g Ignore docs, tests, pip build, dist

- Update MANIFEST.ini to include web, emacs interface directories
  Web directory is used by the web interface which will be exposed by
  the docker container
2022-08-04 00:14:25 +03:00
Debanjum Singh Solanky
7a353066dd Improve name, stages in Github Workflows for Publish and Test 2022-08-03 22:37:16 +03:00
Debanjum Singh Solanky
f55eaf0a53 Remove comma in command to pip install multiple packages 2022-08-03 22:01:18 +03:00
Debanjum Singh Solanky
ee3eddb044 Create Workflow to Publish Application to (test) PyPI 2022-08-03 21:57:27 +03:00