- It is a non-user configurable, app state that is set on app start
- Reduce passing unneeded arguments around. Just set device where
required by looking for ML compute device in global state
- Note: Support for MPS in Pytorch is currently in v1.13.0 nightly builds
- Users will have to wait for PyTorch MPS support to land in stable builds
- Until then the code can be tweaked and tested to make use of the GPU
acceleration on newer Macs
- Pass device to load models onto from app state.
- SentenceTransformer models accept device to load models onto during initialization
- Pass device to load corpus embeddings onto from app state
- CLIP Image score and XMP metadata score are not combining well.
When combined they give non sensical results. Enable only once
figure how best to combine the two.
- Show scores with higher precision for image search
- Image search scores seem to be mostly be between 0.2 - 0.3 for some reason
- Higher precision scores make it easier to understand the quality
of returned results perceived by the model itself
- Allows adding multiple image directories via GUI
- Allow adding multiple files in different directories via GUI
- Previously users couldn't add multiple directories via GUI
They'd have to manually append to input field if multiple files, directories
- To clear/overwrite is much easier.
The user can just select text to delete in input area
- Issue
Fix configuring image search from Desktop GUI. It was broken before.
The Desktop GUI was updating input-files field under content-type > image.
This field is not used for image search. So image search couldn't be
configured from the Desktop GUI
- Fix
- Set input-directories when field of search type image is set from GUI
- Otherwise set input-files field in config
- Show a helpful error message in the GUI to the user, instead of the
crashing if loading config fails, for e.g if file wasn't found
- Collate GUI errors into an ErrorType enum class
- Remove previous error messages before showing the new one
Previously if the embeddings were already there only the khoj.yml
config file would get updated. The embeddings would remain old.
1. This results in a stale app state where the config doesn't
match the embeddings
2. Currently the user cannot update their config from the config
screen. They'd have to use a combination of config screen and web
interface>regenerate button to trigger it or delete their ~/.khoj dir
This commit should resolve the above issues
- Prevent immediate overwrite of re-ranked results by
incremental-search without rerank triggered via post-command-hook.
- This triggers right after the reranking results are rendered, so
user never ends up seeing them
- Add docstrings, mention args in them. Make docstring crisper
- prefix funcs, variables with khoj--
- Require emacs >27.1 for json-parse-buffer
- Use lexical binding
- Add quickstart docs to elisp file itself
- Bump version of khoj.el
- What
- Convert the config screen into the main application window
with configuration as just one of the functionality it provides
- Rename config screen to main window to match new designation
- Why
- System Tray isn't available everywhere (e.g Linux)
- This requires moving functionality into a normal window for cross-compat
- What
- On Linux
- Show Configure Screen, even if not first run experience
- Do no show system tray on Linux
- Quit app on closing Configure Screen
- On Windows, Mac
- Show Configure screen only if first run experience
- Show system tray always
- Do not quit app on closing Configure Screen
- Why
- Configure screen is the only GUI element on Linux. So closing it
should close the application
- On Windows, Mac the system tray exists, so app should not be closed
on closing configure screen
- Start evolving configure screen away from just being a configure screen
- Update Window Title to just say Khoj
- Allow Opening Web Interface to Search from Khoj configure screen
- Rename "Start" Button to more accurate "Configure"
- Disable Search button on first run and while configuring app
- Issue
- In the previous form, updates to self.current_config would update
default_config as python does a shallow copy
- So self.current_config is just referencing the values of default_config
- Hence updates to current_config updates the default_config values too
- This is not what we want
- Fix
- Deep copy the default_config values. Now updates to
self.current_config wouldn't affect the default_config
- Generating embeddings takes time
- If user enables a content type and clicks start.
The app starts to generate embeddings when loading the new settings
- Run this function in a separate thread to keep config screen responsive
- But disable start button to prevent re-entrant threads
- Also show a minimal visual indication that the app is saving state
- Convert Input Fields into PlainTextEdit
- Display Each Selected File on a Separate Line in Field
- Set Height of FileBrowser Input Field based on Number of Lines/Files
- This fixes the field expanding when configure screen is expanded
- Allows for reusability of the labelled text field
- Simplifies the logic to save settings for conversation processor
- Avoid having to pass the khoj_sample.yml data file into pip, native apps
- Packaging data files into python packages is annoying.
- There's `MANIFEST.in`, `data_files` and `package_data` in setup.py
- Bdist, wheel, generated source tarball use different set of these fields
and put the data files in different locations
- Rather just code the default config into a constant. Avoid
pointless file reads as well this way
- Assume path is absolute in yaml util module while saving, loading file
- This follows same convention as jsonl. Which just operates on
passed file path, assuming it is of appropriate form.
Responsibility to put it in appropriate form is on the caller, for now
- Include khoj_sample.yml in pip package to load default config from
- Create khoj config directory if it doesn't exist
- Load config from khoj_sample.yml if khoj.yml config doesn't exist
Conflicts:
- src/main.py
- router functions have moved to router
- move logic to handle null query perf timer variables into
router.py
- set main.py to current branch, not master
- If a user manually edits the input file lines, clicking start should
use that. Currently it just looks at the files selected last via file
browser
- We want to allow users to manually enter file paths in field. Which
is why the field hasn't been set to read-only
- Track current (saved/loaded) config separate from the new config (to
be written) when user clicks Start
- Fallback to using default config when no config for the specific
content type or processor is specified in khoj.yml
- Earlier were only loading default config on first run, not after
- Create Child CheckBox, LineEdit classes for Processor Widgets
- Create ProcessorType, similar to SearchType
- Track ProcessorType the widgets are associated with
- Simplify update, save, load of config based on type
