sij/khoj
1
0
Fork 0
mirror of https://github.com/khoj-ai/khoj.git synced 2025-02-17 08:04:21 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
Mirror of khoj from Github
68 commits 15 branches 128 tags 142 MiB
Python 50.1%
TypeScript 37.6%
Emacs Lisp 3.1%
HTML 3%
JavaScript 2.9%
Other 3.3%
Find a file
Debanjum Singh Solanky d8abbc0552 Use XMP metadata in images to improve image search 
- Details
  - The CLIP model can represent images, text in the same vector space

  - Enhance CLIP's image understanding by augmenting the plain image
    with it's text based metadata.
    Specifically with any subject, description XMP tags on the image

  - Improve results by combining plain image similarity score with
    metadata similarity scores for the highest ranked images

- Minor Fixes
  - Convert verbose to integer from bool in image_search.
    It's already passed as integer from the main program entrypoint

  - Process images with ".jpeg" extensions too
2021-09-16 08:55:20 -07:00
src Use XMP metadata in images to improve image search 2021-09-16 08:55:20 -07:00
.gitignore Add Readme, License. Update .gitignore 2021-08-15 22:52:37 -07:00
environment.yml Enable Semantic Search on Images 2021-08-22 21:42:37 -07:00
LICENSE Add Readme, License. Update .gitignore 2021-08-15 22:52:37 -07:00
README.org Update Readme to state can now query beancount transactions, images 2021-08-22 21:50:27 -07:00
sample_config.yml Update sample config to add minimal config for ledger, image search 2021-08-22 21:54:49 -07:00

README.org

Semantic Search

Allow natural language search on user content like notes, images, transactions using transformer based models

All data is processed locally. User can interface with semantic-search app via Emacs, API or Commandline

Dependencies

Install 

git clone https://github.com/debanjum/semantic-search && cd semantic-search
conda env create -f environment.yml
conda activate semantic-search

Run

Load ML model, generate embeddings and expose API to query specified org-mode files

python3 src/main.py -c=sample_config.yml --verbose

Use

Upgrade 

  cd semantic-search
  git pull origin master
  conda env update -f environment.yml
  conda activate semantic-search

Acknowledgments