khoj/src/processor
Debanjum Singh Solanky 2f7a6af56a Support incremental update of org-mode entries and embeddings
- What
  - Hash the entries and compare to find new/updated entries
  - Reuse embeddings encoded for existing entries
  - Only encode embeddings for updated or new entries
  - Merge the existing and new entries and embeddings to get the updated
    entries, embeddings

- Why
  - Given most note text entries are expected to be unchanged
    across time. Reusing their earlier encoded embeddings should
    significantly speed up embeddings updates
  - Previously we were regenerating embeddings for all entries,
    even if they had existed in previous runs
2022-09-10 20:58:33 +03:00
..
conversation Convert search_filter, conversation dir to proper modules 2022-08-02 20:23:42 +03:00
ledger Support incremental update of org-mode entries and embeddings 2022-09-10 20:58:33 +03:00
markdown Support incremental update of org-mode entries and embeddings 2022-09-10 20:58:33 +03:00
org_mode Support incremental update of org-mode entries and embeddings 2022-09-10 20:58:33 +03:00
__init__.py Move application files under src directory. Update Readmes 2021-08-17 04:11:03 -07:00