khoj/src/processor/org_mode
Debanjum Singh Solanky 2f7a6af56a Support incremental update of org-mode entries and embeddings
- What
  - Hash the entries and compare to find new/updated entries
  - Reuse embeddings encoded for existing entries
  - Only encode embeddings for updated or new entries
  - Merge the existing and new entries and embeddings to get the updated
    entries, embeddings

- Why
  - Given most note text entries are expected to be unchanged
    across time. Reusing their earlier encoded embeddings should
    significantly speed up embeddings updates
  - Previously we were regenerating embeddings for all entries,
    even if they had existed in previous runs
2022-09-10 20:58:33 +03:00
..
__init__.py Move application files under src directory. Update Readmes 2021-08-17 04:11:03 -07:00
org_to_jsonl.py Support incremental update of org-mode entries and embeddings 2022-09-10 20:58:33 +03:00
orgnode.py Indent Timestamps, Drawers at Body Level in OrgNode Entry Representation 2022-08-10 18:55:37 +03:00