Debanjum Singh Solanky
d9f60c00bf
Warn if any input files to org-to-json are potentially non org-mode files
...
That is, if the file paths in the input set don't end with .org
2021-08-16 13:49:39 -07:00
Debanjum Singh Solanky
3aa0c30fee
Use absolute file path to open files in org-to-jsonl.py, asymmetric.py
...
Exit script if neither org_files, org_file_filter is present
2021-08-16 13:49:39 -07:00
Debanjum Singh Solanky
e773611558
Remove unused jsonl_file argument from convert_org_entries_to_jsonl
2021-08-16 13:49:35 -07:00
Debanjum Singh Solanky
8b29e272d3
Standardize interface, better default args for org-to-json.py script
...
- Remove non-standard, unnecessary argument for org-directory
Pass path each file in org-files and org-files-filter argument directly
- Allow shorthand -i, -o for input files, output files
- Default to compress, unless user explicitly specifies not to
2021-08-16 11:29:08 -07:00
Debanjum Singh Solanky
7547e90745
Minor doc updates after merging emacs package with main repository
2021-08-16 02:02:26 -07:00
Debanjum Singh Solanky
ec157ea0ff
Add Emacs interface to semantic-search directly to main repository
...
Too much overhead to maintain multiple repositories, especially when
the Emacs library for semantic-search is a single file.
Import Readme from the emacs-semantic-search repository too
2021-08-16 01:27:46 -07:00
Debanjum Singh Solanky
dcf7b2d04f
Remove requirements.txt for now as virtualenv setup doesn't work
...
Haven't gotten it to work on Mac or Ubuntu. Remove to avoid confusion
for now. Application depends on miniconda for now
2021-08-16 00:15:10 -07:00
Debanjum Singh Solanky
3b81fafa3e
Use updated path to MiniLM bi-encoder model on hugging-face
2021-08-15 23:57:22 -07:00
Debanjum Singh Solanky
4839153086
Acknowledge ML models used for search. Simplify path used in commands
2021-08-15 23:56:18 -07:00
Debanjum Singh Solanky
c58c1d96aa
Change default install directory to current, fix open file code
2021-08-15 23:01:55 -07:00
Debanjum Singh Solanky
ae15e429b5
Reduce indentation from 4 to 2 in Readme.md.
...
Prevent everything looking like code blocks due to 4 space indentations
2021-08-15 22:56:36 -07:00
Debanjum Singh Solanky
636b6195cc
Add Readme, License. Update .gitignore
2021-08-15 22:52:37 -07:00
Debanjum Singh Solanky
354c541b62
Add org processor to generate compressed jsonl from org-mode files
...
The corpus embeddings are generated from this compressed JSONL
using the specified transformer ML model
2021-08-15 22:52:31 -07:00
Debanjum Singh Solanky
b74cb9a104
Move install.py to new utils dir as it's for cmdline ease of use only
2021-08-15 19:10:30 -07:00
Debanjum Singh Solanky
ec92f3e146
Move different search types into search_types directory
2021-08-15 19:09:50 -07:00
Debanjum Singh Solanky
4d681c86ec
Update requirements.txt for users wanting to use pip install
2021-08-15 18:45:37 -07:00
Debanjum Singh Solanky
d75df54385
Create API interface for Semantic Search
...
Use FastAPI, Uvicorn to create app with API endpoint at /search
Example Query: http://localhost:8000/?q= "why sleep?"&t="notes'&n=5
2021-08-15 18:11:48 -07:00
Debanjum Singh Solanky
e3088c8cf8
Create environment.yml to install prerequisites for app via conda
2021-08-15 17:48:38 -07:00
Debanjum Singh Solanky
660e6c3937
Add explicit filters to asymmetric search
...
User can filter results to ones which include, exclude specified words
To show entries which include, exclude specific words, user should prepend
a '+', '-' before the word. E.g "+hello -bye"
2021-08-15 17:48:38 -07:00
Debanjum Singh Solanky
91a2c598fe
Resolve paths to absolute paths once. Use pathlib glob directly
2021-08-09 00:39:33 -07:00
Debanjum Singh Solanky
ca0a22f4dd
Search for images similar to query image provided by the user
...
Example user passes path to an image in query. e.g ~/Pictures/photo.jpg
The script should return images in images_embedding most similar to
the query image
2021-08-09 00:21:02 -07:00
Debanjum Singh Solanky
00d0065c5b
Allow user to search images via text queries
2021-08-08 23:02:30 -07:00
Debanjum Singh Solanky
181cab89d2
Ignore Title Notes i.e notes with just headings from compute
2021-08-04 21:30:09 -07:00
Debanjum Singh Solanky
d6d7b9d6a8
Make installed script executable. Minor clean-up of duplicate code
2021-08-04 18:29:20 -07:00
Debanjum Singh Solanky
2eb029a7b0
Create script to install semantic-search as a program
2021-08-02 00:29:09 -07:00
Debanjum Singh Solanky
13d5100ce6
Rename script similarity to symmetric
2021-07-31 20:37:07 -07:00
Debanjum Singh Solanky
ad7e90bec3
Modularize script, provide cmdline control, improve results rendering
2021-07-31 17:13:39 -07:00
Debanjum Singh Solanky
eb03f57917
Save, Load Embeddings to/from file to speed up script load time
2021-07-31 10:13:41 -07:00
Debanjum Singh Solanky
0914f284bb
Re-rank using cross encoder to get even more relevant results
...
The cross encoder re-ranked results are much better for more distant queries.
It does take more time with the cross-encoder re-ranking but it seems
worth it to get more relevant results
2021-07-31 03:10:44 -07:00
Debanjum Singh Solanky
9864a2b551
Retrieve most relevant entries for a query using MSMarco based bi-encoder
...
Returns best 3 results ranked by MSMarco based biencoder score of
query match to entries from org-mode notes
2021-07-31 00:20:37 -07:00
debanjum
0ef5495701
Use Sentence Transformers to Encode, Query Schedule.org Headings
2021-04-04 04:53:03 -07:00
