sij/khoj
1
0
Fork 0
mirror of https://github.com/khoj-ai/khoj.git synced 2024-12-18 18:47:11 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 23
1472 commits 9 branches 114 tags 135 MiB
Commit graph

1472 commits

This branch
This branch
All branches
Author SHA1 Message Date
Saba
7ffa70a001 Add button to modify text, not save 2021-11-27 18:00:47 -05:00
Saba
d150bd9ad7 Add stylesheet to config page 2021-11-27 14:39:05 -05:00
Debanjum Singh Solanky
67c3cd7372 Wire up GPT understand method to /chat API. Log conversation metadata too 2021-11-28 00:04:39 +05:30
Saba
49461a0c97 Working example for reading and display the config.yml 2021-11-27 11:14:49 -05:00
Saba
fa15d8dcf6 Rename readWriteConfig to just config.js 2021-11-27 10:52:54 -05:00
Saba
2e4a56daa3 Add script for reading config 2021-11-27 10:50:05 -05:00
Saba
3db06eee3f Basic example of serving conifg as JSON and retriving on button click 2021-11-27 10:49:33 -05:00
Saba
3d4471e107 Merge branch 'master' of github.com:debanjum/semantic-search into saba/configui 2021-11-27 08:52:48 -05:00
Debanjum Singh Solanky
882e0f81b4 Skip running the inconsistent image search test 2021-11-27 18:38:44 +05:30
Debanjum Singh Solanky
ccfb97e1a7 Wire up minimal conversation processor. Expose it over /chat API endpoint 
Ensure conversation history persists across application restart
 2021-11-27 18:12:01 +05:30
Debanjum Singh Solanky
a99b4b3434 Make conversation processor configurable 2021-11-27 18:12:01 +05:30
Debanjum Singh Solanky
d4e1120b22 Add GPT based conversation processor to understand intent and converse with user 
- Allow conversing with user using GPT's contextually aware, generative capability
- Extract metadata, user intent from user's messages using GPT's general understanding
 2021-11-27 18:12:01 +05:30
Saba
3edecf91f2 Image reader, placeholder 2021-11-26 15:21:02 -05:00
Saba
baee52648d Set up basic ui page with no functionality 2021-11-26 14:51:11 -05:00
Debanjum Singh Solanky
6244ccc01a Add Configuring Application Section, Update Run command in Readme 2021-11-18 19:25:50 +05:30
debanjum
46661b3057 Ensure top_k never more than total entries to run symmetric search on 2021-11-16 11:32:21 -08:00
debanjum
8c858d1a94 Reduce symmetric search results for cross-encoder to re-rank to improve search speed 2021-11-16 11:31:19 -08:00
Debanjum Singh Solanky
f3fd5ae978 Improve code comments. Do not import unused modules in asymmetric search 2021-11-17 00:58:31 +05:30
Debanjum Singh Solanky
8cf2465e8e Ensure top_k never more than total entries to search from 2021-11-17 00:56:31 +05:30
Debanjum Singh Solanky
4d37ace3d6 Reduce search results for cross-encoder to re-rank to improve search speed 
Search time on my notes reduced from 14s to 4s. Cross-encoder
re-ranking step takes majority time, not the cosine similarity search
 2021-11-17 00:50:28 +05:30
Debanjum Singh Solanky
0082631f16 Fixup caching step of Github workflow. Correct path to environment.yml 2021-10-03 01:38:37 -07:00
Debanjum Singh Solanky
faebde41d2 Cache conda dependencies for Github workflow execution 2021-10-03 01:04:14 -07:00
Debanjum Singh Solanky
5c92db7562 Try see if tests pass on a macos build for github action 2021-10-03 00:38:39 -07:00
Debanjum Singh Solanky
d896664fdb Lock packages python dependencies to their current major version 
- This may help reproduce test failures seen on Github locally
- Interfaces shouldn't break within bounds of minor version updates of
  dependent packages, assuming their following semantic versioning
 2021-10-02 20:34:07 -07:00
Debanjum Singh Solanky
9dcffe3e8e Rename test_main to test_client. It only contains client specific tests 2021-10-02 20:32:53 -07:00
Debanjum Singh Solanky
7e0d9bafa7 Split test_main into client and search type specific test files 
- Move search config fixture to conftests.py to be shared across tests
- Move image search type specific tests to test_image_search.py file
- Move, create asymmetric search type specific tests in new file
 2021-10-02 20:28:33 -07:00
Debanjum Singh Solanky
da33e9e743 Create test directory with model data to reuse for pytest session 
- Use pytest fixture with session scope
- Instantiate test directory with model data to reuse for tests
 2021-10-02 19:46:29 -07:00
Debanjum Singh Solanky
1832e418e5 Use raw string for regex in orgnode to fix deprecation warning 2021-10-02 17:38:31 -07:00
Debanjum Singh Solanky
f59e321419 Update CLIP model load path 2021-10-02 16:50:06 -07:00
Debanjum Singh Solanky
31e21a6e4e Update gitignore to ignore src/data and vscode directories 2021-10-02 16:30:13 -07:00
Debanjum Singh Solanky
c47a8cdf16 Allow configuring host, port or unix socket of server via CLI 2021-10-02 16:16:33 -07:00
Debanjum Singh Solanky
866ccb5cd3 Add all configurables to sample_config. Add test music, ledger data 
- Get ledger sample from github.com/debanjum/company-ledger
- Get music sample from github.com/debanjum/org-music
 2021-10-02 16:11:27 -07:00
Debanjum Singh Solanky
e933a5d3d0 Fix badge in readme, post workflow rename 2021-09-30 04:58:18 -07:00
Debanjum Singh Solanky
0a02dba25f Rename workflow to build from tests 2021-09-30 04:57:32 -07:00
Debanjum Singh Solanky
cedd723721 Add tests badge to readme. Simplify name of tests workflow 2021-09-30 04:51:47 -07:00
Debanjum Singh Solanky
266a03b03d Test application on Push or PR using Github Actions 2021-09-30 04:37:24 -07:00
Debanjum Singh Solanky
e8b7f06709 Ignore less files. Make gitignore more specific 2021-09-30 04:35:22 -07:00
Debanjum Singh Solanky
516f28b082 Merge branch 'master' of github.com:debanjum/semantic-search 2021-09-30 04:17:32 -07:00
Debanjum Singh Solanky
c200189385 Test notes search with explicit include, exclude filters 2021-09-30 04:13:40 -07:00
Debanjum Singh Solanky
d2905c4be6 Move tests out to project root. Use absolute import in project 
tests/ directory in project root is more standard.
Just had to use absolute path for internal module imports to get it to
work
 2021-09-30 04:12:14 -07:00
Debanjum Singh Solanky
58bb420f69 Fix image_metadata argument ordering bug. Add E2E image search test 
- Image search test seems a little flaky
- Interchanged argument was causing inaccurate results earlier
 2021-09-30 03:30:47 -07:00
Debanjum Singh Solanky
d5597442f4 Modularize Code. Wrap Search, Model Config in Classes. Add Tests 
Details
  - Rename method query_* to query in search_types for standardization
  - Wrapping Config code in classes simplified mocking test config
  - Reduce args beings passed to a function by passing it as single
    argument wrapped in a class
  - Minimize setup in main.py:__main__. Put most of it into functions
    These functions can be mocked if required in tests later too

Setup Flow:
  CLI_Args|Config_YAML -> (Text|Image)SearchConfig -> (Text|Image)SearchModel
 2021-09-30 02:04:04 -07:00
Debanjum Singh Solanky
f4dd9cd117 Use type specific model for other search types too. Expose them via SearchModels 
- Wrap Image, Music, Ledger search into the type of SearchModel they use
  Similar to what was done for notes model by wrapping it's config
  into an AsymmetricSearchModel.

- Use the uber wrapper class to expose all type specific search models
 2021-09-29 21:09:42 -07:00
Debanjum Singh Solanky
352d2930ee Use multiple threads to generate model embeddings. Other minor formating 2021-09-29 20:47:58 -07:00
Debanjum Singh Solanky
e22e0b41e3 Wrap asymmetric search model into SearchModels. Test notes search end-to-end 
- Wrap asymmetric search model parameters into AsymmetricSearchModel class
- Create wrapper for all search type models. Put notes search model into it
- Test notes search end-to-end from client API layer to results.
  Use model build on test data
 2021-09-29 20:47:35 -07:00
Debanjum Singh Solanky
cde11a2331 Wrap search type enablement status in a search settings class 
- Cleaner, more idiomatic usage of a global variable
- Simplifies mocking when testing client in pytest as setting wrapped
  in object rather than a simple type. So passed around by reference
 2021-09-29 19:18:33 -07:00
Debanjum Singh Solanky
81ce0cacc3 Only allow supported search types to /search, /regenerate APIs 
- Use a SearchType to limit types that can be passed by user
- FastAPI automatically validates type passed in query param
- Available type options show up in Swagger UI, FastAPI docs
- controller code looks neater instead of doing string comparisons for type
- Test invalid, valid search types via pytest
 2021-09-29 19:12:56 -07:00
Debanjum Singh Solanky
5db08c5293 Set query as heading of notes search results in Emacs Org buffer 2021-09-29 13:30:15 -07:00
Debanjum Singh Solanky
150593c776 Update Readme. Acknowledger PyExifTool and Minor Fixes 2021-09-16 12:39:42 -07:00
Debanjum Singh Solanky
fdb60a8dcf Set Query as Heading of Image Search Results Emacs Buffer 2021-09-16 12:30:06 -07:00