Debanjum Singh Solanky
1785047ea6
Improve understand primer and load understand response as dict
2021-11-28 13:04:16 +05:30
Saba
64645c3ac1
Begin type checking/input validation effort
2021-11-27 21:47:56 -05:00
Saba
9a0264b7fc
Add a dummy POST config endpoint, integrate with editable UI
2021-11-27 20:36:03 -05:00
Saba
f3b03ea5b7
Make raw data reactive to changes
2021-11-27 19:17:15 -05:00
Saba
7ffa70a001
Add button to modify text, not save
2021-11-27 18:00:47 -05:00
Saba
d150bd9ad7
Add stylesheet to config page
2021-11-27 14:39:05 -05:00
Debanjum Singh Solanky
67c3cd7372
Wire up GPT understand method to /chat API. Log conversation metadata too
2021-11-28 00:04:39 +05:30
Saba
49461a0c97
Working example for reading and display the config.yml
2021-11-27 11:14:49 -05:00
Saba
fa15d8dcf6
Rename readWriteConfig to just config.js
2021-11-27 10:52:54 -05:00
Saba
2e4a56daa3
Add script for reading config
2021-11-27 10:50:05 -05:00
Saba
3db06eee3f
Basic example of serving conifg as JSON and retriving on button click
2021-11-27 10:49:33 -05:00
Saba
3d4471e107
Merge branch 'master' of github.com:debanjum/semantic-search into saba/configui
2021-11-27 08:52:48 -05:00
Debanjum Singh Solanky
882e0f81b4
Skip running the inconsistent image search test
2021-11-27 18:38:44 +05:30
Debanjum Singh Solanky
ccfb97e1a7
Wire up minimal conversation processor. Expose it over /chat API endpoint
...
Ensure conversation history persists across application restart
2021-11-27 18:12:01 +05:30
Debanjum Singh Solanky
a99b4b3434
Make conversation processor configurable
2021-11-27 18:12:01 +05:30
Debanjum Singh Solanky
d4e1120b22
Add GPT based conversation processor to understand intent and converse with user
...
- Allow conversing with user using GPT's contextually aware, generative capability
- Extract metadata, user intent from user's messages using GPT's general understanding
2021-11-27 18:12:01 +05:30
Saba
3edecf91f2
Image reader, placeholder
2021-11-26 15:21:02 -05:00
Saba
baee52648d
Set up basic ui page with no functionality
2021-11-26 14:51:11 -05:00
Debanjum Singh Solanky
6244ccc01a
Add Configuring Application Section, Update Run command in Readme
2021-11-18 19:25:50 +05:30
debanjum
46661b3057
Ensure top_k never more than total entries to run symmetric search on
2021-11-16 11:32:21 -08:00
debanjum
8c858d1a94
Reduce symmetric search results for cross-encoder to re-rank to improve search speed
2021-11-16 11:31:19 -08:00
Debanjum Singh Solanky
f3fd5ae978
Improve code comments. Do not import unused modules in asymmetric search
2021-11-17 00:58:31 +05:30
Debanjum Singh Solanky
8cf2465e8e
Ensure top_k never more than total entries to search from
2021-11-17 00:56:31 +05:30
Debanjum Singh Solanky
4d37ace3d6
Reduce search results for cross-encoder to re-rank to improve search speed
...
Search time on my notes reduced from 14s to 4s. Cross-encoder
re-ranking step takes majority time, not the cosine similarity search
2021-11-17 00:50:28 +05:30
Debanjum Singh Solanky
0082631f16
Fixup caching step of Github workflow. Correct path to environment.yml
2021-10-03 01:38:37 -07:00
Debanjum Singh Solanky
faebde41d2
Cache conda dependencies for Github workflow execution
2021-10-03 01:04:14 -07:00
Debanjum Singh Solanky
5c92db7562
Try see if tests pass on a macos build for github action
2021-10-03 00:38:39 -07:00
Debanjum Singh Solanky
d896664fdb
Lock packages python dependencies to their current major version
...
- This may help reproduce test failures seen on Github locally
- Interfaces shouldn't break within bounds of minor version updates of
dependent packages, assuming their following semantic versioning
2021-10-02 20:34:07 -07:00
Debanjum Singh Solanky
9dcffe3e8e
Rename test_main to test_client. It only contains client specific tests
2021-10-02 20:32:53 -07:00
Debanjum Singh Solanky
7e0d9bafa7
Split test_main into client and search type specific test files
...
- Move search config fixture to conftests.py to be shared across tests
- Move image search type specific tests to test_image_search.py file
- Move, create asymmetric search type specific tests in new file
2021-10-02 20:28:33 -07:00
Debanjum Singh Solanky
da33e9e743
Create test directory with model data to reuse for pytest session
...
- Use pytest fixture with session scope
- Instantiate test directory with model data to reuse for tests
2021-10-02 19:46:29 -07:00
Debanjum Singh Solanky
1832e418e5
Use raw string for regex in orgnode to fix deprecation warning
2021-10-02 17:38:31 -07:00
Debanjum Singh Solanky
f59e321419
Update CLIP model load path
2021-10-02 16:50:06 -07:00
Debanjum Singh Solanky
31e21a6e4e
Update gitignore to ignore src/data and vscode directories
2021-10-02 16:30:13 -07:00
Debanjum Singh Solanky
c47a8cdf16
Allow configuring host, port or unix socket of server via CLI
2021-10-02 16:16:33 -07:00
Debanjum Singh Solanky
866ccb5cd3
Add all configurables to sample_config. Add test music, ledger data
...
- Get ledger sample from github.com/debanjum/company-ledger
- Get music sample from github.com/debanjum/org-music
2021-10-02 16:11:27 -07:00
Debanjum Singh Solanky
e933a5d3d0
Fix badge in readme, post workflow rename
2021-09-30 04:58:18 -07:00
Debanjum Singh Solanky
0a02dba25f
Rename workflow to build from tests
2021-09-30 04:57:32 -07:00
Debanjum Singh Solanky
cedd723721
Add tests badge to readme. Simplify name of tests workflow
2021-09-30 04:51:47 -07:00
Debanjum Singh Solanky
266a03b03d
Test application on Push or PR using Github Actions
2021-09-30 04:37:24 -07:00
Debanjum Singh Solanky
e8b7f06709
Ignore less files. Make gitignore more specific
2021-09-30 04:35:22 -07:00
Debanjum Singh Solanky
516f28b082
Merge branch 'master' of github.com:debanjum/semantic-search
2021-09-30 04:17:32 -07:00
Debanjum Singh Solanky
c200189385
Test notes search with explicit include, exclude filters
2021-09-30 04:13:40 -07:00
Debanjum Singh Solanky
d2905c4be6
Move tests out to project root. Use absolute import in project
...
tests/ directory in project root is more standard.
Just had to use absolute path for internal module imports to get it to
work
2021-09-30 04:12:14 -07:00
Debanjum Singh Solanky
58bb420f69
Fix image_metadata argument ordering bug. Add E2E image search test
...
- Image search test seems a little flaky
- Interchanged argument was causing inaccurate results earlier
2021-09-30 03:30:47 -07:00
Debanjum Singh Solanky
d5597442f4
Modularize Code. Wrap Search, Model Config in Classes. Add Tests
...
Details
- Rename method query_* to query in search_types for standardization
- Wrapping Config code in classes simplified mocking test config
- Reduce args beings passed to a function by passing it as single
argument wrapped in a class
- Minimize setup in main.py:__main__. Put most of it into functions
These functions can be mocked if required in tests later too
Setup Flow:
CLI_Args|Config_YAML -> (Text|Image)SearchConfig -> (Text|Image)SearchModel
2021-09-30 02:04:04 -07:00
Debanjum Singh Solanky
f4dd9cd117
Use type specific model for other search types too. Expose them via SearchModels
...
- Wrap Image, Music, Ledger search into the type of SearchModel they use
Similar to what was done for notes model by wrapping it's config
into an AsymmetricSearchModel.
- Use the uber wrapper class to expose all type specific search models
2021-09-29 21:09:42 -07:00
Debanjum Singh Solanky
352d2930ee
Use multiple threads to generate model embeddings. Other minor formating
2021-09-29 20:47:58 -07:00
Debanjum Singh Solanky
e22e0b41e3
Wrap asymmetric search model into SearchModels. Test notes search end-to-end
...
- Wrap asymmetric search model parameters into AsymmetricSearchModel class
- Create wrapper for all search type models. Put notes search model into it
- Test notes search end-to-end from client API layer to results.
Use model build on test data
2021-09-29 20:47:35 -07:00
Debanjum Singh Solanky
cde11a2331
Wrap search type enablement status in a search settings class
...
- Cleaner, more idiomatic usage of a global variable
- Simplifies mocking when testing client in pytest as setting wrapped
in object rather than a simple type. So passed around by reference
2021-09-29 19:18:33 -07:00
