Commit graph

28 commits

Author SHA1 Message Date
Debanjum Singh Solanky
dcb6fe479e Fix date_filter query, entry in query range check. Add tests for it
- Fix date_filter date_in_entry within query range check
  - Extracted_date_range is in [included_date, excluded_date) format
  - But check was checking for date_in_entry <= excluded_date
  - Fixed it to do date_in_entry < excluded_date

- Fix removal of date filter from query
- Add tests for date_filter
2022-07-14 20:01:35 +04:00
Debanjum Singh Solanky
011f81fac5 Fix date_filter to handle non overlapping date ranges 2022-07-14 18:53:38 +04:00
Debanjum Singh Solanky
70ac35b2a5 Compute Date Range to filter entries to, from Comparators, Dates in Query 2022-07-14 18:20:09 +04:00
Debanjum Singh Solanky
e6db3e3d00 Prefer Dates From Future only when specific words in date string
- Default to looking at dates from past, as most notes are from past
- Look for dates in future for cases where it's obvious query is for
  dates in the future but dateparser's parse doesn't parse it at all.
  E.g parse('5 months from now') returns nothing

- Setting PREFER_DATES_FROM_FUTURE in this case and passing just
  parse('5 months') to dateparser.parse works as expected
2022-07-14 18:13:12 +04:00
Debanjum Singh Solanky
4a201d52af Add, test date filter regex and date parsing to get natural date range 2022-07-14 16:47:32 +04:00
Debanjum Singh Solanky
741fca0e6b Fix asymmetric search test to pass entries returned by query to collate_results 2022-07-12 18:48:49 +04:00
Debanjum Singh Solanky
8bb9a49994 Cleanup Test Asymmetric Search to Fix Test
- test_regenerate_with_valid_content failed when run after test_asymmetric_search
- test_asymmetric_search did't clean the temporary update to config it had made
- This was resulting in regenerate looking for a file that didn't exist
2022-07-07 01:25:31 +04:00
Debanjum Singh Solanky
eda4b65ddb Improve Query Speed. Normalize Embeddings, Moving them to Cuda GPU
- Move embeddings to CUDA GPU for compute, when available
- Normalize embeddings and Use Dot Product instead of Cosine
2022-06-30 00:59:57 +04:00
Debanjum Singh Solanky
2f7ef08b11 Add Unit Tests to verify the Reload API functions as desired 2022-06-29 23:47:17 +04:00
Debanjum Singh Solanky
85fbe1c42b Normalize org notes path to be relative to home directory
- This is still clunky but it should be commitable
- General enough that it'll work even when a users notes are not in the home directory
- While solving for the special case where:
  - Notes are being processed on a different machine and used on a different machine
  - But the notes directory is in the same location relative to home on both the machines
2022-06-28 19:16:11 +04:00
Debanjum Singh Solanky
f66192f2a7 Test OrgNode Parsing and Rendering 2022-06-17 19:13:11 +03:00
Debanjum Singh Solanky
79c2224eaa Improve test data organization and update correspoding conftests
- Put test data for each content type into separate directories
- Makes config.yml for docker and local host consistent
  - Prepending tests to /data in sample_config.yml makes application
    run on local host using test data
  - Allows mounting separate volume for each content type in docker-compose
- Ignore gitignore to only add tests content, not generated models or embeddings
2022-01-29 02:03:17 -05:00
Debanjum Singh Solanky
179153dc5a Rename RawConfig Types for Consistency
- Naming convention - [ContentType][ConfigType]Config
  - Where [ConfigType] ~ Content, Search, Processor
  - Where [ContentType] ~ Text, Image, Asymmetric, Symmetric, Conversation

- Current Configs:
  - Content:
    - Org Notes
    - Org Music
    - Image
    - Ledger/Beancount

  - Search:
     - Asymmetric
     - Symmetric
     - Image

  - Processor:
    - Conversation
2022-01-14 20:54:38 -05:00
Debanjum Singh Solanky
ed144f7984 Setup Search with Search_Config to Fix Tests
- Rename pytest fixture search_config to more appropriate
content_config
- Create search_config pytest fixture
- Use search_config where search being setup, used in tests
2022-01-14 20:13:14 -05:00
Debanjum Singh Solanky
2e53fbc844 Fix the user intent extraction prompt for GPT. Clean up chatbot test 2022-01-12 10:36:01 -05:00
Debanjum
ef911aa6be
Skip Flaky Image Search Test
Image search doesn't always return expected image path.
Should resolve remaining issues with failing cloud test. See #11
2021-12-12 02:15:20 -08:00
Saba
ba8dc9ed5f Update the search_config instantiated for tests in conftest 2021-12-11 14:14:31 -05:00
Saba
d65190c3ee Update unit tests, files with removing model suffix to config types 2021-12-09 08:50:38 -05:00
Saba
76e9e9da2f Update unit tests to use the new BaseModel types 2021-12-05 09:31:39 -05:00
Saba
c4cd4b57f1 Update types used in conftest.py 2021-12-04 12:02:19 -05:00
Debanjum Singh Solanky
882e0f81b4 Skip running the inconsistent image search test 2021-11-27 18:38:44 +05:30
Debanjum Singh Solanky
d4e1120b22 Add GPT based conversation processor to understand intent and converse with user
- Allow conversing with user using GPT's contextually aware, generative capability
- Extract metadata, user intent from user's messages using GPT's general understanding
2021-11-27 18:12:01 +05:30
Debanjum Singh Solanky
9dcffe3e8e Rename test_main to test_client. It only contains client specific tests 2021-10-02 20:32:53 -07:00
Debanjum Singh Solanky
7e0d9bafa7 Split test_main into client and search type specific test files
- Move search config fixture to conftests.py to be shared across tests
- Move image search type specific tests to test_image_search.py file
- Move, create asymmetric search type specific tests in new file
2021-10-02 20:28:33 -07:00
Debanjum Singh Solanky
da33e9e743 Create test directory with model data to reuse for pytest session
- Use pytest fixture with session scope
- Instantiate test directory with model data to reuse for tests
2021-10-02 19:46:29 -07:00
Debanjum Singh Solanky
866ccb5cd3 Add all configurables to sample_config. Add test music, ledger data
- Get ledger sample from github.com/debanjum/company-ledger
- Get music sample from github.com/debanjum/org-music
2021-10-02 16:11:27 -07:00
Debanjum Singh Solanky
c200189385 Test notes search with explicit include, exclude filters 2021-09-30 04:13:40 -07:00
Debanjum Singh Solanky
d2905c4be6 Move tests out to project root. Use absolute import in project
tests/ directory in project root is more standard.
Just had to use absolute path for internal module imports to get it to
work
2021-09-30 04:12:14 -07:00