Commit graph

2055 commits

Author SHA1 Message Date
sabaimran
38144a7a69 pull_request path should be src/khoj rather than src/ 2023-11-21 11:33:07 -08:00
Debanjum
e5130fb3f3
Fix ranking search results on Obsidian (#560)
This bug was causing the search results on the Obsidian client to be shown in the reverse order of their actual relevance.

It reversed since entry scores returned by Khoj server are a distance metric since the move to Postgres. So lesser distance is better. Previously higher score was better.
2023-11-21 11:32:47 -08:00
sabaimran
333cb3445c Use colon rather than equals to indicate typing 2023-11-21 11:28:51 -08:00
Debanjum Singh Solanky
645fd96634 Search across all content types from Khoj Obsidian client
Previously it was only searching for PDF and Markdown files. This was
meant to show only content from current vault as results.

But it has not scaled well as other clients also allow syncing PDF and
markdown files now. So remove this content type filter for now.

A proper solution would limit by using file/dir filters on server or
client side.
2023-11-21 11:19:33 -08:00
sabaimran
a1460a5bf9 Set operations to typed empty list in migration file 2023-11-21 11:14:40 -08:00
sabaimran
8932fc0c36 Ignore w004 check to bypass pypi warnings for check-wheel-contents
- PyPi doesn't like to have files that start with numbers, however all of the generated django migration files start with numbers. To accommodate, skip this check.
- Refer to https://pypi.org/project/check-wheel-contents/ for documentation and recommendation
2023-11-21 11:12:50 -08:00
sabaimran
71e794c26f Remove the sys.append line in the main.py file, as it's not required 2023-11-21 10:57:21 -08:00
sabaimran
a474c31e02 Move the django app into the src/khoj folder for better organization and functionality
- Our pypi package currently does not work because the django app and associated database is not included. To remedy this issue, move the app into the src/khoj folder. This has the added benefit of improved organization of the codebase, as all server related code is now in a single folder
- Update associated file paths and system references
2023-11-21 10:56:04 -08:00
Debanjum Singh Solanky
c89bd49973 Fix ranking search results on Obsidian
It's reversed since score of entries is now a distance metric on
Khoj server. So lesser distance is better. Previously higher score was
better
2023-11-21 01:24:59 -08:00
Debanjum
6d8e889917
Improve Self Hosted Khoj Setup (#557)
- c07401cf Fix, Improve chat config via CLI on first run by using defaults
- d61b0dd5 Add Khoj Django app package to sys path to load Django module via pip install
- 4e98acbc Update minimum pydantic version to one with model_validate function
2023-11-20 17:25:53 -08:00
Daniel Grossmann-Kavanagh
f142999bce fix khoj local server usage 2023-11-20 17:07:30 -08:00
Debanjum Singh Solanky
c07401cf76 Fix, Improve chat config via CLI on first run by using defaults
- Fix setting prompt size for online chat
- generally improve chat config via cli by using default chat model,
  prompt size for online and offline chat
2023-11-20 17:01:20 -08:00
sabaimran
b142de15a8 Merge branch 'features/internet-enabled-search' of github.com:khoj-ai/khoj into features/reflective-suggested-questions 2023-11-20 15:56:09 -08:00
sabaimran
a9623ef85a Add requisite imports in order to instantiate offline model in adapters file 2023-11-20 15:27:42 -08:00
sabaimran
a8f13f334f Fix merging issues with base after popping the stash 2023-11-20 15:22:50 -08:00
sabaimran
8fa0b69c67 Resolve merge issue with adapters methods 2023-11-20 15:21:06 -08:00
sabaimran
fee99779bf Add subqueries for internet-connected search results and update client-side code accordingly
- Add a wrapper method to help make direct queries to the LLM and determine any intermediate responses needed for handling the request
2023-11-20 15:19:15 -08:00
Debanjum Singh Solanky
d61b0dd55c Add Khoj Django app package to sys path to load Django module via pip install 2023-11-20 14:55:00 -08:00
Debanjum Singh Solanky
4e98acbca7 Update minimum pydantic version to one with model_validate function 2023-11-20 14:52:37 -08:00
sabaimran
b8e6883a81 Merge branch 'master' of github.com:khoj-ai/khoj into features/internet-enabled-search 2023-11-19 16:20:08 -08:00
sabaimran
237195e20e Make all name-related fields nullable within the GoogleUser 2023-11-19 14:22:32 -08:00
sabaimran
4def8cce36
Merge pull request #541 from asim-shrestha/patch-1
Add test separators
2023-11-19 14:14:34 -08:00
Debanjum
71799add0b
Index Parent Headings of Org-Mode Entries to Improve Search Context (#548)
### Overview
The parent hierarchy of org-mode entries can store important context. 
This change updates OrgNode to track parent headings for each org entry and adds the parent outline for each entry to the index

### Details
- Test search uses ancestor headings as context for improved results
- Add ancestor headings of each org-mode entry to their compiled form
- Track ancestor headings for each org-mode entry in org-node parser

Resolves #85
2023-11-19 13:18:19 -08:00
sabaimran
e398a76779 Fix test word filter 2023-11-19 13:14:58 -08:00
sabaimran
33a9304428 Resolve merge conflicts 2023-11-19 12:57:55 -08:00
sabaimran
cfd76b8472 Add open graph links to configure Khoj Docs preview 2023-11-19 12:16:59 -08:00
sabaimran
ef5e9d66c1 Resolve merge conflicts in dependency imports 2023-11-19 11:42:20 -08:00
Debanjum Singh Solanky
c3465d6982 Release Khoj version 1.0.0 2023-11-19 09:50:25 -08:00
Debanjum
736744be3a
Update documentation to reflect new multi-user config scenario (#550)
- Update docs to show how to use Khoj Cloud
- Move self-hosting Khoj to separate section
- Add page to setup Desktop app
- Set default URL to Khoj Cloud URL in Obsidian, Emacs clients
2023-11-18 18:22:46 -08:00
Debanjum Singh Solanky
d0e84385f2 Simplify links in Khoj docs to use page_name.md with no prefixes
This allows jumping to page via VSCode IDE and on docs website
2023-11-18 18:17:46 -08:00
Debanjum Singh Solanky
fc65d8a9fe Add documentation page for the Khoj Desktop client 2023-11-18 18:17:35 -08:00
Debanjum Singh Solanky
35b469e488 Simplify setup, features since Khoj cloud in docs
- No Khoj server setup required to start using Khoj from Obsidian, Emacs
- Use tabs for install, upgrade in Emacs with different package
  managers
- Use default subtitles in Khoj Docs
- Deduplicate query filters, remove backend setup instructions in
  plugin pages
- Remove stale Setup demo on Khoj Obsidian plugin docs
2023-11-18 17:25:52 -08:00
Debanjum Singh Solanky
e1bf1f0e86 Update default Khoj server URL to Khoj cloud on Emacs, Obsidian clients 2023-11-18 16:25:45 -08:00
Debanjum Singh Solanky
8775ce730a Use URL fragments to allow jumping to config page sections on Web app 2023-11-18 16:25:45 -08:00
sabaimran
a5613cb08a
Merge pull request #554 from khoj-ai/fix/issues-with-prod-chat
Fix misc. issues with chat configuration
2023-11-18 14:45:06 -08:00
sabaimran
f792b1e301 Remove already defined identical function 2023-11-18 14:08:50 -08:00
sabaimran
e2fff5dc47 Don't explicitly use value to get the model type value 2023-11-18 14:01:01 -08:00
sabaimran
a8a25ceac2 Honor user's chat settings when running the extract questions phase
- Add marginally better error handling when GPT gives a messed up respones to the extract questions method
- Remove debug log lines
2023-11-18 13:31:51 -08:00
sabaimran
67156e6aec Add new logs for debugging issues with chat references 2023-11-18 12:10:50 -08:00
sabaimran
5de2ab6098 Change parse_obj calls to use model_validate per new pydantic specification 2023-11-18 12:10:36 -08:00
sabaimran
ebdb423d3e
Merge pull request #553 from khoj-ai/features/validation-errors
Update types of base config models for pydantic 2.0
2023-11-18 00:42:56 -08:00
sabaimran
6d249645a6 Fix interpretation of the default search type 2023-11-18 00:04:18 -08:00
sabaimran
f180b2ba94 Resolve mypy errors for various data types 2023-11-17 23:26:15 -08:00
sabaimran
3328a41f08 Update types of base config models for pydantic 2.0 2023-11-17 23:08:52 -08:00
sabaimran
f688529150 Update the default configuration for the AppConfig 2023-11-17 19:26:31 -08:00
sabaimran
11ccb92755 Fix formatting of welcome message to use markdown 2023-11-17 18:55:59 -08:00
Debanjum Singh Solanky
ca87b4ede9 Wrap common API query parameters into shared class to deduplicate code
- Upgrade FastAPI to >= latest version. Required upgrade of FastAPI.
  Earlier version didn't support wrapping common query params in class

- Use per fixture app instead of a global FastAPI app in conftest

- Upgrade minimum required Django version

- Fix no notes chat director test with updated no notes message
  No notes message was updated in commit 118f1143
2023-11-17 18:43:49 -08:00
sabaimran
262f3ccb59 Resolve mypy issues with formatting 2023-11-17 17:11:00 -08:00
sabaimran
a7e00898cb Fix rendering even when no online context references are returned 2023-11-17 16:41:28 -08:00
sabaimran
0fcf234f07 Add support for using serper.dev for online queries
- Use the knowledgeGraph, answerBox, peopleAlsoAsk and organic responses of serper.dev to provide online context for queries made with the /online command
- Add it as an additional tool for doing Google searches
- Render the results appropriately in the chat web window
- Pass appropriate reference data down to the LLM
2023-11-17 16:19:11 -08:00