Commit graph

4041 commits

Author SHA1 Message Date
sabaimran
9c403d24e1 Fix reference to directory in the eval workflow for starting terrarium 2024-12-08 13:03:05 -08:00
sabaimran
7cd2855146 Make attributes optional in the knowledge graph model 2024-12-08 12:23:17 -08:00
sabaimran
2af687d1c5 Allow snippetHighlighted to also be nullable 2024-12-08 11:51:24 -08:00
sabaimran
efa23a8ad8 Update validation requirements for online searches 2024-12-08 11:30:17 -08:00
sabaimran
6940c6379b Add sudo when running installations in order to install relevant packages
add --legacy-peer-deps temporarily to see if it helps mitigate the issue
2024-12-08 11:11:13 -08:00
sabaimran
4c4b7120c6 Use Khoj terrarium fork instead of building from official Cohere repo 2024-12-08 11:06:33 -08:00
sabaimran
a138845fea Merge branch 'master' of github.com:khoj-ai/khoj into features/allow-multi-outputs-in-chat 2024-12-08 10:57:16 -08:00
sabaimran
19832a3ed0 Add note to uncomment line when using the prod image
Some checks failed
dockerize / Publish Khoj Docker Images (push) Has been cancelled
build and deploy github pages for documentation / deploy (push) Has been cancelled
2024-12-05 18:16:01 -08:00
Debanjum
65c5b163c9 Add khoj_lantern svg to web public assets for use by new admin panel
Some checks failed
dockerize / Publish Khoj Docker Images (push) Waiting to run
build and deploy github pages for documentation / deploy (push) Waiting to run
pypi / Publish Python Package to PyPI (push) Has been cancelled
2024-12-05 10:57:18 -08:00
Debanjum
354dc12b3b
Style the Admin Panel with a modern theme and Khoj branding (#999)
Some checks failed
dockerize / Publish Khoj Docker Images (push) Waiting to run
build and deploy github pages for documentation / deploy (push) Waiting to run
pypi / Publish Python Package to PyPI (push) Waiting to run
pre-commit / Setup Application and Lint (push) Has been cancelled
test / Run Tests (push) Has been cancelled
Overview
- The default django admin panel UI looks pretty dated and didn't
  have any Khoj specific branding
- Used the Unfold Django admin panel theme for a modern look
- Used the Khoj logo and name in Admin panel title, headings, favicons

Details:
All models shown on Admin panel need to inherit from unfold's
ModelAdmin to get styling applied. So

- Make all models on Admin panel inherit from unfold's ModelAdmin
- Subclassed UserAdmin to inherit from unfold's ModelAdmin
- Deregistered the unused Auth Group model from the Admin panel
  We can add it back when its actually used. Avoid confusion for now
- Explicitly register DjangoJobExecution on admin panel and again make
  it inherit from the unfold.admin.ModelAdmin
2024-12-04 23:53:43 -08:00
Matias Forbord
9cc79c0fb7
Fix broken doc links to query filters from emacs docs page (#1000)
* docs: repair query filters link

* change docs link to be relative
2024-12-04 23:52:27 -08:00
sabaimran
8953ac03ec Rename additional context for llm response to program execution context 2024-12-04 18:43:41 -08:00
sabaimran
886fe4a0c9 Merge branch 'master' of github.com:khoj-ai/khoj into features/allow-multi-outputs-in-chat 2024-12-03 21:37:00 -08:00
sabaimran
df5e34615a Fix processing of images field when construct chat messages 2024-12-03 21:26:55 -08:00
sabaimran
3552032827 Rename additional context to additional_context_for_llm_response 2024-12-03 21:23:15 -08:00
Debanjum
9f7cb335c5
Create Android app for Khoj (#991)
Some checks failed
pypi / Publish Python Package to PyPI (push) Has been cancelled
dockerize / Publish Khoj Docker Images (push) Has been cancelled
build and deploy github pages for documentation / deploy (push) Has been cancelled
Use bubblewrap to publish the Khoj Progressive Web App (PWA) as a Trusted Web Activity (TWA) to the Android Play Store
2024-12-02 15:45:04 -08:00
Debanjum
ee28d7f125 Fix Android Studio build warnings by using newer gradle, mavenCentral 2024-12-02 12:48:33 -08:00
Debanjum
5d6fb07066 Fix app icons, orientation. Improve name, id, description in webmanifest
- Use bubblewrap generated splash screen, notification icons from
  1200x1200 high res khoj icon in assets.khoj.dev.
- Discard bubblewrap generated launcher icons.
- Fix orientation to respect phone orientation settings. "any" was not.
2024-12-02 12:48:25 -08:00
Debanjum
147c8e9115 Release v3 with high-res splash screen. More details in web, app manifest
- Add 512, 192 Khoj maskable icons to web app manifest for android rendering
- Add id, categories etc suggested by pwabuilder
- Use higher quality icon images for splash screen than what
  bubblewrap creates by default
2024-12-02 11:28:35 -08:00
Debanjum
d333e10e64 Encode request params as utf-8 to fix multibyte char error in khoj.el
Some checks failed
dockerize / Publish Khoj Docker Images (push) Waiting to run
build and deploy github pages for documentation / deploy (push) Waiting to run
pypi / Publish Python Package to PyPI (push) Waiting to run
test / Run Tests (push) Has been cancelled
test khoj.el / test (27.1) (push) Has been cancelled
build khoj.el / build (push) Has been cancelled
pre-commit / Setup Application and Lint (push) Has been cancelled
test khoj.el / test (27.2) (push) Has been cancelled
test khoj.el / test (28.1) (push) Has been cancelled
test khoj.el / test (28.2) (push) Has been cancelled
test khoj.el / test (snapshot) (push) Has been cancelled
Encode api key in header, POST request body and GET query param for
search as utf-8 to avoid the multibyte char in request issue when
making API calls from khoj.el to khoj server.

Resolves #935
2024-12-02 02:00:14 -08:00
Debanjum
db29894038
Do not wrap filepath in Path to fix indexing markdown files on Windows (#993)
### Issue
- Path with / are converted to \\ on Windows using the `Path' operator.
- The `markdown_to_entries' module was trying to normalize file paths with`Path'  for some reason.
  This would store the file paths in DB Entry differently than the file to entries map if Khoj ran on Windows.
  That'd result in a KeyError when trying to look up the entry file path from `file_to_text_map' in the `text_to_entries:update_embeddings()' function.

### Fix
- Removing the unnecessary OS dependent Path normalization in `markdown_to_entries' should keep the file path storage consistent across `file_to_text_map' var, `FileObjectAdaptor', `Entry' DB tables on Windows for Markdown files as well.

This issue will affect users hosting Khoj server on Windows and attempting to index markdown files.

Resolves #984
2024-12-02 01:02:58 -08:00
Debanjum
47c926b0ff Add more typing to org|md_to_entries. Remove redundant f-string wraps
- Add type hints to improve maintainability of stabilzed indexing code
- It shouldn't be necessary to wrap string variables in an f-string

This change aims to improve code quality. It should not affect
functionality.
2024-12-01 23:02:52 -08:00
Debanjum
dffdd81345 Do not wrap filepath in Path to fix indexing markdown files on Windows
Issue
- Path with / are converted to \\ on Windows using the Path operator.
- The markdown to entries method for some reason was doing this.
  This would store the file paths in DB entry differently than the file
  to entries map. Resulting in a KeyError when trying to look up the
  entry file path from file_to_text_map in the
  text_to_entries:update_embeddings() function.

Fix
- Removing the unnecessary OS dependendent Path normalization in
  markdown_to_entries should keep the file path storage consistent
  across file_to_text_map var, FileObjectAdaptor, Entry DB tables on
  Windows for Markdown files as well

This issue would only affect users hosting Khoj server on Windows and
attempting to index markdown files.

Resolves #984
2024-12-01 23:00:31 -08:00
sabaimran
c87fce5930 Add a migration to use the new image storage format for past conversations
- Added it to the Django migrations so that it auto-triggers when someone updates their server and starts it up again for the first time. This will require that they update their clients as well in order to view/consume image content.
- Remove server-side references in the code that allow to parse the text-to-image intent as it will no longer be necessary, given the chat logs will be migrated
2024-12-01 18:35:31 -08:00
Debanjum
9e0a2c7a98 Restrict generated chat title to 200 chars limit allowed for chat slug
Some checks are pending
dockerize / Publish Khoj Docker Images (push) Waiting to run
build and deploy github pages for documentation / deploy (push) Waiting to run
pre-commit / Setup Application and Lint (push) Waiting to run
pypi / Publish Python Package to PyPI (push) Waiting to run
test / Run Tests (push) Waiting to run
2024-11-30 19:12:03 -08:00
Debanjum
8b8e2be82d Only create subscription object when it does not exist for user
This avoid unnecessarily throwing an internal server error when the
user tries to sign-up using multiple mechanisms (e.g first by email, then
by google oauth)
2024-11-30 19:08:34 -08:00
Debanjum
fc6be543bd Improve GPQA eval prompt to imrpove parsing answer from Khoj response
Some checks are pending
build and deploy github pages for documentation / deploy (push) Waiting to run
pre-commit / Setup Application and Lint (push) Waiting to run
2024-11-30 17:21:09 -08:00
sabaimran
00f48dc1e8 If in the new images format, show the response text in obsidian instead of the inferred query 2024-11-30 14:39:51 -08:00
sabaimran
224abd14e0 Only add the image_url to the constructed chat message if it is a url 2024-11-30 14:39:27 -08:00
sabaimran
991577aa17 Allow a None turnId to accommodate historic chat messages 2024-11-30 14:39:08 -08:00
sabaimran
a539761c49 Fix processing of excalidrawdiagram in json response chunking 2024-11-30 12:35:13 -08:00
sabaimran
dc4a9ee3e1 Ensure that the generated assets are maintained in the chat window after streaming is completed. 2024-11-30 12:31:20 -08:00
sabaimran
e3aee50cf3 Fix parsing of generated_asset response 2024-11-29 18:41:53 -08:00
sabaimran
2b32f0e80d Remove commented out code blocks 2024-11-29 18:11:50 -08:00
sabaimran
df855adc98 Update response handling in Obsidian to work with new format 2024-11-29 18:10:47 -08:00
sabaimran
512cf535e0 Collapse train of thought when completed during live stream 2024-11-29 18:10:35 -08:00
sabaimran
a0b00ce4a1 Don't include null attributes when filling in stored conversation metadata
- Prompt adjustments to indicate to LLM what context it has
2024-11-29 18:10:14 -08:00
sabaimran
c5329d76ba Merge branch 'master' of github.com:khoj-ai/khoj into features/allow-multi-outputs-in-chat 2024-11-29 14:12:03 -08:00
sabaimran
46f647d91d Improve image rendering for khoj generated images. FIx typing of stored excalidraw image. 2024-11-29 14:11:48 -08:00
Debanjum
fdf69b7049 Publish second version with new upload key 2024-11-28 22:04:10 -08:00
Debanjum
faf15072b6 Create first version of Khoj Android app from PWA using Bubblewrap 2024-11-28 22:04:10 -08:00
sabaimran
4f6d1211ba Fix additional context type in anthropic chat 2024-11-28 20:16:36 -08:00
sabaimran
6f408948d3 Fix typing of generated_fiels parameters 2024-11-28 20:15:10 -08:00
sabaimran
439b18c21f Release Khoj version 1.30.10
Some checks failed
build khoj.el / build (push) Has been cancelled
desktop / 🖥️ Build, Release Desktop App (push) Has been cancelled
dockerize / Publish Khoj Docker Images (push) Has been cancelled
build and deploy github pages for documentation / deploy (push) Has been cancelled
pypi / Publish Python Package to PyPI (push) Has been cancelled
test khoj.el / test (27.1) (push) Has been cancelled
test khoj.el / test (27.2) (push) Has been cancelled
test khoj.el / test (28.1) (push) Has been cancelled
test khoj.el / test (28.2) (push) Has been cancelled
test khoj.el / test (snapshot) (push) Has been cancelled
2024-11-28 19:43:06 -08:00
sabaimran
2dfd163430 Add more explicity run strategies in the runner matrix 2024-11-28 19:31:34 -08:00
sabaimran
80cd902c86 Since linux/amd64 images aren't being created, try setting a custom description on the image
Refer to this GH documentation on working with multi arch images in the container registry:
https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#adding-a-description-to-multi-arch-images
2024-11-28 19:14:06 -08:00
sabaimran
40d8a7a581 Release Khoj version 1.30.9 2024-11-28 18:45:50 -08:00
sabaimran
87aa653c7f Add additional steps in prod.Dockerfile to ensure dependencies are copied over 2024-11-28 18:37:08 -08:00
sabaimran
d91935c880 Initial commit of a functional but not yet elegant prototype for this concept 2024-11-28 17:28:23 -08:00
Debanjum
a552543f4f Use json5 to parse llm generated questions to query docs and web
Some checks failed
build khoj.el / build (push) Waiting to run
desktop / 🖥️ Build, Release Desktop App (push) Waiting to run
dockerize / Publish Khoj Docker Images (push) Waiting to run
build and deploy github pages for documentation / deploy (push) Waiting to run
pypi / Publish Python Package to PyPI (push) Waiting to run
test khoj.el / test (27.1) (push) Waiting to run
test khoj.el / test (27.2) (push) Waiting to run
test khoj.el / test (28.1) (push) Waiting to run
test khoj.el / test (28.2) (push) Waiting to run
test khoj.el / test (snapshot) (push) Waiting to run
test / Run Tests (push) Has been cancelled
pre-commit / Setup Application and Lint (push) Has been cancelled
json5 is more forgiving, handles double quotes, newlines in raw json
string
2024-11-28 14:35:34 -08:00