sij/khoj

mirror of https://github.com/khoj-ai/khoj.git synced 2024-12-01 03:13:01 +01:00

Author	SHA1	Message	Date
sabaimran	439b18c21f	Release Khoj version 1.30.10 Some checks failed build khoj.el / build (push) Has been cancelled Details desktop / 🖥️ Build, Release Desktop App (push) Has been cancelled Details dockerize / Publish Khoj Docker Images (push) Has been cancelled Details build and deploy github pages for documentation / deploy (push) Has been cancelled Details pypi / Publish Python Package to PyPI (push) Has been cancelled Details test khoj.el / test (27.1) (push) Has been cancelled Details test khoj.el / test (27.2) (push) Has been cancelled Details test khoj.el / test (28.1) (push) Has been cancelled Details test khoj.el / test (28.2) (push) Has been cancelled Details test khoj.el / test (snapshot) (push) Has been cancelled Details	2024-11-28 19:43:06 -08:00
sabaimran	2dfd163430	Add more explicity run strategies in the runner matrix	2024-11-28 19:31:34 -08:00
sabaimran	80cd902c86	Since linux/amd64 images aren't being created, try setting a custom description on the image Refer to this GH documentation on working with multi arch images in the container registry: https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#adding-a-description-to-multi-arch-images	2024-11-28 19:14:06 -08:00
sabaimran	40d8a7a581	Release Khoj version 1.30.9	2024-11-28 18:45:50 -08:00
sabaimran	87aa653c7f	Add additional steps in prod.Dockerfile to ensure dependencies are copied over	2024-11-28 18:37:08 -08:00
Debanjum	a552543f4f	Use json5 to parse llm generated questions to query docs and web Some checks failed build khoj.el / build (push) Waiting to run Details desktop / 🖥️ Build, Release Desktop App (push) Waiting to run Details dockerize / Publish Khoj Docker Images (push) Waiting to run Details build and deploy github pages for documentation / deploy (push) Waiting to run Details pypi / Publish Python Package to PyPI (push) Waiting to run Details test khoj.el / test (27.1) (push) Waiting to run Details test khoj.el / test (27.2) (push) Waiting to run Details test khoj.el / test (28.1) (push) Waiting to run Details test khoj.el / test (28.2) (push) Waiting to run Details test khoj.el / test (snapshot) (push) Waiting to run Details test / Run Tests (push) Has been cancelled Details pre-commit / Setup Application and Lint (push) Has been cancelled Details json5 is more forgiving, handles double quotes, newlines in raw json string	2024-11-28 14:35:34 -08:00
Debanjum	0a69af4f61	Update to latest ToDesktop runtime	2024-11-28 13:56:14 -08:00
Debanjum	1d0fe141dc	Release Khoj version 1.30.8	2024-11-28 13:37:30 -08:00
Debanjum	29e801c381	Add MATH500 dataset to eval Evaluate simpler MATH500 responses with gemini 1.5 flash This improves both the speed and cost of running this eval	2024-11-28 12:48:25 -08:00
Debanjum	22aef9bf53	Add GPQA (diamond) dataset to eval	2024-11-28 12:48:25 -08:00
Debanjum	f1190ccf32	Improve parsing complex json strings returned by LLM (#989 ) - Improve escaping to load complex json objects - Fallback to a more forgiving [json5](https://json5.org/) loader if `json.loads` cannot parse complex json str This should reduce failures to pick research tool and run code by agent	2024-11-28 11:01:39 -08:00
Debanjum	8c120a5139	Fallback to json5 loader if json.loads cannot parse complex json str JSON5 spec is more flexible, try to load using a fast json5 parser if the stricter json.loads from the standard library can't load the raw complex json string into a python dictionary/list	2024-11-26 21:17:00 -08:00
Debanjum	70b7e7c73a	Improve load of complex json objects. Use it to pick tool, run code Gemini doesn't work well when trying to output json objects. Using it to output raw json strings with complex, multi-line structures requires more intense clean-up of raw json string for parsing	2024-11-26 17:37:57 -08:00
Debanjum	8cb0db0051	Fix llama-cpp-python install by pytest github workflow Some checks failed dockerize / Publish Khoj Docker Images (push) Has been cancelled Details build and deploy github pages for documentation / deploy (push) Has been cancelled Details pre-commit / Setup Application and Lint (push) Has been cancelled Details pypi / Publish Python Package to PyPI (push) Has been cancelled Details test / Run Tests (push) Has been cancelled Details - Use pre-built wheels for torch and llama-cpp-python - Install and link musl as it's used by llama-cpp-python pre-built wheel instead of glibc - Join Install git and Install Dependencies steps in pytest workflow To remove unnecessary steps	2024-11-26 02:04:36 -08:00
Debanjum	29315f44e7	Add assetlinks.json to link android app to app.khoj.dev domain Add sha cert of android upload, signing keys to open debug, prod apps as TWA in fullscreen on android phones	2024-11-26 01:57:54 -08:00
Debanjum	a97a45bf20	Align agent personality with recently updated khoj personality See update to Khoj personality in commit `6eb59464da`	2024-11-26 00:06:16 -08:00
Debanjum	e088fcbc7b	Build for arm64 on arm64 runner. Parallelize arm64, x64 docker builds Some checks failed dockerize / Publish Khoj Docker Images (push) Has been cancelled Details build and deploy github pages for documentation / deploy (push) Has been cancelled Details pypi / Publish Python Package to PyPI (push) Has been cancelled Details - Building arm64 image on an ubuntu arm64 runner reduces `yarn build' step time by 75% from 12mins to 3mins. - This is because no QEMU emulation for arm64 on x86 is required now - Parallelizing x64 and arm64 platform builds halves build time on top - Revert to use standard ubuntu-latest runner as large x64 runner doesn't give much more speed improvements This results an effective additional 50%-66% reduction in build time on top of #987. So a full dockerize workflow run now takes 10 mins vs previous 35+mins. This is a total of 72% improvement in max dockerize run time. Get additional speed improvements when docker layer cache hit.	2024-11-24 23:18:55 -08:00
Debanjum	5723a3778e	Speed up Docker image builds using multi-stage parallel pipelines (#987 ) ## Objective Improve build speed and size of khoj docker images ## Changes ### Improve docker image build speeds - Decouple web app and server build steps - Build the web app and server in parallel - Cache docker layers for reuse across dockerize github workflow runs - Split Docker build layers for improved cacheability (e.g separate `yarn install` and `yarn build` steps) ### Reduce size of khoj docker images - Use an up-to-date `.dockerignore` to exclude unnecessary directories - Do not installing cuda python packages for cpu builds ### Improve web app builds - Use consistent mechanism to get fonts for web app - Make tailwind extensions production instead of dev dependencies - Make next.js create production builds for the web app (via `NODE_ENV=production` env var)	2024-11-24 21:49:46 -08:00
Debanjum	4a5646c8da	Cache docker layers, nextjs builds in dockerize github workflow	2024-11-24 21:06:22 -08:00
Debanjum	6a39651ad3	Standardize loading fonts locally across pages on web app	2024-11-24 20:41:15 -08:00
sabaimran	9368699b2c	Migrate the pre-commit config Some checks failed dockerize / Publish Khoj Docker Images (push) Waiting to run Details build and deploy github pages for documentation / deploy (push) Waiting to run Details pypi / Publish Python Package to PyPI (push) Waiting to run Details test / Run Tests (push) Has been cancelled Details pre-commit / Setup Application and Lint (push) Has been cancelled Details	2024-11-24 14:54:26 -08:00
sabaimran	6eb59464da	Add additional reinforcement to coax gemini into giving a minimum helpful response	2024-11-24 14:53:53 -08:00
sabaimran	15f062b34a	Remove print statement for agent style map	2024-11-24 14:53:53 -08:00
sabaimran	d7e68a2d1b	Wait for iplcodata to load before first message - Fix the console khoj ai ascii art - Remove some not so good suggested prompt	2024-11-24 14:53:53 -08:00
Debanjum	f51e0f7859	Make Next.js create production builds of web app for Docker images	2024-11-24 13:59:40 -08:00
Debanjum	710e00ad9e	Make tailwind extensions prod, instead of dev, deps of web app	2024-11-24 13:59:40 -08:00
Debanjum	4b486ea5f6	Exclude unnecessary directories from final docker builds	2024-11-24 13:59:40 -08:00
Debanjum	78d8ca49ec	Skip Nvidia GPU python packages during Server install in Dockerfiles	2024-11-24 13:59:39 -08:00
Debanjum	37887a175a	Speed up Docker image builds using multi-stage parallel pipelines Decouple web app, server builds in parallel to speed up Docker builds	2024-11-24 12:48:30 -08:00
Debanjum	7c77d65d35	Improve logic to disable telemetry via KHOJ_TELEMETRY_DISABLE env var Some checks failed dockerize / Publish Khoj Docker Images (push) Waiting to run Details build and deploy github pages for documentation / deploy (push) Waiting to run Details pre-commit / Setup Application and Lint (push) Waiting to run Details pypi / Publish Python Package to PyPI (push) Waiting to run Details test / Run Tests (push) Waiting to run Details build khoj.el / build (push) Has been cancelled Details desktop / 🖥️ Build, Release Desktop App (push) Has been cancelled Details test khoj.el / test (27.1) (push) Has been cancelled Details test khoj.el / test (27.2) (push) Has been cancelled Details test khoj.el / test (28.1) (push) Has been cancelled Details test khoj.el / test (28.2) (push) Has been cancelled Details test khoj.el / test (snapshot) (push) Has been cancelled Details The newly added KHOJ_TELEMETRY_DISABLE env var knob to disable telemetry should override old config mechanism when set	2024-11-24 00:54:16 -08:00
sabaimran	2d683898c2	Release Khoj version 1.30.7	2024-11-23 22:51:10 -08:00
sabaimran	914ff994f7	Fix cost addition to chat_metadata Some checks are pending build khoj.el / build (push) Waiting to run Details desktop / 🖥️ Build, Release Desktop App (push) Waiting to run Details dockerize / Publish Khoj Docker Images (push) Waiting to run Details build and deploy github pages for documentation / deploy (push) Waiting to run Details pre-commit / Setup Application and Lint (push) Waiting to run Details pypi / Publish Python Package to PyPI (push) Waiting to run Details test / Run Tests (push) Waiting to run Details test khoj.el / test (27.1) (push) Waiting to run Details test khoj.el / test (27.2) (push) Waiting to run Details test khoj.el / test (28.1) (push) Waiting to run Details test khoj.el / test (28.2) (push) Waiting to run Details test khoj.el / test (snapshot) (push) Waiting to run Details	2024-11-23 22:50:45 -08:00
Debanjum	caaa127dcf	Release Khoj version 1.30.6	2024-11-23 21:07:00 -08:00
Debanjum	57b8273002	Fix apt install for musl-dev in prod.Dockerfile	2024-11-23 21:06:09 -08:00
Debanjum	8f966b11ec	Release Khoj version 1.30.5	2024-11-23 20:49:05 -08:00
Debanjum	498895a47d	Fix libmusl error using pre-built llama-cpp-python wheel in prod Docker	2024-11-23 20:47:41 -08:00
Debanjum	e5b211a743	Release Khoj version 1.30.4	2024-11-23 19:48:21 -08:00
Debanjum	9848d89d03	Try build docker images with github high cpu, ram runner	2024-11-23 19:09:36 -08:00
Debanjum	04bb3d6f15	Fix libmusl error using pre-built llama-cpp-python wheel via Docker Seems like llama-cpp-python pre-built wheels need libmusl. Otherwise you run into runtime errors on Khoj startup via Docker.	2024-11-23 18:46:44 -08:00
Debanjum	8dd2122817	Set sample size to 200 for automated eval runs as well	2024-11-23 14:48:38 -08:00
Debanjum	c4ef31d86f	Release Khoj version 1.30.3 Some checks are pending build khoj.el / build (push) Waiting to run Details desktop / 🖥️ Build, Release Desktop App (push) Waiting to run Details dockerize / Publish Khoj Docker Images (push) Waiting to run Details build and deploy github pages for documentation / deploy (push) Waiting to run Details pypi / Publish Python Package to PyPI (push) Waiting to run Details test khoj.el / test (27.1) (push) Waiting to run Details test khoj.el / test (27.2) (push) Waiting to run Details test khoj.el / test (28.1) (push) Waiting to run Details test khoj.el / test (28.2) (push) Waiting to run Details test khoj.el / test (snapshot) (push) Waiting to run Details	2024-11-23 14:40:06 -08:00
Debanjum	15ae22bdcf	Use pre-built llama-cpp-python wheel in Khoj docker images Reduces build time and resolves FileNotFoundError 'ninja' during llama-cpp-python local build.	2024-11-23 14:38:07 -08:00
sabaimran	4ac49ca90f	Release Khoj version 1.30.2	2024-11-23 12:00:28 -08:00
Debanjum	5aa5cb1941	Add "New" section with latest updates to Readme Some checks are pending build and deploy github pages for documentation / deploy (push) Waiting to run Details	2024-11-23 01:36:50 -08:00
sabaimran	7f5bf35806	Disambiguate renewal_date type. Previously, being used as None, False, and Datetime in different places. Some checks failed dockerize / Publish Khoj Docker Images (push) Waiting to run Details build and deploy github pages for documentation / deploy (push) Waiting to run Details pypi / Publish Python Package to PyPI (push) Waiting to run Details pre-commit / Setup Application and Lint (push) Has been cancelled Details test / Run Tests (push) Has been cancelled Details	2024-11-22 12:06:20 -08:00
sabaimran	5e8c824ecc	Improve the experience for finding past conversation - add a conversation title search filter, and an agents filter, for finding conversations - in the chat session api, return relevant agent style data	2024-11-22 12:03:01 -08:00
sabaimran	a761865724	Fix handling of customer.subscription.updated event to process new renewal end date	2024-11-22 12:03:01 -08:00
sabaimran	6a054d884b	Add quicker/easier filtering on auth	2024-11-22 12:03:01 -08:00
Debanjum	b9a889ab69	Fix Khoj responses when code generated charts in response context The current fix should improve Khoj responses when charts in response context. It truncates code context before sharing with response chat actors. Previously Khoj would respond with it not being able to create chart but than have a generated chart in it's response in default mode. The truncate code context was added to research chat actor for decision making but it wasn't added to conversation response generation chat actors. When khoj generated charts with code for its response, the images in the context would exceed context window limits. So the truncation logic to drop all past context, including chat history, context gathered for current response. This would result in chat response generator 'forgetting' all for the current response when code generated images, charts in response context.	2024-11-21 14:43:52 -08:00
Debanjum	5475a262d4	Move truncate code context func for reusability across modules It needs to be used across routers and processors. It being in run_code tool makes it hard to be used in other chat provider contexts due to circular dependency issues created by send_message_to_model_wrapper func	2024-11-21 14:27:39 -08:00

1 2 3 4 5 ...

3997 commits