Mirror of khoj from Github
Find a file
Debanjum Singh Solanky e47922e53a Aggregate webpage extract queries to run once for each distinct webpage
This should reduce webpage read and response generation time.

Previously, we'd run separate webpage read and extract relevant
content pipes for each distinct (query, url) pair.

Now we aggregate all queries for each url to extract information from
and run the webpage read and extract relevant content pipes once for
each distinct url.

Even though the webpage content extraction pipes were previously being
in parallel. They increased response time by
1. adding more context for the response generation chat actor to
   respond from
2. and by being more susceptible to page read and extract latencies of
   the parallel jobs

The aggregated retrieval of context for all queries for a given
webpage could result in some hit to context quality. But it should
improve and reduce variability in response time, quality and costs.
2024-10-17 17:40:49 -07:00
.github Remove tools cache in dockerize.yml workflow 2024-09-29 00:27:37 -07:00
documentation Upgrade documentation website dependencies 2024-10-17 11:58:52 -07:00
scripts Update bump version script to bump new next.js web app version too 2024-08-05 16:20:47 +05:30
src Aggregate webpage extract queries to run once for each distinct webpage 2024-10-17 17:40:49 -07:00
tests Intelligently initialize a decent default set of chat model options 2024-09-19 20:32:08 -07:00
.dockerignore Use pypi khoj to fix docker builds and dockerize github workflow 2023-02-19 01:57:01 -06:00
.gitattributes Exclude tests data file from programming stats on Github 2023-08-28 11:00:52 -07:00
.gitignore Cycle through chat history in chat input on Obsidian (#861) 2024-08-12 23:55:25 -07:00
.pre-commit-config.yaml Add isort to the pre-commit configuration and apply it to the whole project (#595) 2023-12-28 18:04:02 +05:30
docker-compose.yml Intelligently initialize a decent default set of chat model options 2024-09-19 20:32:08 -07:00
Dockerfile Reduce size of Khoj Docker images by removing layers and caches 2024-09-29 04:06:35 -07:00
gunicorn-config.py Bump gunicorn workers per server up to 2 2024-04-18 11:32:51 +05:30
LICENSE Change license to GNU AGPLv3 from GNU GPLv3 2023-11-16 11:14:06 -08:00
manifest.json Release Khoj version 1.25.0 2024-10-10 18:07:30 -07:00
prod.Dockerfile Reduce size of Khoj Docker images by removing layers and caches 2024-09-29 04:06:35 -07:00
pyproject.toml Upgrade Django version used by Khoj server 2024-10-17 11:58:52 -07:00
pytest.ini Move the django app into the src/khoj folder for better organization and functionality 2023-11-21 10:56:04 -08:00
README.md Use Khoj icons. Add automation & improve agent text on web login page 2024-10-17 11:58:52 -07:00
versions.json Release Khoj version 1.25.0 2024-10-10 18:07:30 -07:00

Khoj Logo

test docker pypi discord

Your AI second brain

📑 Docs   •   🌐 Web   •   🔥 App   •   💬 Discord   •   ✍🏽 Blog


Khoj is a personal AI app to extend your capabilities. It smoothly scales up from an on-device personal AI to a cloud-scale enterprise AI.

  • Chat with any local or online LLM (e.g llama3, qwen, gemma, mistral, gpt, claude, gemini).
  • Get answers from the internet and your docs (including image, pdf, markdown, org-mode, word, notion files).
  • Access it from your Browser, Obsidian, Emacs, Desktop, Phone or Whatsapp.
  • Create agents with custom knowledge, persona, chat model and tools to take on any role.
  • Automate away repetitive research. Get personal newsletters and smart notifications delivered to your inbox.
  • Find relevant docs quickly and easily using our advanced semantic search.
  • Generate images, talk out loud, play your messages.
  • Khoj is open-source, self-hostable. Always.
  • Run it privately on your computer or try it on our cloud app.

See it in action

demo_chat

Go to https://app.khoj.dev to see Khoj live.

Full feature list

You can see the full feature list here.

Self-Host

To get started with self-hosting Khoj, read the docs.

Contributors

Cheers to our awesome contributors! 🎉

Made with contrib.rocks.

Interested in Contributing?

We are always looking for contributors to help us build new features, improve the project documentation, or fix bugs. If you're interested, please see our Contributing Guidelines and check out our Contributors Project Board.