Mirror of khoj from Github
Find a file
Debanjum c015eeb5dd
Improve Online Search: Parallelize Search, Use Jina Reader API by default (#832)
- Overview
  Khoj wil be able to do online search out of the box, even for self-hosted users
  - Default to Jina search, reader API when no Serper.dev, Olostep API keys
  - Run online searches in parallel to process multiple queries faster

- Details
  - Jina provides a [reader API](https://github.com/jina-ai/reader) for online search and web page reading
     It requires no API key. This provides a good default to enable 
     online search for self-hosted readers requiring no additional setup. 

  - Jina search API also returns webpage contents with the results, so
     just use those directly when Jina Search API used instead of
     trying to read webpages separately. The extract relevant content from
     webpage step using a chat model is still used from the
    `read_webpage_and_extract_content' func in this case.

  - Parse search results from Jina search API into same format as
     Serper.dev for accurate rendering of online references by clients

  - Run online searches in parallel with AsyncIO to process multiple queries faster
2024-07-02 17:44:51 +05:30
.github In dockerize.yml file, revert the added configuration 2024-06-22 21:11:25 +05:30
documentation Update tests and documentation with Jina reader API usage and info 2024-07-02 17:19:09 +05:30
scripts Use Yarn instead of NPM to bump Desktop, Obsidian client versions 2024-06-26 10:37:58 +05:30
src Improve Online Search: Parallelize Search, Use Jina Reader API by default (#832) 2024-07-02 17:44:51 +05:30
tests Update tests and documentation with Jina reader API usage and info 2024-07-02 17:19:09 +05:30
.dockerignore Use pypi khoj to fix docker builds and dockerize github workflow 2023-02-19 01:57:01 -06:00
.gitattributes Exclude tests data file from programming stats on Github 2023-08-28 11:00:52 -07:00
.gitignore Add developer support for using next.js to serve generated static files (#814) 2024-06-22 20:12:41 +05:30
.pre-commit-config.yaml Add isort to the pre-commit configuration and apply it to the whole project (#595) 2023-12-28 18:04:02 +05:30
docker-compose.yml Spell fix s/e.g/e.g./ across code, tests and docs 2024-06-24 15:24:45 +05:30
Dockerfile Add developer support for using next.js to serve generated static files (#814) 2024-06-22 20:12:41 +05:30
gunicorn-config.py Bump gunicorn workers per server up to 2 2024-04-18 11:32:51 +05:30
LICENSE Change license to GNU AGPLv3 from GNU GPLv3 2023-11-16 11:14:06 -08:00
manifest.json Release Khoj version 1.15.0 2024-06-26 10:38:16 +05:30
prod.Dockerfile Add developer support for using next.js to serve generated static files (#814) 2024-06-22 20:12:41 +05:30
pyproject.toml Install pydantic with optional email validator package 2024-06-24 16:12:20 +05:30
pytest.ini Move the django app into the src/khoj folder for better organization and functionality 2023-11-21 10:56:04 -08:00
README.md Add links for contributors to use in the readme 2024-04-12 12:49:12 +05:30
versions.json Release Khoj version 1.15.0 2024-06-26 10:38:16 +05:30

Khoj Logo

test dockerize pypi Discord

The open-source, personal AI for your digital brain

🤖 Read Docs   •   🏮 Khoj Cloud   •   💬 Get Involved   •   📚 Read Blog


Khoj is an application that creates always-available, personal AI agents for you to extend your capabilities.

  • You can share your notes and documents to extend your digital brain.
  • Your AI agents have access to the internet, allowing you to incorporate realtime information.
  • Khoj is accessible on Desktop, Emacs, Obsidian, Web and Whatsapp.
  • You can share pdf, markdown, org-mode, notion files and github repositories.
  • You'll get fast, accurate semantic search on top of your docs.
  • Your agents can create deeply personal images and understand your speech.
  • Khoj is open-source, self-hostable. Always.

See it in action

Khoj Demo

Go to https://app.khoj.dev to see Khoj live.

Full feature list

You can see the full feature list here.

Self-Host

To get started with self-hosting Khoj, read the docs.

Contributors

Cheers to our awesome contributors! 🎉

Made with contrib.rocks.

Interested in Contributing?

We are always looking for contributors to help us build new features, improve the project documentation, or fix bugs. If you're interested, please see our Contributing Guidelines and check out our Contributors Project Board.

Sponsors

Shout out to our brilliant sponsors! 🌈