5 commits

Author	SHA1	Message	Date
Debanjum Singh Solanky	e9f86e320b	Fix and improve offline chat actor, director tests ... - Use updated references schema with compiled key - Enable director tests that are now expected to pass and that do pass (with Gemma 2 at least)	2024-07-18 03:43:09 +05:30
Debanjum Singh Solanky	f2db8d7d99	Fix offline chat actor tests ... Do not check for original q in extracted questions. Since this was removed in a previous commit	2024-04-24 09:40:00 +05:30
sabaimran	60658a8037	Get rid of enable flag for the offline chat processor config ... - Default, assume that offline chat is enabled if there is an offline chat model option configured	2024-04-23 23:08:29 +05:30
Debanjum Singh Solanky	886d49e3a4	Merge branch 'master' into migrate-to-llama-cpp-for-offline-chat	2024-03-31 00:59:20 +05:30
Debanjum Singh Solanky	8ca39a436c	Use llama.cpp for offline chat models ... - Benefits of moving to llama-cpp-python from gpt4all: - Support for all GGUF format chat models - Support for AMD, Nvidia, Mac, Vulcan GPU machines (instead of just Vulcan, Mac) - Supports models with more capabilities like tools, schema enforcement, speculative ddecoding, image gen etc. - Upgrade default chat model, prompt size, tokenizer for new supported chat models - Load offline chat model when present on disk without requiring internet - Load model onto GPU if not disabled and device has GPU - Load model onto CPU if loading model onto GPU fails - Create helper function to check and load model from disk, when model glob is present on disk. `Llama.from_pretrained' needs internet to get repo info from HuggingFace. This isn't required, if the model is already downloaded Didn't find any existing HF or llama.cpp method that looked for model glob on disk without internet	2024-03-26 22:33:01 +05:30

Renamed from tests/test_gpt4all_chat_actors.py (Browse further)