sij/khoj
1
0
Fork 0
mirror of https://github.com/khoj-ai/khoj.git synced 2024-11-27 17:35:07 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

Set >=6Gb RAM required for offline chat

Browse source 
Llama v2 7B with 4bit quantization technically needs ~3.5Gb RAM (7B * 0.5byte), practically a system with 6Gb of RAM should suffice
This commit is contained in:
Debanjum 2023-10-18 12:05:54 -07:00 committed by GitHub
parent 8346e1193c
commit d93395ae48
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
docs/chat.md
View file

@ -10,7 +10,7 @@
Offline chat stays completely private and works without internet. But it is slower, lower quality and more compute intensive.
> **System Requirements**:
> - You need at least **16 GB of RAM** and **4 GB of Disk**
> - Machine with at least **6 GB of RAM** and **4 GB of Disk** available
> - A CPU supporting [AVX or AVX2 instructions](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions) is required
> - A Mac M1+ or [Vulcan supported GPU](https://vulkan.gpuinfo.org/) should significantly speed up chat response times