sij/khoj
1
0
Fork 0
mirror of https://github.com/khoj-ai/khoj.git synced 2024-11-23 23:48:56 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 16

Set >=6Gb RAM required for offline chat

Browse source 
Llama v2 7B with 4bit quantization technically needs ~3.5Gb RAM (7B * 0.5byte), practically a system with 6Gb of RAM should suffice
This commit is contained in:
Debanjum 2023-10-18 12:05:54 -07:00 committed by GitHub
parent 8346e1193c
commit d93395ae48
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
docs/chat.md
View file

@ -10,7 +10,7 @@
Offline chat stays completely private and works without internet. But it is slower, lower quality and more compute intensive. Offline chat stays completely private and works without internet. But it is slower, lower quality and more compute intensive.
> **System Requirements**: > **System Requirements**:
> - You need at least **16 GB of RAM** and **4 GB of Disk** > - Machine with at least **6 GB of RAM** and **4 GB of Disk** available
> - A CPU supporting [AVX or AVX2 instructions](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions) is required > - A CPU supporting [AVX or AVX2 instructions](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions) is required
> - A Mac M1+ or [Vulcan supported GPU](https://vulkan.gpuinfo.org/) should significantly speed up chat response times > - A Mac M1+ or [Vulcan supported GPU](https://vulkan.gpuinfo.org/) should significantly speed up chat response times