sij/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2025-03-13 21:52:22 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions1
1201 commits 35 branches 12 tags 149 MiB
Commit graph

3 commits

Author SHA1 Message Date
Timothy Carambat
d1ca16f7f8
Add tokenizer improvments via Singleton class and estimation () 
* Add tokenizer improvments via Singleton class
linting

* dev build

* Estimation fallback when string exceeds a fixed byte size

* Add notice to tiktoken on backend
 2025-01-30 17:55:03 -08:00
Sean Hatfield
9b86bbd2b8
[FIX] PDFLoader module bug fix () 
use pdf.js by importing it from pdf-parse and fix custom PDFLoader module
 2024-07-16 13:09:43 -07:00
Sean Hatfield
79656718b2
[FEAT] Create custom pdfloader () 
* implement custom PDFLoader to remove LC dep

* remove unneeded comment

* remove pdfjs as dep and fix page splitting using pdf-parse

* linting + export rename for desktop compat

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
 2024-07-11 12:26:11 -07:00