sij/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2025-03-13 13:42:23 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions1
anything-llm/collector/utils/extensions
History
Timothy Carambat d1ca16f7f8
Add tokenizer improvments via Singleton class and estimation () 
* Add tokenizer improvments via Singleton class
linting

* dev build

* Estimation fallback when string exceeds a fixed byte size

* Add notice to tiktoken on backend
2025-01-30 17:55:03 -08:00
..
Confluence Add tokenizer improvments via Singleton class and estimation () 2025-01-30 17:55:03 -08:00
RepoLoader Add tokenizer improvments via Singleton class and estimation () 2025-01-30 17:55:03 -08:00
WebsiteDepth Add tokenizer improvments via Singleton class and estimation () 2025-01-30 17:55:03 -08:00
YoutubeTranscript Add tokenizer improvments via Singleton class and estimation () 2025-01-30 17:55:03 -08:00