Commit graph

80 commits

Author SHA1 Message Date
Timothy Carambat
ad01df8790
Reranker option for RAG ()
* Reranker WIP

* add cacheing and singleton loading

* Add field to workspaces for vectorSearchMode
Add UI for lancedb to change mode
update all search endpoints to pass in reranker prop if provider can use it

* update hint text

* When reranking, swap score to rerank score

* update optchain
2025-01-02 14:27:52 -08:00
timothycarambat
d52249aff8 support attachments via thread API chat/stream-chat endpoints 2024-12-31 09:52:21 -08:00
Sean Hatfield
f8885a43e4
Append writable fields to dev API new workspace endpoint ()
* add writible fields to dev api new workspace endpoint

* lint

* implement validations for workspace model

* update swagger comments

* simplify validations for workspace on frontend and API

* cleanup validations

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-12-16 15:12:20 -08:00
Timothy Carambat
dd7c4675d3
LLM performance metric tracking ()
* WIP performance metric tracking

* fix: patch UI trying to .toFixed() null metric
Anthropic tracking migraiton
cleanup logs

* Apipie implmentation, not tested

* Cleanup Anthropic notes, Add support for AzureOpenAI tracking

* bedrock token metric tracking

* Cohere support

* feat: improve default stream handler to track for provider who are actually OpenAI compliant in usage reporting
add deepseek support

* feat: Add FireworksAI tracking reporting
fix: improve handler when usage:null is reported (why?)

* Add token reporting for GenericOpenAI

* token reporting for koboldcpp + lmstudio

* lint

* support Groq token tracking

* HF token tracking

* token tracking for togetherai

* LiteLLM token tracking

* linting + Mitral token tracking support

* XAI token metric reporting

* native provider runner

* LocalAI token tracking

* Novita token tracking

* OpenRouter token tracking

* Apipie stream metrics

* textwebgenui token tracking

* perplexity token reporting

* ollama token reporting

* lint

* put back comment

* Rip out LC ollama wrapper and use official library

* patch images with new ollama lib

* improve ollama offline message

* fix image handling in ollama llm provider

* lint

* NVIDIA NIM token tracking

* update openai compatbility responses

* UI/UX show/hide metrics on click for user preference

* update bedrock client

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
2024-12-16 14:31:17 -08:00
Timothy Carambat
ff024286d0
Add new workspace user management endpoint ()
deprecate old endpoint which users workspace ID vs slug
resolves 
2024-12-16 11:50:34 -08:00
Sean Hatfield
79cdcd4e64
Add thread name to workspaces get endpoint ()
add thread name to workspaces get endpoint

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-12-13 17:53:54 -08:00
Timothy Carambat
a46eb2418a
add Gitlab to watchable documents () 2024-12-12 10:44:11 -08:00
Timothy Carambat
538078747d
Add vector search API endpoint ()
* Add vector search API endpoint

* Add missing import

* Modify the data that is returned

* Change similarityThreshold to scoreThreshold

As this is what is actually returned by the search

* Removing logging (oops!)

* chore: regen swagger docs for new endpoint
fix: update function to sanity check values to prevent crashes during search

---------

Co-authored-by: Scott Bowler <scott@dcsdigital.co.uk>
2024-12-12 10:12:32 -08:00
timothycarambat
a0c5d898f0 patch docs with proper input for OpenAI compatible embeddings
resolves 
2024-12-11 08:15:52 -08:00
Sean Hatfield
e719d05027
Add optional limit and orderBy to dev api chat endpoint ()
add optional limit and orderBy to dev api chat endpoint
2024-10-30 14:47:24 -07:00
Mr Simon C
ccde891aa2
Feature/update api example outputs ()
* /v1/workspace/{slug} actually returns an array - update example output

* update swagger example to show array output
2024-10-29 16:50:19 -07:00
Timothy Carambat
2c9cb28d5f
Simple SSO feature for login flows from external services ()
* Simple SSO feature for login flows from external services

* linting
2024-10-29 15:30:53 -07:00
Timothy Carambat
40800631e3
Patch v1/document/upload filename charset encoding () 2024-10-25 12:03:19 -07:00
Timothy Carambat
72ba9f7f28
Add filtering to sessionID for workspace chats () 2024-10-24 15:10:50 -07:00
Timothy Carambat
e71392d83f
Feature/thread creation slug name ()
* thread creation additional params name and slug, with api

* typo fix

* Rebuild openai Swagger docs
Handle validations for fields to prevent invalid field inputs for .new
Enforce sluggification of `slug` to prevent breaking of URL structs

---------

Co-authored-by: abrakadobr <abrakadobr@gmail.com>
2024-10-21 13:09:55 -07:00
Sean Hatfield
be6289d141
Daily message limit per user ()
* set message limit per user

* remove old limit user messages + unused admin page

* fix daily message validation

* refactor message limit input
refactor canSendChat on user to a method on user model

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-10-15 14:01:29 -07:00
Sean Hatfield
e6c4eb3f1c
Support attachments in developer API ()
* support attachments in developer api

* lint

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-09-25 13:44:26 -07:00
Timothy Carambat
0cbe4d0b04
Make userId actually optional for workspaceThread endpoint () 2024-09-12 13:33:41 -07:00
Mr Simon C
7594841dac
Bug/make swagger json output openapi 3 compliant ()
update source to ensure swagger.json is openapi 3.0.0 compliant
2024-09-04 15:40:24 -07:00
timothycarambat
47a5c7126c Patch path traversal in move-files that can be used by administrator level attacker only 2024-08-27 16:19:12 -07:00
Timothy Carambat
fdc3add53c
Api session id support ()
* Refactor api endpoint chat handler to its own function
remove legacy `chatWithWorkspace` and cleanup `index.js`

* Add `sessionId` in dev API to partition chats logically statelessly
2024-08-21 15:25:47 -07:00
Timothy Carambat
1f96b837b3
Refactor api endpoint chat handler to its own function ()
remove legacy `chatWithWorkspace` and cleanup `index.js`
2024-08-21 14:47:06 -07:00
timothycarambat
17abbe97e5 remove #swagger.path from API docs 2024-08-20 12:48:08 -07:00
timothycarambat
c9e19264eb fix validation issue 2024-08-07 14:24:24 -07:00
Sean Hatfield
665553af5a
Developer API access for embed chat widgets ()
* developer api access for embedded chat widgets and embed chats

* lint

* uncheck files

* sanitize embed api inputs

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-08-07 11:36:32 -07:00
Timothy Carambat
d072875e43
Add piperTTS in-browser text-to-speech ()
* Add piperTTS in-browser text-to-speech

* update vite config

* Add voice default + change prod public URL

* uncheck file

* Error handling
bump package for better quality and voices

* bump package

* Remove pre-packed WASM - will not support offline first solution for docker

* attach TTSProvider telem
2024-08-07 11:09:51 -07:00
Timothy Carambat
ae58a2cb0d
Return threads for API workspace calls ()
* Return threads for API workspace calls

* remove import
2024-07-26 14:18:40 -07:00
timothycarambat
d0713e15f1 patch response body for swagger 2024-07-22 15:21:32 -07:00
timothycarambat
7b42a67acf docs: update requirement for /raw-text endpoint
resolves 
2024-07-22 13:19:57 -07:00
Timothy Carambat
3198718975
Update references to new domain () 2024-07-22 11:05:34 -07:00
Johnny Chen
c4158b97d6
bugfix: API new user return 400 if user creation failed ()
* return 400 if user creation failed

The client might not check the error field unless they realize it's a 4xx error instead of 200 success.

* Update index.js

* use one-liner style

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-07-19 15:27:21 -07:00
Timothy Carambat
29c9eeaa5c
Add winston logging for production () 2024-07-03 16:39:33 -07:00
Sean Hatfield
77916d920b
[FEAT] Remove users_can_delete_workspaces feature ()
remove all references to users_can_delete_workspaces option
2024-06-25 16:46:06 -07:00
Timothy Carambat
88a0335fd2
Add OpenAI compatible API support () 2024-06-24 18:28:21 -07:00
timothycarambat
c176fe38d2 remove async on dumpEnv 2024-06-20 20:50:14 -07:00
timothycarambat
ed53bbcf15 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2024-06-20 18:03:33 -07:00
timothycarambat
9411791fe5 Simplify ENV management
auto write ENV on changes
Dont double-track envs that are already tracked
2024-06-20 18:03:21 -07:00
Sean Hatfield
12bf576e16
[FEAT] Workspace threads developer API endpoints ()
* WIP workspace threads developer api endpoints

* fix swagger docs

* fix broken endpoints and test on single/multi user modes

* support for chatting with workspace thread by userId

* update swagger json
2024-06-20 16:27:35 -07:00
Sean Hatfield
1c473621aa
[FEAT] Add user management endpoints to list all users via developer API ()
* add user management endpoints to list all users via developer api

* rename users to user management in swagger docs

* linting

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-06-17 16:24:41 -07:00
timothycarambat
0d5cc558c9 linting
patch UI exception with invalid return value from failed parse
resolves 
2024-06-17 13:55:10 -07:00
lewismacnow
38441f4b21
1693 get workspace users api ()
* Enhance API - add GET for users with access to workspace

Adds GET request endpoint for retrieving a list of users with permissions to access the specified workspace

* Update Swagger for users with access to workspace

Adds swagger docs for the endpoint used to retrieve users with access to workspace. "v1/admin/workspaces/:workspaceId/users"
2024-06-17 13:52:01 -07:00
timothycarambat
29263dab52 patch: Fix API delete documents 2024-06-11 08:14:13 -07:00
timothycarambat
3434bf4686 linting 2024-06-07 10:09:35 -07:00
yonathan suarez
bb0a4b0531
Add API Endpoint for Deleting Documents in system ()
* fix: "remove document system api"

* fix: "update text system system "

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-06-07 09:22:03 -07:00
Timothy Carambat
a89812703b
repatch path normalization () 2024-05-23 12:52:04 -07:00
Timothy Carambat
c2d37ccce5
Limit return object of user when returned in some endpoints () 2024-05-22 12:32:39 -05:00
Sean Hatfield
5bf4b4db58
[FEAT] Add support for Voyage AI embedder ()
* add support for voyageai embedder

* remove unneeded import

* linting

* Add ENV examples
Update how chunks are processed for Voyage
use correct langchain import
Add data handling

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-05-19 13:20:23 -05:00
yonathan suarez
ecd5d3cb8a
Add API endpoint for updating pin status () 2024-05-19 12:47:47 -05:00
timothycarambat
7c6c272051 add logging on more api endpoints for exceptions 2024-05-16 12:56:19 -07:00
timothycarambat
a87978d1d9 Make LanceDB the vector database default provider in backend to prevent issues where somehow this key is not set by the user resulting in a Pinecone error even though they never said they wanted Pinecone to be their vector db 2024-05-13 12:22:53 -07:00