Commit graph

212 commits

Author SHA1 Message Date
Timothy Carambat
ad01df8790
Reranker option for RAG ()
* Reranker WIP

* add cacheing and singleton loading

* Add field to workspaces for vectorSearchMode
Add UI for lancedb to change mode
update all search endpoints to pass in reranker prop if provider can use it

* update hint text

* When reranking, swap score to rerank score

* update optchain
2025-01-02 14:27:52 -08:00
timothycarambat
d52249aff8 support attachments via thread API chat/stream-chat endpoints 2024-12-31 09:52:21 -08:00
Sean Hatfield
8d302c3f67
Patch custom models endpoint ()
* prevent non admin users from changing llm settings via custom-models endpoint

* permission middleware to JSDOC

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-12-30 14:58:26 -08:00
Sean Hatfield
696af19c45
Patch unauthorized access to other user's pfps ()
* patch unauthorized viewing of other user's pfps

* inline return responses

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-12-30 14:36:51 -08:00
Timothy Carambat
d54b5dfc62
Remove fine-tuning flow ()
remove fine-tuning flow
2024-12-18 10:24:02 -08:00
Sean Hatfield
f8885a43e4
Append writable fields to dev API new workspace endpoint ()
* add writible fields to dev api new workspace endpoint

* lint

* implement validations for workspace model

* update swagger comments

* simplify validations for workspace on frontend and API

* cleanup validations

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-12-16 15:12:20 -08:00
Timothy Carambat
dd7c4675d3
LLM performance metric tracking ()
* WIP performance metric tracking

* fix: patch UI trying to .toFixed() null metric
Anthropic tracking migraiton
cleanup logs

* Apipie implmentation, not tested

* Cleanup Anthropic notes, Add support for AzureOpenAI tracking

* bedrock token metric tracking

* Cohere support

* feat: improve default stream handler to track for provider who are actually OpenAI compliant in usage reporting
add deepseek support

* feat: Add FireworksAI tracking reporting
fix: improve handler when usage:null is reported (why?)

* Add token reporting for GenericOpenAI

* token reporting for koboldcpp + lmstudio

* lint

* support Groq token tracking

* HF token tracking

* token tracking for togetherai

* LiteLLM token tracking

* linting + Mitral token tracking support

* XAI token metric reporting

* native provider runner

* LocalAI token tracking

* Novita token tracking

* OpenRouter token tracking

* Apipie stream metrics

* textwebgenui token tracking

* perplexity token reporting

* ollama token reporting

* lint

* put back comment

* Rip out LC ollama wrapper and use official library

* patch images with new ollama lib

* improve ollama offline message

* fix image handling in ollama llm provider

* lint

* NVIDIA NIM token tracking

* update openai compatbility responses

* UI/UX show/hide metrics on click for user preference

* update bedrock client

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
2024-12-16 14:31:17 -08:00
Timothy Carambat
ff024286d0
Add new workspace user management endpoint ()
deprecate old endpoint which users workspace ID vs slug
resolves 
2024-12-16 11:50:34 -08:00
Sean Hatfield
79cdcd4e64
Add thread name to workspaces get endpoint ()
add thread name to workspaces get endpoint

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-12-13 17:53:54 -08:00
Timothy Carambat
a46eb2418a
add Gitlab to watchable documents () 2024-12-12 10:44:11 -08:00
Timothy Carambat
538078747d
Add vector search API endpoint ()
* Add vector search API endpoint

* Add missing import

* Modify the data that is returned

* Change similarityThreshold to scoreThreshold

As this is what is actually returned by the search

* Removing logging (oops!)

* chore: regen swagger docs for new endpoint
fix: update function to sanity check values to prevent crashes during search

---------

Co-authored-by: Scott Bowler <scott@dcsdigital.co.uk>
2024-12-12 10:12:32 -08:00
timothycarambat
a0c5d898f0 patch docs with proper input for OpenAI compatible embeddings
resolves 
2024-12-11 08:15:52 -08:00
Sean Hatfield
05c530221b
Community hub integration ()
* wip hub connection page fe + backend

* lint

* implement backend for local hub items + placeholder endpoints to fetch hub app data

* fix hebrew translations

* revamp community integration flow

* change sidebar

* Auto import if id in URL param
remove preview in card screen and instead go to import flow

* get user's items + team items from hub + ui improvements to hub settings

* lint

* fix merge conflict

* refresh hook for community items

* add fallback for user items

* Disable bundle items by default on all instances

* remove translations (will complete later)

* loading skeleton

* Make community hub endpoints admin only
show visibility on items
combine import/apply for items to they are event logged for review

* improve middleware and import flow

* community hub ui updates

* Adjust importing process

* community hub to dev

* Add webscraper preload into imported plugins

* add runtime property to plugins

* Fix button status on imported skill change
show alert on skill change
Update markdown type and theme on import of agent skill

* update documentaion paths

* remove unused import

* linting

* review loading state

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-11-26 09:59:43 -08:00
Sean Hatfield
727d802779
Light/dark mode UI overhaul ()
* Refactor workspace sidebar component styles ()

rely on css for conditional styles

* New sidebar colors ()

new sidebar colors

* Main container color update ()

* Dark mode setup themes ()

* setup generic tailwind theme + ability to add new themes

* add theme context

* use correct colors from design for sidebar + fix padding

* Settings sidebar UI updates ()

settings sidebar ui updates

* fix sidebar resizing/truncate issue on hover

* Dark mode chat window ()

* Support XLSX files ()

* support xlsx files

* lint

* create seperate docs for each xlsx sheet

* lint

* use node-xlsx pkg for parsing xslx files

* lint

* update error handling

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* wip chat window

* ux+ux improvements and update new colors

* chat window dark mode

* remove comment

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* Dark mode welcome page ()

* dark mode welcome page styles + refactor

* remove AI_BACKGROUND_COLOR and USER_BACKGROUND_COLOR constants

* Dark mode UI for admin and tools pages + mobile view improvements ()

* dark mode ui for admin and tools pages + mobile view improvements

* lint

* ai provider pages + options darkmode ui

* placeholder generic class

* appearance settings styles

* ai providers mobile margin

* dark mode styles for agent skills + experimental features

* mobile styles on security settings

* fine tune flow ui dark mode

* workspace settings page

* lint

* Dark mode onboarding ()

dark mode onboarding

* update all modals + normalize styles ()

* lint

* Dark mode privacy & experimental pages ()

* document watch + privacy pages ui + mobile modal darkmode

* lint

* Dark mode login screens ()

* multi-user auth screen ui update

* dark mode login screen + recovery key modals

* remove unneeded import

* Workspace preset commands modals dark mode ()

update workspace preset dark mode modal

* Document pinning modal ui update ()

document pinning modal ui update

* Experimental agreement modal dark mode ()

experimental agreement modal dark mode

* Serp options dark mode ()

serp options dark mode

* field fixes

* attempt light mode wip

* setting sidebar

* Toasts and threads

* main page content and privacy page

* force rewrite for light

* add border for light mode rightside content

* more fixes

* wip

* wip

* wip light mode implementation

* wip dark light mode file picker

* document picker light mode ui

* slight ui tweaks

* light mode fine tuning flow

* light mode tweaks + qa fixes

* fix md rendering of light mode + tooltip fixes

* lint

* qa bug fixes

* Add developer hook for theme
move provider to outmost layer

* qa light mode bug fixes

* Linting and hotfixes for UI

* Light mode to dev

* accept invite light mode ui fix

* Fix onboarding inputs in dark mode

* fix close icons
last minute items

* patch z-index on tooltips

* patch light mode citations

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-11-18 15:40:18 -08:00
Sean Hatfield
e719d05027
Add optional limit and orderBy to dev api chat endpoint ()
add optional limit and orderBy to dev api chat endpoint
2024-10-30 14:47:24 -07:00
Mr Simon C
ccde891aa2
Feature/update api example outputs ()
* /v1/workspace/{slug} actually returns an array - update example output

* update swagger example to show array output
2024-10-29 16:50:19 -07:00
Timothy Carambat
2c9cb28d5f
Simple SSO feature for login flows from external services ()
* Simple SSO feature for login flows from external services

* linting
2024-10-29 15:30:53 -07:00
Timothy Carambat
40800631e3
Patch v1/document/upload filename charset encoding () 2024-10-25 12:03:19 -07:00
Timothy Carambat
72ba9f7f28
Add filtering to sessionID for workspace chats () 2024-10-24 15:10:50 -07:00
Timothy Carambat
0524aadf58
Enable the ability to disable the chat history UI ()
* Enable the ability to disable the chat history UI

* forgot files
2024-10-21 13:19:19 -07:00
Timothy Carambat
e71392d83f
Feature/thread creation slug name ()
* thread creation additional params name and slug, with api

* typo fix

* Rebuild openai Swagger docs
Handle validations for fields to prevent invalid field inputs for .new
Enforce sluggification of `slug` to prevent breaking of URL structs

---------

Co-authored-by: abrakadobr <abrakadobr@gmail.com>
2024-10-21 13:09:55 -07:00
Timothy Carambat
c3723ce2ff
Add backfilling on query for chat widget to improve UX () 2024-10-15 14:37:44 -07:00
Sean Hatfield
be6289d141
Daily message limit per user ()
* set message limit per user

* remove old limit user messages + unused admin page

* fix daily message validation

* refactor message limit input
refactor canSendChat on user to a method on user model

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-10-15 14:01:29 -07:00
Sean Hatfield
e6c4eb3f1c
Support attachments in developer API ()
* support attachments in developer api

* lint

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-09-25 13:44:26 -07:00
Sean Hatfield
4ebc37b4e3
Export embedded chat history ()
export embedded chat history

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-09-24 16:05:34 -07:00
Timothy Carambat
0cbe4d0b04
Make userId actually optional for workspaceThread endpoint () 2024-09-12 13:33:41 -07:00
Timothy Carambat
d1103e2b71
Add support for custom agent skills via plugins ()
* Add support for custom agent skills via plugins
Update Admin.systemPreferences to updated endpoint (legacy has deprecation notice

* lint

* dev build

* patch safeJson
patch label loading

* allow plugins with no config options

* lint

* catch invalid setupArgs in frontend

* update link to docs page for agent skills

* remove unneeded files

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
2024-09-10 17:06:02 -07:00
Mr Simon C
7594841dac
Bug/make swagger json output openapi 3 compliant ()
update source to ensure swagger.json is openapi 3.0.0 compliant
2024-09-04 15:40:24 -07:00
timothycarambat
47a5c7126c Patch path traversal in move-files that can be used by administrator level attacker only 2024-08-27 16:19:12 -07:00
Timothy Carambat
29df483a27
AnythingLLM Chrome Extension ()
* initial commit for chrome extension

* wip browser extension backend

* wip frontend browser extension settings

* fix typo for browserExtension route

* implement verification codes + frontend panel for browser extension keys

* reorganize + state management for all connection states

* implement embed to workspace

* add send page to anythingllm extension option + refactor

* refactor connection string auth + update context menus + organize background.js into models

* popup extension from main app and save if successful

* fix hebrew translation misspelling

* fetch custom logo inside chrome extension

* delete api keys on disconnect of extension

* use correct apiUrl constant in frontend + remove unneeded comments

* remove upload-link endpoint and send inner text html to raw text collector endpoint

* update readme

* fix readme link

* fix readme typo

* update readme

* handle deletion of browser keys with key id and DELETE endpoint

* move event string to constant

* remove tablename and writable fields from BrowserExtensionApiKey backend model

* add border-none to all buttons and inputs for desktop compatibility

* patch prisma injections

* update delete endpoints to delete keys by id

* remove unused prop

* add button to attempt browser extension connection + remove max active keys

* wip multi user mode support

* multi user mode support

* clean up backend + show created by in frotend browser extension page

* show multi user warning message on key creation + hide context menus when no workspaces

* show browser extension options to managers

* small backend changes and refactors

* extension cleanup

* rename submodule

* extension updates & docs

* dev docker build

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
2024-08-27 14:58:47 -07:00
Timothy Carambat
fdc3add53c
Api session id support ()
* Refactor api endpoint chat handler to its own function
remove legacy `chatWithWorkspace` and cleanup `index.js`

* Add `sessionId` in dev API to partition chats logically statelessly
2024-08-21 15:25:47 -07:00
Timothy Carambat
1f96b837b3
Refactor api endpoint chat handler to its own function ()
remove legacy `chatWithWorkspace` and cleanup `index.js`
2024-08-21 14:47:06 -07:00
timothycarambat
17abbe97e5 remove #swagger.path from API docs 2024-08-20 12:48:08 -07:00
Timothy Carambat
a8d25c7dd3
Allow readable username passed in script widget () 2024-08-16 16:55:42 -07:00
Sean Hatfield
1dad4d9409
Fix multi-user setup bug for invalid username/password ()
* fix multiuser setup bug for invalid username/password

* handle user creation failed gracefully and delete user on fail
2024-08-16 12:55:14 -07:00
Timothy Carambat
4430ddb059
Encryption in JWT for single-user password mode ()
* wip encrypting jwt value

* Encrypt/Decrypt pass in JWT value for verification in single-user password mode
2024-08-13 17:54:12 -07:00
timothycarambat
c9e19264eb fix validation issue 2024-08-07 14:24:24 -07:00
Sean Hatfield
665553af5a
Developer API access for embed chat widgets ()
* developer api access for embedded chat widgets and embed chats

* lint

* uncheck files

* sanitize embed api inputs

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-08-07 11:36:32 -07:00
Timothy Carambat
d072875e43
Add piperTTS in-browser text-to-speech ()
* Add piperTTS in-browser text-to-speech

* update vite config

* Add voice default + change prod public URL

* uncheck file

* Error handling
bump package for better quality and voices

* bump package

* Remove pre-packed WASM - will not support offline first solution for docker

* attach TTSProvider telem
2024-08-07 11:09:51 -07:00
Timothy Carambat
38fc181238
Add multimodality support ()
* Add multimodality support

* Add Bedrock, KoboldCpp,LocalAI,and TextWebGenUI multi-modal

* temp dev build

* patch bad import

* noscrolls for windows dnd

* noscrolls for windows dnd

* update README

* update README

* add multimodal check
2024-07-31 10:47:49 -07:00
Timothy Carambat
d877d2b7ad
Add drag-and-drop to chat window ()
* Add drag-and-drop to chat window

* add uploader icon and remove empty space text when attachments are present

* color theme

* color update
2024-07-30 10:26:16 -07:00
Timothy Carambat
ae58a2cb0d
Return threads for API workspace calls ()
* Return threads for API workspace calls

* remove import
2024-07-26 14:18:40 -07:00
Timothy Carambat
42235fcd8a
GitLab Hosted and Local Connector ()
* Add support for GitLab repo collection as well as Github Repo collection
* Refactor for repo collectors to be more compact

---------

Co-authored-by: Emil Rofors <emirof@gmail.com>
2024-07-23 12:23:51 -07:00
Timothy Carambat
f8e54b2328
Fine tuning order flow (cloud-based) ()
* WIP fine-tuning order flow

* flow patches and typo

* Refine steps
add fine tuning CTA on chat page
add fine tuning banner and validation endpoints
add finetuning banner on relevant pages with perms check

* Add prod firebase url
2024-07-22 17:59:17 -07:00
timothycarambat
d0713e15f1 patch response body for swagger 2024-07-22 15:21:32 -07:00
timothycarambat
7b42a67acf docs: update requirement for /raw-text endpoint
resolves 
2024-07-22 13:19:57 -07:00
Timothy Carambat
3198718975
Update references to new domain () 2024-07-22 11:05:34 -07:00
Timothy Carambat
8180787c2e
Enable editing and setting of meta-tag information ()
* Enable editing and setting of meta-tag information

* cleanup

* tmp build for testing

* finally always refresh

* unset workflow

* dev build

* rm tmp build
2024-07-19 15:58:43 -07:00
Johnny Chen
c4158b97d6
bugfix: API new user return 400 if user creation failed ()
* return 400 if user creation failed

The client might not check the error field unless they realize it's a 4xx error instead of 200 success.

* Update index.js

* use one-liner style

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-07-19 15:27:21 -07:00
Sean Hatfield
ab56bae1bb
[FEAT] Ability to delete messages from conversation history ()
* implement delete message feature

* add new delete chat endpoint that just hides messages from ui and fix perms

* refactor: add delete-event hooks for delete so we dont need to pass so much
refactor: implmentation of delete endpoint PUT

* update animation
linting
remove element on animiation to prevent popin

* linting and cleanup imports

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-07-03 18:28:08 -07:00