Add /reload API to reload model embeddings and entries from file

- The reload API adds the ability to separate out the loading of
  embeddings from file without having to restart app or (re-)generate embeddings

- Before this the only way to load model from file was by restarting app
- The other way to reload the model embeddings by regenerating them
  was to expensive for larger datasets

- This unlocks at least 1 use-case, where
  - we regenerate model via an app instance running on a separate server and
  - just reload the generated embeddings on the client device

  - This allows us to offload the expensive embedding generation
    compute to a background server while letting

  - This avoids having to (re-)restart application on client device or
    be forced to generate embeddings on the client device itself

  - But it requires the model relevant files to be synced to the client device
    This can be done with any file syncing application like Syncthing

  - We can then call /regenerate on server and /reload client on a
    regular schedule to keep our data up to date on semantic search
This commit is contained in:
Debanjum Singh Solanky 2022-06-29 22:46:17 +04:00
parent f5d6d1e752
commit b89fc2f4ac

View file

@ -90,6 +90,13 @@ def search(q: str, n: Optional[int] = 5, t: Optional[SearchType] = None):
return {}
@app.get('/reload')
def regenerate(t: Optional[SearchType] = None):
global model
model = initialize_search(config, regenerate=False, t=t)
return {'status': 'ok', 'message': 'reload completed'}
@app.get('/regenerate')
def regenerate(t: Optional[SearchType] = None):
global model