mirror of
https://github.com/khoj-ai/khoj.git
synced 2024-11-23 15:38:55 +01:00
c53c3db96b
- Track input, output token usage and cost for interactions
via chat api with openai, anthropic and google chat models
- Get usage metadata from OpenAI using stream_options
- Handle openai proxies that do not support passing usage in response
- Add new usage, end response events returned by chat api.
- This can be optionally consumed by clients at a later point
- Update streaming clients to mark message as completed after new
end response event, not after end llm response event
- Ensure usage data from final response generation step is included
- Pass usage data after llm response complete. This allows gathering
token usage and cost for the final response generation step across
streaming and non-streaming modes
|
||
|---|---|---|
| .. | ||
| app | ||
| database | ||
| interface | ||
| migrations | ||
| processor | ||
| routers | ||
| search_filter | ||
| search_type | ||
| utils | ||
| __init__.py | ||
| configure.py | ||
| main.py | ||
| manage.py | ||