浏览代码

fix: use Gemini response metadata for token counting (#11226)

tags/0.13.0
Shota Totsuka 11 个月前
父节点
当前提交
594666eb61
没有帐户链接到提交者的电子邮件
共有 1 个文件被更改,包括 6 次插入2 次删除
  1. 6
    2
      api/core/model_runtime/model_providers/google/llm/llm.py

+ 6
- 2
api/core/model_runtime/model_providers/google/llm/llm.py 查看文件

assistant_prompt_message = AssistantPromptMessage(content=response.text) assistant_prompt_message = AssistantPromptMessage(content=response.text)


# calculate num tokens # calculate num tokens
prompt_tokens = self.get_num_tokens(model, credentials, prompt_messages)
completion_tokens = self.get_num_tokens(model, credentials, [assistant_prompt_message])
if response.usage_metadata:
prompt_tokens = response.usage_metadata.prompt_token_count
completion_tokens = response.usage_metadata.candidates_token_count
else:
prompt_tokens = self.get_num_tokens(model, credentials, prompt_messages)
completion_tokens = self.get_num_tokens(model, credentials, [assistant_prompt_message])


# transform usage # transform usage
usage = self._calc_response_usage(model, credentials, prompt_tokens, completion_tokens) usage = self._calc_response_usage(model, credentials, prompt_tokens, completion_tokens)

正在加载...
取消
保存