Ollama LLM¶

Ollama ¶

Bases: ChatLLM[dict, dict, dict]

适配本地使用Ollama运行的各种单机小模型 | Ollama LLM (Local Language Model)

stop_words `property` ¶

stop_words: list[str]

获取停止词列表，需要与ToolPrompt（如果存在的话）整合

Returns:

Type	Description
`list[str]`	list[str]: 停止词列表

partial_params `property` ¶

partial_params: dict

返回部分参数

Returns:

Type	Description
`dict`	dict

reformat_request_msg_to_api ¶

reformat_request_msg_to_api(msg: BaseLLMMessage) -> dict

将LLMUserMessage转换为Ollama请求消息格式。Ollama接口对于格式有自己的要求，与当前TFRobot协议并不完全一致。因此需要进行转换。

Parameters:

Name	Type	Description	Default
`msg`	`BaseLLMMessage`	BaseLLMMessage	required

Returns:

Name	Type	Description
`dict`	`dict`	Ollama请求消息格式

Source code in tfrobot/brain/chain/llms/ollama.py

def reformat_request_msg_to_api(self, msg: BaseLLMMessage) -> dict:
    """
    将LLMUserMessage转换为Ollama请求消息格式。Ollama接口对于格式有自己的要求，与当前TFRobot协议并不完全一致。因此需要进行转换。

    Args:
        msg (BaseLLMMessage): BaseLLMMessage

    Returns:
        dict: Ollama请求消息格式
    """
    if isinstance(msg, LLMUserMessage) and isinstance(msg.content, list):
        # 如果消息内容是列表类型，说明是极有可能是图片消息，需要进行处理
        contents: list[str] = []
        images: list[str] = []
        for part in msg.content:
            if part.part_type == "text":
                contents.append(part.text)
            elif part.part_type == "image_url":
                img_url = part.image_url.url
                if not is_base64(img_url):
                    # 读取url中的二进制数据，转换为bytes写入images | Read binary data from url and convert to bytes for images
                    result = self.load_image_from_uri(img_url)
                    if result:
                        # 关键点: load_image_from_uri 已经返回 base64 编码的 bytes | Key point: load_image_from_uri already returns base64 encoded bytes
                        base64_bytes, _ = result
                        base64_image = base64_bytes.decode("utf-8")
                        images.append(base64_image)
                    else:
                        # 如果加载失败，记录警告但继续处理 | If loading fails, log warning but continue
                        warnings.warn(f"Failed to load image from URI: {img_url}")
                else:
                    images.append(img_url)
        return {"role": "user", "content": "\n".join(contents), "images": images}
    return msg.model_dump()

extract_thinking `staticmethod` ¶

extract_thinking(text: str) -> tuple[str, str]

从字符串中提取第一个...标签内的内容及剩余部分。

Parameters:

Name	Type	Description	Default
`text`	`str`	要处理的原始字符串	required

Returns:

Type	Description
`tuple[str, str]`	tuple[str, str]: 第一个元素是标签内的内容，第二个是去除整个标签后的剩余字符串

Source code in tfrobot/brain/chain/llms/ollama.py

@staticmethod
def extract_thinking(text: str) -> tuple[str, str]:
    """
    从字符串中提取第一个<thinking>...</thinking>标签内的内容及剩余部分。

    Args:
        text (str): 要处理的原始字符串

    Returns:
        tuple[str, str]: 第一个元素是标签内的内容，第二个是去除整个标签后的剩余字符串
    """
    pattern = re.compile(r"<think>(.*?)</think>", re.DOTALL)
    match = pattern.search(text)

    if not match:
        return "", text

    start, end = match.start(), match.end()
    return match.group(1), text[:start] + text[end:]

construct_llm_result ¶

construct_llm_result(
    raw_completion: dict,
    *,
    params: dict,
    response_format: Optional[LLMResponseFormat] = None
) -> LLMResult

Construct chat result | 构造聊天结果

因为Ollama提供的HttpAPI参数中不支持stop停止词。所以在生成内容中手动实现了此能力。但需要注意这会导致completion_tokens过高，因为是生成后，手动终止的停止词。

Parameters:

Name	Type	Description	Default
`raw_completion`	`dict`	Response from Ollama \| Ollama的响应	required
`params`	`dict`	Request to Ollama \| Ollama的请求	required
`response_format`	`Optional[LLMResponseFormat]`	Response format \| 响应格式.	`None`

Returns:

Name	Type	Description
`LLMResult`	`LLMResult`	The result of completion \| 完成的结果

Source code in tfrobot/brain/chain/llms/ollama.py

def construct_llm_result(
    self, raw_completion: dict, *, params: dict, response_format: Optional[LLMResponseFormat] = None
) -> LLMResult:
    """
    Construct chat result | 构造聊天结果

    因为Ollama提供的HttpAPI参数中不支持stop停止词。所以在生成内容中手动实现了此能力。
    但需要注意这会导致completion_tokens过高，因为是生成后，手动终止的停止词。

    Args:
        raw_completion (dict): Response from Ollama | Ollama的响应
        params (dict): Request to Ollama | Ollama的请求
        response_format (Optional[LLMResponseFormat]): Response format | 响应格式.

    Returns:
        LLMResult: The result of completion | 完成的结果
    """
    import tiktoken

    # 使用Tiktoken近似计算Token消耗
    all_messages = str(params.get("messages"))

    encoding = tiktoken.get_encoding("gpt2")
    prompt_tokens = encoding.encode(all_messages)
    completion_tokens = encoding.encode(raw_completion["message"]["content"])

    split_content = raw_completion["message"]["content"]
    if self.stop_words:
        split_content = split_on_stopwords(split_content, self.stop_words)

    # 尝试进行格式化操作
    response_format = response_format or (
        self.response_format if self.response_format != inspect.Parameter.empty else None
    )
    if response_format:
        json_obj = extract_json_from_nature_language(
            split_content,
            schema=cast(dict, response_format.get("json_schema", {})).get("schema") if response_format else None,
        )
        if json_obj:
            split_content = json.dumps(json_obj, ensure_ascii=False)

    usage = TokenUsage(
        prompt_tokens=len(prompt_tokens),
        completion_tokens=len(completion_tokens),
        total_tokens=len(prompt_tokens) + len(completion_tokens),
        prompt_cost=len(prompt_tokens) * self.input_price,
        completion_cost=len(completion_tokens) * self.input_price,
        total_cost=len(prompt_tokens) * self.input_price + len(completion_tokens) * self.input_price,
    )
    reasoning_content, split_content = self.extract_thinking(split_content) if split_content else (None, "")
    llm_res = LLMResult(
        generations=[
            Generation(
                text=split_content,
                reasoning_content=reasoning_content,
                tool_calls=[
                    ToolCall(
                        function=FunctionCall(
                            name=tc["function"]["name"], parameters=json.dumps(tc["function"]["arguments"])
                        )
                    )
                    for tc in raw_completion["message"]["tool_calls"]
                ]
                if raw_completion["message"].get("tool_calls")
                else None,
            )
        ],
        usage=usage,
        prompt=params,
        meta_info={
            "model": self.name,
            "created": raw_completion.get("created_at"),
            "done": raw_completion.get("done"),
            "total_duration": raw_completion.get("total_duration"),
            "load_duration": raw_completion.get("load_duration"),
            "prompt_eval_duration": raw_completion.get("prompt_eval_duration"),
            "eval_count": raw_completion.get("eval_count"),
            "eval_duration": raw_completion.get("eval_duration"),
            "id": None,
            "system_fingerprint": None,
        },
    )
    if self.used_tool_prompt:
        for prompt in self.all_prompts:
            if isinstance(prompt, ToolPrompt):
                prompt.extract_tool_call(llm_res)
    if self.enable_tfl_mode:
        for prompt in self.all_prompts:
            if isinstance(prompt, TFLPrompt):
                prompt.extract_tool_call(llm_res)
    return llm_res

OllamaWithTools ¶

Bases: Ollama

OllamaWithTools相较于Ollama加强了对Tools的控制能力。

construct_request_params ¶

construct_request_params(
    current_input: BaseMessage,
    conversation: Optional[list[BaseMessage]] = None,
    elements: Optional[list[DocElement]] = None,
    knowledge: Optional[str] = None,
    tools: Optional[list[BaseTool]] = None,
    intermediate_msgs: Optional[list[BaseMessage]] = None,
    response_format: Optional[LLMResponseFormat] = None,
) -> dict

Construct chat request parameters | 构造聊天请求参数

最终的会话排序如下： 1. 首条System Message 2. 用户历史会话内容 3. 当前输入前的提示 4. 当前输入 5. 当前输入后的提示 6. 中间消息

Parameters:

Name	Type	Description	Default
`current_input`	`BaseMessage`	Current input \| 当前输入	required
`conversation`	`Optional[list[BaseMessage]]`	Conversation history \| 会话历史	`None`
`elements`	`Optional[list[DocElement]]`	Document elements \| 文档元素	`None`
`knowledge`	`Optional[str]`	Related knowledge \| 相关知识	`None`
`tools`	`Optional[list[BaseTool]]`	Available tool list \| 可用的工具列表	`None`
`intermediate_msgs`	`Optional[list[BaseMessage]]`	Intermediate messages, it will be generated during llm work in chain, list tool's response or other system message, those messages will not save into memory, but usefully during chain work \| 中间消息，它将在链式工作中生成，列表工具的响应或其他系统消息，这些消息不会保存到记忆体中，但在链式工作中非常有用	`None`
`response_format`	`Optional[LLMResponseFormat]`	Response format \| 响应格式如果在此指定响应格式，会覆盖LLM的默认响应格式。	`None`

Returns:

Name	Type	Description
`dict`	`dict`	Request parameters \| 请求参数

Source code in tfrobot/brain/chain/llms/ollama.py

@overrides.override
def construct_request_params(
    self,
    current_input: BaseMessage,
    conversation: Optional[list[BaseMessage]] = None,
    elements: Optional[list[DocElement]] = None,
    knowledge: Optional[str] = None,
    tools: Optional[list[BaseTool]] = None,
    intermediate_msgs: Optional[list[BaseMessage]] = None,
    response_format: Optional[LLMResponseFormat] = None,
) -> dict:
    """
    Construct chat request parameters | 构造聊天请求参数

    最终的会话排序如下：
    1. 首条System Message
    2. 用户历史会话内容
    3. 当前输入前的提示
    4. 当前输入
    5. 当前输入后的提示
    6. 中间消息

    Args:
        current_input(BaseMessage): Current input | 当前输入
        conversation(Optional[list[BaseMessage]]): Conversation history | 会话历史
        elements(Optional[list[DocElement]]): Document elements | 文档元素
        knowledge(Optional[str]): Related knowledge | 相关知识
        tools(Optional[list[BaseTool]]): Available tool list | 可用的工具列表
        intermediate_msgs(Optional[list[BaseMessage]]): Intermediate messages, it will be generated during llm work
            in chain, list tool's response or other system message, those messages will not save into memory, but
            usefully during chain work | 中间消息，它将在链式工作中生成，列表工具的响应或其他系统消息，这些消息不会保存到记忆体中，
            但在链式工作中非常有用
        response_format(Optional[LLMResponseFormat]): Response format | 响应格式
            如果在此指定响应格式，会覆盖LLM的默认响应格式。

    Returns:
        dict: Request parameters | 请求参数
    """
    super_res = super().construct_request_params(
        current_input, conversation, elements, knowledge, tools, intermediate_msgs, response_format
    )
    if tools := super_res.get("tools"):
        for i in range(len(tools) - 1, -1, -1):
            tool_name = tools[i]["function"]["name"]
            if self.exclude_tools and tool_name in self.exclude_tools:
                tools.pop(i)
                continue
            if self.available_tools is not None and tool_name not in self.available_tools:
                tools.pop(i)
        if not super_res.get("tools"):
            del super_res["tools"]
    return super_res

split_on_stopwords ¶

split_on_stopwords(
    text: str, stopwords: str | list[str]
) -> str

Split text on stopwords | 在停止词上拆分文本

Parameters:

Name	Type	Description	Default
`text`	`str`	Text to split \| 要拆分的文本	required
`stopwords`	`str \| list[str]`	Stopwords \| 停止词	required

Returns:

Name	Type	Description
`str`	`str`	Splitted text \| 拆分后的文本

Source code in tfrobot/brain/chain/llms/ollama.py

def split_on_stopwords(text: str, stopwords: str | list[str]) -> str:
    """
    Split text on stopwords | 在停止词上拆分文本

    Args:
        text (str): Text to split | 要拆分的文本
        stopwords (str | list[str]): Stopwords | 停止词

    Returns:
        str: Splitted text | 拆分后的文本
    """
    if not stopwords:
        return text
    if isinstance(stopwords, str):
        stopwords = [stopwords]  # 确保停止词是列表形式
    if stopwords:  # 只有当列表非空时执行
        # 移除重复单词并构建正则表达式
        stopwords = list(set(stopwords))
        pattern = r"(" + "|".join(re.escape(word) for word in stopwords if word) + r")"
        parts = re.split(pattern, text, 1)
        if len(parts) > 1 and parts[0].strip():
            # 注意if判断条件里 parts[0].strip() 很重要，很多时候大语言模型会进行指令跟随，可能停止词会出现开头的位置。这种情况下往往是对一个结果的反馈，不需要进行停止
            return parts[0]
    return text  # 如果没有分割发生或停止词列表为空

Ollama LLM¶

Ollama ¶

stop_words property ¶

partial_params property ¶

reformat_request_msg_to_api ¶

extract_thinking staticmethod ¶

construct_llm_result ¶

OllamaWithTools ¶

construct_request_params ¶

split_on_stopwords ¶

stop_words `property` ¶

partial_params `property` ¶

extract_thinking `staticmethod` ¶