Skip to content

Openai like vllm

OpenAILikeVllm(*args, **kwargs)

Bases: LLM

Simple wrapper around OpenAI-like LLMs to indicate vLLM usage in extractors.extract_from_text

Source code in src/kibad_llm/llms/openai_like_vllm.py 
20
21
def __init__(self, *args, **kwargs) -> None:
    self.model = OpenAILike(*args, **kwargs)

get_reasoning_from_chat_response(response)

Extract reasoning from a chat response.

Source code in src/kibad_llm/llms/openai_like_vllm.py 
52
53
54
55
56
57
58
59
60
61
62
63
64
def get_reasoning_from_chat_response(self, response: ChatResponse) -> str:
    """Extract reasoning from a chat response."""

    raw_msg = self.get_raw_message_from_chat_response(response)

    # vLLM: prefer `reasoning`, fallback to legacy `reasoning_content`
    result = getattr(raw_msg, "reasoning", None) or getattr(raw_msg, "reasoning_content", None)
    if not isinstance(result, str):
        raise ReasoningExtractionError("Could not extract reasoning from chat response.")
    if not result.strip():
        raise EmptyReasoningError("Extracted reasoning is empty.")

    return result