Errors

Metrics for collecting and summarizing prediction error messages.

Classes:

Name	Description
ErrorCollector	Collect error messages from predictions and summarize their frequencies.

ErrorCollector(show_errors=False, type_separator=': ')

Bases: Metric

Collect error messages from model predictions and summarize their occurrences.

Parameters:

Name	Type	Description	Default
show_errors	bool	If True, log each collected error message.	False
type_separator	str	Separator used to split error messages into type and details.	': '

Parameters:

Name	Type	Description	Default
show_errors	bool	If True, log each collected error message as it is collected.	False
type_separator	str	Separator used to split error messages into type and details.	': '

Source code in src/kibad_llm/metrics/errors.py

def __init__(self, show_errors: bool = False, type_separator: str = ": ") -> None:
    """Initialize the error collector.

    Args:
        show_errors: If `True`, log each collected error message as it is collected.
        type_separator: Separator used to split error messages into type and details.
    """
    self.show_errors = show_errors
    self.type_separator = type_separator
    self.reset()

reset()

Reset the collected error state.

Source code in src/kibad_llm/metrics/errors.py

def reset(self) -> None:
    """Reset the collected error state."""
    self.state: list[list[str]] = []