Skip to content

ListF1Score metric

Bases: CategoricalFBetaScore

Computes F-1 Score on categorical (list / label) structures.

Formula:

    f1_score = 2 * (precision * recall) / (precision + recall)

This is the harmonic mean of precision and recall. Its output range is [0, 1]. It operates at a label level and can be used for classification or retrieval pipelines.

The difference between this metric and F1Score is that this one considers each element of the list (or the string value) as one label.

If labels is provided, accumulation is performed per-label (sklearn-style) and result() returns a {label: score} dict for average=None. See CategoricalFBetaScore for details.

Example:

    # for single label classification

    class ListClassification(synalinks.DataModel):
        label: Literal["label", "label_1", "label_2"]

    # for multi label classification

    class ListClassification(synalinks.DataModel):
        labels: List[Literal["label", "label_1", "label_2"]]

    # or use it with retrieval pipelines, in that case make sure to mask
    # the correct fields.

    class AnswerWithReferences(synalinks.DataModel):
        sources: List[str]
        answer: str

Compilation example:

program.compile(
    metrics=[
        synalinks.metrics.CategoricalF1Score(),
    ],
)

Parameters:

Name Type Description Default
average str

Type of averaging to be performed. Acceptable values are None, "micro", "macro" and "weighted". Defaults to None.

None
labels list

(Optional) Explicit list of label names to track. When provided, accumulation is per-label across all batches and result() returns a {label: score} dict for average=None.

None
name str

(Optional) string name of the metric instance.

'categorical_f1_score'
in_mask list

(Optional) list of keys to keep to compute the metric.

None
out_mask list

(Optional) list of keys to remove to compute the metric.

None
in_mask_pattern str

(Optional) Regex pattern; fields whose names match are kept (combined with in_mask via OR).

None
out_mask_pattern str

(Optional) Regex pattern; fields whose names match are dropped (combined with out_mask via OR).

None
Source code in synalinks/src/metrics/f_score_metrics.py
@synalinks_export(
    [
        "synalinks.metrics.CategoricalF1Score",
        "synalinks.metrics.ListF1Score",
    ]
)
class CategoricalF1Score(CategoricalFBetaScore):
    """Computes F-1 Score on categorical (list / label) structures.

    Formula:
    ```python
        f1_score = 2 * (precision * recall) / (precision + recall)
    ```

    This is the harmonic mean of precision and recall.
    Its output range is `[0, 1]`. It operates at a label level
    and can be used for **classification** or **retrieval pipelines**.

    The difference between this metric and `F1Score` is that this one considers
    each element of the list (or the string value) as **one label**.

    If `labels` is provided, accumulation is performed per-label (sklearn-style)
    and `result()` returns a `{label: score}` dict for `average=None`. See
    `CategoricalFBetaScore` for details.

    Example:

    ```python

        # for single label classification

        class ListClassification(synalinks.DataModel):
            label: Literal["label", "label_1", "label_2"]

        # for multi label classification

        class ListClassification(synalinks.DataModel):
            labels: List[Literal["label", "label_1", "label_2"]]

        # or use it with retrieval pipelines, in that case make sure to mask
        # the correct fields.

        class AnswerWithReferences(synalinks.DataModel):
            sources: List[str]
            answer: str
    ```


    Compilation example:

    ```python
    program.compile(
        metrics=[
            synalinks.metrics.CategoricalF1Score(),
        ],
    )
    ```

    Args:
        average (str): Type of averaging to be performed.
            Acceptable values are `None`, `"micro"`, `"macro"` and
            `"weighted"`. Defaults to `None`.
        labels (list): (Optional) Explicit list of label names to track.
            When provided, accumulation is per-label across all batches and
            `result()` returns a `{label: score}` dict for `average=None`.
        name (str): (Optional) string name of the metric instance.
        in_mask (list): (Optional) list of keys to keep to compute the metric.
        out_mask (list): (Optional) list of keys to remove to compute the metric.
        in_mask_pattern (str): (Optional) Regex pattern; fields whose names match
            are kept (combined with ``in_mask`` via OR).
        out_mask_pattern (str): (Optional) Regex pattern; fields whose names match
            are dropped (combined with ``out_mask`` via OR).
    """

    def __init__(
        self,
        average=None,
        labels=None,
        name="categorical_f1_score",
        in_mask=None,
        out_mask=None,
        in_mask_pattern=None,
        out_mask_pattern=None,
    ):
        super().__init__(
            average=average,
            beta=1.0,
            labels=labels,
            name=name,
            in_mask=in_mask,
            out_mask=out_mask,
            in_mask_pattern=in_mask_pattern,
            out_mask_pattern=out_mask_pattern,
        )

    def get_config(self):
        """Return the serializable config of the metric.

        Returns:
            (dict): The config dict.
        """
        base_config = super().get_config()
        del base_config["beta"]
        return base_config

get_config()

Return the serializable config of the metric.

Returns:

Type Description
dict

The config dict.

Source code in synalinks/src/metrics/f_score_metrics.py
def get_config(self):
    """Return the serializable config of the metric.

    Returns:
        (dict): The config dict.
    """
    base_config = super().get_config()
    del base_config["beta"]
    return base_config