Skip to content

Base Reward class

Bases: SynalinksSaveable

Reward base class.

This is the class to subclass in order to create new custom rewards.

Parameters:

Name Type Description Default
name str

Optional name for the reward instance.

None
reduction str

Optional. One of "mean", "sum", "min", "max", "none" or None. Applied by __call__ when invoked on a batch directly (standalone evaluation) and propagated through compile to control how the trainer/optimizer reduce per-sample rewards into the scalar shown in progress logs and used for candidate scoring. Use "min" to score by the worst sample (robust/pessimistic) or "max" for the best (optimistic / best-of-N). "none"/ None falls back to "mean" for those scalar consumers (per-sample values are always preserved for the optimizer's RL bookkeeping).

'mean'
in_mask list

Optional. List of exact field names to keep before computing the reward.

None
out_mask list

Optional. List of exact field names to drop before computing the reward.

None
in_mask_pattern str

Optional. Regex pattern; fields whose names match are kept (combined with in_mask via OR).

None
out_mask_pattern str

Optional. Regex pattern; fields whose names match are dropped (combined with out_mask via OR).

None

To be implemented by subclasses:

  • call(): Contains the logic for eval calculation using y_true, y_pred.
Source code in synalinks/src/rewards/reward.py
@synalinks_export(["synalinks.Reward", "synalinks.rewards.Reward"])
class Reward(SynalinksSaveable):
    """Reward base class.

    This is the class to subclass in order to create new custom rewards.

    Args:
        name (str): Optional name for the reward instance.
        reduction (str): Optional. One of ``"mean"``, ``"sum"``, ``"min"``,
            ``"max"``, ``"none"`` or ``None``. Applied by ``__call__`` when
            invoked on a batch directly (standalone evaluation) and
            propagated through ``compile`` to control how the
            trainer/optimizer reduce per-sample rewards into the scalar
            shown in progress logs and used for candidate scoring. Use
            ``"min"`` to score by the worst sample (robust/pessimistic) or
            ``"max"`` for the best (optimistic / best-of-N). ``"none"``/
            ``None`` falls back to ``"mean"`` for those scalar consumers
            (per-sample values are always preserved for the optimizer's RL
            bookkeeping).
        in_mask (list): Optional. List of exact field names to keep before
            computing the reward.
        out_mask (list): Optional. List of exact field names to drop before
            computing the reward.
        in_mask_pattern (str): Optional. Regex pattern; fields whose names
            match are kept (combined with ``in_mask`` via OR).
        out_mask_pattern (str): Optional. Regex pattern; fields whose names
            match are dropped (combined with ``out_mask`` via OR).

    To be implemented by subclasses:

    * `call()`: Contains the logic for eval calculation using `y_true`,
        `y_pred`.
    """

    def __init__(
        self,
        name=None,
        reduction="mean",
        in_mask=None,
        out_mask=None,
        in_mask_pattern=None,
        out_mask_pattern=None,
    ):
        self.name = name or auto_name(self.__class__.__name__)
        self.reduction = standardize_reduction(reduction)
        self.in_mask = in_mask
        self.out_mask = out_mask
        self.in_mask_pattern = in_mask_pattern
        self.out_mask_pattern = out_mask_pattern

    async def __call__(self, y_true, y_pred):
        with ops.name_scope(self.name):
            y_true, y_pred = apply_masks(
                y_true,
                y_pred,
                in_mask=self.in_mask,
                in_mask_pattern=self.in_mask_pattern,
                out_mask=self.out_mask,
                out_mask_pattern=self.out_mask_pattern,
            )
            rewards = await self.call(y_true, y_pred)
            return reduce_values(
                rewards,
                reduction=self.reduction,
            )

    async def call(self, y_true, y_pred):
        raise NotImplementedError

    def get_config(self):
        return {
            "name": self.name,
            "reduction": self.reduction,
            "in_mask": self.in_mask,
            "out_mask": self.out_mask,
            "in_mask_pattern": self.in_mask_pattern,
            "out_mask_pattern": self.out_mask_pattern,
        }

    @classmethod
    def from_config(cls, config):
        return cls(**config)

    def _obj_type(self):
        return "Reward"