Skip to content

CachedTokens metric

Bases: LMOperationalMetric

Prompt tokens served from provider-side prompt cache during this run.

For Anthropic this is reported as cache_read_input_tokens; for OpenAI as cached_tokens. LiteLLM normalizes both into usage.prompt_tokens_details.cached_tokens.

Example:

program.compile(
    metrics=[
        synalinks.metrics.CachedTokens(),
    ],
)
Source code in synalinks/src/metrics/lm_metrics.py
@synalinks_export("synalinks.metrics.CachedTokens")
class CachedTokens(LMOperationalMetric):
    """Prompt tokens served from provider-side prompt cache during this run.

    For Anthropic this is reported as `cache_read_input_tokens`; for OpenAI
    as `cached_tokens`. LiteLLM normalizes both into
    `usage.prompt_tokens_details.cached_tokens`.

    Example:

    ```python
    program.compile(
        metrics=[
            synalinks.metrics.CachedTokens(),
        ],
    )
    ```
    """

    def __init__(self, name="cached_tokens"):
        super().__init__(name=name)

    def result(self):
        return int(self._delta("cached_tokens"))