Bases: LMOperationalMetric
Prompt tokens served from provider-side prompt cache during this run.
For Anthropic this is reported as cache_read_input_tokens; for OpenAI
as cached_tokens. LiteLLM normalizes both into
usage.prompt_tokens_details.cached_tokens.
Example:
program.compile(
metrics=[
synalinks.metrics.CachedTokens(),
],
)
Source code in synalinks/src/metrics/lm_metrics.py
| @synalinks_export("synalinks.metrics.CachedTokens")
class CachedTokens(LMOperationalMetric):
"""Prompt tokens served from provider-side prompt cache during this run.
For Anthropic this is reported as `cache_read_input_tokens`; for OpenAI
as `cached_tokens`. LiteLLM normalizes both into
`usage.prompt_tokens_details.cached_tokens`.
Example:
```python
program.compile(
metrics=[
synalinks.metrics.CachedTokens(),
],
)
```
"""
def __init__(self, name="cached_tokens"):
super().__init__(name=name)
def result(self):
return int(self._delta("cached_tokens"))
|