Bases: LMOperationalMetric
Fraction of LM calls that failed: failed / (succeeded + failed).
The headline reliability signal: successful calls bump calls, failures
bump failed_calls, so the error rate is observable even though failures
leave the token / cost / latency counters untouched.
Example:
program.compile(
metrics=[
synalinks.metrics.ErrorRate(),
],
)
Source code in synalinks/src/metrics/lm_metrics.py
| @synalinks_export("synalinks.metrics.ErrorRate")
class ErrorRate(LMOperationalMetric):
"""Fraction of LM calls that failed: failed / (succeeded + failed).
The headline reliability signal: successful calls bump `calls`, failures
bump `failed_calls`, so the error rate is observable even though failures
leave the token / cost / latency counters untouched.
Example:
```python
program.compile(
metrics=[
synalinks.metrics.ErrorRate(),
],
)
```
"""
def __init__(self, name="error_rate"):
super().__init__(name=name)
def result(self):
failed = self._delta("failed_calls")
total = self._delta("calls") + failed
if total <= 0:
return 0.0
return failed / total
|