Bases: LMOperationalMetric
Average whole-trajectory time-to-first-token (s) for streamed agents.
Like AvgTimeToFirstToken, but the clock starts at the outermost agent's
trajectory start rather than at the final LM request -- so it includes every
tool-calling round before the final answer, capturing the user-perceived
"how long until the agent starts answering". Measured per streamed call that
runs inside an agent trajectory_scope, then averaged over those calls.
Only streamed final answers produced inside an agent contribute, so this
reads 0.0 for runs without a streaming agent. For a streamed call with no
tool-calling rounds it coincides with AvgTimeToFirstToken.
Example:
program.compile(
metrics=[
synalinks.metrics.AvgTrajectoryTimeToFirstToken(),
],
)
Source code in synalinks/src/metrics/lm_metrics.py
| @synalinks_export("synalinks.metrics.AvgTrajectoryTimeToFirstToken")
class AvgTrajectoryTimeToFirstToken(LMOperationalMetric):
"""Average *whole-trajectory* time-to-first-token (s) for streamed agents.
Like `AvgTimeToFirstToken`, but the clock starts at the outermost agent's
trajectory start rather than at the final LM request -- so it includes every
tool-calling round before the final answer, capturing the user-perceived
"how long until the agent starts answering". Measured per streamed call that
runs inside an agent `trajectory_scope`, then averaged over those calls.
Only streamed final answers produced inside an agent contribute, so this
reads 0.0 for runs without a streaming agent. For a streamed call with no
tool-calling rounds it coincides with `AvgTimeToFirstToken`.
Example:
```python
program.compile(
metrics=[
synalinks.metrics.AvgTrajectoryTimeToFirstToken(),
],
)
```
"""
def __init__(self, name="avg_trajectory_time_to_first_token"):
super().__init__(name=name)
def result(self):
calls = self._delta("trajectory_calls")
if calls <= 0:
return 0.0
return self._delta("trajectory_ttft_s") / calls
|