Decision Models
Decision Models
Most steps of a program write: an answer, a summary, a plan. But many steps only decide: is this ticket about billing? Which branch should handle this query? Is this answer grounded in its context? A language model can do both, but for a decision it is overkill: it generates tokens you then have to trust to be one of your labels.
A decision model does only the second job. It never writes a sentence: it answers typed questions about its input, a yes or a no, one option out of a list, or a score along ordered levels, each with calibrated probabilities. Its answer is always one of the options you gave, so there is nothing to parse and no invalid label to recover from, and it is much faster and cheaper than a language model.
The mental model is a form, not an essay. A language model writes the essay; a decision model ticks the boxes of a form you designed. When a step of your program is a form, give it to a decision model.
Decision Models vs. Language Models
| Language model | Decision model | |
|---|---|---|
| Output | Generated text (or JSON) | Answers to typed questions |
| Guarantee | Constrained by the schema | Always one of your options |
| Confidence | None | Calibrated probabilities |
| Reasoning | Step by step, tools, images | None |
| Cost and latency | Higher | Much lower |
| Best for | Writing, reasoning, acting | Routing, classification, guards, grading |
The two are complementary, and the best programs use both: a decision model for every decision on the path, a language model only where something has to be written.
Fields Are Questions
A decision model is driven by a data model, like a language model
(Guide 2). The
difference is that each field is a question, asked with the field's
description:
- a
boolfield is a yes/no question, - a
Literal(or stringEnum) field picks one of its options, - a score field (
synalinks.Rating,synalinks.Score, ...) rates along its scale, and the answer is a value of that scale.
class Triage(synalinks.DataModel):
is_billing: bool = synalinks.Field(
description="Is the ticket about billing?",
)
urgency: Literal["low", "medium", "high"] = synalinks.Field(
description="How urgent is the ticket?",
)
frustration: synalinks.Rating = synalinks.Field(
description="How frustrated is the customer, from 1 (calm) to 5 (angry)?",
)
For a score, the decision model does not pick a single level: it spreads its
probability over the levels, and the answer is the level at the center of
that spread. A customer torn between "annoyed" and "angry" lands between the
two, rather than on whichever a model happened to generate. Say what the
scale means in the description, as above: that is how the decision model
knows which end is which.
What a decision model cannot answer is a field that has to be written: a free-form string, a number, a list. Synalinks refuses such a data model up front instead of failing at run time.
All the questions of a data model are answered in one call, in parallel, so put every question about an input in the same data model.
The decision_model Argument
A decision model is set up like a language model, from any provider of litellm's
decisions API or a local Ollama decision model (here kev, after
ollama pull kev). A hosted one such as typesafe/jev-latest reads its API
key from the environment (TYPESAFE_API_KEY).
The modules that can decide take it through their own decision_model
argument: Generator, Decision, Branch, MultiDecision, SelfCritique
and RubricsAsJudge. It is never passed as a language_model, so a decision
model cannot end up in a module that needs to write. With
synalinks.set_default_decision_model(...), these modules use it by default
when no default language model is set: a default language model wins over a
default decision model, and only a decision_model given to the module wins
over the default language model.
Decision Models in Modules
Given a decision model, each of these modules keeps its job but drops its
free-text part: there is no thinking to write before a decision, and no
critique to write before a grade. What remains is the decision itself,
which the decision model makes faster and more reliably. The examples below
run on a support ticket (x0 = synalinks.Input(data_model=Ticket)).
Generator
The Generator is the most general of them: give it a data model made of
questions, and the decision model answers them all at once. The
instructions and examples you give the Generator still apply: they form
the system message the questions are answered in, exactly as for a language
model. Use it when a step needs several answers about the same input, like
triaging a ticket.
triage = await synalinks.Generator(
data_model=Triage,
decision_model=decision_model,
instructions="Triage the support tickets of an online shop.",
)(x0)
# {"is_billing": true, "urgency": "high", "frustration": 4}
Decision
A Decision picks one label out of a closed list: single-label
classification. With a language model, it first writes its reasoning in a
thinking field, then picks. With a decision model, the question is asked
as is, the labels are the options, and the output is only the choice. Since
the question is all the decision model reads besides the input, phrase it
about the input, and give labels that speak for themselves.
team = await synalinks.Decision(
question="Which team should handle the ticket?",
labels=["billing", "technical", "sales"],
decision_model=decision_model,
)(x0)
# {"choice": "billing"}
A decision model also knows how sure it is. Set min_confidence, and a
decision it is not sure enough about is not taken: the Decision returns
None instead of a guess, so the program can hand the input to something
else, a stronger model or a human.
team = await synalinks.Decision(
question="Which team should handle the ticket?",
labels=["billing", "technical", "sales"],
decision_model=decision_model,
min_confidence=0.7,
)(x0)
# {"choice": "billing"}, or None when the model is unsure
MultiDecision
A MultiDecision picks every label that applies: multi-label
classification. A decision model answers it as one yes/no question per
label ("Which topics does the ticket mention? Does the label 'payment'
apply?"), and the labels whose probability reaches the threshold (0.5 by
default) are kept. Raise it to keep only the labels the model is sure about,
lower it to miss fewer. As each label is judged on its own, the result can
hold several labels, or none at all when nothing applies.
topics = await synalinks.MultiDecision(
question="Which topics does the ticket mention?",
labels=["payment", "delivery", "account"],
decision_model=decision_model,
)(x0)
# {"choices": ["payment"]}
Branch
A Branch is a Decision wired to one module per label: the module of the
chosen label runs, and the other branches return None (see
Guide 5). The
decision model only makes the choice: the branches keep their own models, so
a branch can still be a language model writing an answer. This is where a
decision model pays off most, since the routing runs on every input. A
Branch takes a min_confidence too (a threshold with a MultiDecision):
when the decision is not taken, no branch runs.
(billing, technical) = await synalinks.Branch(
question="Which team should handle the ticket?",
labels=["billing", "technical"],
branches=[billing_agent, technical_agent],
decision_model=decision_model,
)(x0)
SelfCritique
A SelfCritique grades its inputs, typically an answer, with a reward
between 0 and 1. A decision model rates the inputs on five levels, from
"Very bad." to "Very good.", and the reward follows the probability of each
level, so an answer the model hesitates about lands between two levels
rather than on one. It does not write the critique a language model would,
so it only works with a reward (return_reward=True, the default).
RubricsAsJudge
A RubricsAsJudge grades an answer against named criteria and combines them
by weight (Guide 13).
A decision model asks each criterion as a question ("How well does the
answer meet this criterion: ...?") over five levels, from "Does not meet the
criterion at all." to "Fully meets the criterion.", and grades all the
criteria in a single call, however many there are. Instead of a written
critique, the critique lists the level and confidence of each criterion,
which is often all you need to see why an answer lost points. The built-in
rubric rewards, such as Faithfulness or Toxicity, take a decision_model
the same way.
reward = synalinks.rewards.RubricsAsJudge(
rubrics=[
{"name": "correct", "description": "The answer is correct.", "weight": 2},
{"name": "concise", "description": "No preamble, no filler."},
],
decision_model=decision_model,
)
toxicity = synalinks.rewards.Toxicity(decision_model=decision_model)
Route Cheap, Write Expensive
The pattern this enables is the one below: a decision model routes each query (Guide 5), and only the queries that need it pay for a language model reasoning step by step.
graph LR
Q[Query] --> D{Branch<br/>decision model}
D -->|recall a fact| A1[Generator<br/>language model]
D -->|reason step by step| A2[ChainOfThought<br/>language model]
A1 --> O[Answer]
A2 --> OThe routing decision runs on every request, so making it cheap and fast matters more than anywhere else in the program.
How Sure Is Sure Enough?
min_confidence and threshold trade one kind of mistake for another. Set
them low, and the program always decides, including when it should not. Set
them high, and it only decides when it is sure, at the price of handing more
inputs over to a slower path. Where the balance lies depends on what a wrong
decision costs you, compared with an unanswered one, and on your data: there
is no good default.
These thresholds are hyperparameters, not parameters: training does not change them, you fix them before it starts. So rather than guessing, search for them, like any other hyperparameter, with the KerasTuner tuners of Guide 17. The one thing to get right is the reward. If an unanswered input scored 0, like a wrong one, the search would learn that abstaining never pays and set the threshold to 0. Give it a score between a right and a wrong decision, reflecting what an abstention is worth to you:
import synalinks
synalinks.disable_keras_backend() # before using the tuners (see Guide 17)
async def triage_reward(y_true, y_pred):
if y_pred is None:
return 0.5 # not decided: better than wrong, worse than right
return 1.0 if y_pred.get("choice") == y_true.get("choice") else 0.0
async def build_program(hp):
min_confidence = hp.Float("min_confidence", 0.0, 0.9, step=0.1)
synalinks.clear_session()
inputs = synalinks.Input(data_model=Ticket)
outputs = await synalinks.Decision(
question="Which team should handle the ticket?",
labels=["billing", "technical", "sales"],
decision_model=decision_model,
min_confidence=min_confidence,
)(inputs)
program = synalinks.Program(inputs=inputs, outputs=outputs)
program.compile(reward=triage_reward)
return program
tuner = synalinks.tuners.GridSearch(
build_program,
objective=synalinks.tuners.Objective("val_reward", direction="max"),
directory="tuning",
project_name="triage_min_confidence",
)
tuner.search(x=x_train, y=y_train, validation_data=(x_val, y_val), epochs=1)
The grid tries every threshold from 0 to 0.9, and the best one is the balance that fits your data, not a number picked by hand. Decision model calls are cheap, so trying many thresholds costs little.
Learning in Context
A decision model answers in the context of the whole conversation: the system message, with the instructions and the few-shot examples, then the inputs. So it learns in context like a language model: show it a few tickets labeled the way your team triages them, and it follows that labeling.
That means a program answered by a decision model trains the same way
(Guide 15). The
trainable variables are the same instructions and examples, and every
optimizer works on them: RandomFewShot selects the examples, and OMEGA
evolves the instructions, with its own language model writing the candidates
while the decision model answers with each of them. As evaluating a candidate
only costs decision model calls, many candidates can be tried cheaply.
A decision model is also a natural judge
(Guide 13): a
RubricsAsJudge given one grades every criterion in one cheap call, which
makes it a practical reward to train with.
Complete Example
The example below runs every module above with a decision model:
- Triage: a
Generatoranswers the questions of its data model about a support ticket, aDecisionpicks the team that handles it, and aMultiDecisionthe topics it mentions. - Routing: a
Branchdecided by the decision model sends the queries that only recall a fact to a plainGenerator, and the ones that need reasoning to aChainOfThought, both answered by a language model. - Grading: a
SelfCritiquegrades a reply, and aRubricsAsJudgegrades an answer against a reference answer, on weighted criteria, in one call.
Take-Home Summary
- A decision model answers typed questions (yes/no, one of a list, a score) with calibrated probabilities; it never writes text.
- Fields are questions:
bool,Literaland score (synalinks.Rating,synalinks.Score) fields, each asked with itsdescription. A field that has to be written needs a language model. - Modules that decide take it through their
decision_modelargument, never as alanguage_model;set_default_decision_model()sets a default. - Route cheap, write expensive: decide with a decision model, write with a language model only where needed.
- It knows how sure it is:
min_confidenceandthresholdlet a decision be skipped rather than guessed, and KerasTuner finds the values that fit your data. - It learns in context, so programs using it train like any other, with
every optimizer,
OMEGAincluded, and it makes a fast, cheap judge.
API References
Answer
Query
Ticket
Triage
Bases: DataModel
The questions a decision model answers about a ticket.
Source code in guides/33_decision_models.py
Source
import asyncio
from typing import Literal
from dotenv import load_dotenv
import synalinks
# =============================================================================
# Data Models
# =============================================================================
class Ticket(synalinks.DataModel):
"""A support ticket."""
message: str = synalinks.Field(description="The customer message")
class Triage(synalinks.DataModel):
"""The questions a decision model answers about a ticket."""
is_billing: bool = synalinks.Field(
description="Is the ticket about billing?",
)
urgency: Literal["low", "medium", "high"] = synalinks.Field(
description="How urgent is the ticket?",
)
frustration: synalinks.Rating = synalinks.Field(
description="How frustrated is the customer, from 1 (calm) to 5 (angry)?",
)
class Query(synalinks.DataModel):
"""User request."""
query: str = synalinks.Field(description="The user query")
class Answer(synalinks.DataModel):
"""Final answer."""
answer: str = synalinks.Field(description="The correct answer")
# =============================================================================
# Main Demonstration
# =============================================================================
async def main():
load_dotenv()
synalinks.clear_session()
# Log every module call to the console.
synalinks.enable_logging()
# A local Ollama decision model (`ollama pull kev`), no API key needed.
decision_model = synalinks.DecisionModel(model="ollama/kev")
language_model = synalinks.LanguageModel(model="ollama/mistral:latest")
# -------------------------------------------------------------------------
# 1. A Generator with a decision model: every field is a question
# -------------------------------------------------------------------------
print("=" * 60)
print("Triage: a Generator, a Decision and a MultiDecision")
print("=" * 60)
inputs = synalinks.Input(data_model=Ticket)
answers = await synalinks.Generator(
data_model=Triage,
decision_model=decision_model,
instructions="Triage the support tickets of an online shop.",
)(inputs)
team = await synalinks.Decision(
question="Which team should handle the ticket?",
labels=["billing", "technical", "sales"],
decision_model=decision_model,
)(inputs)
topics = await synalinks.MultiDecision(
question="Which topics does the ticket mention?",
labels=["payment", "delivery", "account"],
decision_model=decision_model,
)(inputs)
# The three read the same input, so they run in parallel.
triage = synalinks.Program(
inputs=inputs,
outputs=answers + team + topics,
name="ticket_triage",
description="Triage the support tickets",
)
result = await triage(
Ticket(message="I was charged twice for my order. Please fix this today!")
)
print(result.prettify_json())
# -------------------------------------------------------------------------
# 2. Route with a decision model, write with a language model
# -------------------------------------------------------------------------
print("=" * 60)
print("Routing: a Branch decided by a decision model")
print("=" * 60)
inputs = synalinks.Input(data_model=Query)
(recall, reasoning) = await synalinks.Branch(
question="How should the query be answered?",
labels=["recall a fact", "reason step by step"],
branches=[
synalinks.Generator(
data_model=Answer,
language_model=language_model,
),
synalinks.ChainOfThought(
data_model=Answer,
language_model=language_model,
),
],
decision_model=decision_model,
return_decision=False,
)(inputs)
outputs = recall | reasoning
router = synalinks.Program(
inputs=inputs,
outputs=outputs,
name="conditional_reasoning",
description="Think step by step only when the query needs it",
)
for query in [
"What is the capital of France?",
"A train leaves at 9:40 and arrives at 13:15. How long is the trip?",
]:
result = await router(Query(query=query))
print(result.prettify_json())
# -------------------------------------------------------------------------
# 3. Grade a reply, then grade an answer against a reference with rubrics
# -------------------------------------------------------------------------
print("=" * 60)
print("Grading: SelfCritique and RubricsAsJudge with a decision model")
print("=" * 60)
inputs = synalinks.Input(data_model=Answer)
outputs = await synalinks.SelfCritique(decision_model=decision_model)(inputs)
critic = synalinks.Program(
inputs=inputs,
outputs=outputs,
name="self_critique",
description="Grade a reply",
)
graded = await critic(Answer(answer="The trip takes 3 hours and 35 minutes."))
print(graded.prettify_json())
reward = synalinks.rewards.RubricsAsJudge(
decision_model=decision_model,
rubrics=[
{
"name": "correct",
"description": "The answer matches the reference answer.",
"weight": 2.0,
},
{
"name": "concise",
"description": "No preamble, no restating the question.",
},
],
)
# The reference answer, then the answer to grade: every criterion is
# graded in one decision model call.
grades = await reward.program(
[
Answer(answer="The trip takes 3 hours and 35 minutes."),
Answer(answer="Sure! The trip takes 3 hours and 35 minutes."),
]
)
print(grades.prettify_json())
# The same reward trains a program: `program.compile(reward=reward, ...)`
# then `await program.fit(...)`.
print(f"Decision model calls: {decision_model.cumulated_calls}")
print(f"Decision model cost: ${decision_model.cumulated_cost:.6f}")
if __name__ == "__main__":
asyncio.run(main())
Run log
This guide calls synalinks.enable_logging(), so a full run traces every
module call. The log below is the unedited output of running the guide
above with local models.
Full run log: guides/33_decision_models.log
(DEBUG) [Synalinks]
Call ID: fb811e1e-bf5c-4eff-b8a5-d5a82c39dd4d
Parent call ID: None
Module: Generator
Module Name: generator
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON Schema:
[
{
"additionalProperties": false,
"description": "A support ticket.",
"properties": {
"message": {
"description": "The customer message",
"title": "Message",
"type": "string"
}
},
"required": [
"message"
],
"title": "Ticket",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: fb811e1e-bf5c-4eff-b8a5-d5a82c39dd4d
Parent call ID: None
Module: Generator
Module Name: generator
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON Schema:
[
{
"$defs": {
"Rating": {
"description": "A discretized rating on a 5-level integer scale from 1 to 5.\n\n`Rating` is the integer, Likert-style counterpart of `Score`: instead\nof a float between 0.0 and 1.0, the language model picks one of five\nwhole numbers, 1 (worst) to 5 (best). Because `Rating` is both an `int`\nand an `Enum`, the JSON schema constrains the model to exactly those\nfive values while Python code can use the value in arithmetic\ndirectly (e.g. averaging several ratings, or dividing by 5 to get a\nscore between 0.2 and 1.0).\n\nReach for `Rating` when you want the familiar \"rate this from 1 to 5\"\nframing (star ratings, Likert surveys, rubric grades), and for `Score`\nor `FineScore` when you want a normalized float.\n\nThe labels are the spelled-out numbers `ONE` (1) through `FIVE` (5).\n\nExample:\n\n```python\nimport synalinks\n\nclass Review(synalinks.DataModel):\n rating: synalinks.Rating = synalinks.Field(\n description=\"Overall quality of the answer, from 1 (worst) to 5 (best)\",\n )\n\n# Rating values are real ints, usable in arithmetic.\nassert synalinks.Rating.FOUR == 4\nnormalized = synalinks.Rating.FOUR / 5 # 0.8\n```",
"enum": [
1,
2,
3,
4,
5
],
"title": "Rating",
"type": "integer"
}
},
"additionalProperties": false,
"description": "The questions a decision model answers about a ticket.",
"properties": {
"is_billing": {
"description": "Is the ticket about billing?",
"title": "Is Billing",
"type": "boolean"
},
"urgency": {
"description": "How urgent is the ticket?",
"enum": [
"low",
"medium",
"high"
],
"title": "Urgency",
"type": "string"
},
"frustration": {
"$ref": "#/$defs/Rating",
"description": "How frustrated is the customer, from 1 (calm) to 5 (angry)?"
}
},
"required": [
"is_billing",
"urgency",
"frustration"
],
"title": "Triage",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: f300b795-872a-48b4-a1d5-aae3922968e5
Parent call ID: None
Module: Decision
Module Name: decision
Module Description: Perform a decision on the given input based on a question and a list of labels.
Data Model JSON Schema:
[
{
"additionalProperties": false,
"description": "A support ticket.",
"properties": {
"message": {
"description": "The customer message",
"title": "Message",
"type": "string"
}
},
"required": [
"message"
],
"title": "Ticket",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: f300b795-872a-48b4-a1d5-aae3922968e5
Parent call ID: None
Module: Decision
Module Name: decision
Module Description: Perform a decision on the given input based on a question and a list of labels.
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"choice": {
"enum": [
"billing",
"technical",
"sales"
],
"title": "Choice",
"type": "string"
}
},
"required": [
"choice"
],
"title": "DecisionModelAnswer",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: 8acf926d-42d5-4a11-a926-3e3ff2a6e65c
Parent call ID: None
Module: MultiDecision
Module Name: multi_decision
Module Description: Perform a multi-label selection on the given input.
Data Model JSON Schema:
[
{
"additionalProperties": false,
"description": "A support ticket.",
"properties": {
"message": {
"description": "The customer message",
"title": "Message",
"type": "string"
}
},
"required": [
"message"
],
"title": "Ticket",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: 8acf926d-42d5-4a11-a926-3e3ff2a6e65c
Parent call ID: None
Module: MultiDecision
Module Name: multi_decision
Module Description: Perform a multi-label selection on the given input.
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"choices": {
"description": "The labels whose probability reaches the threshold.",
"items": {
"type": "string",
"enum": [
"payment",
"delivery",
"account"
]
},
"title": "Choices",
"type": "array"
}
},
"required": [
"choices"
],
"title": "MultiDecisionModelAnswer",
"type": "object"
}
]
[Synalinks]
Call ID: b633fab7-1ef1-413e-9649-94d7bc4b0324
Parent call ID: None
Module: Functional
Module Name: ticket_triage
Module Description: Triage the support tickets
Data Model JSON:
[
{
"message": "I was charged twice for my order. Please fix this today!"
}
]
[Synalinks]
Call ID: 22631180-7afe-4387-afc7-7ed57bebc205
Parent call ID: b633fab7-1ef1-413e-9649-94d7bc4b0324
Module: Generator
Module Name: generator
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON:
[
{
"message": "I was charged twice for my order. Please fix this today!"
}
]
[Synalinks]
Call ID: a06fbd75-cecd-4d46-8474-901c62b1bf8a
Parent call ID: 22631180-7afe-4387-afc7-7ed57bebc205
Module: DecisionModel
Module Name: decision_model
Module Description: A decision model API wrapper.
Data Model JSON:
[
{
"messages": [
{
"role": "system",
"content": "\n<instructions>\nTriage the support tickets of an online shop.\n</instructions>\n"
},
{
"role": "user",
"content": "{'message': 'I was charged twice for my order. Please fix this today!'}"
}
]
}
]
[Synalinks]
Call ID: 4df67ad7-4d9d-4190-91fe-7d72c25b2f3f
Parent call ID: a06fbd75-cecd-4d46-8474-901c62b1bf8a
Module: Decision
Module Name: decision
Module Description: Perform a decision on the given input based on a question and a list of labels.
Data Model JSON:
[
{
"message": "I was charged twice for my order. Please fix this today!"
}
]
[Synalinks]
Call ID: f19af8a4-51c9-4e7f-91c5-eff472f4e8cc
Parent call ID: 4df67ad7-4d9d-4190-91fe-7d72c25b2f3f
Module: Generator
Module Name: generator_decision
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON:
[
{
"message": "I was charged twice for my order. Please fix this today!",
"question": "Which team should handle the ticket?"
}
]
[Synalinks]
Call ID: 83e31e1b-9920-4dc2-88e1-a80c6d84d399
Parent call ID: f19af8a4-51c9-4e7f-91c5-eff472f4e8cc
Module: DecisionModel
Module Name: decision_model
Module Description: A decision model API wrapper.
Data Model JSON:
[
{
"messages": [
{
"role": "system",
"content": "\n<instructions>\nYou will be given a question, your task is to answer step-by-step to choose\none the following labels: ['billing', 'technical', 'sales']\n</instructions>\n"
},
{
"role": "user",
"content": "{'message': 'I was charged twice for my order. Please fix this today!', 'question': 'Which team should handle the ticket?'}"
}
]
}
]
[Synalinks]
Call ID: a06fbd75-cecd-4d46-8474-901c62b1bf8a
Parent call ID: 22631180-7afe-4387-afc7-7ed57bebc205
Module: DecisionModel
Module Name: decision_model
Module Description: A decision model API wrapper.
Data Model JSON:
[
{
"is_billing": true,
"urgency": "high",
"frustration": 4
}
]
[Synalinks]
Call ID: 22631180-7afe-4387-afc7-7ed57bebc205
Parent call ID: b633fab7-1ef1-413e-9649-94d7bc4b0324
Module: Generator
Module Name: generator
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON:
[
{
"is_billing": true,
"urgency": "high",
"frustration": 4
}
]
[Synalinks]
Call ID: 83e31e1b-9920-4dc2-88e1-a80c6d84d399
Parent call ID: f19af8a4-51c9-4e7f-91c5-eff472f4e8cc
Module: DecisionModel
Module Name: decision_model
Module Description: A decision model API wrapper.
Data Model JSON:
[
{
"decision": {
"choice": "billing",
"probabilities": {
"billing": 0.9713759425582361,
"technical": 0.009066127235268012,
"sales": 0.019557930206495877
},
"confidence": 0.957063913837354
}
}
]
[Synalinks]
Call ID: f19af8a4-51c9-4e7f-91c5-eff472f4e8cc
Parent call ID: 4df67ad7-4d9d-4190-91fe-7d72c25b2f3f
Module: Generator
Module Name: generator_decision
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON:
[
{
"decision": {
"choice": "billing",
"probabilities": {
"billing": 0.9713759425582361,
"technical": 0.009066127235268012,
"sales": 0.019557930206495877
},
"confidence": 0.957063913837354
}
}
]
[Synalinks]
Call ID: 4df67ad7-4d9d-4190-91fe-7d72c25b2f3f
Parent call ID: a06fbd75-cecd-4d46-8474-901c62b1bf8a
Module: Decision
Module Name: decision
Module Description: Perform a decision on the given input based on a question and a list of labels.
Data Model JSON:
[
{
"choice": "billing"
}
]
[Synalinks]
Call ID: 9bb29f68-f70d-4976-b32b-43b71c5183de
Parent call ID: 83e31e1b-9920-4dc2-88e1-a80c6d84d399
Module: MultiDecision
Module Name: multi_decision
Module Description: Perform a multi-label selection on the given input.
Data Model JSON:
[
{
"message": "I was charged twice for my order. Please fix this today!"
}
]
[Synalinks]
Call ID: 08f6527f-dd40-44da-a349-84bde383bf16
Parent call ID: 9bb29f68-f70d-4976-b32b-43b71c5183de
Module: Generator
Module Name: generator_multi_decision
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON:
[
{
"message": "I was charged twice for my order. Please fix this today!",
"question": "Which topics does the ticket mention?"
}
]
[Synalinks]
Call ID: 4305e965-1653-40eb-a3e0-5e45b2c956b4
Parent call ID: 08f6527f-dd40-44da-a349-84bde383bf16
Module: DecisionModel
Module Name: decision_model
Module Description: A decision model API wrapper.
Data Model JSON:
[
{
"messages": [
{
"role": "system",
"content": "\n<instructions>\nYou will be given a question, your task is to answer step-by-step to choose\none or more of the following labels: ['payment', 'delivery', 'account']\n</instructions>\n"
},
{
"role": "user",
"content": "{'message': 'I was charged twice for my order. Please fix this today!', 'question': 'Which topics does the ticket mention?'}"
}
]
}
]
[Synalinks]
Call ID: 4305e965-1653-40eb-a3e0-5e45b2c956b4
Parent call ID: 08f6527f-dd40-44da-a349-84bde383bf16
Module: DecisionModel
Module Name: decision_model
Module Description: A decision model API wrapper.
Data Model JSON:
[
{
"label_0": {
"noul": 0.9726211692854263
},
"label_1": {
"noul": 0.08889118337160506
},
"label_2": {
"noul": 0.26273200232540217
}
}
]
[Synalinks]
Call ID: 08f6527f-dd40-44da-a349-84bde383bf16
Parent call ID: 9bb29f68-f70d-4976-b32b-43b71c5183de
Module: Generator
Module Name: generator_multi_decision
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON:
[
{
"label_0": {
"noul": 0.9726211692854263
},
"label_1": {
"noul": 0.08889118337160506
},
"label_2": {
"noul": 0.26273200232540217
}
}
]
[Synalinks]
Call ID: 9bb29f68-f70d-4976-b32b-43b71c5183de
Parent call ID: 83e31e1b-9920-4dc2-88e1-a80c6d84d399
Module: MultiDecision
Module Name: multi_decision
Module Description: Perform a multi-label selection on the given input.
Data Model JSON:
[
{
"choices": [
"payment"
]
}
]
[Synalinks]
Call ID: b633fab7-1ef1-413e-9649-94d7bc4b0324
Parent call ID: None
Module: Functional
Module Name: ticket_triage
Module Description: Triage the support tickets
Data Model JSON:
[
{
"is_billing": true,
"urgency": "high",
"frustration": 4,
"choice": "billing",
"choices": [
"payment"
]
}
]
(DEBUG) [Synalinks]
Call ID: 03988b31-54da-45cf-a384-5154243ed9d3
Parent call ID: None
Module: Branch
Module Name: branch
Module Description: Use a `LanguageModel` to select which module(s) to call based on an
Data Model JSON Schema:
[
{
"additionalProperties": false,
"description": "User request.",
"properties": {
"query": {
"description": "The user query",
"title": "Query",
"type": "string"
}
},
"required": [
"query"
],
"title": "Query",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: cf464f19-7354-43b8-bb48-dd99b7917249
Parent call ID: 03988b31-54da-45cf-a384-5154243ed9d3
Module: Decision
Module Name: decision_branch
Module Description: Perform a decision on the given input based on a question and a list of labels.
Data Model JSON Schema:
[
{
"additionalProperties": false,
"description": "User request.",
"properties": {
"query": {
"description": "The user query",
"title": "Query",
"type": "string"
}
},
"required": [
"query"
],
"title": "Query",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: cf464f19-7354-43b8-bb48-dd99b7917249
Parent call ID: 03988b31-54da-45cf-a384-5154243ed9d3
Module: Decision
Module Name: decision_branch
Module Description: Perform a decision on the given input based on a question and a list of labels.
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"choice": {
"enum": [
"recall a fact",
"reason step by step"
],
"title": "Choice",
"type": "string"
}
},
"required": [
"choice"
],
"title": "DecisionModelAnswer",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: 1d830b87-a94c-41d0-9666-94920a1b7cb2
Parent call ID: cf464f19-7354-43b8-bb48-dd99b7917249
Module: Generator
Module Name: generator_1
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"query": {
"description": "The user query",
"title": "Query",
"type": "string"
},
"choice": {
"enum": [
"recall a fact",
"reason step by step"
],
"title": "Choice",
"type": "string"
}
},
"required": [
"query",
"choice"
],
"title": "Query",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: 1d830b87-a94c-41d0-9666-94920a1b7cb2
Parent call ID: cf464f19-7354-43b8-bb48-dd99b7917249
Module: Generator
Module Name: generator_1
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON Schema:
[
{
"additionalProperties": false,
"description": "Final answer.",
"properties": {
"answer": {
"description": "The correct answer",
"title": "Answer",
"type": "string"
}
},
"required": [
"answer"
],
"title": "Answer",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: fbc21e05-70c4-4d5e-8b7b-7fc8b6dd53c1
Parent call ID: 1d830b87-a94c-41d0-9666-94920a1b7cb2
Module: ChainOfThought
Module Name: chain_of_thought
Module Description: Useful to answer in a step by step manner.
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"query": {
"description": "The user query",
"title": "Query",
"type": "string"
},
"choice": {
"enum": [
"recall a fact",
"reason step by step"
],
"title": "Choice",
"type": "string"
}
},
"required": [
"query",
"choice"
],
"title": "Query",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: 4838eda4-68c2-44b1-a8e4-e2785eb43b7b
Parent call ID: fbc21e05-70c4-4d5e-8b7b-7fc8b6dd53c1
Module: Generator
Module Name: generator_chain_of_thought
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"query": {
"description": "The user query",
"title": "Query",
"type": "string"
},
"choice": {
"enum": [
"recall a fact",
"reason step by step"
],
"title": "Choice",
"type": "string"
}
},
"required": [
"query",
"choice"
],
"title": "Query",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: 4838eda4-68c2-44b1-a8e4-e2785eb43b7b
Parent call ID: fbc21e05-70c4-4d5e-8b7b-7fc8b6dd53c1
Module: Generator
Module Name: generator_chain_of_thought
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"thinking": {
"description": "Your step by step thinking",
"title": "Thinking",
"type": "string"
},
"answer": {
"description": "The correct answer",
"title": "Answer",
"type": "string"
}
},
"required": [
"thinking",
"answer"
],
"title": "Thinking",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: f36b2aad-5043-450e-9399-ade09fa6715d
Parent call ID: 4838eda4-68c2-44b1-a8e4-e2785eb43b7b
Module: Generator
Module Name: generator_chain_of_thought
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"query": {
"description": "The user query",
"title": "Query",
"type": "string"
},
"choice": {
"enum": [
"recall a fact",
"reason step by step"
],
"title": "Choice",
"type": "string"
}
},
"required": [
"query",
"choice"
],
"title": "Query",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: f36b2aad-5043-450e-9399-ade09fa6715d
Parent call ID: 4838eda4-68c2-44b1-a8e4-e2785eb43b7b
Module: Generator
Module Name: generator_chain_of_thought
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"thinking": {
"description": "Your step by step thinking",
"title": "Thinking",
"type": "string"
},
"answer": {
"description": "The correct answer",
"title": "Answer",
"type": "string"
}
},
"required": [
"thinking",
"answer"
],
"title": "Thinking",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: fbc21e05-70c4-4d5e-8b7b-7fc8b6dd53c1
Parent call ID: 1d830b87-a94c-41d0-9666-94920a1b7cb2
Module: ChainOfThought
Module Name: chain_of_thought
Module Description: Useful to answer in a step by step manner.
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"thinking": {
"description": "Your step by step thinking",
"title": "Thinking",
"type": "string"
},
"answer": {
"description": "The correct answer",
"title": "Answer",
"type": "string"
}
},
"required": [
"thinking",
"answer"
],
"title": "Thinking",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: 7f60e940-4a10-4a72-9029-567cbaf86632
Parent call ID: f36b2aad-5043-450e-9399-ade09fa6715d
Module: Decision
Module Name: decision_branch
Module Description: Perform a decision on the given input based on a question and a list of labels.
Data Model JSON Schema:
[
{
"additionalProperties": false,
"description": "User request.",
"properties": {
"query": {
"description": "The user query",
"title": "Query",
"type": "string"
}
},
"required": [
"query"
],
"title": "Query",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: 7f60e940-4a10-4a72-9029-567cbaf86632
Parent call ID: f36b2aad-5043-450e-9399-ade09fa6715d
Module: Decision
Module Name: decision_branch
Module Description: Perform a decision on the given input based on a question and a list of labels.
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"choice": {
"enum": [
"recall a fact",
"reason step by step"
],
"title": "Choice",
"type": "string"
}
},
"required": [
"choice"
],
"title": "DecisionModelAnswer",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: d2f80dd6-0de3-49f8-bbc9-d54f646b102f
Parent call ID: 7f60e940-4a10-4a72-9029-567cbaf86632
Module: Generator
Module Name: generator_1
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"query": {
"description": "The user query",
"title": "Query",
"type": "string"
},
"choice": {
"enum": [
"recall a fact",
"reason step by step"
],
"title": "Choice",
"type": "string"
}
},
"required": [
"query",
"choice"
],
"title": "Query",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: d2f80dd6-0de3-49f8-bbc9-d54f646b102f
Parent call ID: 7f60e940-4a10-4a72-9029-567cbaf86632
Module: Generator
Module Name: generator_1
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON Schema:
[
{
"additionalProperties": false,
"description": "Final answer.",
"properties": {
"answer": {
"description": "The correct answer",
"title": "Answer",
"type": "string"
}
},
"required": [
"answer"
],
"title": "Answer",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: 7d2b3ed7-acb0-491e-afdf-fe50e395f82d
Parent call ID: d2f80dd6-0de3-49f8-bbc9-d54f646b102f
Module: ChainOfThought
Module Name: chain_of_thought
Module Description: Useful to answer in a step by step manner.
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"query": {
"description": "The user query",
"title": "Query",
"type": "string"
},
"choice": {
"enum": [
"recall a fact",
"reason step by step"
],
"title": "Choice",
"type": "string"
}
},
"required": [
"query",
"choice"
],
"title": "Query",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: 02c38c28-91ed-444c-b798-0b9c160c661d
Parent call ID: 7d2b3ed7-acb0-491e-afdf-fe50e395f82d
Module: Generator
Module Name: generator_chain_of_thought
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"query": {
"description": "The user query",
"title": "Query",
"type": "string"
},
"choice": {
"enum": [
"recall a fact",
"reason step by step"
],
"title": "Choice",
"type": "string"
}
},
"required": [
"query",
"choice"
],
"title": "Query",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: 02c38c28-91ed-444c-b798-0b9c160c661d
Parent call ID: 7d2b3ed7-acb0-491e-afdf-fe50e395f82d
Module: Generator
Module Name: generator_chain_of_thought
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"thinking": {
"description": "Your step by step thinking",
"title": "Thinking",
"type": "string"
},
"answer": {
"description": "The correct answer",
"title": "Answer",
"type": "string"
}
},
"required": [
"thinking",
"answer"
],
"title": "Thinking",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: 7d2b3ed7-acb0-491e-afdf-fe50e395f82d
Parent call ID: d2f80dd6-0de3-49f8-bbc9-d54f646b102f
Module: ChainOfThought
Module Name: chain_of_thought
Module Description: Useful to answer in a step by step manner.
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"thinking": {
"description": "Your step by step thinking",
"title": "Thinking",
"type": "string"
},
"answer": {
"description": "The correct answer",
"title": "Answer",
"type": "string"
}
},
"required": [
"thinking",
"answer"
],
"title": "Thinking",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: 03988b31-54da-45cf-a384-5154243ed9d3
Parent call ID: None
Module: Branch
Module Name: branch
Module Description: Use a `LanguageModel` to select which module(s) to call based on an
Data Model JSON Schema:
[
{
"additionalProperties": false,
"description": "Final answer.",
"properties": {
"answer": {
"description": "The correct answer",
"title": "Answer",
"type": "string"
}
},
"required": [
"answer"
],
"title": "Answer",
"type": "object"
},
{
"additionalProperties": false,
"properties": {
"thinking": {
"description": "Your step by step thinking",
"title": "Thinking",
"type": "string"
},
"answer": {
"description": "The correct answer",
"title": "Answer",
"type": "string"
}
},
"required": [
"thinking",
"answer"
],
"title": "Thinking",
"type": "object"
}
]
[Synalinks]
Call ID: 78f119ce-3ea8-4a6d-b354-ea36691ccf62
Parent call ID: None
Module: Functional
Module Name: conditional_reasoning
Module Description: Think step by step only when the query needs it
Data Model JSON:
[
{
"query": "What is the capital of France?"
}
]
[Synalinks]
Call ID: b40adafa-b13b-4677-93e6-99a1216e6242
Parent call ID: 78f119ce-3ea8-4a6d-b354-ea36691ccf62
Module: Branch
Module Name: branch
Module Description: Use a `LanguageModel` to select which module(s) to call based on an
Data Model JSON:
[
{
"query": "What is the capital of France?"
}
]
[Synalinks]
Call ID: b298b57b-f844-487e-a0c3-abeb2354a1a2
Parent call ID: b40adafa-b13b-4677-93e6-99a1216e6242
Module: Decision
Module Name: decision_branch
Module Description: Perform a decision on the given input based on a question and a list of labels.
Data Model JSON:
[
{
"query": "What is the capital of France?"
}
]
[Synalinks]
Call ID: 12fdaf6e-4aff-4063-bdc1-755628f012b0
Parent call ID: b298b57b-f844-487e-a0c3-abeb2354a1a2
Module: Generator
Module Name: generator_decision_branch
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON:
[
{
"query": "What is the capital of France?",
"question": "How should the query be answered?"
}
]
[Synalinks]
Call ID: 36065484-72c7-471c-b30f-c68d3dbe230d
Parent call ID: 12fdaf6e-4aff-4063-bdc1-755628f012b0
Module: DecisionModel
Module Name: decision_model
Module Description: A decision model API wrapper.
Data Model JSON:
[
{
"messages": [
{
"role": "system",
"content": "\n<instructions>\nYou will be given a question, your task is to answer step-by-step to choose\none the following labels: ['recall a fact', 'reason step by step']\n</instructions>\n"
},
{
"role": "user",
"content": "{'query': 'What is the capital of France?', 'question': 'How should the query be answered?'}"
}
]
}
]
[Synalinks]
Call ID: 36065484-72c7-471c-b30f-c68d3dbe230d
Parent call ID: 12fdaf6e-4aff-4063-bdc1-755628f012b0
Module: DecisionModel
Module Name: decision_model
Module Description: A decision model API wrapper.
Data Model JSON:
[
{
"decision": {
"choice": "recall a fact",
"probabilities": {
"recall a fact": 0.9835272907701689,
"reason step by step": 0.016472709229830983
},
"confidence": 0.9670545815403377
}
}
]
[Synalinks]
Call ID: 12fdaf6e-4aff-4063-bdc1-755628f012b0
Parent call ID: b298b57b-f844-487e-a0c3-abeb2354a1a2
Module: Generator
Module Name: generator_decision_branch
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON:
[
{
"decision": {
"choice": "recall a fact",
"probabilities": {
"recall a fact": 0.9835272907701689,
"reason step by step": 0.016472709229830983
},
"confidence": 0.9670545815403377
}
}
]
[Synalinks]
Call ID: b298b57b-f844-487e-a0c3-abeb2354a1a2
Parent call ID: b40adafa-b13b-4677-93e6-99a1216e6242
Module: Decision
Module Name: decision_branch
Module Description: Perform a decision on the given input based on a question and a list of labels.
Data Model JSON:
[
{
"choice": "recall a fact"
}
]
[Synalinks]
Call ID: 6530c108-87c6-45bc-9015-c14bd8ca6e05
Parent call ID: 36065484-72c7-471c-b30f-c68d3dbe230d
Module: Generator
Module Name: generator_1
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON:
[
{
"query": "What is the capital of France?",
"choice": "recall a fact"
}
]
[Synalinks]
Call ID: 0d4c39fb-2081-45d0-8beb-2e436678a548
Parent call ID: 6530c108-87c6-45bc-9015-c14bd8ca6e05
Module: LanguageModel
Module Name: language_model
Module Description: A language model API wrapper.
Data Model JSON:
[
{
"messages": [
{
"role": "system",
"content": "\n<instructions>\nYour task is to answer with a JSON containing the following keys: ['answer']\n</instructions>\n"
},
{
"role": "user",
"content": "{'query': 'What is the capital of France?', 'choice': 'recall a fact'}"
}
]
}
]
[Synalinks]
Call ID: 0d4c39fb-2081-45d0-8beb-2e436678a548
Parent call ID: 6530c108-87c6-45bc-9015-c14bd8ca6e05
Module: LanguageModel
Module Name: language_model
Module Description: A language model API wrapper.
Data Model JSON:
[
{
"answer": "The capital of France is Paris."
}
]
[Synalinks]
Call ID: 6530c108-87c6-45bc-9015-c14bd8ca6e05
Parent call ID: 36065484-72c7-471c-b30f-c68d3dbe230d
Module: Generator
Module Name: generator_1
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON:
[
{
"answer": "The capital of France is Paris."
}
]
[Synalinks]
Call ID: b40adafa-b13b-4677-93e6-99a1216e6242
Parent call ID: 78f119ce-3ea8-4a6d-b354-ea36691ccf62
Module: Branch
Module Name: branch
Module Description: Use a `LanguageModel` to select which module(s) to call based on an
Data Model JSON:
[
{
"answer": "The capital of France is Paris."
}
]
[Synalinks]
Call ID: 78f119ce-3ea8-4a6d-b354-ea36691ccf62
Parent call ID: None
Module: Functional
Module Name: conditional_reasoning
Module Description: Think step by step only when the query needs it
Data Model JSON:
[
{
"answer": "The capital of France is Paris."
}
]
[Synalinks]
Call ID: 0d23700b-5538-4642-b485-01e2464eeea5
Parent call ID: None
Module: Functional
Module Name: conditional_reasoning
Module Description: Think step by step only when the query needs it
Data Model JSON:
[
{
"query": "A train leaves at 9:40 and arrives at 13:15. How long is the trip?"
}
]
[Synalinks]
Call ID: 1f7c91fc-8acd-46b2-879b-840a7b8592c8
Parent call ID: 0d23700b-5538-4642-b485-01e2464eeea5
Module: Branch
Module Name: branch
Module Description: Use a `LanguageModel` to select which module(s) to call based on an
Data Model JSON:
[
{
"query": "A train leaves at 9:40 and arrives at 13:15. How long is the trip?"
}
]
[Synalinks]
Call ID: fe374d56-c0db-4b49-90eb-26d3746faa00
Parent call ID: 1f7c91fc-8acd-46b2-879b-840a7b8592c8
Module: Decision
Module Name: decision_branch
Module Description: Perform a decision on the given input based on a question and a list of labels.
Data Model JSON:
[
{
"query": "A train leaves at 9:40 and arrives at 13:15. How long is the trip?"
}
]
[Synalinks]
Call ID: 76f9df0b-ec19-42cf-bdc8-e4cb4a5fcbae
Parent call ID: fe374d56-c0db-4b49-90eb-26d3746faa00
Module: Generator
Module Name: generator_decision_branch
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON:
[
{
"query": "A train leaves at 9:40 and arrives at 13:15. How long is the trip?",
"question": "How should the query be answered?"
}
]
[Synalinks]
Call ID: cbec027f-fa89-4ea5-bda7-401cf3fd1823
Parent call ID: 76f9df0b-ec19-42cf-bdc8-e4cb4a5fcbae
Module: DecisionModel
Module Name: decision_model
Module Description: A decision model API wrapper.
Data Model JSON:
[
{
"messages": [
{
"role": "system",
"content": "\n<instructions>\nYou will be given a question, your task is to answer step-by-step to choose\none the following labels: ['recall a fact', 'reason step by step']\n</instructions>\n"
},
{
"role": "user",
"content": "{'query': 'A train leaves at 9:40 and arrives at 13:15. How long is the trip?', 'question': 'How should the query be answered?'}"
}
]
}
]
[Synalinks]
Call ID: cbec027f-fa89-4ea5-bda7-401cf3fd1823
Parent call ID: 76f9df0b-ec19-42cf-bdc8-e4cb4a5fcbae
Module: DecisionModel
Module Name: decision_model
Module Description: A decision model API wrapper.
Data Model JSON:
[
{
"decision": {
"choice": "reason step by step",
"probabilities": {
"recall a fact": 0.06994936985427253,
"reason step by step": 0.9300506301457274
},
"confidence": 0.8601012602914548
}
}
]
[Synalinks]
Call ID: 76f9df0b-ec19-42cf-bdc8-e4cb4a5fcbae
Parent call ID: fe374d56-c0db-4b49-90eb-26d3746faa00
Module: Generator
Module Name: generator_decision_branch
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON:
[
{
"decision": {
"choice": "reason step by step",
"probabilities": {
"recall a fact": 0.06994936985427253,
"reason step by step": 0.9300506301457274
},
"confidence": 0.8601012602914548
}
}
]
[Synalinks]
Call ID: fe374d56-c0db-4b49-90eb-26d3746faa00
Parent call ID: 1f7c91fc-8acd-46b2-879b-840a7b8592c8
Module: Decision
Module Name: decision_branch
Module Description: Perform a decision on the given input based on a question and a list of labels.
Data Model JSON:
[
{
"choice": "reason step by step"
}
]
[Synalinks]
Call ID: a2462c48-8611-4fe3-8240-985d13418f4e
Parent call ID: cbec027f-fa89-4ea5-bda7-401cf3fd1823
Module: ChainOfThought
Module Name: chain_of_thought
Module Description: Useful to answer in a step by step manner.
Data Model JSON:
[
{
"query": "A train leaves at 9:40 and arrives at 13:15. How long is the trip?",
"choice": "reason step by step"
}
]
[Synalinks]
Call ID: 256d5613-8428-42ba-b29a-a313c3a77d63
Parent call ID: a2462c48-8611-4fe3-8240-985d13418f4e
Module: Generator
Module Name: generator_chain_of_thought
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON:
[
{
"query": "A train leaves at 9:40 and arrives at 13:15. How long is the trip?",
"choice": "reason step by step"
}
]
[Synalinks]
Call ID: e9ecc5d7-f04b-4042-a458-42ee774826a3
Parent call ID: 256d5613-8428-42ba-b29a-a313c3a77d63
Module: LanguageModel
Module Name: language_model
Module Description: A language model API wrapper.
Data Model JSON:
[
{
"messages": [
{
"role": "system",
"content": "\n<instructions>\nYour task is to answer with a JSON containing the following keys: ['thinking', 'answer']\n</instructions>\n"
},
{
"role": "user",
"content": "{'query': 'A train leaves at 9:40 and arrives at 13:15. How long is the trip?', 'choice': 'reason step by step'}"
}
]
}
]
[Synalinks]
Call ID: e9ecc5d7-f04b-4042-a458-42ee774826a3
Parent call ID: 256d5613-8428-42ba-b29a-a313c3a77d63
Module: LanguageModel
Module Name: language_model
Module Description: A language model API wrapper.
Data Model JSON:
[
{
"thinking": "Let's break down the problem: The train leaves at 9:40 and arrives at 13:15. We first convert both times to a 24-hour clock format, as the time span may cross the 12-hour boundary. 9:40 and 13:15 are already in 24-hour format, so we don't need to convert them. Subtracting the departure time from the arrival time gives us the duration of the trip in hours. Finally, we convert the result into minutes by multiplying the total number of hours by 60 and adding the remaining minutes.",
"answer": "The duration of the trip is 3 hours and 35 minutes."
}
]
[Synalinks]
Call ID: 256d5613-8428-42ba-b29a-a313c3a77d63
Parent call ID: a2462c48-8611-4fe3-8240-985d13418f4e
Module: Generator
Module Name: generator_chain_of_thought
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON:
[
{
"thinking": "Let's break down the problem: The train leaves at 9:40 and arrives at 13:15. We first convert both times to a 24-hour clock format, as the time span may cross the 12-hour boundary. 9:40 and 13:15 are already in 24-hour format, so we don't need to convert them. Subtracting the departure time from the arrival time gives us the duration of the trip in hours. Finally, we convert the result into minutes by multiplying the total number of hours by 60 and adding the remaining minutes.",
"answer": "The duration of the trip is 3 hours and 35 minutes."
}
]
[Synalinks]
Call ID: a2462c48-8611-4fe3-8240-985d13418f4e
Parent call ID: cbec027f-fa89-4ea5-bda7-401cf3fd1823
Module: ChainOfThought
Module Name: chain_of_thought
Module Description: Useful to answer in a step by step manner.
Data Model JSON:
[
{
"thinking": "Let's break down the problem: The train leaves at 9:40 and arrives at 13:15. We first convert both times to a 24-hour clock format, as the time span may cross the 12-hour boundary. 9:40 and 13:15 are already in 24-hour format, so we don't need to convert them. Subtracting the departure time from the arrival time gives us the duration of the trip in hours. Finally, we convert the result into minutes by multiplying the total number of hours by 60 and adding the remaining minutes.",
"answer": "The duration of the trip is 3 hours and 35 minutes."
}
]
[Synalinks]
Call ID: 1f7c91fc-8acd-46b2-879b-840a7b8592c8
Parent call ID: 0d23700b-5538-4642-b485-01e2464eeea5
Module: Branch
Module Name: branch
Module Description: Use a `LanguageModel` to select which module(s) to call based on an
Data Model JSON:
[
{
"thinking": "Let's break down the problem: The train leaves at 9:40 and arrives at 13:15. We first convert both times to a 24-hour clock format, as the time span may cross the 12-hour boundary. 9:40 and 13:15 are already in 24-hour format, so we don't need to convert them. Subtracting the departure time from the arrival time gives us the duration of the trip in hours. Finally, we convert the result into minutes by multiplying the total number of hours by 60 and adding the remaining minutes.",
"answer": "The duration of the trip is 3 hours and 35 minutes."
}
]
[Synalinks]
Call ID: 0d23700b-5538-4642-b485-01e2464eeea5
Parent call ID: None
Module: Functional
Module Name: conditional_reasoning
Module Description: Think step by step only when the query needs it
Data Model JSON:
[
{
"thinking": "Let's break down the problem: The train leaves at 9:40 and arrives at 13:15. We first convert both times to a 24-hour clock format, as the time span may cross the 12-hour boundary. 9:40 and 13:15 are already in 24-hour format, so we don't need to convert them. Subtracting the departure time from the arrival time gives us the duration of the trip in hours. Finally, we convert the result into minutes by multiplying the total number of hours by 60 and adding the remaining minutes.",
"answer": "The duration of the trip is 3 hours and 35 minutes."
}
]
(DEBUG) [Synalinks]
Call ID: 384508e4-0ccc-4b6c-884f-66c9617cbc09
Parent call ID: None
Module: SelfCritique
Module Name: self_critique
Module Description: Useful to critique the given inputs.
Data Model JSON Schema:
[
{
"additionalProperties": false,
"description": "Final answer.",
"properties": {
"answer": {
"description": "The correct answer",
"title": "Answer",
"type": "string"
}
},
"required": [
"answer"
],
"title": "Answer",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: 601ec0b5-c555-4e09-a338-b1393c7e9786
Parent call ID: 384508e4-0ccc-4b6c-884f-66c9617cbc09
Module: Generator
Module Name: generator_self_critique
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON Schema:
[
{
"additionalProperties": false,
"description": "Final answer.",
"properties": {
"answer": {
"description": "The correct answer",
"title": "Answer",
"type": "string"
}
},
"required": [
"answer"
],
"title": "Answer",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: 601ec0b5-c555-4e09-a338-b1393c7e9786
Parent call ID: 384508e4-0ccc-4b6c-884f-66c9617cbc09
Module: Generator
Module Name: generator_self_critique
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"answer": {
"description": "The correct answer",
"title": "Answer",
"type": "string"
},
"reward": {
"title": "ScoreAnswer",
"type": "object",
"description": "How good are the provided inputs?",
"properties": {
"score": {
"title": "Score",
"type": "number",
"minimum": 0.0,
"maximum": 4.0,
"description": "The probability-weighted level index."
},
"legend": {
"title": "Legend",
"type": "object",
"description": "The description of each level, keyed by level index.",
"properties": {
"0": {
"title": "0",
"type": "string",
"description": "Very bad."
},
"1": {
"title": "1",
"type": "string",
"description": "Bad."
},
"2": {
"title": "2",
"type": "string",
"description": "Acceptable."
},
"3": {
"title": "3",
"type": "string",
"description": "Good."
},
"4": {
"title": "4",
"type": "string",
"description": "Very good."
}
},
"required": [
"0",
"1",
"2",
"3",
"4"
],
"additionalProperties": false
},
"probabilities": {
"title": "Probabilities",
"type": "object",
"description": "The probability of each level, keyed by level index.",
"properties": {
"0": {
"title": "0",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
},
"1": {
"title": "1",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
},
"2": {
"title": "2",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
},
"3": {
"title": "3",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
},
"4": {
"title": "4",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
}
},
"required": [
"0",
"1",
"2",
"3",
"4"
],
"additionalProperties": false
},
"confidence": {
"title": "Confidence",
"type": "number",
"minimum": 0.0,
"maximum": 1.0,
"description": "How certain the decision model is, from 0 to 1."
}
},
"required": [
"score",
"legend",
"probabilities",
"confidence"
],
"additionalProperties": false
}
},
"required": [
"answer",
"reward"
],
"title": "Answer",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: 874a07f3-9632-44a3-816c-65c324ccf9af
Parent call ID: 601ec0b5-c555-4e09-a338-b1393c7e9786
Module: Generator
Module Name: generator_self_critique
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON Schema:
[
{
"additionalProperties": false,
"description": "Final answer.",
"properties": {
"answer": {
"description": "The correct answer",
"title": "Answer",
"type": "string"
}
},
"required": [
"answer"
],
"title": "Answer",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: 874a07f3-9632-44a3-816c-65c324ccf9af
Parent call ID: 601ec0b5-c555-4e09-a338-b1393c7e9786
Module: Generator
Module Name: generator_self_critique
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"answer": {
"description": "The correct answer",
"title": "Answer",
"type": "string"
},
"reward": {
"title": "ScoreAnswer",
"type": "object",
"description": "How good are the provided inputs?",
"properties": {
"score": {
"title": "Score",
"type": "number",
"minimum": 0.0,
"maximum": 4.0,
"description": "The probability-weighted level index."
},
"legend": {
"title": "Legend",
"type": "object",
"description": "The description of each level, keyed by level index.",
"properties": {
"0": {
"title": "0",
"type": "string",
"description": "Very bad."
},
"1": {
"title": "1",
"type": "string",
"description": "Bad."
},
"2": {
"title": "2",
"type": "string",
"description": "Acceptable."
},
"3": {
"title": "3",
"type": "string",
"description": "Good."
},
"4": {
"title": "4",
"type": "string",
"description": "Very good."
}
},
"required": [
"0",
"1",
"2",
"3",
"4"
],
"additionalProperties": false
},
"probabilities": {
"title": "Probabilities",
"type": "object",
"description": "The probability of each level, keyed by level index.",
"properties": {
"0": {
"title": "0",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
},
"1": {
"title": "1",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
},
"2": {
"title": "2",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
},
"3": {
"title": "3",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
},
"4": {
"title": "4",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
}
},
"required": [
"0",
"1",
"2",
"3",
"4"
],
"additionalProperties": false
},
"confidence": {
"title": "Confidence",
"type": "number",
"minimum": 0.0,
"maximum": 1.0,
"description": "How certain the decision model is, from 0 to 1."
}
},
"required": [
"score",
"legend",
"probabilities",
"confidence"
],
"additionalProperties": false
}
},
"required": [
"answer",
"reward"
],
"title": "Answer",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: 384508e4-0ccc-4b6c-884f-66c9617cbc09
Parent call ID: None
Module: SelfCritique
Module Name: self_critique
Module Description: Useful to critique the given inputs.
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"answer": {
"description": "The correct answer",
"title": "Answer",
"type": "string"
},
"reward": {
"title": "Reward",
"type": "number",
"minimum": 0.0,
"maximum": 1.0,
"description": "The reward value corresponding to the critique, normalized between 0.0 (very bad) and 1.0 (very good)"
}
},
"required": [
"answer",
"reward"
],
"title": "Answer",
"type": "object"
}
]
[Synalinks]
Call ID: c0795b39-404d-4043-aff7-cc2d104e7284
Parent call ID: None
Module: Functional
Module Name: self_critique
Module Description: Grade a reply
Data Model JSON:
[
{
"answer": "The trip takes 3 hours and 35 minutes."
}
]
[Synalinks]
Call ID: 3176f848-8a9b-4a82-9c91-4e468d34cd04
Parent call ID: c0795b39-404d-4043-aff7-cc2d104e7284
Module: SelfCritique
Module Name: self_critique
Module Description: Useful to critique the given inputs.
Data Model JSON:
[
{
"answer": "The trip takes 3 hours and 35 minutes."
}
]
[Synalinks]
Call ID: 77120849-195a-464b-a59e-12dd1cddd2e7
Parent call ID: 3176f848-8a9b-4a82-9c91-4e468d34cd04
Module: Generator
Module Name: generator_self_critique
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON:
[
{
"answer": "The trip takes 3 hours and 35 minutes."
}
]
[Synalinks]
Call ID: c5654a5d-f042-4e47-a2e6-951716f9c81a
Parent call ID: 77120849-195a-464b-a59e-12dd1cddd2e7
Module: DecisionModel
Module Name: decision_model
Module Description: A decision model API wrapper.
Data Model JSON:
[
{
"messages": [
{
"role": "system",
"content": "\n<instructions>\nYour task is to carefully examine the provided inputs and grade them.\n</instructions>\n"
},
{
"role": "user",
"content": "{'answer': 'The trip takes 3 hours and 35 minutes.'}"
}
]
}
]
[Synalinks]
Call ID: c5654a5d-f042-4e47-a2e6-951716f9c81a
Parent call ID: 77120849-195a-464b-a59e-12dd1cddd2e7
Module: DecisionModel
Module Name: decision_model
Module Description: A decision model API wrapper.
Data Model JSON:
[
{
"reward": {
"score": 2.2718707329173458,
"legend": {
"0": "Very bad.",
"1": "Bad.",
"2": "Acceptable.",
"3": "Good.",
"4": "Very good."
},
"probabilities": {
"0": 0.09135757351834706,
"1": 0.15854609827831598,
"2": 0.3015716502641704,
"3": 0.28391737764597774,
"4": 0.1646073002931889
},
"confidence": 0.20467231371052863
}
}
]
[Synalinks]
Call ID: 77120849-195a-464b-a59e-12dd1cddd2e7
Parent call ID: 3176f848-8a9b-4a82-9c91-4e468d34cd04
Module: Generator
Module Name: generator_self_critique
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON:
[
{
"answer": "The trip takes 3 hours and 35 minutes.",
"reward": {
"score": 2.2718707329173458,
"legend": {
"0": "Very bad.",
"1": "Bad.",
"2": "Acceptable.",
"3": "Good.",
"4": "Very good."
},
"probabilities": {
"0": 0.09135757351834706,
"1": 0.15854609827831598,
"2": 0.3015716502641704,
"3": 0.28391737764597774,
"4": 0.1646073002931889
},
"confidence": 0.20467231371052863
}
}
]
[Synalinks]
Call ID: 3176f848-8a9b-4a82-9c91-4e468d34cd04
Parent call ID: c0795b39-404d-4043-aff7-cc2d104e7284
Module: SelfCritique
Module Name: self_critique
Module Description: Useful to critique the given inputs.
Data Model JSON:
[
{
"answer": "The trip takes 3 hours and 35 minutes.",
"reward": 0.5679676832293364
}
]
[Synalinks]
Call ID: c0795b39-404d-4043-aff7-cc2d104e7284
Parent call ID: None
Module: Functional
Module Name: self_critique
Module Description: Grade a reply
Data Model JSON:
[
{
"answer": "The trip takes 3 hours and 35 minutes.",
"reward": 0.5679676832293364
}
]
[Synalinks]
Call ID: 0c8e3114-d800-47cd-a61d-b7d1cc4b976a
Parent call ID: None
Module: RubricsAsJudgeProgram
Module Name: rubrics_as_judge_program
Module Description: Score each criterion with a language (or decision) model, then combine
Data Model JSON:
[
{
"answer": "The trip takes 3 hours and 35 minutes."
},
{
"answer": "Sure! The trip takes 3 hours and 35 minutes."
}
]
(DEBUG) [Synalinks]
Call ID: cd3f33e6-13eb-4685-bd0f-aef3cda2726d
Parent call ID: 0c8e3114-d800-47cd-a61d-b7d1cc4b976a
Module: Generator
Module Name: generator_rubrics_as_judge_program
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON Schema:
[
{
"additionalProperties": false,
"properties": {
"gold_answer": {
"description": "The correct answer",
"title": "Gold Answer",
"type": "string"
},
"answer": {
"description": "The correct answer",
"title": "Answer",
"type": "string"
}
},
"required": [
"gold_answer",
"answer"
],
"title": "Answer",
"type": "object"
}
]
(DEBUG) [Synalinks]
Call ID: cd3f33e6-13eb-4685-bd0f-aef3cda2726d
Parent call ID: 0c8e3114-d800-47cd-a61d-b7d1cc4b976a
Module: Generator
Module Name: generator_rubrics_as_judge_program
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON Schema:
[
{
"title": "RubricGrades",
"type": "object",
"properties": {
"correct": {
"title": "ScoreAnswer",
"type": "object",
"description": "How well does the answer meet this criterion: The answer matches the reference answer.",
"properties": {
"score": {
"title": "Score",
"type": "number",
"minimum": 0.0,
"maximum": 4.0,
"description": "The probability-weighted level index."
},
"legend": {
"title": "Legend",
"type": "object",
"description": "The description of each level, keyed by level index.",
"properties": {
"0": {
"title": "0",
"type": "string",
"description": "Does not meet the criterion at all."
},
"1": {
"title": "1",
"type": "string",
"description": "Meets a small part of the criterion."
},
"2": {
"title": "2",
"type": "string",
"description": "Meets about half of the criterion."
},
"3": {
"title": "3",
"type": "string",
"description": "Meets most of the criterion, with minor gaps."
},
"4": {
"title": "4",
"type": "string",
"description": "Fully meets the criterion."
}
},
"required": [
"0",
"1",
"2",
"3",
"4"
],
"additionalProperties": false
},
"probabilities": {
"title": "Probabilities",
"type": "object",
"description": "The probability of each level, keyed by level index.",
"properties": {
"0": {
"title": "0",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
},
"1": {
"title": "1",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
},
"2": {
"title": "2",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
},
"3": {
"title": "3",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
},
"4": {
"title": "4",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
}
},
"required": [
"0",
"1",
"2",
"3",
"4"
],
"additionalProperties": false
},
"confidence": {
"title": "Confidence",
"type": "number",
"minimum": 0.0,
"maximum": 1.0,
"description": "How certain the decision model is, from 0 to 1."
}
},
"required": [
"score",
"legend",
"probabilities",
"confidence"
],
"additionalProperties": false
},
"concise": {
"title": "ScoreAnswer",
"type": "object",
"description": "How well does the answer meet this criterion: No preamble, no restating the question.",
"properties": {
"score": {
"title": "Score",
"type": "number",
"minimum": 0.0,
"maximum": 4.0,
"description": "The probability-weighted level index."
},
"legend": {
"title": "Legend",
"type": "object",
"description": "The description of each level, keyed by level index.",
"properties": {
"0": {
"title": "0",
"type": "string",
"description": "Does not meet the criterion at all."
},
"1": {
"title": "1",
"type": "string",
"description": "Meets a small part of the criterion."
},
"2": {
"title": "2",
"type": "string",
"description": "Meets about half of the criterion."
},
"3": {
"title": "3",
"type": "string",
"description": "Meets most of the criterion, with minor gaps."
},
"4": {
"title": "4",
"type": "string",
"description": "Fully meets the criterion."
}
},
"required": [
"0",
"1",
"2",
"3",
"4"
],
"additionalProperties": false
},
"probabilities": {
"title": "Probabilities",
"type": "object",
"description": "The probability of each level, keyed by level index.",
"properties": {
"0": {
"title": "0",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
},
"1": {
"title": "1",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
},
"2": {
"title": "2",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
},
"3": {
"title": "3",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
},
"4": {
"title": "4",
"type": "number",
"minimum": 0.0,
"maximum": 1.0
}
},
"required": [
"0",
"1",
"2",
"3",
"4"
],
"additionalProperties": false
},
"confidence": {
"title": "Confidence",
"type": "number",
"minimum": 0.0,
"maximum": 1.0,
"description": "How certain the decision model is, from 0 to 1."
}
},
"required": [
"score",
"legend",
"probabilities",
"confidence"
],
"additionalProperties": false
}
},
"required": [
"correct",
"concise"
],
"additionalProperties": false
}
]
[Synalinks]
Call ID: d055ef8b-80dd-42e8-8a5e-db8be6e86678
Parent call ID: cd3f33e6-13eb-4685-bd0f-aef3cda2726d
Module: Generator
Module Name: generator_rubrics_as_judge_program
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON:
[
{
"gold_answer": "The trip takes 3 hours and 35 minutes.",
"answer": "Sure! The trip takes 3 hours and 35 minutes."
}
]
[Synalinks]
Call ID: a20fc05c-077e-4af5-9a34-64e8bbe0a61f
Parent call ID: d055ef8b-80dd-42e8-8a5e-db8be6e86678
Module: DecisionModel
Module Name: decision_model
Module Description: A decision model API wrapper.
Data Model JSON:
[
{
"messages": [
{
"role": "system",
"content": "<instructions>\nYour task is to grade the given answer against each criterion, independently.\n</instructions>\n\n<rubrics>\n\n<rubric>\n<name>\ncorrect\n</name>\n<description>\nThe answer matches the reference answer.\n</description>\n<weight>\n2\n</weight>\n</rubric>\n\n<rubric>\n<name>\nconcise\n</name>\n<description>\nNo preamble, no restating the question.\n</description>\n</rubric>\n\n</rubrics>\n"
},
{
"role": "user",
"content": "{'gold_answer': 'The trip takes 3 hours and 35 minutes.', 'answer': 'Sure! The trip takes 3 hours and 35 minutes.'}"
}
]
}
]
[Synalinks]
Call ID: a20fc05c-077e-4af5-9a34-64e8bbe0a61f
Parent call ID: d055ef8b-80dd-42e8-8a5e-db8be6e86678
Module: DecisionModel
Module Name: decision_model
Module Description: A decision model API wrapper.
Data Model JSON:
[
{
"correct": {
"score": 3.25396779506259,
"legend": {
"0": "Does not meet the criterion at all.",
"1": "Meets a small part of the criterion.",
"2": "Meets about half of the criterion.",
"3": "Meets most of the criterion, with minor gaps.",
"4": "Fully meets the criterion."
},
"probabilities": {
"0": 0.03824271168868315,
"1": 0.06487773397271288,
"2": 0.09441598852153014,
"3": 0.2095961792214786,
"4": 0.5928673865955952
},
"confidence": 0.3783064958854916
},
"concise": {
"score": 1.5730102590118522,
"legend": {
"0": "Does not meet the criterion at all.",
"1": "Meets a small part of the criterion.",
"2": "Meets about half of the criterion.",
"3": "Meets most of the criterion, with minor gaps.",
"4": "Fully meets the criterion."
},
"probabilities": {
"0": 0.24749292017726782,
"1": 0.236316051579202,
"2": 0.26665406773588757,
"3": 0.19476177006969492,
"4": 0.05477519043794761
},
"confidence": 0.1369882976005602
}
}
]
[Synalinks]
Call ID: d055ef8b-80dd-42e8-8a5e-db8be6e86678
Parent call ID: cd3f33e6-13eb-4685-bd0f-aef3cda2726d
Module: Generator
Module Name: generator_rubrics_as_judge_program
Module Description: Use a `LanguageModel` (or a `DecisionModel`) to generate a data model from an
Data Model JSON:
[
{
"correct": {
"score": 3.25396779506259,
"legend": {
"0": "Does not meet the criterion at all.",
"1": "Meets a small part of the criterion.",
"2": "Meets about half of the criterion.",
"3": "Meets most of the criterion, with minor gaps.",
"4": "Fully meets the criterion."
},
"probabilities": {
"0": 0.03824271168868315,
"1": 0.06487773397271288,
"2": 0.09441598852153014,
"3": 0.2095961792214786,
"4": 0.5928673865955952
},
"confidence": 0.3783064958854916
},
"concise": {
"score": 1.5730102590118522,
"legend": {
"0": "Does not meet the criterion at all.",
"1": "Meets a small part of the criterion.",
"2": "Meets about half of the criterion.",
"3": "Meets most of the criterion, with minor gaps.",
"4": "Fully meets the criterion."
},
"probabilities": {
"0": 0.24749292017726782,
"1": 0.236316051579202,
"2": 0.26665406773588757,
"3": 0.19476177006969492,
"4": 0.05477519043794761
},
"confidence": 0.1369882976005602
}
}
]
[Synalinks]
Call ID: 0c8e3114-d800-47cd-a61d-b7d1cc4b976a
Parent call ID: None
Module: RubricsAsJudgeProgram
Module Name: rubrics_as_judge_program
Module Description: Score each criterion with a language (or decision) model, then combine
Data Model JSON:
[
{
"critique": "correct: Meets most of the criterion, with minor gaps. (score 0.81, confidence 0.38)\nconcise: Meets about half of the criterion. (score 0.39, confidence 0.14)",
"correct": 0.8134919487656475,
"concise": 0.39325256475296305,
"reward": 0.6734121540947527
}
]
============================================================
Triage: a Generator, a Decision and a MultiDecision
============================================================
{
"is_billing": true,
"urgency": "high",
"frustration": 4,
"choice": "billing",
"choices": [
"payment"
]
}
============================================================
Routing: a Branch decided by a decision model
============================================================
{
"answer": "The capital of France is Paris."
}
{
"thinking": "Let's break down the problem: The train leaves at 9:40 and arrives at 13:15. We first convert both times to a 24-hour clock format, as the time span may cross the 12-hour boundary. 9:40 and 13:15 are already in 24-hour format, so we don't need to convert them. Subtracting the departure time from the arrival time gives us the duration of the trip in hours. Finally, we convert the result into minutes by multiplying the total number of hours by 60 and adding the remaining minutes.",
"answer": "The duration of the trip is 3 hours and 35 minutes."
}
============================================================
Grading: SelfCritique and RubricsAsJudge with a decision model
============================================================
{
"answer": "The trip takes 3 hours and 35 minutes.",
"reward": 0.5679676832293364
}
{
"critique": "correct: Meets most of the criterion, with minor gaps. (score 0.81, confidence 0.38)\nconcise: Meets about half of the criterion. (score 0.39, confidence 0.14)",
"correct": 0.8134919487656475,
"concise": 0.39325256475296305,
"reward": 0.6734121540947527
}
Decision model calls: 7
Decision model cost: $0.000000