Program structure¶

A Synalog program is a project: a folder with a layer.toml, holding one predicate per file in three folders. This is the layout of semantic layers, and the one this documentation follows.

shop/
├── tables/           # the data: one file per database table
│   └── orders.l
├── concepts/         # what the data is about: entities, relationships, clean views
│   ├── customer.l
│   └── order_status.l
├── rules/            # what you want to know: counts, rates, rankings
│   ├── customer_revenue.l
│   ├── orders_by_status.l
│   └── top_customers.l
└── layer.toml        # the project's name, description and database

A predicate's kind is the folder it is in:

A file, a predicate¶

Each file is named after the predicate it defines, in snake_case: <folder>/<name>.l (CustomerRevenue is rules/customer_revenue.l; the predicate itself stays PascalCase). It holds, in this order:

1. its front matter: name, the predicate, and description, what its rows are;
2. one import per predicate it builds on, by folder, file and predicate: import tables.orders.Orders;;
3. the predicate's rules, after its directives. Every predicate is ordered with @OrderBy: results are paginated, and without a stable order a page differs between calls.

rules/top_customers.l builds on rules/customer_revenue.l, which builds on a concept and a table:

---
name: CustomerRevenue
description: Total amount ordered by each customer, all statuses included.
keywords: [revenue, spend, sales]
---
import concepts.customer.Customer;
import tables.orders.Orders;

@OrderBy(CustomerRevenue, "customer_id");
CustomerRevenue(customer_id:, total? += amount) distinct :-
  Customer(customer_id:), Orders(customer_id:, amount:);

---
name: TopCustomers
description: The two customers with the highest revenue.
keywords: [best customers, ranking]
---
import rules.customer_revenue.CustomerRevenue;

@OrderBy(TopCustomers, "total", "DESC");
@Limit(TopCustomers, 2);
TopCustomers(customer_id:, total:) :- CustomerRevenue(customer_id:, total:);

A file may also define the intermediate rules its predicate is built from (a recursive relation, a staging join): they are private to the file. A step worth reusing belongs in its own file, with its own name and description.

Tables¶

A table file maps one database table, referenced by its database name (lowercase, as it exists in the database), to a PascalCase predicate listing the columns the project may reference; everything else builds on the predicate:

---
name: Orders
description: One row per order, with its customer, amount and status.
---
@OrderBy(Orders, "order_id");
Orders(order_id:, customer_id:, amount:, status:) :-
  orders(order_id:, customer_id:, amount:, status:);

Table files are generated from the database's schema: synalog introspect writes one per table, ordered by its first column, with a description made from its name until someone writes a better one, and keeps the front matter written by hand when it runs again.

Concepts¶

Concepts extract the entities and relationships hidden in tables:

• Entity concepts are named after the entity: Customer, Product.
• Relationship concepts are named after the relationship: Purchased, WorksIn.

---
name: Customer
description: Every customer who placed at least one order.
keywords: [customer, buyer, client]
---
import tables.orders.Orders;

@OrderBy(Customer, "customer_id");
Customer(customer_id:) distinct :- Orders(customer_id:);

See Knowledge graphs for the full modeling conventions.

Rules¶

Rules derive what you want to know from concepts and other rules. They carry no suffix: CustomerRevenue, TopCustomers.

Running a project¶

Imports resolve from the project's folder, so every file runs on its own, from anywhere in the project:

synalog rules/top_customers.l run TopCustomers

The project's layer.toml says which database it runs on (see Projects). From Python, give the project's folder as the import root:

import synalog

source = open("shop/rules/top_customers.l").read()
errors, warnings = synalog.check(source, import_root=["shop"])
assert errors == []
sql = synalog.compile(source, "TopCustomers", import_root=["shop"])

-- Loaded data/orders.csv into DuckDB table orders (5 rows)

$ synalog tables/orders.l run Orders
| order_id | customer_id | amount | status    |
|----------|-------------|--------|-----------|
| 1        | 100         | 250    | shipped   |
| 2        | 100         | 1200   | pending   |
| 3        | 200         | 80     | shipped   |
| 4        | 300         | 430    | cancelled |
| 5        | 200         | 95     | shipped   |
(5 rows)

$ synalog concepts/customer.l run Customer
| customer_id |
|-------------|
| 100         |
| 200         |
| 300         |
(3 rows)

$ synalog concepts/order_status.l run OrderStatus
| status    |
|-----------|
| cancelled |
| pending   |
| shipped   |
(3 rows)

$ synalog rules/customer_revenue.l run CustomerRevenue
| customer_id | total |
|-------------|-------|
| 100         | 1450  |
| 200         | 175   |
| 300         | 430   |
(3 rows)

$ synalog rules/orders_by_status.l run OrdersByStatus
| status    | count |
|-----------|-------|
| cancelled | 1     |
| pending   | 1     |
| shipped   | 3     |
(3 rows)

$ synalog rules/top_customers.l run TopCustomers
| customer_id | total |
|-------------|-------|
| 100         | 1450  |
| 300         | 430   |
(2 rows)

To try the language without a database, a single file can hold its own data as inline facts, as the examples of this documentation do:

Orders(order_id: 1, customer_id: 100, amount: 250, status: "shipped");
Orders(order_id: 2, customer_id: 100, amount: 1200, status: "pending");

Comments¶

# starts a comment, to the end of the line. What a predicate holds is said in its file's front matter, as its description.

Front matter¶

Every file opens with YAML front matter, as in Markdown: a first line ---, the YAML, and a closing --- line. It says what the file is, and carries metadata for the tools around your project (keywords, ownership, ...). Synalog checks that it is well-formed YAML — a syntax error is reported on its line — and reads two keys, both required: name is the predicate the file is about — it must be one the file defines and orders with @OrderBy — and description says what its rows are, in the words someone would search for (see verification). Other keys are left to the host (keywords, locked, ...), and the file parses as if the block were not there, so error line numbers stay those of the file. It must be the very first thing in the file; an opening --- that is never closed is an error.

---
name: Customer
description: Every customer who placed at least one order.
keywords: [customer, buyer, client]
---
import tables.orders.Orders;

@OrderBy(Customer, "customer_id");
Customer(customer_id:) distinct :- Orders(customer_id:);

To read it from Python, use front_matter.

Reuse and compose predicates¶

The power of logic programming is composition: define a predicate once, in its own file, and build on it everywhere. Never recompute the same expression in several rules: CustomerRevenue is defined once, and TopCustomers imports it (both files are above).

Imports¶

import brings one predicate of another file into scope: import <folder>.<name>.<Name>; reads the project's <folder>/<name>.l and names its predicate <Name>. One import statement names exactly one predicate; every import must be used.

import concepts.customer.Customer;
import tables.orders.Orders;

Add as to rename: import rules.customer_revenue.CustomerRevenue as Revenue;. Directives attached to the imported predicate (its @OrderBy) travel with it, and the imported file's own imports are resolved recursively (circular imports are an error).

The imports are the project's dependency graph: what each predicate builds on, and so what a change affects. A project is self-contained: everything its files import is inside it, its tables included.

Imports resolve from the project's folder (the one holding layer.toml) in the CLI, whichever file runs and wherever from; in the Python API, pass that folder as import_root.

Extract categorical values first¶

For columns like status, type, tier, category or country, extract the distinct values as a concept before writing rules over them. This gives consistency, reuse and discoverability:

---
name: OrderStatus
description: The statuses an order can have (shipped, pending, cancelled).
---
import tables.orders.Orders;

@OrderBy(OrderStatus, "status");
OrderStatus(status:) distinct :- Orders(status:);

---
name: OrdersByStatus
description: Number of orders per status.
---
import concepts.order_status.OrderStatus;
import tables.orders.Orders;

@OrderBy(OrdersByStatus, "status");
OrdersByStatus(status:, count? += 1) distinct :- OrderStatus(status:), Orders(status:);