Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/md/doc/builder-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ The Builder Agent is focused on authoring and evolving semantic tables: defining
- File: [`docs/md/skills/claude-code/bsl-model-builder/SKILL.md`](../skills/claude-code/bsl-model-builder/SKILL.md)
- Use it when you want Claude Desktop to help write new semantic tables, add time dimensions, or compose models.
- The skill includes:
- Python DSL examples showing `SemanticTable(...)`, `.with_dimensions`, `.with_measures`, `.with_calculated_measures`, and `.join()` patterns.
- Python DSL examples showing `to_semantic_table(...)`, `.with_dimensions`, `.with_measures`, `join_one()`, `join_many()`, and `join_cross()` patterns.
- YAML equivalents so you can copy the same logic into declarative configs.
- Gotchas such as "measures must aggregate" and "join keys must be defined dimensions".
- Gotchas such as choosing cardinality from the left side, using source-aware equality predicates, and assigning unique model aliases.

**Workflow:** Load the skill in Claude Desktop, paste the schema or YAML snippet you are editing, and ask "Generate a semantic table for flights with avg delay and join to airports". Claude will respond with both Python and YAML patterns that mirror the documentation.

Expand Down
22 changes: 17 additions & 5 deletions docs/md/doc/compose.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,17 @@ Model composition in BSL is achieved through **joins**. When you join semantic t
Each join creates a new semantic model with the combined dimensions and measures from all joined tables. This allows you to build progressively richer models.
</note>

Choose cardinality from the perspective of the left model: use `join_one()`
when each left row can match at most one right row, and `join_many()` when one
left row can match multiple right rows. Every source model in a join tree must
also have a unique `name`, because that name becomes its field prefix and source
identity.

For source-aware aggregation, non-cross joins support direct field equality
predicates and conjunctions of direct field equalities. Inequality, `OR`, cast,
and transformed predicates are rejected when source pre-aggregation is needed;
aggregate each model first or restate the relationship as equality keys.

## Example: Two-Level Composition

Let's build a composed model step-by-step, showing available dimensions and measures at each level.
Expand Down Expand Up @@ -108,8 +119,8 @@ flights_st.dimensions, flights_st.measures
Join carriers to flights to add carrier information:

```level1_join
# Join carriers to flights
flights_with_carriers = flights_st.join_many(
# Each flight row matches at most one carrier row
flights_with_carriers = flights_st.join_one(
carriers_st,
lambda f, c: f.carrier_code == c.code
)
Expand All @@ -124,8 +135,8 @@ flights_with_carriers.dimensions, flights_with_carriers.measures
Add aircraft information to create a fully composed model:

```level2_join
# Join aircraft to the composed model
full_model = flights_with_carriers.join_many(
# Each flight row matches at most one aircraft row
full_model = flights_with_carriers.join_one(
aircraft_st,
lambda f, a: f.aircraft_id == a.id
)
Expand All @@ -152,9 +163,10 @@ result = (

## Key Takeaways

- **Composition via Joins**: Use `join_many()`, `join_one()`, or `join_cross()` to compose models
- **Composition via Joins**: Use `join_one()` for at-most-one right matches, `join_many()` for multiple right matches, and `join_cross()` for Cartesian products
- **Additive**: Each join adds dimensions and measures from the joined table
- **Table Prefixes**: Dimensions/measures are prefixed with table names (`flights.`, `carriers.`, `aircraft.`)
- **Unique Source Names**: Give every base model in the join tree a distinct `name`
- **No Limit**: Compose as many models as needed for your analysis
- **Incremental**: Build from simple to complex, one join at a time

Expand Down
45 changes: 26 additions & 19 deletions docs/md/doc/reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,17 +144,19 @@ flights_st = flights_st.with_measures(
### all()

```python
st.all()
t.all(measure_or_reduction)
```

Reference the entire dataset within measure definitions. Primarily used for percentage-of-total calculations.
Reference the entire dataset within a measure definition. Pass a declared
measure reference, measure name, or Ibis reduction; `all()` does not have a
zero-argument form. It is primarily used for percentage-of-total calculations.

**Example:**
```python
flights_st = to_semantic_table(data, "flights").with_measures(
flight_count=lambda t: t.count(),
pct_of_total=lambda t: (
t.count() / t.all().count() * 100
t.flight_count / t.all(t.flight_count) * 100
)
)
```
Expand All @@ -168,9 +170,8 @@ Methods for composing semantic tables through joins.
```python
join_many(
other: SemanticTable,
on: Callable,
how: str = "left",
name: str = None
on: Callable | str | Deferred | Sequence[str | Deferred],
how: str = "left"
) -> SemanticTable
```

Expand All @@ -179,32 +180,39 @@ One-to-many relationship join (LEFT JOIN). Use when the left table can match mul
| Parameter | Type | Description |
|-----------|------|-------------|
| `other` | `SemanticTable` | The semantic table to join with |
| `on` | `Callable` | Lambda function defining the join condition |
| `on` | `Callable \| str \| Deferred \| Sequence[str \| Deferred]` | Join predicate: a two-argument lambda, same-name column shorthand, or sequence of shorthands for a compound equijoin |
| `how` | `str` | Join type; only `'left'` is supported |
| `name` | `str` | Optional name for the joined table reference |

**Example:**
```python
flights_st = flights_st.join_many(
carriers_st,
on=lambda l, r: l.carrier == r.code,
name="carrier_info"
carriers_with_flights = carriers_st.join_many(
flights_st,
on=lambda carrier, flight: carrier.code == flight.carrier
)
```

Model names are the source aliases used in prefixed fields. Every base model in
a composed join tree must therefore have a unique `name`; assign explicit names
with `to_semantic_table(..., name="...")` before joining. When the same physical
table serves multiple roles, use both `.view()` and distinct model names.

Source-aware aggregation supports direct field equijoins and conjunctions of
direct field equijoins. Inequality, `OR`, cast, and transformed predicates are
rejected when source pre-aggregation is required because they cannot be safely
reconstructed from key columns alone.

### join_one()

```python
join_one(
other: SemanticTable,
on: Callable,
how: str = "left",
name: str = None
on: Callable | str | Deferred | Sequence[str | Deferred],
how: str = "left"
) -> SemanticTable
```

One-to-one relationship join (LEFT JOIN). Use when each left row can match at
most one row in the right table. Only `how="left"` is supported.
At-most-one-right-match relationship join (LEFT JOIN). Use when each left row
can match at most one row in the right table. Only `how="left"` is supported.

**Example:**
```python
Expand All @@ -218,8 +226,7 @@ flights_st = flights_st.join_one(

```python
join_cross(
other: SemanticTable,
name: str = None
other: SemanticTable
) -> SemanticTable
```

Expand Down
69 changes: 41 additions & 28 deletions docs/md/doc/semantic-table.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,9 @@ result = (
<bslquery code-block="measure_all_demo"></bslquery>

<note type="info">
`t.all()` is a method available on the table parameter `t` in measure definitions. It references the entire dataset regardless of grouping, making it perfect for calculating percentages, or comparing groups to the total.
`t.all(ref)` is available on the table parameter `t` in measure definitions. It
evaluates the supplied measure or reduction over the entire dataset regardless
of grouping, making it useful for percentages and comparisons with the total.
</note>

For more examples, see the [Percent of Total pattern](/advanced/percentage-total).
Expand Down Expand Up @@ -205,14 +207,14 @@ Semantic joins explicitly capture the **relationship type** between tables, rath

**SQL Joins:**
```python
# Specifies HOW to join (LEFT/INNER), but not the relationship
flights.join(carriers, condition, how="left")
# Specifies HOW to join, but not the analytical relationship
flights_tbl.left_join(carriers_tbl, flights_tbl.carrier == carriers_tbl.code)
```

**Semantic Joins:**
```python
# Specifies the relationship: one carrier has many flights
flights.join_many(carriers, lambda f, c: f.carrier == c.code)
# One carrier row can match many flight rows
carriers.join_many(flights_st, lambda c, f: c.code == f.carrier)
```

**What You Get:**
Expand All @@ -225,15 +227,31 @@ After joining, dimensions and measures are prefixed with table names (e.g., `fli
</note>

<note type="warning">
**Joining the same table multiple times?** If you need to join to the same source table via different foreign keys (e.g., pickup and dropoff locations), you must use `.view()` to create distinct table references:
**Give every source in a composed model a unique name.** BSL uses model names as
source aliases for dimensions, measures, and grain metadata, and rejects a join
tree containing duplicate names. If you join the same underlying table more than
once (for example, pickup and dropoff locations), create distinct table references
and assign explicit aliases:

```python
# Create distinct references when joining same table twice
pickup_locs = to_semantic_table(locs_tbl.view(), "pickup_locs")
dropoff_locs = to_semantic_table(locs_tbl.view(), "dropoff_locs")
```

Without `.view()`, you'll encounter an `IbisInputError: Ambiguous field reference` error.
The distinct names prevent ambiguous semantic prefixes; `.view()` prevents Ibis
from treating both roles as the same relation.
</note>

<note type="warning">
**Source-aware aggregation requires equality-key joins.** When BSL aggregates
measures at their source grain before joining, each non-cross join predicate must
be a direct field equality or a conjunction of direct field equalities. String,
Deferred, and compound equality-key shorthands are supported. Predicates using
inequality, `OR`, casts, or transformed expressions are rejected because reducing
them to join-key bridges could change the matched row set. Aggregate the models
first or restate the relationship with plain equality keys; use `join_cross()` for
a Cartesian product.
</note>

Let's get some additional data:
Expand Down Expand Up @@ -273,54 +291,49 @@ carriers = (
Use `join_many()` when one row in the left table can match multiple rows in the right table (LEFT JOIN).

```join_demo
# Join carriers to flights - one carrier has many flights
flights_with_carriers = flights_st.join_many(
carriers,
lambda f, c: f.carrier == c.code
# One carrier row can match many flight rows
carriers_with_flights = carriers.join_many(
flights_st,
lambda c, f: c.code == f.carrier
)

# Inspect available dimensions and measures
flights_with_carriers.dimensions
carriers_with_flights.dimensions
```
<regularoutput code-block="join_demo"></regularoutput>

After joining, all dimensions and measures from both tables are available. Each is prefixed with its table name to avoid conflicts:


### join_one() - One-to-One Relationships
### join_one() - At-Most-One Right Match

Use `join_one()` when rows have a unique matching relationship. Like all
non-cross semantic joins, it uses a LEFT JOIN so unmatched left rows remain
visible to measures.
Use `join_one()` when each row on the left can match at most one row on the
right. Like all non-cross semantic joins, it uses a LEFT JOIN so unmatched left
rows remain visible to measures.

```python
# Many flights → one carrier (each flight has exactly one carrier)
# Many flights → one carrier (each flight matches at most one carrier)
flights_with_carrier = flights_st.join_one(
carriers,
lambda f, c: f.carrier == c.code
)
```

<note type="warning">
**Important Limitation:** Currently, `left_on` and `right_on` must be **COLUMN names**, not dimension names.

If you have a dimension that maps to a different column name, you must use the underlying column name in the join.
**Join predicates resolve physical columns.** A string or Deferred shorthand
names the same underlying column on both sides. If the columns have different
names, use a two-argument lambda instead.

**Example:**
```python
# If users table has column 'id' but dimension 'customer_id':
users = to_semantic_table(users_tbl).with_dimensions(
users = to_semantic_table(users_tbl, "users").with_dimensions(
customer_id=lambda t: t.id # Dimension renamed
)

# ❌ This will fail with a helpful error:
orders.join_one(users, left_on="customer_id", right_on="customer_id")

# ✓ Use the actual column name:
orders.join_one(users, left_on="customer_id", right_on="id")
# Compare the underlying columns explicitly:
orders.join_one(users, on=lambda order, user: order.customer_id == user.id)
```

This is a known limitation tracked in [issue #43](https://github.com/boringdata/boring-semantic-layer/issues/43). If you attempt to use a dimension name that doesn't match a column name, you'll get a helpful error message guiding you to use the correct column name.
</note>

### join_cross() - Cross Join
Expand Down
36 changes: 20 additions & 16 deletions docs/md/prompts/build/system.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,9 +81,10 @@ flights_st = flights_st.with_measures(
)
```

### Percent of Total with all()
### Percent of Total with all(ref)

Use `t.all()` to reference the entire dataset:
Pass a declared measure or reduction to `t.all(...)` to reference its value over
the entire dataset. There is no zero-argument `t.all()` form:

```python
flights_st = flights_st.with_measures(
Expand All @@ -98,16 +99,16 @@ flights_st = flights_st.with_measures(

```python
# One carrier has many flights
flights_with_carriers = flights_st.join_many(
carriers_st,
lambda f, c: f.carrier == c.code
carriers_with_flights = carriers_st.join_many(
flights_st,
lambda c, f: c.code == f.carrier
)
```

### join_one() - One-to-One (LEFT JOIN)
### join_one() - At-Most-One Right Match (LEFT JOIN)

```python
# Each flight has exactly one carrier
# Each flight matches at most one carrier
flights_with_carrier = flights_st.join_one(
carriers_st,
lambda f, c: f.carrier == c.code
Expand All @@ -120,19 +121,21 @@ flights_with_carrier = flights_st.join_one(
all_combinations = flights_st.join_cross(carriers_st)
```

### Custom Joins
### Join Predicates and Source-Aware Aggregation

```python
flights_st.join(
carriers_st,
lambda f, c: f.carrier == c.code,
how="left" # "inner", "left", "right", "outer", "cross"
)
```
`join_one()` and `join_many()` are left joins. Their `on=` argument can be a
column-name string, Deferred key, sequence of equality keys, or a lambda whose
predicate is a direct field equality (or conjunction of direct field
equalities). Source-aware aggregation rejects inequality, `OR`, cast, and
transformed join predicates because pre-aggregating those predicates from key
columns could change the matched rows. Aggregate each model first or restate the
relationship as equality keys. Use `join_cross()` for Cartesian products.

**After joins**: Fields are prefixed with table names (e.g., `flights.origin`, `carriers.name`)

**Multiple joins to same table**: Use `.view()` to create distinct references:
**Unique aliases are required**: Every base model in a composed join tree must
have a distinct `name`. When the same physical table plays multiple roles, use
both `.view()` and explicit model aliases:
```python
pickup_locs = to_semantic_table(locs_tbl.view(), "pickup_locs")
dropoff_locs = to_semantic_table(locs_tbl.view(), "dropoff_locs")
Expand Down Expand Up @@ -193,6 +196,7 @@ flights_sm = models["flights"]
3. **Define composed measures** to avoid repetition
4. **Use YAML** for production models (version control, collaboration)
5. **Use profiles** for database connections (see Profile docs)
6. **Choose join cardinality from the left side** and give every joined source a unique name

## Common Patterns

Expand Down
Loading
Loading