Tables
Tables are a unique feature of Tabsdata. Unlike other data integration tools that discard intermediary data after execution, Tabsdata Tables are persistent, versioned Tablestores that store all data produced by Tabsdata Functions.
Tables allow users to monitor and debug workflows, time travel to inspect data from previous executions, track schema changes over time, and more.
Beyond observability, Tables enable richer context for LLMs, enable more flexibility for function operations, and allow Tabsdata to automatically orchestrate workflows
Table Versions
A Table is not a single set of rows. Every successful Function run that returns data commits a new Version, and each Version carries its own schema and its own data. Nothing is overwritten.
A Function reading a Table takes the latest Version unless it names an older one. That is what makes a run repeatable: the same Version reference returns the same rows however much the Table has changed since.
A run that returns None for one of its output Tables commits nothing for that Table.
The Table keeps the Version it already had, and anything watching it stays put.
input/persons| Column | Type | Description |
|---|---|---|
first_name | Utf8 | Given name |
last_name | Utf8 | Family name |
age | Int64 | Age in years |
nationality | Utf8 | Nationality |
city | Utf8 | City of residence |
Because each Version keeps its own schema, a Publisher whose source gains a column produces a Version with that column while the Versions before it keep the shape they were written with.
Where Tables Come From
Every Table belongs to exactly one Collection, and the Function that writes it declares it as an output. Which kind of Function that is decides where the data came from.
A Publisher reads an external system and writes Tabsdata Tables. It declares no input Tables at all, so a Publisher's outputs are where data enters the server.
A Transformer reads Tabsdata Tables and writes new Tabsdata Tables. Both sides are inside the server, which is what lets a Transformer's output become another Transformer's input.
A Subscriber reads Tabsdata Tables and writes out to an external system. It declares no output Tables, so a Subscriber is where data leaves.
Referring to a Table
A Function names its inputs as [collection/]table[@versions]. Omit the collection and
the name resolves inside the Function's own Collection.
The Version part accepts HEAD for the latest, HEAD^ and HEAD~N for earlier ones,
INITIAL for the first, a fixed Version id, comma-separated lists, and a..b ranges.
Omit it and the Function reads HEAD.
Cross-Collection references work within one Project. A reference to a Table in another
Project reports Dependency table '<x>' does not exist, because row-level security hides
it rather than reporting it as forbidden.
Private Tables
A Table whose name begins with _ is private. It can only be read from inside its own
Collection, which is how a Collection keeps an intermediate result out of reach of the
rest of the Project while still passing it between its own Functions.
Tables and Triggers
A commit to a Table is a signal, not just a write. Transformers and Subscribers are triggered by a commit to any of their input Tables, so committing a Table is what advances the pipeline downstream of it.
A commit does not mean the data changed. A Publisher writing three Tables commits all three on every run, whether or not the rows in a given Table differ from last time, and each of those commits triggers whatever watches that Table. See Working with Triggers for how to narrow which Tables trigger a Function.
Naming
A Table name starts with a letter, then letters, digits, and underscores, up to 100 characters. A leading underscore marks the Table private. Names are unique within their Collection.