Tables
Tables are a unique feature of Tabsdata. Unlike other data integration tools that discard intermediary data after execution, Tabsdata tables are persistent, versioned tablestores that store all data produced by Tabsdata functions.
Tables allow users to monitor and debug workflows, time travel to inspect data from previous executions, track schema changes over time, and more.
Beyond observability, Tables enable richer context for LLMs, enable more flexibility for function operations, and allow Tabsdata to automatically orchestrate workflows.
Table Versions
All tables come with a full version lineage of every commit made onto it since its inception. Every successful function run that returns data commits a new version onto the table, with each version independently carrying its own schema and data.
These versions are accessible to users or other functions, which can query any table version to use for input. Each table version is also bundled with metadata regarding the execution plan that generated it and the specific function code that was used to generate it.
Schema
Table schema defines the structure of a table by listing its columns and their data types.
| Object | Type |
|---|---|
| first_name | str |
| last_name | str |
| age | i64 |
| nationality | str |
| city | str |
Because each table version independently handles its own schema, Tabsdata tables can respond to unexpected schema changes without failing due to schema mismatch.
Tables in Action
Tables on their own only store data. It is their relationship with other resources that enable a Tabsdata workflow
Publisher functions ingest data from external systems and write them into Tabsdata Tables.
Transformer functions read Tabsdata Tables and writes new Tabsdata Tables.
Subscriber functions reads Tabsdata Tables and writes out to an external system.
Referring to a Table
A Function names its inputs as [collection/]table[@versions]. Omit the collection and
the name resolves inside the Function's own Collection.
The Version part accepts HEAD for the latest, HEAD^ and HEAD~N for earlier ones,
INITIAL for the first, a fixed Version id, comma-separated lists, and a..b ranges.
Omit it and the Function reads HEAD.
Cross-Collection references work within one Project. A reference to a Table in another
Project reports Dependency table '<x>' does not exist, because row-level security hides
it rather than reporting it as forbidden.
Private Tables
A Table whose name begins with _ is private. It can only be read from inside its own
Collection, which is how a Collection keeps an intermediate result out of reach of the
rest of the Project while still passing it between its own Functions.
Tables and Triggers
A commit to a Table is a signal, not just a write. Transformers and Subscribers are triggered by a commit to any of their input Tables, so committing a Table is what advances the pipeline downstream of it.
A commit does not mean the data changed. A Publisher writing three Tables commits all three on every run, whether or not the rows in a given Table differ from last time, and each of those commits triggers whatever watches that Table. See Working with Triggers for how to narrow which Tables trigger a Function.