Working with Subscribers

A subscriber is a Tabsdata function that reads tables in the Tabsdata server and writes them to external destinations such as file systems or databases. For example, you can create a subscriber to write the Tabsdata tables that it is subscribed to and write them to an Amazon S3 instance.

Here is an example subscriber named write_sales. It reads three tables from the Tabsdata server, and writes them to the local file system with the same file names. This subscriber executes automatically as soon as a new commit occurs on any of its input tables.

import tabsdata as td


@td.subscriber(
    tables=[
        "us-sales-by-sales-rep",
        "eu-sales-by-sales-rep",
        "emea-sales-by-sales-rep",
    ],
    destination=td.LocalFileDestination(
        [
            "/users/username/opt/sales-staging/output/us/us-sales-by-sales-rep.csv",
            "/users/username/opt/sales-staging/output/eu/eu-sales-by-sales-rep.csv",
            "/users/username/opt/sales-staging/output/emea/emea-sales-by-sales-rep.csv",
        ]
    ),
)
def write_sales(tf1: td.TableFrame, tf2: td.TableFrame, tf3: td.TableFrame):
    return tf1, tf2, tf3

When you define a subscriber, you specify the name of the subscriber, the Tabsdata tables to write, and destination properties to write to. By default a subscriber is triggered by a new commit to any of its input tables.

To create and run a Tabsdata subscriber, complete the following tasks:

Define a subscriber - Define a subscriber to configure the parameters that govern the writing of data from the Tabsdata server. The syntax for defining a subscriber differs depending on the system that the subscriber writes to. For details, see Writing to File Storage or Writing to Database.

Register the subscriber - Register the subscriber with a Tabsdata collection. A subscriber can read from tables outside the collection it is registered with. For information about registering functions, see here.

Execute the subscriber - Execute the subscriber by initiating its trigger. Once executed, the subscriber reads the Tabsdata tables, and writes the output data to the external systems. By default, subscribers are triggered when any of the input tables receive a new commit. However, you can define any Tabsdata table to act as a trigger for a subscriber. For more information, see here.