Postgres

The Postgres connector lets Tabsdata run queries against Postgres and write tables into Postgres.

The PostgresSrc connector can be used by a publisher function to read data from Postgres into a Tabsdata table.

The PostgresDest connector can be used by a subscriber function to write data from a Tabsdata table into Postgres.

Publishers and Subscribers

The PostgresSrc connector can be used by a publisher function to read data from Postgres into a Tabsdata table. See the full publisher walkthrough

Connection

Postgres publishers use PostgresSrcConn to define the database connection and credentials used by the publisher.

conn-postgres.yaml

kind: connectionDef
apiVersion: '1.0'
type: tabsdatak.conn.postgres:PostgresSrcConn
spec:
  uri: 'postgres://db.internal:5432/world'
  credentials:
  kind: userPasswordCredentials
  apiVersion: '1.0'
  type: tabsdatak.conn.common.types:UserPassword
  spec:
    user: 'secret:tabsdata_reader'
    password: 'secret:example-password'

• required
• required
• optional
• optional

Connection Config Parameters

uri

The SQLAlchemy-style connection URI: postgres://host:port/database. Written as a plain value here, but accepts secret: too.

credentials

The database user and password, spliced into the URI at connect time. UserPassword requires a user and password.

entity_whitelist

Limits which source tables are discovered (registered in the collection's table catalog) when the source collection is created: a YAML list of regular expressions, not a comma-separated string. A table is kept if it matches any pattern. Omit it entirely to discover every table in the database.

conn_cfg

Driver-side options forwarded to the connection as SQLAlchemy connect_args.

Publisher

Configure PostgresSrc as the source of a publisher function to select the queries that Tabsdata runs against Postgres.

publish_orders.py

from tabsdatak.api import publisher
from tabsdatak.conn.postgres import PostgresSrc

@publisher(
  source=PostgresSrc(
        queries=[
            "SELECT * FROM orders",
      ],
    ),
  output_tables=["orders"],
)
def publish_orders(orders):
  return orders

• required
• optional
• optional

Function Config Parameters

queries

Defines the SQL queries the publisher runs against Postgres.

Each element in queries represents one source slot and maps positionally to an argument in the publisher function.

For example:

PostgresSrc(
    queries=[
        "SELECT * FROM country",
        "SELECT * FROM city",
    ],
)

initial_values

Values substituted into {placeholder}-style tokens inside queries, so one publisher can read the same tables out of a different schema or catalog without the SQL changing.

src_cfg

Sets additional source configuration.