Version: 2.2.0

Db2CdcStreamSrc

class

class Db2CdcStreamSrc(
    tables: list[Db2TableNameSpec],
    start_from: Db2CdcStartFromSpec,
    columns_format: CdcColumnsFormatSpec = 'columns',
    flatten_columns: bool = True,
    src_cfg: CdcSrcCfgSpec | None = None,
)

Bases: StreamSrc

Categories: source

Db2 CDC stream source for @stream_publisher.

A streaming change-data-capture source: it emits one tableframe-batch per capture-table flush across the tracked CD (change-data) tables. Db2 SQL Replication captures changes into shadow CD tables that the ASN capture program (asncap) populates in the background. tables lists the CD tables (not the source tables); the ingester resolves the source via ASN.IBMSNAP_REGISTER. The decorated @stream_publisher function receives the flushed batch as a list of frames and must NOT declare a ctx parameter.

Parameters

parameter

tableslist[Db2TableNameSpec] (list[str])

Qualified cd_schema.cd_table CD table names.

parameter

start_fromDb2CdcStartFromSpec (Literal['full'] | Literal['head', 'tail'] | CommitSeqPosition | TableCommitSeqPosition | TimestampPosition)

Where to begin -- "full", "head" (replay each tracked CD table from the origin), "tail" (each table's current watermark), a CommitSeqPosition, a TableCommitSeqPosition or a TimestampPosition. Required.

"full" first emits a full snapshot of every tracked source table (rows marked as inserts, preceded by a LOAD control row per table) and then captures changes from the exact position of that snapshot.

To resynchronise a destination later, keep start_from="full" and reset the function's stage (tdk fn update --reset-stage): a fresh stage holds no stream state, so the stream starts again with the snapshot and emits the LOAD rows first. A stage reset with any other start_from restarts there and sends no LOAD row.

Beyond the privileges plain CDC needs, the connecting user must be able to SELECT every tracked source table (plain CDC only reads the CD tables), since the snapshot reads the source tables themselves, and needs EXECUTE on SYSPROC.MON_GET_TRANSACTION_LOG or SYSMON authority to read the log position the snapshot is pinned to. No table is locked.

parameter

columns_formatCdcColumnsFormatSpec (Literal['columns', 'struct', 'map'])

Layout for the new (and old) columns payload on each change row -- "columns" (default), "struct" or "map".

parameter

flatten_columnsbool

When True (default), the new columns are inlined as top-level columns.

parameter

src_cfgCdcSrcCfgSpec | None (Mapping[Literal['tabsdata.cdc.poll.max_secs', 'db2.cdc.logging_level', 'tabsdata.cdc.buffer.max_rows', 'tabsdata.cdc.buffer.max_bytes', 'tabsdata.cdc.buffer.max_secs', 'tabsdata.cdc.poll.idle_secs', 'tabsdata.cdc.max_large_text_length', 'tabsdata.cdc.fetch_size', 'tabsdata.cdc.chunk_size', 'tabsdata.cdc.initial_load.max_window_changes'], Any] | None)

Optional config mapping over the allowed CDC keys only -- buffer thresholds (the batch limits are the @stream_publisher batch settings), polling cadence, server-side fetch size. A byte threshold is measured on every change and slows ingestion down; bounding by rows is the cheap option and the preferred one.

#

Parameters​

#

#

#

#

#

Parameters