StarRocks

The StarRocks connector lets Tabsdata Subscribers write data from Tabsdata Tables into StarRocks.

The connector stages the data as Parquet files in cloud storage, then loads it into StarRocks through its native FILES() table function. Destination tables in StarRocks must already exist as the connector does not create them automatically.

See the full subscriber walkthrough

Connection

StarRocksDestConn defines the StarRocks frontend address, database credentials, and cloud staging location. Register the Connection on the Collection where the Subscriber runs.

conn-starrocks.yaml

kind: connectionDef
apiVersion: '1.0'
type: tabsdatak.conn.starrocks:StarRocksDestConn
spec:
  host: 'starrocks-fe.internal'
  credentials:
  kind: userPasswordCredentials
  apiVersion: '1.0'
  type: tabsdatak.conn.common.types:UserPassword
  spec:
    user: 'secret:tabsdata_writer'
    password: 'secret:example-password'
  staging:
  kind: starRocksS3Location
  apiVersion: '1.0'
  type: tabsdatak.conn.starrocks:S3Location
  spec:
    bucket: 'acme-staging'
    region: 'us-east-1'
    credentials:
      kind: awsAccessSecretKeyCredentials
      apiVersion: '1.0'
      type: tabsdatak.conn.common.types:AwsAccessSecretKey
      spec:
        access_key_id: 'secret:example-access-key'
        secret_access_key: 'secret:example-secret-key'
    base_path: '/starrocks'

• required
• optional
• optional
• required
• required
• optional
• optional

Connection Config Parameters

host, port, and database

host identifies the StarRocks frontend. port is the MySQL-protocol port, from 1 to 65535, and defaults to "9030". database defaults to "default" and supplies the database for unqualified destination table names. These fields accept strings or secret references.

credentials

A UserPassword containing the StarRocks username and password. Cloud storage credentials are configured separately under staging.

staging

Use one of the location classes from tabsdatak.conn.starrocks:

Location	Required fields	Credentials
S3Location	bucket, region, credentials	AwsAccessSecretKey
AzBlobLocation	container, credentials	AzureAccountKey
GCSLocation	bucket, credentials, hmac_credentials	GCPJson for uploads and GCPHmac for StarRocks reads

Tabsdata must be able to upload the staged files, and StarRocks must be able to read them. For GCS, both credential types are required: uploads use the native GCS API, while StarRocks reads through the S3-compatible endpoint.

Each location accepts base_path, which defaults to /, and an optional cfg dictionary for storage-client configuration. A non-root base_path starts with /, has no trailing /, and contains no empty path segments.

entity_whitelist

Optional regex patterns narrowing which tables the connector considers. Unset by default, which filters nothing.

conn_cfg

Optional connector configuration overrides.

Subscriber

Configure StarRocksDest as the destination of a Subscriber Function. Each returned frame maps to one entry in tables, in order.

write_orders.py

from tabsdatak.api import subscriber
from tabsdatak.conn.starrocks import StarRocksDest

@subscriber(
  destination=StarRocksDest(
        tables=[
            "orders",
      ],
    ),
  input_tables=["sales/orders"],
)
def write_orders(orders):
  return orders

• required
• optional
• optional
• optional

Function Config Parameters

tables

Names of existing StarRocks tables, one per returned frame. Names can be table or database.table; an unqualified name uses the Connection's database. Backtick-quote individual name parts containing special characters.

Create the destination tables before running the Subscriber. StarRocks table creation requires a physical layout, including a key definition and distribution strategy, that the connector cannot infer from a frame.

if_table_exists

"append" (the default) inserts rows into the existing table. "replace" truncates the existing table and then inserts the returned rows. Neither mode creates a missing table.

schema_evolution

"strict" (the default) rejects schema changes. "iceberg" applies Iceberg-style schema changes to the existing destination table; it does not change the destination into an Iceberg table.

dest_cfg

Optional destination configuration. The supported key is starrocks.logging_level.