Local File System

The Local File System connector lets Tabsdata read files from and write files to the server's own filesystem.

The LocalFileSrc connector can be used by a publisher function to read data from local files into a Tabsdata table.

The LocalFileDest connector can be used by a subscriber function to write data from a Tabsdata table into local files.

Supported formats

Format	Publish	Subscribe
AUTO	Yes	Yes
CSV	Yes	Yes
JSON	Yes	Yes
AVRO	Yes	Yes
PARQUET	Yes	Yes
LOG	Yes	No

Publishers and Subscribers

The LocalFileSrc connector can be used by a publisher function to read data from local files into a Tabsdata table. See the full publisher walkthrough

Connection

Local File System publishers use LocalFileSrcConn to define the root filesystem path used by the publisher. Unlike the cloud storage connectors, there is no bucket and no credentials: the connector reads directly off the server's own disk.

conn-local.yaml

kind: connectionDef
apiVersion: '1.0'
type: tabsdatak.conn.localfile:LocalFileSrcConn
spec:
  base_path: '/data/hr'

• required
• optional

Connection Config Parameters

base_path

The absolute filesystem path that acts as the root for all paths the publisher reads. Unlike the cloud storage connectors, base_path is required here: there is no bucket to fall back on, so the server needs a starting point on disk.

conn_cfg

Optional connector configuration overrides.

Publisher

Configure LocalFileSrc as the source of a publisher function to select the files that Tabsdata reads from disk.

publish_orders.py

from tabsdatak.api import publisher
from tabsdatak.conn.localfile import LocalFileSrc, SrcFileFormat

@publisher(
  source=LocalFileSrc(
        paths=[
            "incoming/orders.parquet",
      ],
    ),
  output_tables=["orders"],
)
def publish_orders(orders):
  return orders

• required
• optional
• optional
• optional
• optional

Function Config Parameters

paths

Defines the files that the publisher reads from disk.

Each path is relative to the base_path configured in LocalFileSrcConn. Source paths:

• cannot begin or end with /
• cannot contain empty path segments
• can contain a * glob in the final path segment

For example:

LocalFileSrc(
    paths=[
        "orders/2026/*.parquet",
        "customers/customers.csv",
    ]
)

Each element in paths represents one source slot and maps positionally to an argument in the publisher function.

When a path uses a glob to match multiple files, the matching files are provided through the corresponding publisher input.

format

Sets the format used to read the source files.

Supported formats are AUTO, CSV, JSON, AVRO, PARQUET, and LOG. The default is AUTO, which determines the format from the file extension.

format_cfg

Sets format-specific reader options. Configuration is keyed by SrcFileFormat.

initial_last_modified

Sets the initial cutoff for incremental ingestion.

On the first run, Tabsdata imports files modified at or after the specified timestamp. The cutoff then advances so subsequent runs only import newer files.

src_cfg

Sets additional source configuration.

The following options are supported:

Option	Description
tabsdata.file.chunk_size	Controls the size of chunks used when reading files.
tabsdata.src_metadata.drop	When True, excludes the @td.file.path metadata column.

By default, rows read from disk include a @td.file.path metadata column containing the source file's path.