Version: 2.0.0

Transform Strings

Change case

make a new table that takes the persons table and uppercases the name column

Needs a transformer. Getting its guide.

Called tabsdata 2 times (ctrl+o to expand)

Generating — str.to_uppercase() on the column. Aliasing it back to name replaces the column instead of adding one.

Called tabsdata 3 times (ctrl+o to expand)

The transformer, registered in subscriptions:

@transformer(
  input_tables=["persons"],
  output_tables=["persons_upper"],
)
def upper_name(persons: TableFrameSpec) -> TableFrameSpec:
  expr = col("name").str.to_uppercase()
  return persons.with_columns(expr)

Execution committed, 0 failures.

subscriptions/persons_upper is live — name uppercased in place:

┌──────────────┬─────┐
│     name     │ age │
├──────────────┼─────┤
│ ADA LOVELACE │ 36  │
├──────────────┼─────┤
│ GRACE HOPPER │ 45  │
└──────────────┴─────┘

Transformed tables

persons

name	age
ada Lovelace	36
Grace HOPPER	45

2 rows

persons_upper

name	age
ADA LOVELACE	36
GRACE HOPPER	45

2 rows

Test a string against a pattern

make a new table that takes the contacts table and flags the rows whose email column ends in @example.com

Needs a transformer. Getting its guide.

Called tabsdata 2 times (ctrl+o to expand)

Generating — str.ends_with() returns a Boolean per row, so this adds a flag column rather than filtering. Use it inside filter() if you want the rows dropped.

Called tabsdata 3 times (ctrl+o to expand)

The transformer, registered in subscriptions:

@transformer(
  input_tables=["contacts"],
  output_tables=["contacts_flagged"],
)
def flag_internal(contacts: TableFrameSpec) -> TableFrameSpec:
  expr = col("email").str.ends_with("@example.com")
  return contacts.with_columns(expr)

Execution committed, 0 failures.

subscriptions/contacts_flagged is live — a Boolean column, true where the suffix matched:

┌───────┐
│ email │
├───────┤
│ True  │
├───────┤
│ False │
└───────┘

Transformed tables

contacts

email
ada@example.com
bo@other.org

2 rows

contacts_flagged

email
True
False

2 rows

Extract and replace

make a new table that takes the contacts table and pulls the mail domain out of the email column

Needs a transformer. Getting its guide.

Called tabsdata 2 times (ctrl+o to expand)

Generating — str.extract() with a capture group, and group_index=1 picks that group rather than the whole match.

Called tabsdata 3 times (ctrl+o to expand)

The transformer, registered in subscriptions:

@transformer(
  input_tables=["contacts"],
  output_tables=["contacts_domain"],
)
def extract_domain(contacts: TableFrameSpec) -> TableFrameSpec:
  expr = col("email").str.extract(r"@(\w+)\.", group_index=1)
  return contacts.with_columns(expr)

Execution committed, 0 failures.

subscriptions/contacts_domain is live — the captured group, not the whole match:

┌─────────┐
│  email  │
├─────────┤
│ example │
├─────────┤
│ other   │
└─────────┘

Transformed tables

contacts

email
ada@example.com
bo@other.org

2 rows

contacts_domain

email
example
other

2 rows

Trim, pad and slice

make a new table that takes the people table and trims the whitespace off both ends of the name column

Needs a transformer. Getting its guide.

Called tabsdata 2 times (ctrl+o to expand)

Generating — str.strip_chars() with no argument trims whitespace from both ends and leaves the inside alone.

Called tabsdata 3 times (ctrl+o to expand)

The transformer, registered in subscriptions:

@transformer(
  input_tables=["people"],
  output_tables=["people_trimmed"],
)
def trim_name(people: TableFrameSpec) -> TableFrameSpec:
  expr = col("name").str.strip_chars()  # Both sides
  return people.with_columns(expr)

Execution committed, 0 failures.

subscriptions/people_trimmed is live — padding gone from both ends:

┌───────┐
│ name  │
├───────┤
│ Ada   │
├───────┤
│ Grace │
└───────┘

Transformed tables

people

name
Ada
Grace

2 rows

people_trimmed

name
Ada
Grace

2 rows

Measure and locate

make a new table that takes the tickets table and measures the character length of the description column

Needs a transformer. Getting its guide.

Called tabsdata 2 times (ctrl+o to expand)

Generating — str.len_chars() counts characters, which is what a person means by length.

Called tabsdata 3 times (ctrl+o to expand)

The transformer, registered in subscriptions:

@transformer(
  input_tables=["tickets"],
  output_tables=["tickets_lengths"],
)
def measure_chars(tickets: TableFrameSpec) -> TableFrameSpec:
  expr = col("description").str.len_chars()  # Character count
  return tickets.with_columns(expr)

Execution committed, 0 failures.

subscriptions/tickets_lengths is live — the character count per row:

┌─────────────┐
│ description │
├─────────────┤
│ 12          │
├─────────────┤
│ 2           │
└─────────────┘

Transformed tables

tickets

description
disk failure
ok

2 rows

tickets_lengths

description
12
2

2 rows

Parse a log line

make a new table that takes the access table and parses the log_line column into named columns

Needs a transformer. Getting its guide.

Called tabsdata 2 times (ctrl+o to expand)

Generating — grok() on the frame rather than an expression on one column: a grok pattern produces several columns at once, so it is a frame-level operation.

Called tabsdata 3 times (ctrl+o to expand)

The transformer, registered in subscriptions:

@transformer(
  input_tables=["access"],
  output_tables=["access_parsed"],
)
def parse_access(access: TableFrameSpec) -> TableFrameSpec:
  return access.grok(
      "log_line",
      r"%{IP:client_ip} - - \[%{HTTPDATE:timestamp}\] \"%{WORD:method} %{URIPATHPARAM:path}\""
  )

Execution committed, 0 failures.

subscriptions/access_parsed is live — the log line broken into the named fields:

┌──────────┬───────────┬───────────┬────────┬──────┐
│ log_line │ client_ip │ timestamp │ method │ path │
├──────────┼───────────┼───────────┼────────┼──────┤
│          │           │           │        │      │
└──────────┴───────────┴───────────┴────────┴──────┘

Transformed tables

access

log_line
10.0.0.1 - - [12/Mar/2024:09:15:00 +0000] "GET /users"

1 row

access_parsed

log_line	client_ip	timestamp	method	path
10.0.0.1 - - [12/Mar/2024:09:15:00 +0000] "GET /users"	10.0.0.1	12/Mar/2024:09:15:00 +0000	GET	/users

1 row

Change case​

Test a string against a pattern​

Extract and replace​

Trim, pad and slice​

Measure and locate​

Parse a log line​

Change case

Test a string against a pattern

Extract and replace

Trim, pad and slice

Measure and locate

Parse a log line