Version: 1.8.0

tabsdata.dataquality


`AllOk`	A boolean criteria that selects records where all boolean classifiers pass (`True`).
`AnyFailed`	A boolean criteria that selects records where at least one boolean classifier fails (`False`).
`BoolClassifier`	Abstract base class for classifiers that produce a boolean result (`True` or `False`).
`BoolClassifier.supported_dtypes`	Returns the set of data types supported by boolean classifiers.
`BoolCriteria`	A boolean criteria to select records based on classifier results.
`Categorizer`	A base class for categorizers.
`Categorizer.supported_dtypes`	Returns the set of data types supported by this classifier.
`CategoryCriteria`	A category criteria to select records based on classifier results.
`Classifier`	Abstract base class for all data quality classifiers.
`Classifier.supported_dtypes`	Returns the set of data types supported by this classifier.
`Criteria`	Defines the conditions for selecting records based on classifier results.
`DataQuality`	Represents a data quality action to be performed on a table.
`DoesNotMatch`	A boolean classifier that checks if a value does not match a regex pattern.
`DoesNotMatch.supported_dtypes`	Returns the set of data types supported by Matches classifier.
`Enrich`	An operator that adds the data quality classifier columns to the original table frame.
`ExponentialScale`	An exponential scale.
`Fail`	An operator that fails the current transaction if a data quality threshold is met.
`Filter`	An operator that filters rows from a table based on data quality criteria.
`HasLength`	A boolean classifier that checks if the length of a value is within a given range.
`HasLength.supported_dtypes`	Returns the set of data types supported by HasLength classifier.
`IdentityScale`	An identity scale where each integer value corresponds to a bin.
`InBins`	A category criteria that selects records where the value in the specified columns falls into one of the given bins.
`IsBetween`	A boolean classifier that checks if a value is inside a specified range.
`IsBetween.supported_dtypes`	Returns the set of data types supported by IsBetween classifier.
`IsFalse`	A boolean classifier that checks if a value is `False`.
`IsFalse.supported_dtypes`	Returns the set of data types supported by IsFalse classifier.
`IsIn`	A boolean classifier that checks if a value is in a specified set of values.
`IsIn.supported_dtypes`	Returns the set of data types supported by IsIn classifier.
`IsNan`	A boolean classifier that checks if a value is `NaN` (Not a Number).
`IsNan.supported_dtypes`	Returns the set of data types supported by IsNan classifier.
`IsNegative`	A boolean classifier that checks if a numeric value is negative.
`IsNegative.supported_dtypes`	Returns the set of data types supported by IsNegative classifier.
`IsNegativeOrZero`	A boolean classifier that checks if a numeric value is negative or zero.
`IsNegativeOrZero.supported_dtypes`	Returns the set of data types supported by IsNegativeOrZero classifier.
`IsNotBetween`	A boolean classifier that checks if a value is outside a specified range.
`IsNotBetween.supported_dtypes`	Returns the set of data types supported by IsBetween classifier.
`IsNotIn`	A boolean classifier that checks if a value is not in a specified set of values.
`IsNotIn.supported_dtypes`	Returns the set of data types supported by IsIn classifier.
`IsNotNan`	A boolean classifier that checks if a value is not `NaN`.
`IsNotNan.supported_dtypes`	Returns the set of data types supported by IsNotNan classifier.
`IsNotNull`	A boolean classifier that checks if a value is not `NULL`.
`IsNotNull.supported_dtypes`	Returns the set of data types supported by IsNotNull classifier.
`IsNotNullNorNan`	A boolean classifier that checks if a value is neither `NULL` nor `NaN`.
`IsNotNullNorNan.supported_dtypes`	Returns the set of data types supported by IsNotNullNorNan classifier.
`IsNotZero`	A boolean classifier that checks if a numeric value is not zero.
`IsNotZero.supported_dtypes`	Returns the set of data types supported by IsNotZero classifier.
`IsNull`	A boolean classifier that checks if a value is `NULL`.
`IsNull.supported_dtypes`	Returns the set of data types supported by IsNull classifier.
`IsNullOrNan`	A boolean classifier that checks if a value is either `NULL` or `NaN`.
`IsNullOrNan.supported_dtypes`	Returns the set of data types supported by IsNullOrNan classifier.
`IsPositive`	A boolean classifier that checks if a numeric value is positive.
`IsPositive.supported_dtypes`	Returns the set of data types supported by IsPositive classifier.
`IsPositiveOrZero`	A boolean classifier that checks if a numeric value is positive or zero.
`IsPositiveOrZero.supported_dtypes`	Returns the set of data types supported by IsPositiveOrZero classifier.
`IsTrue`	A boolean classifier that checks if a value is `True`.
`IsTrue.supported_dtypes`	Returns the set of data types supported by IsTrue classifier.
`IsZero`	A boolean classifier that checks if a numeric value is zero.
`IsZero.supported_dtypes`	Returns the set of data types supported by IsZero classifier.
`LinearScale`	A linear scale.
`LogarithmicScale`	A logarithmic scale.
`Matches`	A boolean classifier that checks if a value matches a regex pattern.
`Matches.supported_dtypes`	Returns the set of data types supported by Matches classifier.
`MonomialScale`	A monomial (power-law) scale.
`NotInBins`	A category criteria that selects records where the value in the specified columns does not fall into any of the given bins.
`Operator`	Abstract base class for all data quality operators.
`PercentThreshold`	A threshold based on a percentage of total rows.
`RowCountThreshold`	A threshold based on a specific number of rows.
`Scale`	Abstract base class for scales used in categorization.
`ScaleCategorizer`	A classifier that bucketizes data into bins based on a given scale.
`ScaleCategorizer.supported_dtypes`	Returns the set of data types supported by categorizers.
`Select`	An operator that selects rows based on data quality criteria and writes them to another table.
`Summary`	An operator that generates a data quality summary report for a table.
`Threshold`	Abstract base class for thresholds used in the `Fail` operator.

BoolClassifier

classView source ↗

class BoolClassifier(
    column_names: str | tuple[str, str | None] | Annotated[list[str], <class 'Strict'>] | Annotated[list[tuple[str, str | None]], <class 'Strict'>] | NoneType = None,
    on_missing_column: Literal['ignore', 'fail'] = 'ignore',
    on_wrong_type: Literal['ignore', 'fail'] = 'ignore',
    on_wrong_value: Literal['ignore', 'fail'] = 'ignore',
    tags: str | list[str] | NoneType = None,
)

Bases: Classifier, ABC

Abstract base class for classifiers that produce a boolean result (True or False).

property

column_nameslist[tuple[str, str | None]] | None

Returns the list of columns the classifier applies to, including optional destination column names.

property

on_missing_columnLiteral['ignore', 'fail']

Returns what do if the column is missing.

property

on_wrong_typeLiteral['ignore', 'fail']

Returns what do if the column type is incompatible with the classifier.

property

on_wrong_valueLiteral['ignore', 'fail']

Returns what do if the column type is incompatible with the classifier.

property

tagslist[str] | None

Returns the list of tags associated with the classifier.

method

View source ↗

supported_dtypes

def supported_dtypes() -> FrozenSet[Type]

Returns the set of data types supported by boolean classifiers.

BoolCriteria

classView source ↗

class BoolCriteria(
    none_is_ok: bool,
    nan_is_ok: bool,
    tags: str | list[str] | NoneType = None,
)

Bases: Criteria, ABC

A boolean criteria to select records based on classifier results.

Initializes the BoolCriteria operator.

Parameters:

parameter

none_is_ok

If None values should be considered OK.

parameter

nan_is_ok

If NaN values should be considered OK.

parameter

tags

Optional tags to link this operator to specific classifiers.

property

nan_is_okbool

Returns if NaN values should be considered OK.

property

none_is_okbool

Returns if None values should be considered OK.

property

tagslist[str] | None

Returns the list of tags associated with this criteria.

Categorizer

classView source ↗

class Categorizer(
    column_names: str | tuple[str, str | None] | Annotated[list[str], <class 'Strict'>] | Annotated[list[tuple[str, str | None]], <class 'Strict'>] | NoneType = None,
    on_missing_column: Literal['ignore', 'fail'] = 'ignore',
    on_wrong_type: Literal['ignore', 'fail'] = 'ignore',
    on_wrong_value: Literal['ignore', 'fail'] = 'ignore',
    tags: str | list[str] | NoneType = None,
)

Bases: Classifier, ABC

A base class for categorizers.

property

column_nameslist[tuple[str, str | None]] | None

Returns the list of columns the classifier applies to, including optional destination column names.

property

on_missing_columnLiteral['ignore', 'fail']

Returns what do if the column is missing.

property

on_wrong_typeLiteral['ignore', 'fail']

Returns what do if the column type is incompatible with the classifier.

property

on_wrong_valueLiteral['ignore', 'fail']

Returns what do if the column type is incompatible with the classifier.

property

tagslist[str] | None

Returns the list of tags associated with the classifier.

method

View source ↗

supported_dtypes

def supported_dtypes() -> FrozenSet[Type]

Returns the set of data types supported by this classifier.

CategoryCriteria

classView source ↗

class CategoryCriteria(
    column_name: str,
    bins: Annotated[int, Strict(strict=True), FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=100)])] | Literal['none', 'nan', 'underflow', 'overflow'] | Annotated[list[Annotated[int, Strict(strict=True), FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=100)])] | Literal['none', 'nan', 'underflow', 'overflow']], <class 'Strict'>],
)

Bases: Criteria, ABC

A category criteria to select records based on classifier results.

Initializes the CategoryCriteria operator.

Parameters:

parameter

column_name

The column name to check for values.

parameter

bins

A single bin or a list of bins.

property

binsAbstractSet[Annotated[int, Strict(strict=True), FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=100)])] | Literal['none', 'nan', 'underflow', 'overflow']]

Returns a set of bins.

property

column_namestr

Returns the column name to check for values.

Classifier

classView source ↗

class Classifier(
    column_names: str | tuple[str, str | None] | Annotated[list[str], <class 'Strict'>] | Annotated[list[tuple[str, str | None]], <class 'Strict'>] | NoneType = None,
    on_missing_column: Literal['ignore', 'fail'] = 'ignore',
    on_wrong_type: Literal['ignore', 'fail'] = 'ignore',
    on_wrong_value: Literal['ignore', 'fail'] = 'ignore',
    tags: str | list[str] | NoneType = None,
)

Bases: ABC

Abstract base class for all data quality classifiers.

A classifier is applied to one or more columns to produce a classification result, which is stored in a new column. It supports tags to allow data quality operations to selectively act on specific classifiers.

Initializes the classifier.

Parameters:

parameter

column_names

The column(s) to apply the classifier to. - A string or list of strings specifies the source columns. The results are stored in columns with auto-generated names. - A tuple (source, destination) or a list of such tuples allows specifying custom names for the result columns. - If None, the classifier is applied to all columns.

parameter

on_missing_column

What to do if a column name is missing.

parameter

on_wrong_type

Behavior when a column’s type is incompatible with the classifier.

parameter

on_wrong_value

Behavior when a classifier's value is incompatible with the column type.

parameter

tags

A tag or list of tags to associate with the classifier. Tags help filter which classifiers an operation should consider.

property

column_nameslist[tuple[str, str | None]] | None

Returns the list of columns the classifier applies to, including optional destination column names.

property

on_missing_columnLiteral['ignore', 'fail']

Returns what do if the column is missing.

property

on_wrong_typeLiteral['ignore', 'fail']

Returns what do if the column type is incompatible with the classifier.

property

on_wrong_valueLiteral['ignore', 'fail']

Returns what do if the column type is incompatible with the classifier.

property

tagslist[str] | None

Returns the list of tags associated with the classifier.

method

View source ↗

supported_dtypes

def supported_dtypes() -> FrozenSet[Type]

Returns the set of data types supported by this classifier.

Criteria

classView source ↗

class Criteria(*args, **kwargs)

Bases: ABC

Defines the conditions for selecting records based on classifier results.

A Criteria object specifies which rows of a table should be considered for a data quality operation. This can be based on the outcomes of boolean classifiers or the binning results of categorical classifiers.

Operator

classView source ↗

class Operator(*args, **kwargs)

Bases: ABC

Abstract base class for all data quality operators.

This class provides a common structure for operators, including support for tags, which allow for selective application of classifiers.

RowCountThreshold

classView source ↗

class RowCountThreshold(
    row_count: Annotated[int, Strict(strict=True), FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0)])],
)

Bases: Threshold

A threshold based on a specific number of rows.

Initializes the threshold with a maximum row count.

Parameters:

parameter

row_count

The number of rows that, if met or exceeded, will trigger the Fail operation.

property

row_countint

Returns the row count for the threshold.

Scale

classView source ↗

class Scale(
    scale_range: tuple[Annotated[int, Strict(strict=True)] | Annotated[float, Strict(strict=True)], Annotated[int, Strict(strict=True)] | Annotated[float, Strict(strict=True)]],
    bins: Annotated[int, Strict(strict=True), FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1), Le(le=100)])],
    use_bin_zero: bool,
)

Bases: ABC

Abstract base class for scales used in categorization.

A scale defines how numerical data is divided into bins.

property

use_bin_zerobool

Returns whether the minimum value is in bin zero or in bin one.

Threshold

classView source ↗

class Threshold(*args, **kwargs)

Bases: ABC

Abstract base class for thresholds used in the Fail operator.