tabsdatak.dataquality
AllOk | A boolean criteria that selects rows where every boolean classifier passes (True). |
AnyFailed | A boolean criteria that selects rows where at least one boolean classifier fails (False). |
BetweenClassifier | A boolean classifier that checks if a value is within a specified range. |
BinCountSpec | A type alias for the number of bins a Scale will have, between 1 and MAX_BINS. |
BinIdxSpec | A type alias for a zero-based bin index, constrained by MAX_BINS. |
BinsCriteriaSpec | A non-empty set of bins used in criteria. |
BinSpec | A type alias for the bin, either an index [0..100] or a special one [none, nan, underflow, overflow] |
BoolClassifier | Abstract base class for classifiers with a boolean verdict. |
BoolCriteria | Abstract base for criteria over boolean classifier verdicts. |
Categorizer | Abstract base class for classifiers that assign each value to a bin. |
CategoryCriteria | Abstract base for criteria over a categorizer's bin assignments. |
Classifier | Abstract base class for all data quality classifiers. |
ClassifiersSpec | A non-empty list of Classifier instances. |
ColumnNameSpec | A type alias for a single column name, represented as a string. |
ColumnNamesWithAltSpec | A list of column references. |
ColumnNameWithAltSpec | A single column reference: either the source column name, or a (source, destination) tuple naming the column to store the result in (destination may be None to auto-generate it). |
Criteria | Selects rows for a Filter, Select or Fail operator. |
CriteriaOperator | An operator that uses a criteria to select records. |
DataQuality | A data quality check applied to one of a function's output tables. |
DoesNotMatch | A boolean classifier that checks whether a value does not match a regular expression. |
Enrich | An operator that appends each classifier's verdict column to the target table. |
ExponentialScale | A scale whose bin edges are spaced exponentially over [min_val, max_val] (used with ScaleCategorizer). |
Fail | An operator that fails (aborts) the current transaction when the rows selected by criteria reach threshold. |
Filter | An operator that removes the rows selected by criteria from the target table. |
HasLength | A boolean classifier that checks whether a value's length is within a range. |
IdentityScale | A scale where every integer in [min_val, max_val) maps to its own bin (used with ScaleCategorizer). |
InBins | A category criteria that selects rows whose bin is one of the given bins. |
InClassifier | A boolean classifier that checks if a value is in a specified set of values. |
IncludeQualityColumnsSpec | A type alias for quality columns to included in the Select or Filter target table. |
IsBetween | A boolean classifier that checks whether a value is inside a range. |
IsFalse | A boolean classifier that checks whether a value is False. |
IsIn | A boolean classifier that checks whether a value is in a set of values. |
IsNan | A boolean classifier that checks whether a value is NaN (not a number). |
IsNegative | A boolean classifier that checks whether a numeric value is negative. |
IsNegativeOrZero | A boolean classifier that checks whether a numeric value is negative or zero. |
IsNotBetween | A boolean classifier that checks whether a value is outside a range. |
IsNotIn | A boolean classifier that checks whether a value is not in a set of values. |
IsNotNan | A boolean classifier that checks whether a value is not NaN. |
IsNotNull | A boolean classifier that checks whether a value is not null. |
IsNotNullNorNan | A boolean classifier that checks whether a value is neither null nor NaN. |
IsNotZero | A boolean classifier that checks whether a numeric value is not zero. |
IsNull | A boolean classifier that checks whether a value is null. |
IsNullOrNan | A boolean classifier that checks whether a value is null or NaN. |
IsPositive | A boolean classifier that checks whether a numeric value is positive. |
IsPositiveOrZero | A boolean classifier that checks whether a numeric value is positive or zero. |
IsTrue | A boolean classifier that checks whether a value is True. |
IsZero | A boolean classifier that checks whether a numeric value is zero. |
LenSpec | A type alias for a non-negative integer, typically used for specifying length. |
LinearScale | A scale that splits [min_val, max_val] into equal-width bins (used with ScaleCategorizer). |
LogarithmicScale | A scale whose bin edges are spaced logarithmically over [min_val, max_val] (used with ScaleCategorizer). |
MatchClassifier | A boolean classifier that checks if a value matches a regex pattern. |
Matches | A boolean classifier that checks whether a value matches a regular expression. |
MAX_BINS | The maximum number of bins allowed in a bucketing operation. |
MonomialScale | A power-law scale over [min_val, max_val] whose bin widths grow as a power of the value (used with ScaleCategorizer). |
NotInBins | A category criteria that selects rows whose bin is not among the given bins. |
OnInvalidColumnSpec | A type alias that specifies what to do if a column is invalid for the classifier. |
OnInvalidValueSpec | A type alias that specifies what to do if a classifier value is invalid for column. |
Operator | Abstract base class for all data quality operators. |
OperatorsSpec | A non-empty list of Operator instances. |
PercentSpec | A type alias for a percentage value, which can be an integer or a float between 0 and 100. |
PercentThreshold | A Fail threshold expressed as a percentage of the table's rows. |
PythonDataTypeSpec | A type alias for a union of common Python data types used in data quality checks. |
RangeClosedOnSpec | A type alias that specifies which side of a range is closed (inclusive). |
RangeValueSpec | A type alias for a value used in defining a range, based on Python data types. |
RowCountSpec | A type alias for a non-negative row count. |
RowCountThreshold | A Fail threshold expressed as an absolute number of rows. |
Scale | Abstract base class for the scales used by ScaleCategorizer. |
ScaleBaseSpec | A type alias for the base of a scale, which must be a float greater than 0. |
ScaleCategorizer | A classifier that assigns each value to a bin defined by a Scale. |
ScaleIdentityLimitSpec | A type alias for a limit in a scale identity, ranging from 0 to 100. |
ScaleLimitSpec | A type alias for a generic limit in a scale, which can be an integer or a float. |
ScalePowerSpec | A type alias for the power of a scale, which must be a float greater than 0. |
Select | An operator that copies the rows selected by criteria to another table, leaving the target table unchanged. |
SpecialBinSpec | A type alias for the bins for None, NaN, underflow and overflow values. |
Summary | An operator that writes a data quality summary (one row per classifier: verdict tallies, or bin counts for a categorizer) to a table. |
TagOperator | An operator that is scoped to classifiers by tag. |
TagSpec | A type alias for a single tag, represented as a string. |
TagsSpec | A list of tags. |
Threshold | Abstract base class for the thresholds used by the Fail operator. |
ValuesSpec | A non-empty set of values to check membership against (None is allowed, to test for null membership). |
BinCountSpec
type_aliasBinCountSpec: StrictInt
A type alias for the number of bins a Scale will have, between 1 and MAX_BINS.
BinIdxSpec
type_aliasBinIdxSpec: StrictInt
A type alias for a zero-based bin index, constrained by MAX_BINS.
BinsCriteriaSpec
type_aliasBinsCriteriaSpec: frozenset[BinSpec]
Resolved type:
frozenset[StrictInt | Literal['none', 'nan', 'underflow', 'overflow']]
A non-empty set of bins used in criteria.
BinSpec
type_aliasBinSpec: Union[BinIdxSpec, SpecialBinSpec]
Resolved type:
StrictInt | Literal['none', 'nan', 'underflow', 'overflow']
A type alias for the bin, either an index [0..100] or a special one [none, nan, underflow, overflow]
ClassifiersSpec
type_aliasClassifiersSpec: list[Classifier]
A non-empty list of Classifier instances.
ColumnNameSpec
type_aliasColumnNameSpec: str
A type alias for a single column name, represented as a string.
ColumnNamesWithAltSpec
type_aliasColumnNamesWithAltSpec: list[ColumnNameWithAltSpec]
Resolved type:
list[str | tuple[str, str | None]]
A list of column references. Each element is a source column name or a
(source, destination) tuple; the two forms may be mixed.
ColumnNameWithAltSpec
type_aliasColumnNameWithAltSpec: Union[ColumnNameSpec, tuple[ColumnNameSpec, Union[ColumnNameSpec, None]]]
Resolved type:
str | tuple[str, str | None]
A single column reference: either the source column name, or a
(source, destination) tuple naming the column to store the result in
(destination may be None to auto-generate it).
IncludeQualityColumnsSpec
type_aliasIncludeQualityColumnsSpec: Literal['none', 'criteria', 'all']
A type alias for quality columns to included in the Select or Filter target table.
LenSpec
type_aliasLenSpec: StrictInt
A type alias for a non-negative integer, typically used for specifying length.
MAX_BINS
constantMAX_BINS = 100
The maximum number of bins allowed in a bucketing operation.
OnInvalidColumnSpec
type_aliasOnInvalidColumnSpec: Literal['ignore', 'fail']
A type alias that specifies what to do if a column is invalid for the classifier.
OnInvalidValueSpec
type_aliasOnInvalidValueSpec: Literal['ignore', 'fail']
A type alias that specifies what to do if a classifier value is invalid for column.
OperatorsSpec
type_aliasOperatorsSpec: list[Operator]
A non-empty list of Operator instances.
PercentSpec
type_aliasPercentSpec: Union[StrictInt, StrictFloat]
Resolved type:
StrictInt | StrictFloat
A type alias for a percentage value, which can be an integer or a float between 0 and 100.
PythonDataTypeSpec
type_aliasPythonDataTypeSpec: Union[int, float, bool, str, date, time, datetime, timedelta, bytes, Decimal]
Resolved type:
int | float | bool | str | date | time | datetime | timedelta | bytes | Decimal
A type alias for a union of common Python data types used in data quality checks.
RangeClosedOnSpec
type_aliasRangeClosedOnSpec: Literal['none', 'lower', 'upper', 'both']
A type alias that specifies which side of a range is closed (inclusive).
- 'none': The range is open on both sides.
- 'lower': The range is closed on the lower bound.
- 'upper': The range is closed on the upper bound.
- 'both': The range is closed on both sides.
RangeValueSpec
type_aliasRangeValueSpec: PythonDataTypeSpec
Resolved type:
int | float | bool | str | date | time | datetime | timedelta | bytes | Decimal
A type alias for a value used in defining a range, based on Python data types.
RowCountSpec
type_aliasRowCountSpec: StrictInt
A type alias for a non-negative row count.
ScaleBaseSpec
type_aliasScaleBaseSpec: StrictFloat
A type alias for the base of a scale, which must be a float greater than 0.
ScaleIdentityLimitSpec
type_aliasScaleIdentityLimitSpec: StrictInt
A type alias for a limit in a scale identity, ranging from 0 to 100.
ScaleLimitSpec
type_aliasScaleLimitSpec: Union[StrictInt, StrictFloat]
Resolved type:
StrictInt | StrictFloat
A type alias for a generic limit in a scale, which can be an integer or a float.
ScalePowerSpec
type_aliasScalePowerSpec: StrictFloat
A type alias for the power of a scale, which must be a float greater than 0.
SpecialBinSpec
type_aliasSpecialBinSpec: Literal['none', 'nan', 'underflow', 'overflow']
A type alias for the bins for None, NaN, underflow and overflow values.
TagSpec
type_aliasTagSpec: str
A type alias for a single tag, represented as a string.
TagsSpec
type_aliasTagsSpec: list[TagSpec]
Resolved type:
list[str]
A list of tags.
ValuesSpec
type_aliasValuesSpec: frozenset[Union[PythonDataTypeSpec, None]]
Resolved type:
frozenset[int | float | bool | str | date | time | datetime | timedelta | bytes | Decimal | None]
A non-empty set of values to check membership against (None is allowed, to
test for null membership). A list passed to the constructor is coerced to a
frozenset (deduplicated) by pydantic.