Filters
Pipelines can declare filters to filter processed records, i.e., consider only a subset of the records.
Filters are applied to single attributes. Exemplary filters match attribute values against regular expression, verify that values feature a specific suffix, or validate that an attribute holds a specific value.
DataCater ships a set of pre-defined filters for the most common use cases in operational transforms. In addition, DataCater supports custom Python transforms that can be defined inline in the YAML spec of pipelines or can be edited in the UI with full support for previews.
Last modified 4mo ago