In addition to CRUD operations for managing Stream objects, DataCater offers the Inspect operation.
The Inspect operation allows you to view the most recent records of the underlying Apache Kafka topic and is very useful for exploring your data and debugging pipelines.
You can either call the API endpoint for retrieving the raw records of a Stream or inspect a Stream in our UI. The latter lets you explore the values of your records in an interactive grid:
Inspecting the stream external.companies using DataCater's UI.
DataCater offers two methods for sampling the records from a Stream.
This is the default sampling method. When retrieving
Xrecords from a Stream, we fill up the sample set one partition at a time until
xrecords are reached, moving from the first partition,
partition 0, to the second partition,
partition 1, and so on. This could mean that only records from
partition 0will be returned when inspecting the Stream. On the other hand, it is guaranteed, that
Xrecords will be returned, provided that the Stream contains at least
The sampling method
uniformtries to create a sample set that holds an equal number of records from each partition of the Apache Kafka topic. When creating a sample set of
Xrecords for a Kafka topic with
Npartitions, this sample method consumes up to
X / Nrecords from each partition. Please note that this sampling method does not guarantee to return
Xsample records. If at least one of the
Npartitions contains less than
X / Nrecords, the sampling method uniform returns less than
Xrecords. If you do not require an equal distribution of the sample records across the partitions of the Kafka topic , please consider using the sampling method sequenced.