Building a pipeline
Pipelines describe the processing of records (or events) that are streamed between two streams. They can define filters or transforms that are applied to the data.
Please make sure that you have created at least two streams: One that you want to use as the input for your pipeline, and another one that you want to use as the output for your pipeline.
Navigate to Pipelines in the UI and click Create your first pipeline:
Define a name and select both a source and a sink stream:
Finally, click Create pipeline.
Alternatively, you can also use our API to create a pipeline. The following API call creates a pipeline with the name
Process commits that consumes records from the stream with the UUID cf8fda79-a86c-45cf-b769-58c6b625797d and publishes data to the stream with the UUID ef63b4a5-579a-4187-89f6-2c86f24122d6:$ curl http://localhost:8080/api/v1/pipelines/ \
-H'Authorization: Bearer YOUR_TOKEN' \
-XPOST \
-H'Content-Type:application/json' \
-d'{"name":"Process commits","metadata":{"stream-in":"cf8fda79-a86c-45cf-b769-58c6b625797d","stream-out":"ef63b4a5-579a-4187-89f6-2c86f24122d6"},"spec":{}}'
Last modified 3mo ago