Crate grafos_stream 

Stream processing pipelines over fabric resources.

This crate provides a pipeline builder DSL for composing data processing stages (Source → Transform → Sink/Fold) that communicate through FabricQueue inter-stage buffers backed by leased fabric memory.

Stage traits

Trait	Role
Source<T>	Produces items (returns None when exhausted)
Transform<In, Out>	Transforms one item to zero or one output items
Sink<T>	Consumes items (terminal stage)
Fold<T, Acc>	Reduces items into an accumulator (terminal stage)

Pipeline builder

use grafos_stream::prelude::*;

let source = VecSource::new(vec![1u32, 2, 3, 4, 5]);
let result = Pipeline::from_source(source)
    .map(|x| x * 2)
    .filter(|x| *x > 4)
    .collect()
    .run()?;
assert_eq!(result, vec![6, 8, 10]);

Feature flags

Feature	Default	Effect
std	Yes	Enables std in grafos-std
checkpoint	No	Block-backed stage checkpointing

Modules

pipeline

Pipeline builder and execution engine.

placement

Placement constraints for pipeline stages.

prelude

sink

Built-in sink implementations.

source

Built-in source implementations.

stage

Core stage traits for stream processing pipelines.