Expand description
Relocatable stage edges with generation-tracked handoff and durable checkpoints for grafOS pipelines.
This crate provides the core primitives for building pipelines whose inter-stage edges can relocate transparently on failure. The key pattern:
- A producer pushes items into a
grafos_collections::queue::FabricQueuevia aRelocatableQueueEdge. - When the queue’s lease expires or the connection drops, the producer
allocates a new queue, publishes the new
grafos_locator::locator::QueueLocatorvia agrafos_locator::handoff::HandoffWriter, and resumes pushing. - A consumer polls a
grafos_locator::handoff::HandoffReaderto detect the generation bump, rebinds to the new queue, and resumes popping.
Stage state can be durably checkpointed to block storage via
CheckpointedStageState, enabling at-least-once delivery with
idempotent sinks.
§Feature flags
| Feature | Default | Effect |
|---|---|---|
std | Yes | Enables std across all grafos dependencies |
observe | No | Enables grafos-observe integration |
Structs§
- Checkpointed
Stage State - A stage state wrapper with durable checkpoint/restore support.
- Edge
Checkpoint - Checkpoint recording how far a stage has progressed through its input and output queues.
- Relocatable
Queue Edge - A queue edge that can relocate to a new backing queue on failure.
- StageId
- Unique identifier for a pipeline stage.
Enums§
- Edge
Error - Errors that can occur during edge operations.
Type Aliases§
- Edge
Generation - Alias for
grafos_fence::FenceEpochused as the edge generation counter.