QUIC-Stream Constrained Dataplane Profile v0
Status: Implemented (Phases 34-37 Linux, Phase 86 Pi5 bare-metal)
Overview
This profile defines the QUIC-stream transport for memory and block data-plane bindings. It covers two deployment targets:
- Linux (
fabricbiosd) — Quinn-based QUIC with dedicated data-plane endpoints per resource type. - Pi5 bare-metal — Custom QUIC v1 implementation with stream-multiplexed control and data-plane on a single endpoint.
Both targets share the same 56-byte FBMU/FBBU header wire format and the same operation semantics. The differences are in connection topology, stream mapping, and authentication model.
Transport
Linux (fabricbiosd)
- QUIC stack: Quinn (Rust, async, IETF QUIC v1)
- TLS: TLS 1.3 via rustls, integrated into QUIC handshake
- ALPN:
fabricbios/0 - Ports: Data-plane endpoints bind on separate addresses from the control-plane (port 5701). The memory and block data-plane each listen on their own QUIC endpoint (configurable via
--mem-dp-bindand--block-dp-bind).
Pi5 Bare-Metal
- QUIC stack: Custom QUIC v1 implementation (not Quinn), single-threaded,
no_std+alloc - TLS: TLS 1.3 via vendored rustls with RustCrypto provider
- ALPN:
fabricbios/0(same as control plane) - Port: 5701 — a single QUIC endpoint serves both control-plane and data-plane traffic
- Multiplexing: Control and data-plane share the same QUIC connection; stream IDs and magic-byte discrimination distinguish traffic types
Stream Mapping
Linux (fabricbiosd)
Each QUIC connection is bound to a single lease bundle. Within that connection:
-
Stream 0 (first bidi): HELLO handshake. Client sends HELLO with the 16-byte
lease_id; server responds with HELLO_ACK containing status, resource dimensions, and max I/O sizes. Both sides finish (half-close) this stream after the handshake. -
Subsequent bidi streams: Each READ, WRITE, or PING request opens a new bidirectional stream. The client sends a single framed message, finishes its send half, and reads the response from the server’s send half. One stream per request; streams are lightweight and disposable.
The request_id field is preserved from the FBMT/FBBT wire format for correlation, but each stream naturally scopes a single request-response pair.
Pi5 Bare-Metal
A single QUIC connection carries both control-plane and data-plane traffic. Stream assignment:
-
Stream 0 (first client-initiated bidi, stream ID 0): Control-plane operations (PING, GET_IDENTITY, GET_INVENTORY, LEASE_ALLOC, LEASE_RENEW, LEASE_FREE, LEASE_QUERY).
-
Subsequent client-initiated bidi streams (stream IDs 4, 8, 12, …): Data-plane operations. Each stream carries exactly one request-response pair and is self-contained.
Magic-byte discrimination: The first 4 bytes of stream data determine the protocol:
| Magic bytes | Hex | Protocol |
|---|---|---|
FBMU | 0x46424D55 | Memory dataplane (FBMQ) |
FBBU | 0x46424255 | Block dataplane (FBBQ) |
| Other | — | Control frame (existing wire format) |
The server reads the first 4 bytes of each new bidi stream to dispatch to the appropriate handler. This allows a single connection to interleave control ops, memory I/O, and block I/O without dedicated ports.
Stream limits: initial_max_streams_bidi=8. Clients should reuse stream budget by completing requests before opening new ones.
Wire Format
The on-wire framing uses the 56-byte FBMU/FBBU header, shared across all transport paths (QUIC streams, UDP shim).
Header Layout (56 bytes)
Offset Size Field Description────── ──── ───── ─────────── 0 4 magic "FBMU" (0x46424D55) or "FBBU" (0x46424255) 4 1 version Protocol version (currently 1) 5 1 op Operation code 6 2 flags Bit flags (RESP=0x0001, ERROR=0x0002, FRAG_V1=0x0004) 8 2 payload_len Length of payload following header10 2 reserved Must be zero12 4 request_id Client-assigned request correlation ID16 16 lease_id 128-bit lease identifier32 8 nonce Anti-replay nonce40 16 auth_tag HMAC-SHA256 truncated to 128 bits (see Auth Tag below)All multi-byte integers are big-endian.
Auth Tag Handling
| Transport | Auth tag behavior |
|---|---|
| UDP shim (FBMU/FBBU) | HMAC-SHA256 computed over bytes 0-39 using the data-plane key (dp_key) from LEASE_ALLOC. Required for authentication. |
| QUIC stream (FBMQ/FBBQ) | Zeroed (16 bytes of 0x00). TLS 1.3 provides authentication, integrity, and confidentiality. Server MAY skip HMAC verification for QUIC-transported frames. The field is retained for wire compatibility. |
Memory Operations (FBMU/FBMQ)
| Op | Code | Direction | Payload |
|---|---|---|---|
| HELLO | 0x01 | Client→Server | client_caps (u32) |
| HELLO_ACK | 0x02 | Server→Client | Status, resource dimensions, max I/O sizes |
| READ | 0x10 | Client→Server | offset (u64), length (u32) |
| READ_RESP | 0x11 | Server→Client | Status + data |
| WRITE | 0x20 | Client→Server | offset (u64), length (u32) + data |
| WRITE_RESP | 0x21 | Server→Client | Status |
| PING | 0x30 | Client→Server | (empty) |
| PONG | 0x31 | Server→Client | (empty) |
Block Operations (FBBU/FBBQ)
| Op | Code | Direction | Payload |
|---|---|---|---|
| HELLO | 0x01 | Client→Server | (empty) |
| HELLO_ACK | 0x02 | Server→Client | Status, sector count, sector size |
| READ_BLOCK | 0x10 | Client→Server | start_lba (u64), block_count (u32) |
| READ_BLOCK_RESP | 0x11 | Server→Client | Status + data |
| WRITE_BLOCK | 0x20 | Client→Server | start_lba (u64), block_count (u32) + data |
| WRITE_BLOCK_RESP | 0x21 | Server→Client | Status |
Status Codes
| Code | Name | Meaning |
|---|---|---|
| 0 | OK | Success |
| 1 | INVALID | Malformed request |
| 2 | NO_LEASE | Lease not found or expired |
| 3 | RANGE | Offset/LBA out of bounds |
| 4 | REPLAY | Duplicate request_id (replay detected) |
Connection Lifecycle
Linux (fabricbiosd)
- Lease validation: The server re-validates the lease on every READ/WRITE request. If the lease has expired or been revoked, the server sends a rejection response and closes the QUIC connection with reason
lease_expired. - Fail-closed: Lease expiry always terminates the QUIC connection. No new streams can be opened after closure. This matches the RDMA fencing semantics.
- Fencing: A fenced resource rejects HELLO handshakes with an error status, preventing new connections.
Pi5 Bare-Metal
- Single connection: One QUIC connection handles control-plane and all data-plane streams. The connection persists across multiple lease operations.
- Lease validation: The server validates
lease_idfrom every data-plane request header against its local lease table (same table used by control-plane LEASE_ALLOC/RENEW/FREE/QUERY). - Fail-closed: Invalid lease returns an error response on the individual stream. The QUIC connection stays open for other operations (control-plane and data-plane streams for valid leases).
- Replay protection:
request_id-based deduplication for write operations using the existing replay cache. - LEASE_ALLOC binding: LEASE_ALLOC returns a binding TLV with
port=5701, indicating that the data-plane uses QUIC streams on the same endpoint (as opposed toport=5702/5703for UDP shim).
Security Model
Linux (fabricbiosd)
- Authentication: TLS 1.3 mutual authentication is built into the QUIC handshake. The same mTLS configuration (CA-based or TOFU) used for the control plane applies to data-plane endpoints.
- Authorization: The HELLO handshake binds the connection to a specific lease. The server checks the lease against its local lease store. Only the lease holder (authenticated via TLS) can access the granted resource region.
- No separate HMAC: Unlike the bare-metal UDP shim (FBMU) path which uses HMAC auth tags, the QUIC data-plane relies entirely on TLS for authentication. The TLS session provides confidentiality, integrity, and peer identity.
Pi5 Bare-Metal
- Authentication: mTLS via QUIC/TLS 1.3 handshake. The same certificate used for control-plane authentication applies to data-plane streams within the same connection.
- Authorization: Every data-plane operation includes a
lease_idin the header. The server validates this against the lease table on each request. No HELLO handshake is required for the QUIC path — the lease_id in each operation header is sufficient since the TLS session already authenticates the peer. - No separate dp_key HMAC: TLS provides confidentiality, integrity, and peer identity. The
auth_tagfield in the header is zeroed for QUIC-transported frames. - Feature flag interaction: When
fbmu-authis enabled, mTLS is required — clients must present certificates. Without client certificates, the TLS handshake fails, which cascades to TX ring exhaustion on Pi5 (GEM HRESP error).
Implementation
Linux (fabricbiosd)
| Component | File | LOC |
|---|---|---|
| QUIC memory server + client | crates/fabricbios-platform-linux/src/quic_memory.rs | ~715 |
| QUIC block server + client | crates/fabricbios-platform-linux/src/quic_block.rs | ~658 |
| fabricbiosd wiring | crates/fabricbiosd/src/main.rs (control-server subcommand) | integrated |
| Server TLS/QUIC config | crates/fabricbiosd/src/quic_server.rs | shared with control plane |
Pi5 Bare-Metal
| Component | File |
|---|---|
| QUIC server (control + dataplane dispatch) | crates/fabricbios-pi5-bringup/src/main.rs |
| FBMU memory handler | crates/fabricbios-platform-rpi-baremetal/src/quic_fbmu.rs |
| FBBU block handler | crates/fabricbios-platform-rpi-baremetal/src/quic_fbbu.rs |
| Header codec (shared) | crates/fabricbios-core/src/fbmu.rs, crates/fabricbios-core/src/fbbu.rs |
| Interop client | crates/fabricbios-quic-interop/src/ |
Relationship to Other Profiles
| Profile | Transport | Nodes | Port(s) | Status |
|---|---|---|---|---|
| QUIC-stream / Linux (this doc) | Quinn / QUIC v1, dedicated endpoints | Linux (fabricbiosd) | 5701 (control), configurable (data) | Active |
| QUIC-stream / Pi5 (this doc) | Custom QUIC v1, stream-multiplexed | Pi5 bare metal | 5701 (unified) | Active |
| FBMU/FBBU (UDP shim) | Raw UDP frames | Pi5 bare metal | 5702 (mem), 5703 (block) | Deprecated (compat-udp-dataplane feature flag) |
| FBMT/FBBT (TCP) | TCP + TLS 1.3 | Linux | — | Removed (Phase 37) |
Deprecation: UDP Shim
The original FBMU/FBBU UDP shim paths are retained behind the compat-udp-dataplane feature flag for backwards compatibility. QUIC-stream dataplane (FBMQ/FBBQ) is the default for new deployments.
The binding TLV endpoint port in LEASE_ALLOC responses distinguishes transport:
| Port | Transport | Protocol names |
|---|---|---|
| 5701 | QUIC stream (default) | FBMQ (memory), FBBQ (block) |
| 5702 | UDP shim (deprecated) | FBMU (memory) |
| 5703 | UDP shim (deprecated) | FBBU (block) |
Clients receiving a binding with port 5701 open a new QUIC bidi stream on the existing connection and write the appropriate magic bytes (FBMU for memory, FBBU for block) as the first 4 bytes. Clients receiving port 5702/5703 fall back to the UDP shim path.