Skip to content
grafOS docs

CPU Isolation Wire Format

Status: design decision. Commits the wire shape for per-lease CPU isolation policy. Implementation is a separate wave gated on this note.

Addendum to: docs/spec/resource-isolation-and-exclusivity.md §6.2 (lease requests) and §7 (SDK consequences).

1. Problem recap

The resource isolation design establishes that CPU capacity, execution mode, and isolation are three independent axes, and that isolation must be a first-class request input rather than an implicit consequence of wider leases. Today CpuIsolationPolicy exists as a daemon-wide CLI knob (fabricbiosd --cpu-isolation-policy ...) with three classes:

  • BestEffort — default; no pinning beyond cgroup quota
  • WholeCore — lease owns a full core; no SMT sibling sharing
  • StrictIsolated — WholeCore plus topology/NUMA constraints

Clients cannot currently request these per-lease. This note answers: how should a LEASE_ALLOC request carry a per-lease CPU isolation policy independent of execution width?

2. Decision

Extend the existing LeaseAllocRequest TLV params blob with a new TLV_LEASE_CPU_ISOLATION (0x0902) tag carrying a single-byte policy class. Mirror the shape already used by TLV_LEASE_INTENT_KV_CACHE (0x0901), defined in the lease-intent parser.

2.1 TLV layout

+--------+--------+--------+--------+
| tag=0x0902 (u16 BE)               |
+--------+--------+--------+--------+
| len=0x0001 (u16 BE)               |
+--------+--------+--------+--------+
| class (u8)                        |
+--------+--------+--------+--------+

Class encoding:

ValueClassNotes
0x00BestEffortEquivalent to omitting the TLV.
0x01WholeCoreExclusive core; no sibling sharing.
0x02StrictIsolatedWholeCore + topology/NUMA constraints.
0x03..0xFEreservedFail closed per §3.
0xFFreserved sentinelNever valid on the wire.

2.2 Absent TLV

If the TLV is absent, the request inherits the daemon-wide default set by --cpu-isolation-policy. This preserves backwards compatibility: existing clients that never emit the TLV observe unchanged behavior.

3. Fail-closed rules

Per design note §6.2:

  1. Unknown class byte → reject with LeaseError::InvalidIntent (reusing the existing error; no new variant required for this wave).
  2. Class not supported by the runtime → reject with LeaseError::InvalidArgs and a rejection-reason TLV naming the unsupported class. “Silently downgrade to BestEffort” is explicitly forbidden.
  3. Bare-metal runtimes that cannot enforce WholeCore/StrictIsolated → reject the request. They MUST NOT accept-and-ignore. The runtime semantics for bare metal are addressed separately; this note only commits that the wire path fails closed until that decision lands.
  4. Malformed TLV length → reject with LeaseError::InvalidIntent, matching existing TLV parse behavior.

4. Rationale for the TLV approach

Four options were considered. This note chooses option 2 (TLV) for the following reasons.

4.1 Precedent already exists

TLV_LEASE_INTENT_KV_CACHE is the load-bearing precedent: optional, advisory-but-validated, parsed via lease_intent::parse_kv_cache_intent, carried on the same LeaseAllocRequest params blob. CPU isolation is structurally identical (optional classifier attached to a lease request) and should not invent a second mechanism for the same shape.

4.2 Avoids a wire break

Option 1 (new struct field on LeaseAllocRequest) would require regenerating all vectors/v0/ golden vectors per the Wire Compatibility Discipline in CLAUDE.md, and every client/server that decodes the struct would need synchronized updates. The TLV path adds a new optional tag with zero impact on existing encoders.

4.3 Avoids op-code proliferation

Option 3 (dedicated LEASE_ALLOC_ISOLATED sub-op) duplicates handler logic and forces the same daemon-wide vs per-lease policy merge on two code paths. Rejected.

4.4 Cap-rights are the wrong granularity

Option 4 (encode via cap-entry rights bits) collapses three distinct isolation classes into binary flags, which cannot represent StrictIsolated without either a second flag (fragile) or overloading WholeCore (dishonest). Rejected.

5. What this note does NOT commit to

  • Runtime semantics on bare metal. Currently unsupported. This note only commits that unsupported classes fail closed.
  • Inventory advertisement of which classes a node supports. Not yet exposed. The TLV wire shape here is a precondition but the inventory side is a separate design decision — specifically, how RESOURCE_SUMMARY or an equivalent exposes the set of honored classes, which echoes the GPU exclusivity inventory question.
  • SDK knob. Not yet exposed in the SDK. The expected shape is CpuBuilder::new().isolation(IsolationPolicy::WholeCore).acquire() per design note §7, but the builder change is a separate wave.
  • Scheduler admission using the class. Currently unsupported by the scheduler; the TLV is only honored by the node-side lease handler on the single-cell path.
  • Interaction with grafos_scheduler::tenant::Priority::Standard. These are unrelated concepts that share a name and must not be folded in.

6. Implementation checklist for the follow-on wave

When the implementation wave is picked up:

  1. Add TLV_LEASE_CPU_ISOLATION = 0x0902 to the lease-intent parser (or a new sibling module — prefer the sibling to keep KV-cache intent parsing narrow).
  2. Add a cpu_isolation: Option<CpuIsolationClass> field to LeaseAllocRequest populated by parse().
  3. Reject unknown class bytes and malformed TLVs per §3.
  4. Thread the class through to the node-side lease handler and apply it via the existing Linux CpuIsolationPolicy enforcement path. Bare-metal handlers reject (fail closed) until runtime support lands.
  5. Add golden-vector coverage: one positive case per class, one negative case for an unknown class byte, one negative case for a truncated TLV. Regenerate vectors/v0/ per CLAUDE.md discipline.
  6. Document the TLV in docs/spec/fabricbios-wire-encoding-v0.md.

Out of scope for that wave: SDK builder knob, inventory advertisement, scheduler policy.

  • docs/spec/resource-isolation-and-exclusivity.md — the three-axis model this wire shape serves
  • The lease-intent TLV parser provides the precedent
  • LeaseAllocRequest carries the params blob this TLV is added to
  • The Linux CpuIsolationPolicy enum this TLV maps onto
  • docs/spec/fabricbios-wire-encoding-v0.md — needs an entry for TLV_LEASE_CPU_ISOLATION when implementation lands