grafos admin
Fabric-administrator surface (CA bootstrap, cert issuance, tenant management on the scheduler).
grafos admin
Fabric administration: CA bootstrap, cert issuance
Usage: grafos admin [OPTIONS] <COMMAND>
Commands: init-ca Generate a new fabric CA (key + self-signed certificate) issue-cert Issue a client certificate for a tenant, signed by the fabric CA issue-server-cert Issue a server certificate for scheduler/orchestrator create-tenant Register a tenant with the scheduler list-tenants List all registered tenants with quotas and usage set-quota Set or update resource quotas for a tenant delete-tenant Delete a tenant. Refuses with HTTP 409 if any active leases reference the tenant; revoke or wait for them to expire first. The orchestrator removes the tenant immediately; cell-side cleanup is eventual on the next long-poll cycle (~30s) audit-query Query the audit log. Hits the `/api/v1/admin/audit` endpoint with optional filters; works against both orchestrator and cell-scheduler endpoints (same wire shape on both) list-fenced-leases list fenced leases unfence-lease explicitly clear a fence image-provenance image provenance record builder help Print this message or the help of the given subcommand(s)
Options: --fabric <FABRIC> Fabric address (host:port). Overrides GRAFOS_FABRIC env var [env: GRAFOS_FABRIC=] --json Output in JSON format for scripting --wide Show additional columns in table output --no-color Disable color output --pool <POOL> Pool name (default: "default") [default: default] -h, --help Print help -V, --version Print versiongrafos admin init-ca
Generate a new fabric CA (key + self-signed certificate).
Creates an ECDSA P-256 CA key pair and self-signed certificate. The CA is used to sign all client and server certificates in the fabric.
Output files: ca-key.pem — CA private key (keep secure!) ca.pem — CA certificate (distribute to all hosts)
Example: grafos admin init-ca --out /etc/fabricbios/ca --days 3650
Usage: grafos admin init-ca [OPTIONS]
Options: --fabric <FABRIC> Fabric address (host:port). Overrides GRAFOS_FABRIC env var
[env: GRAFOS_FABRIC=]
--out <OUT> Output directory for ca-key.pem and ca.pem
[default: .]
--cn <CN> CA common name
[default: "fabricBIOS Fabric CA"]
--json Output in JSON format for scripting
--days <DAYS> Validity in days
[default: 3650]
--wide Show additional columns in table output
--no-color Disable color output
--pool <POOL> Pool name (default: "default")
[default: default]
-h, --help Print help (see a summary with '-h')
-V, --version Print versiongrafos admin issue-cert
Issue a client certificate for a tenant, signed by the fabric CA.
The certificate embeds the tenant identity as a URI SAN: urn:fabricbios:tenant:<name>
When the client presents this certificate to a scheduler with mTLS enabled, the scheduler extracts the tenant name from the SAN and uses it for quota enforcement and audit. The --tenant flag in deploy commands becomes unnecessary.
Output files: client-key.pem — client private key client.pem — client certificate
Example: grafos admin issue-cert --ca-cert ca/ca.pem --ca-key ca/ca-key.pem \ --tenant acme --out /home/acme/.grafos/
Usage: grafos admin issue-cert [OPTIONS] --ca-cert <CA_CERT> --ca-key <CA_KEY>
Options: --ca-cert <CA_CERT> Path to CA certificate (PEM)
--fabric <FABRIC> Fabric address (host:port). Overrides GRAFOS_FABRIC env var
[env: GRAFOS_FABRIC=]
--ca-key <CA_KEY> Path to CA private key (PEM)
--json Output in JSON format for scripting
--tenant <TENANT> Tenant name. Required for --role tenant (default); ignored for --role fabric-observer and --role fabric-admin since those principals are fabric-wide and have no tenant scope
[default: ]
--wide Show additional columns in table output
--no-color Disable color output
--role <ROLE> Principal role encoded in the cert's URI SAN:
- `tenant` (default, backward compatible) → SAN `urn:fabricbios:tenant:<tenant>`. The scheduler classifies the caller as [`Role::TenantUser`] scoped to `<tenant>`. - `fabric-observer` → SAN `urn:fabricbios:fabric-observer`. The scheduler classifies the caller as [`Role::FabricObserver`]: cross-tenant read-only access, NO lease mint. Use for operator dashboards that show fleet-wide billing / SLA / audit. - `fabric-admin` → SAN `urn:fabricbios:fabric-admin`. Full [`Role::FabricAdmin`] via mTLS. Prefer token-based admin for day-to-day ops; this flag is for long-lived operator workloads. - `cell` → SAN `urn:fabricbios:cell:<provider>:<cell_id>`. Required for outbound provider cell agents. Authorizes the `(provider, cell_id)` pair for `/api/v1/cells/{register, work/poll, work/complete}` and nothing else.
[default: tenant]
--pool <POOL> Pool name (default: "default")
[default: default]
--provider <PROVIDER> Provider name for `--role cell`. Must be lowercase letters, digits, or `-` (matches the orchestrator's provider name rules)
[default: ]
--cell-id <CELL_ID> Numeric cell id for `--role cell`. Must be a u64 decimal
--out <OUT> Output directory for client-key.pem and client.pem
[default: .]
--days <DAYS> Validity in days
[default: 365]
-h, --help Print help (see a summary with '-h')
-V, --version Print versiongrafos admin issue-server-cert
Issue a server certificate for scheduler/orchestrator.
The certificate includes IP address and/or DNS SANs so that clients can verify the server's identity during the TLS handshake.
Output files: server-key.pem — server private key server.pem — server certificate
Example: grafos admin issue-server-cert --ca-cert ca/ca.pem --ca-key ca/ca-key.pem \ --san 192.168.0.55,10.10.0.20,127.0.0.1 --out /etc/fabricbios/tls/
Usage: grafos admin issue-server-cert [OPTIONS] --ca-cert <CA_CERT> --ca-key <CA_KEY>
Options: --ca-cert <CA_CERT> Path to CA certificate (PEM)
--fabric <FABRIC> Fabric address (host:port). Overrides GRAFOS_FABRIC env var
[env: GRAFOS_FABRIC=]
--ca-key <CA_KEY> Path to CA private key (PEM)
--json Output in JSON format for scripting
--san <SAN> Subject alternative names (IP addresses and DNS names, comma-separated)
--wide Show additional columns in table output
--no-color Disable color output
--out <OUT> Output directory for server-key.pem and server.pem
[default: .]
--days <DAYS> Validity in days
[default: 365]
--pool <POOL> Pool name (default: "default")
[default: default]
-h, --help Print help (see a summary with '-h')
-V, --version Print versiongrafos admin create-tenant
Register a tenant with the scheduler.
Tenants must be pre-registered before they can deploy programs or request leases (when mTLS is enabled). Optionally set an initial memory quota.
Example: grafos admin create-tenant --scheduler http://localhost:9100 \ --name acme --quota-mem 1073741824
Usage: grafos admin create-tenant [OPTIONS] --scheduler <SCHEDULER> --name <NAME>
Options: --fabric <FABRIC> Fabric address (host:port). Overrides GRAFOS_FABRIC env var
[env: GRAFOS_FABRIC=]
--scheduler <SCHEDULER> Scheduler URL
[env: GRAFOS_SCHEDULER=]
--json Output in JSON format for scripting
--name <NAME> Tenant name
--quota-mem <QUOTA_MEM> Initial memory quota in bytes (0 = unlimited)
[default: 0]
--wide Show additional columns in table output
--cert <CERT> Client cert for mTLS to scheduler
[env: GRAFOS_CERT=]
--no-color Disable color output
--key <KEY> Client key for mTLS to scheduler
[env: GRAFOS_KEY=]
--pool <POOL> Pool name (default: "default")
[default: default]
--ca <CA> CA cert for verifying scheduler
[env: GRAFOS_CA=]
--bearer <BEARER> FabricAdmin bearer token (required under mtls+token for POST /tenants)
[env: GRAFOS_BEARER=]
-h, --help Print help (see a summary with '-h')
-V, --version Print versiongrafos admin list-tenants
List all registered tenants with quotas and usage.
Example: grafos admin list-tenants --scheduler http://localhost:9100
Usage: grafos admin list-tenants [OPTIONS] --scheduler <SCHEDULER>
Options: --fabric <FABRIC> Fabric address (host:port). Overrides GRAFOS_FABRIC env var
[env: GRAFOS_FABRIC=]
--scheduler <SCHEDULER> Scheduler URL
[env: GRAFOS_SCHEDULER=]
--cert <CERT> Client cert for mTLS to scheduler
[env: GRAFOS_CERT=]
--json Output in JSON format for scripting
--key <KEY> Client key for mTLS to scheduler
[env: GRAFOS_KEY=]
--wide Show additional columns in table output
--ca <CA> CA cert for verifying scheduler
[env: GRAFOS_CA=]
--no-color Disable color output
--bearer <BEARER> FabricAdmin bearer token (optional — GET /tenants is read-only)
[env: GRAFOS_BEARER=]
--pool <POOL> Pool name (default: "default")
[default: default]
-h, --help Print help (see a summary with '-h')
-V, --version Print versiongrafos admin set-quota
Set or update resource quotas for a tenant.
Example: grafos admin set-quota --scheduler http://localhost:9100 \ --tenant acme --mem 2147483648
Usage: grafos admin set-quota [OPTIONS] --scheduler <SCHEDULER> --tenant <TENANT>
Options: --fabric <FABRIC> Fabric address (host:port). Overrides GRAFOS_FABRIC env var
[env: GRAFOS_FABRIC=]
--scheduler <SCHEDULER> Scheduler URL
[env: GRAFOS_SCHEDULER=]
--json Output in JSON format for scripting
--tenant <TENANT> Tenant name
--mem <MEM> Memory quota in bytes
[default: 0]
--wide Show additional columns in table output
--cpu <CPU> CPU quota in cores
[default: 0]
--no-color Disable color output
--force set the quota even if the requested value is below the tenant's current usage. Existing leases are NOT revoked; only new admissions are blocked until usage drops back under the cap. Without `--force`, the cell-scheduler refuses with HTTP 409 `quota_below_usage`
--pool <POOL> Pool name (default: "default")
[default: default]
--cert <CERT> Client cert for mTLS to scheduler
[env: GRAFOS_CERT=]
--key <KEY> Client key for mTLS to scheduler
[env: GRAFOS_KEY=]
--ca <CA> CA cert for verifying scheduler
[env: GRAFOS_CA=]
--bearer <BEARER> FabricAdmin bearer token (required under mtls+token for POST /tenants/{name}/quota)
[env: GRAFOS_BEARER=]
-h, --help Print help (see a summary with '-h')
-V, --version Print versiongrafos admin delete-tenant
Delete a tenant. Refuses with HTTP 409 if any active leases reference the tenant; revoke or wait for them to expire first. The orchestrator removes the tenant immediately; cell-side cleanup is eventual on the next long-poll cycle (~30s).
Example: grafos admin delete-tenant --scheduler https://scheduler:9200 \ --name acme
Usage: grafos admin delete-tenant [OPTIONS] --scheduler <SCHEDULER> --name <NAME>
Options: --fabric <FABRIC> Fabric address (host:port). Overrides GRAFOS_FABRIC env var
[env: GRAFOS_FABRIC=]
--scheduler <SCHEDULER> Scheduler URL (orchestrator)
[env: GRAFOS_SCHEDULER=]
--json Output in JSON format for scripting
--name <NAME> Tenant name to delete
--wide Show additional columns in table output
--yes Skip the "are you sure" prompt
--force force-delete the tenant even if cells hold active leases for it. Without `--force`, cells refuse with HTTP 409 (`tenant_has_active_leases`) and the orchestrator- driven cell delete retries every poll cycle until leases drain. With `--force`, cells release active leases server-side (admission_gate + WAL); the data-plane reclaims released allocations on the next health tick
--no-color Disable color output
--cert <CERT> [env: GRAFOS_CERT=]
--pool <POOL> Pool name (default: "default")
[default: default]
--key <KEY> [env: GRAFOS_KEY=]
--ca <CA> [env: GRAFOS_CA=]
--bearer <BEARER> FabricAdmin bearer token (required under mtls+token for DELETE /tenants/<name>)
[env: GRAFOS_BEARER=]
-h, --help Print help (see a summary with '-h')
-V, --version Print versiongrafos admin audit-query
Query the audit log. Hits the `/api/v1/admin/audit` endpoint with optional filters; works against both orchestrator and cell-scheduler endpoints (same wire shape on both).
Example: grafos admin audit-query --scheduler https://scheduler:9200 \ --kind tenant --since 1700000000 --limit 50
Usage: grafos admin audit-query [OPTIONS] --scheduler <SCHEDULER>
Options: --fabric <FABRIC> Fabric address (host:port). Overrides GRAFOS_FABRIC env var
[env: GRAFOS_FABRIC=]
--scheduler <SCHEDULER> Scheduler URL (orchestrator at:9200 or cell at:9100; both expose /api/v1/admin/audit with the same shape)
[env: GRAFOS_SCHEDULER=]
--json Output in JSON format for scripting
--kind <KIND> Filter to a specific audit kind. Repeat `--kind` to admit multiple. Valid values: `auth`, `admin`, `tenant`, `token`, `admission`, `billing`, `lease` (subset depends on which side you query)
--tenant-id <TENANT_ID> Filter to a specific tenant_id (numeric)
--wide Show additional columns in table output
--no-color Disable color output
--principal <PRINCIPAL> Filter to entries containing this substring in the `principal` field
--outcome <OUTCOME> Filter to a specific outcome string. Repeatable
--pool <POOL> Pool name (default: "default")
[default: default]
--since <SINCE> Lower bound (Unix epoch seconds, inclusive)
--until <UNTIL> Upper bound (Unix epoch seconds, inclusive)
--skip <SKIP> Skip N matching entries (pagination)
--limit <LIMIT> Cap the result count. Default 100
[default: 100]
--all when set against an orchestrator, fan out to every registered cell-scheduler and merge results newest-first. Each entry is tagged with its `source` (URL of the orchestrator or cell). The response also carries a `sources` array reporting per-source success/failure so an unreachable cell shows up explicitly instead of silently missing entries. Has no effect against a cell-scheduler directly (cells have no other cells to fan out to)
--cert <CERT> [env: GRAFOS_CERT=]
--key <KEY> [env: GRAFOS_KEY=]
--ca <CA> [env: GRAFOS_CA=]
--bearer <BEARER> FabricAdmin bearer (required under mtls+token)
[env: GRAFOS_BEARER=]
-h, --help Print help (see a summary with '-h')
-V, --version Print versiongrafos admin list-fenced-leases
list fenced leases.
Hits `/api/v1/admin/fenced-leases` against a cell-scheduler or `/api/v1/admin/fenced-leases/all` against the orchestrator (with `--all`). Each entry carries lease_id, tenant_id, node_id, resource_kind, fence_reason (fenced/failed/skipped), fenced_at, capacity, holder, and expires_at.
Usage: grafos admin list-fenced-leases [OPTIONS] --scheduler <SCHEDULER>
Options: --fabric <FABRIC> Fabric address (host:port). Overrides GRAFOS_FABRIC env var
[env: GRAFOS_FABRIC=]
--scheduler <SCHEDULER> Scheduler URL — orchestrator (with `--all`) or cell-scheduler
[env: GRAFOS_SCHEDULER=]
--all When set against an orchestrator, fan out to every registered cell-scheduler and merge results, tagging each entry with its source cell URL. Has no effect against a cell-scheduler
--json Output in JSON format for scripting
--cert <CERT> [env: GRAFOS_CERT=]
--wide Show additional columns in table output
--key <KEY> [env: GRAFOS_KEY=]
--no-color Disable color output
--ca <CA> [env: GRAFOS_CA=]
--pool <POOL> Pool name (default: "default")
[default: default]
--bearer <BEARER> [env: GRAFOS_BEARER=]
-h, --help Print help (see a summary with '-h')
-V, --version Print versiongrafos admin unfence-lease
explicitly clear a fence.
Operator action: once you've verified the holder is gone (powered down the node, manually torn down the binding), call this to release the quarantined capacity. Hits `POST /api/v1/admin/fenced-leases/<lease_id>/unfence` on the cell-scheduler that holds the fence record.
Usage: grafos admin unfence-lease [OPTIONS] --scheduler <SCHEDULER> --lease-id <LEASE_ID>
Options: --fabric <FABRIC> Fabric address (host:port). Overrides GRAFOS_FABRIC env var
[env: GRAFOS_FABRIC=]
--scheduler <SCHEDULER> Cell-scheduler URL (NOT the orchestrator — fences live on the cell that holds the lease record)
[env: GRAFOS_SCHEDULER=]
--json Output in JSON format for scripting
--lease-id <LEASE_ID> Lease id (hex, with or without `0x` prefix)
--wide Show additional columns in table output
--yes Skip the "are you sure" prompt
--cert <CERT> [env: GRAFOS_CERT=]
--no-color Disable color output
--key <KEY> [env: GRAFOS_KEY=]
--pool <POOL> Pool name (default: "default")
[default: default]
--ca <CA> [env: GRAFOS_CA=]
--bearer <BEARER> [env: GRAFOS_BEARER=]
-h, --help Print help (see a summary with '-h')
-V, --version Print versiongrafos admin image-provenance
image provenance record builder.
Produces a stable JSON record (`schema_version: 1`) describing exactly what is in a grafOS cell image: source commit, dirty-tree marker, per-binary SHA-256s, build host/user/time, and reserved fields the AWS/GCP/ Azure pipelines (and the conformance gate) flow through unchanged. The same record is meant to ride alongside the AMI / GCP custom image / Compute Gallery image so deploy/runtime can verify the binaries match what was conformance-tested.
Subcommands: build — compute the record from a binary list verify — re-hash binaries and check against a record
Usage: grafos admin image-provenance [OPTIONS] <COMMAND>
Commands: build Compute provenance for a set of binaries plus the source repo state, write a JSON record to `--output`. Run inside a build (AMI bake, GCP image build, etc.) AFTER the binaries are produced and BEFORE the image snapshot verify Re-hash binaries on disk and assert they match an existing provenance record. Refuses with non-zero exit on mismatch — used by deploy-time / runtime checks to ensure the image hasn't been swapped out from under the scheduler. Fail-closed by design help Print this message or the help of the given subcommand(s)
Options: --fabric <FABRIC> Fabric address (host:port). Overrides GRAFOS_FABRIC env var
[env: GRAFOS_FABRIC=]
--json Output in JSON format for scripting
--wide Show additional columns in table output
--no-color Disable color output
--pool <POOL> Pool name (default: "default")
[default: default]
-h, --help Print help (see a summary with '-h')
-V, --version Print versiongrafos admin image-provenance build
Compute provenance for a set of binaries plus the source repo state, write a JSON record to `--output`. Run inside a build (AMI bake, GCP image build, etc.) AFTER the binaries are produced and BEFORE the image snapshot
Usage: grafos admin image-provenance build [OPTIONS] --image-kind <IMAGE_KIND> --image-name <IMAGE_NAME>
Options: --fabric <FABRIC> Fabric address (host:port). Overrides GRAFOS_FABRIC env var [env: GRAFOS_FABRIC=] --image-kind <IMAGE_KIND> Image kind tag (`aws-ami` | `gcp-custom-image` | `azure-compute-gallery` | `lab-debian-image`). Free-form today; later 205.c slices may pin the set --image-name <IMAGE_NAME> Human-meaningful image name (e.g. the AMI name the bake script produced). Free-form. Lets later tooling correlate records with cloud-side image ids --json Output in JSON format for scripting --binary <PATH> Path to a binary that ships in the image. Repeatable. Order is preserved so consumers can pin known binaries (e.g. fabricbiosd before grafos-scheduler) --wide Show additional columns in table output --no-color Disable color output --output <OUTPUT> Output JSON path. Use `-` for stdout (handy in pipelines that pipe straight into a signing tool) [default: provenance.json] --pool <POOL> Pool name (default: "default") [default: default] --repo <REPO> Repo root to inspect for commit + dirty-tree markers. Defaults to the current directory; image-build scripts pass the path they tarred [default: .] -h, --help Print help -V, --version Print versiongrafos admin image-provenance verify
Re-hash binaries on disk and assert they match an existing provenance record. Refuses with non-zero exit on mismatch — used by deploy-time / runtime checks to ensure the image hasn't been swapped out from under the scheduler. Fail-closed by design
Usage: grafos admin image-provenance verify [OPTIONS] --input <INPUT>
Options: --fabric <FABRIC> Fabric address (host:port). Overrides GRAFOS_FABRIC env var [env: GRAFOS_FABRIC=] --input <INPUT> Path to the provenance record to verify against --binary <PATH> Optional override for binary paths. By default `verify` re-hashes the same paths the record names, which is the right thing on the build host. Pass `--binary <path>` (repeatable, order-matched against the record) when the runtime location differs from the build location --json Output in JSON format for scripting --wide Show additional columns in table output --no-color Disable color output --pool <POOL> Pool name (default: "default") [default: default] -h, --help Print help -V, --version Print version