Skip to content

SERVER-API.md

Latest commit

 

History

History
297 lines (250 loc) · 14.3 KB

SERVER-API.md

File metadata and controls

297 lines (250 loc) · 14.3 KB
Raw

DistFS Server API Documentation (v1)

This document is the authoritative technical specification for the DistFS protocol. It provides every detail necessary for a clean-room implementation of a Server (Metadata/Data) or Client.

1. Cryptographic Foundations & Wire Formats

1.1 Primitive Standards

  • Identity/Signing: ML-DSA-65 (FIPS 204). Signature size: 2420 bytes.
  • Asymmetric Encryption: ML-KEM-768 (FIPS 203). Ciphertext size: 1088 bytes.
  • Symmetric Encryption: AES-256-GCM. 12-byte random nonces. 16-byte tags.
  • Hashing: SHA-256.

1.2 Serialization & Canonicalization

  • JSON: All JSON payloads used in signatures or encryption MUST be minified (no whitespace) and use lexicographically sorted keys (JCS).
  • Binary: Multi-byte integers are Big-Endian by default unless specified otherwise.

1.3 The "Sealing" Protocol (L7 E2EE)

Provides end-to-end encryption for sensitive Metadata API requests.

1.3.1 Standard Sealing (Asymmetric)

Used for initial login and requests where no session is established.

Inner Payload Format (Binary): [Timestamp (8b BigEndian uint64)][Signature (2420b ML-DSA-65)][Plaintext JSON]

  • Signature: Calculated over [Timestamp][Plaintext JSON].

Encrypted Wrapper (Binary): [KEM_CT (1088b)][NONCE (12b)][AES_GCM_CT + TAG]

  • KEM_CT: Encapsulates a 32-byte Shared Secret for the recipient's ML-KEM-768 public key.
  • AES_GCM_CT: The Inner Payload encrypted using the Shared Secret.

1.3.2 Symmetric Sealing (Forward Secret)

Used for requests within an established session. Provides improved performance and forward secrecy.

Encrypted Wrapper (Binary): [KEM_CT (1088b zeroed)][NONCE (12b)][AES_GCM_CT + TAG]

  • KEM_CT: MUST be zero-filled to signal symmetric mode.
  • AES_GCM_CT: The Inner Payload encrypted using the ephemeral Session Key derived during Login.

HTTP Request/Response Wrappers:

  • Request: {"uid": "user_id", "sealed": "base64_enc_wrapper"}
  • Response: {"sealed": "base64_enc_wrapper"}
  • Header: X-DistFS-Sealed: true MUST be set for all sealed requests.
  • Header: Session-Token MUST be provided for symmetric mode.

2. Exhaustive Endpoint Catalog

2.1 Public & Discovery (Unsealed)

These endpoints remain accessible on their original paths to support bootstrapping and cluster coordination.

Method Path Auth Description
GET /v1/health None Node health status.
GET /v1/meta/key None Cluster ML-KEM Public Key.
GET /v1/meta/key/sign None Cluster ML-DSA Public Key.
GET /v1/meta/key/world None Public Key for the 'world' identity.
GET /v1/node/info None Node ID and protocol version.
GET /v1/auth/config None OIDC Issuer and endpoint configuration.
GET /v1/cluster/stats None Cluster-wide storage and node metrics.
POST /v1/user/register OIDC JWT Register a new user and public keys.
POST /v1/auth/challenge None Request a login challenge for a User ID.
POST /v1/login Challenge Establish a session and establishment a shared secret.
POST /v1/user/keysync Session + E2EE Store encrypted configuration backup.
GET /v1/user/keysync Bearer JWT Retrieve encrypted configuration backup.
POST /v1/node Raft Secret Internal cluster management.

2.2 The Unified Invoke Endpoint

To prevent network analysis and preserve user privacy, all authenticated metadata and identity operations are routed through a single, non-descript endpoint.

  • Endpoint: POST /v1/invoke
  • Authentication: Required (Layer 7 E2EE + Session Token)
  • Request Format: SealedEnvelope (Decrypted inner payload)
  • Response Format: SealedResponse (Encrypted)

2.2.1 Sealed Envelope Schema

{
  "action": "ActionName",
  "payload": { ... }
}

2.2.2 Supported Actions

Action Description
GetInode Fetch metadata for a specific inode. Payload: {"id": "hex"}.
GetInodes Fetch metadata for multiple inodes. Payload: InodeIDBatchRequest.
IssueToken Request a storage access token. Payload: TokenRequest.
Batch Apply a sequence of Raft commands atomically. Payload: []LogCommand.
AllocateChunk Request storage nodes for a new chunk. Payload: {}.
AcquireLeases Acquire distributed leases. Payload: AcquireLeaseRequest.
ReleaseLeases Release distributed leases. Payload: ReleaseLeaseRequest.
GetUser Fetch user profile and usage. Payload: {"id": "hex"}.
ListGroups List groups the user belongs to. Payload: {}.
GetGroup Fetch group metadata and registry. Payload: {"id": "hex"}.
AllocateGID Allocate a new Global Group ID. Payload: {}.
GetWorldPrivate Fetch the private key for the "world" group. Payload: {}.
AdminUsers List all users (Admin only). Payload: {}.
AdminGroups List all groups (Admin only). Payload: {"cursor": "...", "limit": int}.
AdminLeases List all active leases (Admin only). Payload: {}.
AdminNodes List all storage nodes (Admin only). Payload: {}.
AdminStatus Get cluster health and stats (Admin only). Payload: {}.
AdminPromote Promote a user to Admin (Admin only). Payload: {"user_id": "hex"}.
AdminUserLock Lock/Unlock a user account (Admin only). Payload: AdminSetUserLockRequest.
AdminUserQuota Set user storage quota (Admin only). Payload: SetUserQuotaRequest.
AdminGroupQuota Set group storage quota (Admin only). Payload: SetGroupQuotaRequest.
AdminClusterJoin Join a new node to the cluster (Admin only). Payload: {"address": "url"}.
AdminClusterRemove Remove a node from the cluster (Admin only). Payload: {"id": "hex"}.
AdminAudit Stream cluster audit logs (Admin only). Payload: {}.

2.3 Data Node API

Method Path Auth Description
GET /v1/data/{id} Signed Token Download encrypted chunk.
PUT /v1/data/{id} Signed Token Upload encrypted chunk. Optional Query: replicas=csv_node_addresses.
DELETE /v1/data/{id} Signed Token Permanently remove chunk.
POST /v1/data/{id}/replicate Signed Token Trigger P2P replication to target nodes.

Consistency Guarantees:

  • Metadata: Linearizable (Raft).
  • Data: Sub-Quorum Persistent ($R=3, W=2$). A PUT returns 201 Created once the local write and at least one remote replica succeed. Remaining replication completes in the background.
  • Write-Through: Metadata updates MUST only be submitted AFTER the Data sub-quorum is reached.

3. Core Object Schemas

3.1 Inode

{
  "id": "32_char_hex",
  "links": {"parentID:nameHMAC": true},
  "type": 0, // 0:File, 1:Dir, 2:Symlink
  "owner_id": "user_id",
  "group_id": "group_id",
  "mode": 420, // octal 0644
  "size": 1024,
  "ctime": 1772517680144,
  "nlink": 1,
  "client_blob": "base64_enc_metadata",
  "children": {"nameHMAC": {"id": "childID", "enc_name": "base64", "nonce": "base64"}},
  "manifest": [{"id": "chunk_hash", "nodes": ["node_id"]}],
  "chunk_pages": ["page_id"],
  "lockbox": {"recipient_id": {"kem": "base64", "dem": "base64"}},
  "version": 5,
  "is_system": false,
  "leases": {"nonce": {"id": "session_id", "exp": 123, "type": 1}},
  "unlinked": false,
  "user_sig": "base64_sig",
  "group_sig": "base64_sig",
  "owner_delegation_sig": "base64_sig",
  "cluster_sig": "base64_sig"
}

3.2 client_blob (Internal Metadata)

Clients MUST use this schema inside the encrypted client_blob:

{
  "symlink_target": "path",
  "inline_data": "base64",
  "mtime": int64_ns,
  "uid": uint32,
  "gid": uint32
}

3.3 User

{
  "id": "derived_user_id",
  "uid": 1001,
  "sign_key": "base64_pub_dsa",
  "enc_key": "base64_pub_kem",
  "usage": {"inodes": 10, "bytes": 1048576},
  "quota": {"max_inodes": 1000, "max_bytes": 1073741824}
}

3.4 Group

{
  "id": "group_id",
  "gid": 5001,
  "owner_id": "owner_user_id",
  "members": {"user_id": true},
  "enc_key": "base64_pub_kem",
  "sign_key": "base64_pub_dsa",
  "lockbox": {"recipient_id": {"kem": "base64", "dem": "base64"}},
  "version": 1,
  "signer_id": "user_id",
  "signature": "base64_sig",
  "quota_enabled": true
}

4. Manifest Integrity (Hashing Algorithms)

All multi-byte integers in hashes are Big-Endian.

4.1 Inode Hash

The user_sig is over the SHA-256 hash of these fields concatenated exactly:

  1. []byte("id:" + id + "|")
  2. []byte("v:") + BigEndian(uint64(version)) + []byte("|")
  3. []byte("mode:") + BigEndian(uint32(mode)) + []byte("|")
  4. []byte("gid_str:" + group_id + "|")
  5. []byte("sys:" + (is_system ? "1" : "0") + "|")
  6. []byte("client_blob:") + raw_encrypted_bytes + []byte("|")
  7. []byte("owner:" + owner_id + "|")
  8. []byte("type:") + BigEndian(uint32(type)) + []byte("|")
  9. []byte("links:") + SortedCSV(parentID:nameHMAC) + []byte("|")
  10. []byte("children:") + SortedCSV(nameHMAC:childID,hex_enc_name,hex_nonce|) // Each entry is separated by |
  11. []byte("manifest:") + CSV(chunk_id) + []byte("|") // Nodes are EXCLUDED from user-signed content hash.
  12. []byte("pages:") + SortedCSV(chunk_page_ids) + []byte("|")
  13. []byte("lockbox:") + SortedCSV(id:kem+dem) + []byte("|")

4.2 Group Hash

The group signature is over the SHA-256 hash of these fields concatenated exactly:

  1. []byte("DistFS-Group-v2|")
  2. []byte("group-id:" + id + "|")
  3. []byte("v:") + BigEndian(uint64(version)) + []byte("|")
  4. []byte("nonce:") + nonce_bytes + []byte("|")
  5. []byte("sys:" + (is_system ? "1" : "0") + "|")
  6. []byte("epoch:") + BigEndian(uint32(epoch)) + []byte("|")
  7. []byte("enc_epoch_seed:") + enc_epoch_seed_bytes + []byte("|")
  8. []byte("client_blob:") + client_blob_bytes + []byte("|")
  9. []byte("owner:" + owner_id + "|")
  10. []byte("signer:" + signer_id + "|")
  11. []byte("anonymous_lockbox:") + CSV(ciphertext) + []byte("|")
  12. []byte("anonymous_registry:") + anonymous_registry_bytes + []byte("|")
  13. []byte("enc_key:") + enc_key_bytes + []byte("|")
  14. []byte("sign_key:") + sign_key_bytes + []byte("|")
  15. []byte("hsk:") + SortedCSV(epoch:sign_key) + []byte("|")
  16. []byte("enc_sign_key:") + enc_sign_key_bytes + []byte("|")
  17. []byte("lockbox:") + SortedCSV(id:kem+dem) + []byte("|")
  18. []byte("registry_lockbox:") + SortedCSV(id:kem+dem) + []byte("|")
  19. []byte("enc_registry:") + enc_registry_bytes + []byte("|")
  20. []byte("quota_enabled:" + (quota_enabled ? "1" : "0") + "|")

5. Zero-Knowledge Resolution

To hide filenames from the server, all map keys in children and links are HMACs: nameHMAC = Hex(HMAC-SHA256(Parent_FileKey, plaintext_filename))

6. Coordination and Concurrency

6.1 Leases (Action AcquireLeases)

  • Request: {"inode_ids": ["..."], "type": 1, "duration": ns, "nonce": "..."}
  • Types: 0: Shared, 1: Exclusive.
  • Rule: Mutations are rejected if an Exclusive lease is held by a different session.

6.2 Storage Tokens (Action IssueToken)

  • Request: {"inode_id": "...", "chunks": ["hash"], "mode": "R|W|D"}
  • Response: {"payload": "base64_minified_json", "sig": "base64_cluster_sig"}
  • Capability Schema: {"chunks": ["hash"], "mode": "RWD", "exp": unix_ts}

7. Error Code Registry

The DistFS API maps FSM sentinel errors to structured JSON HTTP responses (APIErrorResponse). The following table defines the authoritative conditions under which each error MUST be triggered.

Code HTTP FSM Sentinel Intended Trigger Condition
DISTFS_NOT_FOUND 404 ErrNotFound The requested Inode, User, or Group ID does not exist in the BoltDB state.
DISTFS_EXISTS 409 ErrExists Attempting to create an Inode, User, or Group ID that already exists.
DISTFS_VERSION_CONFLICT 409 ErrConflict Optimistic concurrency failure: the Version field in an update command does not strictly equal the current Version in the BoltDB state + 1.
DISTFS_LEASE_REQUIRED 409 ErrLeaseRequired A mutation (update/delete) was attempted on an Inode, but the request's sessionID does not hold a valid, unexpired LeaseExclusive for that Inode.
DISTFS_QUOTA_EXCEEDED 403 ErrQuotaExceeded The operation would cause the debtor's (User or Group) Usage to strictly exceed their Quota. Note: Groups with quota_enabled: true are always the debtors for their files.
DISTFS_QUOTA_DISABLED 403 ErrQuotaDisabled Attempted to set a quota on a group that was created with quota_enabled: false.
DISTFS_UNAUTHORIZED 401 (Auth Middleware) The client failed to provide a valid signature, a valid session JWT, or the OIDC token is invalid/expired.
DISTFS_FORBIDDEN 403 (ACL Checks) The authenticated user is not the owner of the Inode/Group, is not an Admin, or is attempting a restricted action (e.g. promoting an admin).
DISTFS_NOT_LEADER 503 raft.ErrNotLeader The HTTP request was routed to a node that is not the current Raft leader. Clients MUST retry against a different node.
DISTFS_STRUCTURAL_INCONSISTENCY 409 ErrStructuralInconsistency An atomic batch mutation would result in a corrupted filesystem graph. Examples: a parent directory claims a child, but the child does not have a reciprocal links entry; an Inode is deleted but still has an active lease.
DISTFS_ATOMIC_ROLLBACK 500 / 409 ErrAtomicRollback An explicitly marked atomic CmdBatch encountered a failure in one of its sub-commands (e.g., a version conflict). The FSM MUST trigger a BoltDB tx.Rollback() and return this error wrapping the original sub-command failure.
DISTFS_INTERNAL_ERROR 500 (Various) Unexpected errors, panics, cryptographic seal verification failures, or disk I/O errors.

8. Header Behaviors

Header Usage Effect
Session-Token Mandatory for auth routes Identifies the session and lease-owner.
X-DistFS-Sealed Mandatory for mutations Signals request body is a SealedRequest.
X-DistFS-Admin-Bypass Optional (Admins only) Bypasses ownership checks for recovery.
X-DistFS-Sealed Optional for GET If true, server returns a SealedResponse.