Creating a cluster
Spread objects across multiple machines for durability — survive drive and node loss with erasure coding.
The shape of a cluster
A KerPlace cluster has two roles:
- Drive nodes store raw shards and serve simple positioned reads and writes. They never see plaintext and never hold keys.
- A gateway speaks S3 to clients and does the work — encryption, erasure coding, hashing — spreading each object across the drive nodes.
Durability through erasure coding
Each object is split into K data shards plus M parity shards across the nodes, so the object survives losing any M of them. Every shard carries a checksum, so silent corruption (“bitrot”) is detected and repaired from the surviving shards. This is more space-efficient than keeping full copies.
Bringing it up
Start the drive nodes (each storing one shard slot, internal RPC only), then start the gateway pointing at them. Clients talk only to the gateway, exactly as they would to a single server — the distribution is invisible to them.
Securing the internal mesh
The gateway↔node traffic rides an encrypted overlay so the shards never cross the network in the clear. A production runbook covers joining nodes, locking down the internal ports, and systemd units.
Full detail, source & scripts in the repository → https://github.com/agalletero/kerplace/blob/main/docs/CLUSTERING.md