SegmentTransferService
Inter-node bulk segment transfer: the transport for replication repair, node resync, and (per shard) migration. The source streams a segment's pieces to the target, which verifies each piece's checksum, decodes it, and writes it to local storage. BitTorrent-inspired: a target that has received pieces can in turn serve them to other targets, turning linear transfer into O(log N) fanout. This is an internal cluster RPC (node-to-node), not a client-facing API.
Proto source
Methods
TransferPieces
Push one segment to this node. The caller (source) opens the stream with a header frame carrying the segment id and its transfer manifest, then sends one piece frame per piece in index order. The callee (target) verifies each wire piece against the manifest's per-piece checksum, decodes it, assembles the segment, verifies the whole-segment checksum, and replies once with the outcome. A checksum failure aborts the transfer (corrupt data is never written).
Request: stream PieceData
| Field | Type | Description |
|---|---|---|
header | SegmentTransferHeader | First frame: segment identity and transfer manifest. Exactly one per stream, before any piece frame. |
piece | PieceFrame | A single wire (encoded) piece. Sent in ascending index order after the header. |
Response: TransferAck
| Field | Type | Description |
|---|---|---|
ok | bool | True when every piece verified, decoded, and the whole-segment checksum matched; the segment is durably written. False on any failure. |
error | string | Human-readable failure cause when ok is false (checksum mismatch, decode error, short stream, local I/O). Empty when ok is true. |
pieces_received | uint32 | Number of pieces the target received and verified before replying. |
GetSegmentManifest
Receiver-driven swarm pull. A node that needs a segment (replication repair, resync, the target of a migration) asks a peer holding it for the transfer manifest plus the set of pieces that peer can serve, then pulls individual pieces by index. Issued in parallel against several peers, this drives the rarest-first, multi-source download (every receiver also becomes a source). All sources produce byte-identical pieces for the same descriptor + split parameters (export is a deterministic key-range scan), so per-piece hashes match across peers and pieces fetched from different sources interleave.
Request: SegmentManifestRequest
| Field | Type | Description |
|---|---|---|
segment | SegmentDescriptorRef | The segment + split parameters to build the manifest for. |
Response: SegmentManifestReply
| Field | Type | Description |
|---|---|---|
manifest | SegmentTransferHeader | The transfer manifest (per-piece + whole-segment checksums, encoding, sizes) — same shape the streaming push sends as its leading header. |
piece_bitfield | bytes | Bitfield over the segment's pieces, little-endian bit order: bit i set means this peer can serve piece i. A peer holding the whole segment sets every bit. |
GetPiece
Fetch one wire (encoded) piece by index for the receiver-driven pull above.
Request: PieceRequest
| Field | Type | Description |
|---|---|---|
segment | SegmentDescriptorRef | The segment + split parameters; must match the manifest request so the served piece's hash matches the manifest entry. |
index | uint32 | Zero-based piece index to fetch. |
Response: PieceFrame
| Field | Type | Description |
|---|---|---|
index | uint32 | Piece index (0-based). Frames arrive in ascending order. |
wire | bytes | Encoded piece bytes, hashed (over these wire bytes) by the matching piece_hashes entry in the header. |
Types
PieceData
One frame of a transfer stream: the leading header, then one frame per piece.
| Field | Type | Description |
|---|---|---|
header | SegmentTransferHeader | First frame: segment identity and transfer manifest. Exactly one per stream, before any piece frame. |
piece | PieceFrame | A single wire (encoded) piece. Sent in ascending index order after the header. |
PieceFrame
A single encoded piece on the wire.
| Field | Type | Description |
|---|---|---|
index | uint32 | Piece index (0-based). Frames arrive in ascending order. |
wire | bytes | Encoded piece bytes, hashed (over these wire bytes) by the matching piece_hashes entry in the header. |
PieceRequest
Request one piece of a segment from a peer.
| Field | Type | Description |
|---|---|---|
segment | SegmentDescriptorRef | The segment + split parameters; must match the manifest request so the served piece's hash matches the manifest entry. |
index | uint32 | Zero-based piece index to fetch. |
SegmentDescriptorRef
Addresses a segment to serve by its placement-descriptor coordinates plus the split parameters every source must agree on. The coordinates let a peer build the segment from local storage on demand; the split parameters make every peer's pieces byte-identical so their checksums match in a multi-source pull.
| Field | Type | Description |
|---|---|---|
segment_id | uint64 | Placement-layer segment id (SegmentDescriptor.id). |
partition | uint32 | Partition wire tag the segment's entries live in. |
range_start | bytes | Half-open key range [range_start, range_end) the segment covers. |
range_end | bytes | Exclusive upper bound; empty means unbounded above (to end of partition). |
piece_size | uint32 | Raw (pre-encoding) bytes per piece, last piece excepted. Every source must use the same value or piece boundaries (and thus hashes) diverge. |
encoding | uint32 | Wire encoding applied to each piece, as the swarm PieceEncoding discriminant: 0 = none, 1 = lz4, 2 = zstd. |
zstd_level | uint32 | zstd effort when encoding = 2, as the ZstdLevel discriminant (0 = fastest, 1 = default, 2 = better). Ignored otherwise. |
SegmentManifestReply
A peer's manifest for a segment plus which of its pieces it can serve.
| Field | Type | Description |
|---|---|---|
manifest | SegmentTransferHeader | The transfer manifest (per-piece + whole-segment checksums, encoding, sizes) — same shape the streaming push sends as its leading header. |
piece_bitfield | bytes | Bitfield over the segment's pieces, little-endian bit order: bit i set means this peer can serve piece i. A peer holding the whole segment sets every bit. |
SegmentManifestRequest
Request a segment's transfer manifest from a peer.
| Field | Type | Description |
|---|---|---|
segment | SegmentDescriptorRef | The segment + split parameters to build the manifest for. |
SegmentTransferHeader
Segment identity plus the transfer manifest the target needs to verify and decode the incoming pieces. Carries only what the target cannot compute itself: the per-piece and whole-segment checksums and the wire encoding.
| Field | Type | Description |
|---|---|---|
segment_id | uint64 | Placement-layer segment id (SegmentDescriptor.id) being transferred. |
encoding | uint32 | Wire encoding applied to every piece, as the swarm PieceEncoding discriminant: 0 = none (raw), 1 = lz4, 2 = zstd. |
zstd_level | uint32 | zstd effort when encoding = 2 (zstd), as the ZstdLevel discriminant: 0 = fastest, 1 = default, 2 = better. Ignored for other encodings. The target does not need the level to decode (the zstd frame is self-describing) but it is carried for observability and parity with the source's manifest. |
piece_hashes | uint64[] | xxh3-64 of each wire (encoded) piece, in index order. The piece count is this list's length; the target verifies each arriving piece against its entry before decoding. |
total_hash | uint64 | xxh3-64 of the whole raw (decoded) segment, verified after assembly. |
piece_size | uint32 | Raw (pre-encoding) size of each piece in bytes, last piece excepted. Carried so the target can reconstruct the full manifest (size-hinting the output); not required to verify or decode. |
total_len | uint64 | Total raw segment length in bytes. Used by the target to pre-size the output sink; not required to verify or decode. |
TransferAck
The target's one-shot reply after the stream completes.
| Field | Type | Description |
|---|---|---|
ok | bool | True when every piece verified, decoded, and the whole-segment checksum matched; the segment is durably written. False on any failure. |
error | string | Human-readable failure cause when ok is false (checksum mismatch, decode error, short stream, local I/O). Empty when ok is true. |
pieces_received | uint32 | Number of pieces the target received and verified before replying. |