Malachi.Cluster.ReplicaTracker (malachi v0.8.13)

View Source

Pure quorum-commit tracking for the replication of a single segment, the deterministic core of the data-plane replication mechanism, with no processes or network.

A segment is replicated across an ordered replica_set (chosen by Malachi.Cluster.Placement). The first broker is the primary: it accepts writes and forwards records to the followers. Each replica reports the highest offset it has durably stored (its match offset); this module folds those reports into the segment's commit offset: the highest offset present on a quorum (majority) of replicas, which is the point up to which the data is durable and may be acknowledged to producers.

Quorum (rather than all-replicas) means a write commits once ⌊N/2⌋+1 replicas hold it, tolerating ⌊(N-1)/2⌋ failures without blocking on the slowest replica. The primary is just a replica that is always ahead, so it needs no special-casing in the commit math.

This is the counterpart, for segment data, of the deterministic Malachi.Metadata machine for metadata: the subtle "what is committed" logic lives here, pure and property-tested, before any transport (primary → followers over distributed Erlang) is wired on top. Primary failover and replica-set changes are a later concern; this models a fixed replica set.

Summary

Types

offset()

The highest contiguous offset a replica has durably stored.

t()

Functions

ack(tracker, broker, offset)

Records that broker has durably stored up to offset. Monotonic: a stale report (lower than what was already recorded) is ignored. {:error, :unknown_replica} if broker is not in the replica set.

commit_offset(tracker)

The segment's commit offset: the highest offset stored on at least quorum/1 replicas, or :none if no offset has reached a quorum yet. This is monotonic across ack/3s.

committed?(tracker, offset)

Whether offset is committed (stored on a quorum of replicas).

new(replica_set)

Builds a tracker for a segment replicated across replica_set (the first broker is the primary). Duplicates are ignored. Raises ArgumentError on an empty/invalid replica set: Malachi.Cluster.Placement.place/3 always yields a non-empty, distinct set.

primary(replica_tracker)

The primary broker (the first of the replica set), which accepts writes.

quorum(replica_tracker)

The number of replicas that must hold an offset for it to commit (a strict majority).

Types

offset()

@type offset() :: non_neg_integer()

The highest contiguous offset a replica has durably stored.

t()

@type t() :: %Malachi.Cluster.ReplicaTracker{
  match: %{required(Malachi.Metadata.broker()) => offset()},
  replica_set: [Malachi.Metadata.broker()]
}

Functions

ack(tracker, broker, offset)

@spec ack(t(), Malachi.Metadata.broker(), offset()) ::
  {:ok, t()} | {:error, :unknown_replica}

Records that broker has durably stored up to offset. Monotonic: a stale report (lower than what was already recorded) is ignored. {:error, :unknown_replica} if broker is not in the replica set.

commit_offset(tracker)

@spec commit_offset(t()) :: offset() | :none

The segment's commit offset: the highest offset stored on at least quorum/1 replicas, or :none if no offset has reached a quorum yet. This is monotonic across ack/3s.

committed?(tracker, offset)

@spec committed?(t(), offset()) :: boolean()

Whether offset is committed (stored on a quorum of replicas).

new(replica_set)

@spec new([Malachi.Metadata.broker()]) :: t()

Builds a tracker for a segment replicated across replica_set (the first broker is the primary). Duplicates are ignored. Raises ArgumentError on an empty/invalid replica set: Malachi.Cluster.Placement.place/3 always yields a non-empty, distinct set.

primary(replica_tracker)

@spec primary(t()) :: Malachi.Metadata.broker()

The primary broker (the first of the replica set), which accepts writes.

quorum(replica_tracker)

@spec quorum(t()) :: pos_integer()

The number of replicas that must hold an offset for it to commit (a strict majority).