Many concurrency designs look unrelated at first. A map protected by locks, a Kafka consumer group, a worker pool and a database shard appear to solve different problems. Underneath, they are combinations of the same few decisions.
The most useful correction to my original mental model was this:
If exactly one worker owns a shard, that shard does not need a mutex.
The word owns matters. A shard does not require a lock merely because it contains mutable state. It requires protection when multiple goroutines can access that state concurrently. If every operation for the shard is routed to one goroutine and no other goroutine touches its map, access is already serialized by ownership.
I now start with two questions:
1. How is work distributed?2. Who owns or protects the state?A third question completes the model:
3. Is the scheduling fixed or dynamic?These questions lead to five patterns:
- A shared sharded map protected by per-shard locks.
- Keyed channels with worker-owned state.
- Static slice-range partitioning.
- A shared queue consumed by a worker pool.
- Worker-local state followed by a merge.
They are useful inside one Go process. They also form a path toward database sharding, Kafka partition ownership, consistent hashing, task queues and MapReduce.
Before the patterns: routing, ownership and scheduling
It helps to keep three responsibilities separate.
Routing decides where work goes
A router can use a key:
shardID := hash(key) % shardCountIt can use a fixed index range:
start := len(items) * workerID / workerCountend := len(items) * (workerID + 1) / workerCountOr it can avoid choosing a worker and place the next job on a shared queue. The first available worker takes it.
Ownership decides how state stays safe
There are several valid ownership models:
Shared state → protect it with a lock or atomic operationSingle owner → route all operations through that ownerDisjoint state → give every worker a non-overlapping sectionLocal state → let workers build private results, then mergeImmutable state → publish a new snapshot instead of mutating itThe synchronization mechanism follows from the ownership model. Choosing a mutex, channel or atomic operation before deciding ownership usually makes the design harder to reason about.
Scheduling decides who performs the next unit of work
Static scheduling assigns work ahead of time. It has almost no coordination overhead, but a slow range can delay the whole batch.
Dynamic scheduling lets workers take work as they become free. It adapts to uneven job duration, but needs a queue and usually gives up deterministic result order.
With those distinctions in place, the five patterns become easier to derive.
Pattern 1: shared sharded map with per-shard locks
Start with a concurrent counter. One global map and one global mutex are correct:
type Counter struct { mu sync.RWMutex values map[string]uint64}The limitation is contention. An update for tenant-a blocks an unrelated update for tenant-z because both operations need the same lock.
Sharding divides the map into independent lock domains:
many goroutines │ ▼ hash(key) % N │ ├── shard 0: mutex + map ├── shard 1: mutex + map └── shard N: mutex + mapAny goroutine may access any shard, so each shard still needs a mutex.
Go implementation
package counter import ( "hash/fnv" "sync") type counterShard struct { mu sync.RWMutex values map[string]uint64} type ShardedCounter struct { shards []counterShard} func NewShardedCounter(shardCount int) *ShardedCounter { if shardCount < 1 { panic("shard count must be positive") } c := &ShardedCounter{ shards: make([]counterShard, shardCount), } for i := range c.shards { c.shards[i].values = make(map[string]uint64) } return c} func hashKey(key string) uint32 { h := fnv.New32a() _, _ = h.Write([]byte(key)) return h.Sum32()} func (c *ShardedCounter) shardFor(key string) *counterShard { index := hashKey(key) % uint32(len(c.shards)) return &c.shards[index]} func (c *ShardedCounter) Add(key string, delta uint64) uint64 { shard := c.shardFor(key) shard.mu.Lock() defer shard.mu.Unlock() shard.values[key] += delta return shard.values[key]} func (c *ShardedCounter) Get(key string) uint64 { shard := c.shardFor(key) shard.mu.RLock() defer shard.mu.RUnlock() return shard.values[key]}The public operation remains synchronous. Add returns only after the value has changed. Get reads the latest value protected by the same shard lock.
What this pattern guarantees
- Operations on a single key are serialized by its shard lock.
- Operations on different shards can proceed concurrently.
- Every map access is protected.
- The caller gets an immediate result.
The number of shards is a trade-off. Too few shards leave contention. Too many increase bookkeeping and do not help a hot key because one key always maps to one shard.
An RWMutex is not automatically faster than a Mutex. It helps only when the workload has enough overlapping reads to repay its extra coordination cost. The choice should come from measurement.
Multi-shard operations
Suppose an operation transfers a value between two keys. It may need two shard locks. If different goroutines take those locks in different orders, they can deadlock.
The normal fix is a deterministic lock order:
first, second := sourceShard, destinationShardif sourceIndex > destinationIndex { first, second = second, first} first.mu.Lock()second.mu.Lock()defer second.mu.Unlock()defer first.mu.Unlock()If both keys land on the same shard, it must be locked only once. Cross-shard transactions are where an apparently simple design begins to acquire distributed-database-like complexity.
How database sharding grows from this pattern
Replace each in-memory shard with a database server:
application goroutines application instances │ │ ▼ ▼ hash(key) % N shard router / metadata │ │ ▼ ▼ mutex + local map database shard + local locksThe routing idea is the same. A shard key chooses an ownership domain. Inside that database shard, transactions, row locks or a single leader protect concurrent changes.
The local mutex does not literally become a network-wide mutex. The analogy is about the boundary: operations inside one shard are coordinated locally, while operations across shards require an additional protocol.
Two common database routing strategies also follow from the router:
Hashed sharding distributes keys using a hash. It tends to spread load evenly, but nearby keys no longer live together. A range query may have to contact many shards.
Range sharding assigns contiguous key ranges. It supports targeted range queries and locality, but a monotonically increasing key can direct all new writes to the last range and create a hotspot.
Use this pattern when the API must be synchronous, callers can access arbitrary keys and measured contention makes one global lock too expensive.
Pattern 2: keyed channels with worker-owned state
The second pattern changes ownership. Instead of allowing every goroutine to touch every shard, route each key to exactly one worker.
producers │ ▼hash(key) % N │ ├── channel 0 → worker 0 → private map 0 ├── channel 1 → worker 1 → private map 1 └── channel N → worker N → private map NOnly the owning worker accesses its map. The map does not need a mutex.
This is no map mutex, not no synchronization. Sending and receiving through a channel coordinates the producers with the owner. The runtime also synchronizes access to the channel itself.
Go implementation
package counter import ( "context" "errors") var ErrStopped = errors.New("counter workers stopped") type addRequest struct { key string delta int64 reply chan int64} type OwnedCounters struct { inputs []chan addRequest done <-chan struct{}} func NewOwnedCounters(ctx context.Context, workerCount, queueSize int) *OwnedCounters { if workerCount < 1 { panic("worker count must be positive") } c := &OwnedCounters{ inputs: make([]chan addRequest, workerCount), done: ctx.Done(), } for i := range c.inputs { input := make(chan addRequest, queueSize) c.inputs[i] = input go runCounterOwner(ctx, input) } return c} func runCounterOwner(ctx context.Context, input <-chan addRequest) { state := make(map[string]int64) // This worker is the only owner. for { select { case <-ctx.Done(): return case request := <-input: state[request.key] += request.delta value := state[request.key] select { case request.reply <- value: case <-ctx.Done(): return } } }} func (c *OwnedCounters) Add( ctx context.Context, key string, delta int64,) (int64, error) { workerID := hashKey(key) % uint32(len(c.inputs)) reply := make(chan int64, 1) request := addRequest{key: key, delta: delta, reply: reply} select { case c.inputs[workerID] <- request: case <-ctx.Done(): return 0, ctx.Err() case <-c.done: return 0, ErrStopped } select { case value := <-reply: return value, nil case <-ctx.Done(): return 0, ctx.Err() case <-c.done: return 0, ErrStopped }}The state map is created inside runCounterOwner, which makes accidental access from another goroutine less likely. Every request for the same key uses the same hash and reaches the same channel.
The reply channel is buffered so the owner does not become permanently blocked if the caller cancels after submitting the request. For fire-and-forget aggregation, the reply can be removed.
What this pattern guarantees
- A key has one owner while the assignment is stable.
- Events for that key enter one channel and are processed sequentially.
- The private map needs no mutex.
- A bounded channel provides backpressure when the owner cannot keep up.
The queue does not make overload disappear. A hot key can saturate its owner while other workers remain idle. Increasing the channel buffer only postpones the point at which producers slow down. It also increases memory use and the amount of queued work lost during an ungraceful shutdown.
This model is a natural fit for continuous asynchronous streams, aggregation, deduplication and per-key state machines. It is less natural when every operation needs a low-latency synchronous response. The request-and-reply example supports that API, but each operation now takes a queue round trip.
How Kafka partition ownership grows from this pattern
The structure of a traditional Kafka consumer group is similar:
record key │ ▼partitioner │ ▼partition log → one assigned consumer in the group → local stateRecords with the same key are normally routed to the same partition when the producer uses keyed partitioning. Kafka preserves order within a partition, not across the whole topic. A traditional consumer group assigns a partition exclusively to one consumer at a time, which lets that consumer process partition state without a cross-consumer mutex.
The distributed version adds concerns that the in-process channel example does not have:
- The log persists messages instead of keeping them only in memory.
- Offsets record processing progress.
- A rebalance transfers partition ownership when consumers join or leave.
- Replicas protect partition data from broker failure.
- Delivery and commit semantics determine what happens after a crash.
Ownership transfer is the hard part. The old owner must stop changing state before the new owner starts from the handoff point. In a stateful stream processor, local state may need to be restored from a changelog before processing resumes.
Current Kafka also has share groups, where records from a partition can be processed by multiple consumers under a different acknowledgement model. That design is closer to the shared worker queue in pattern 4. It should not be confused with exclusive partition ownership in a traditional consumer group.
Actors and event loops are the same ownership decision
An actor owns private state and processes one mailbox message at a time. A UI event loop owns UI state and applies events serially. A Redis event loop owns much of the command execution path. The implementation details differ, but the core decision is identical:
Do not share mutable state.Share requests that are routed to the state owner.Use this pattern when the input is continuous, per-key ordering matters and asynchronous processing is acceptable.
Pattern 3: static slice-range partitioning
Some work does not need keyed routing at all. If the input is a known finite slice, divide its index range before starting the workers.
worker 0 → items [0:1000]worker 1 → items [1000:2000]worker 2 → items [2000:3000]Each worker reads a disjoint part of the input and writes to the matching disjoint part of the output. No queue is needed. No mutex protects the item data because the ranges do not overlap.
Go implementation
package batch import ( "crypto/sha256" "sync") func HashAll(items [][]byte, workerCount int) [][32]byte { results := make([][32]byte, len(items)) if len(items) == 0 { return results } if workerCount < 1 { workerCount = 1 } if workerCount > len(items) { workerCount = len(items) } var workers sync.WaitGroup workers.Add(workerCount) for workerID := 0; workerID < workerCount; workerID++ { start := len(items) * workerID / workerCount end := len(items) * (workerID + 1) / workerCount go func(start, end int) { defer workers.Done() for index := start; index < end; index++ { results[index] = sha256.Sum256(items[index]) } }(start, end) } workers.Wait() return results}Every worker writes different results indexes. The WaitGroup still matters: the caller must wait until all writes are complete before reading the result slice.
This works well when each item has roughly equal processing cost. Hashing fixed-size blocks, encoding equal-sized records and applying the same image transform to equal-sized tiles are good examples.
The straggler problem
Suppose worker 0 receives ten expensive items while every other range contains cheap items. The fast workers finish, but the whole batch still waits for worker 0.
worker 0: ████████████████████worker 1: ███worker 2: ████worker 3: ██ ↑ batch completes hereStatic ranges optimize for low coordination and locality, not load balancing. Smaller ranges can reduce skew, but once workers dynamically claim those ranges the design is moving toward a worker queue.
How range partitioning grows into database ranges
Static slice ranges and database range sharding share the idea of non-overlapping ownership:
local indexes database keys[0, 1000) → worker 0 [A, G) → shard 0[1000, 2000)→ worker 1 [G, P) → shard 1[2000, 3000)→ worker 2 [P, Z) → shard 2The database version needs metadata to locate each range. It also needs range splitting, migration, replication and failure handling. A distributed query may contact several owners and merge their responses.
Range ownership preserves locality. A query such as created_at between X and Y can target the relevant ranges. The downside is skew: popular ranges or steadily increasing keys can receive disproportionate traffic.
Use static partitioning for a finite input when work is predictable and each worker can touch a disjoint output region.
Pattern 4: shared worker queue
Static ranges perform poorly when job durations vary. A shared queue changes the scheduling rule:
┌── worker 0one job channel ──┼── worker 1 └── worker 2Workers do not own predetermined ranges. Each worker takes the next available job. A slow worker holds only its current job while faster workers continue draining the queue.
Go implementation
package pool import ( "context" "sync") type Job struct { ID string Run func(context.Context) error} type Result struct { JobID string WorkerID int Err error} func Run( ctx context.Context, workerCount int, jobs <-chan Job,) <-chan Result { results := make(chan Result) var workers sync.WaitGroup if workerCount < 1 { workerCount = 1 } workers.Add(workerCount) for workerID := 0; workerID < workerCount; workerID++ { go func(workerID int) { defer workers.Done() for { select { case <-ctx.Done(): return case job, ok := <-jobs: if !ok { return } result := Result{ JobID: job.ID, WorkerID: workerID, Err: job.Run(ctx), } select { case results <- result: case <-ctx.Done(): return } } } }(workerID) } go func() { workers.Wait() close(results) }() return results}The producer owns the job channel lifecycle and closes it after sending all jobs:
jobs := make(chan pool.Job, 64)results := pool.Run(ctx, 8, jobs) go func() { defer close(jobs) for _, file := range files { file := file jobs <- pool.Job{ ID: file.Name, Run: func(ctx context.Context) error { return processFile(ctx, file) }, } }}() for result := range results { if result.Err != nil { log.Printf("%s failed: %v", result.JobID, result.Err) }}This is a scheduling pattern, not a state-ownership pattern by itself. Jobs should be independent or should use a separate mechanism to protect shared state.
What this pattern gives up
It does not preserve input order. Job 10 may finish before job 2. It does not preserve per-key order either. Two events for one account can run concurrently on different workers.
If the caller needs output in input order, include a sequence number and reorder the completed results. If per-key ordering matters, route that key to an owner as in pattern 2.
A bounded job channel creates backpressure. When all workers and queue slots are busy, producers wait instead of allowing memory usage to grow without limit. Production queues also need decisions for timeouts, retry limits, idempotency and poison jobs.
How task queues grow from this pattern
Replace the Go channel with a durable broker:
Go worker pool distributed task queue------------- ----------------------buffered channel durable queuegoroutines worker processeschannel receive message claim or leaseresult channel acknowledgementcontext cancellation lease timeout / redeliveryThe broker adds persistence and failure recovery. If a worker dies before acknowledging a job, another worker can claim it. This normally creates at-least-once delivery, so the handler should tolerate retries.
HTTP crawling, thumbnail generation, email delivery and transaction verification with uneven costs all fit this model.
This is also the right comparison for Kafka share groups or queue-like broker modes where multiple consumers can process records originating from the same partition. It is not the right model for a traditional consumer group that gives one consumer exclusive partition ownership.
Use a shared queue for independent jobs when work duration varies or input arrives continuously without a per-key ordering requirement.
Pattern 5: worker-local state followed by a merge
The fifth pattern avoids shared writes during processing. Each worker accumulates into private state. After every worker finishes, one goroutine combines the partial results.
worker 0 → local map 0 ─┐worker 1 → local map 1 ─┼→ merge after Waitworker 2 → local map 2 ─┘This is especially effective for counts, histograms, summaries and other reductions.
Go implementation
package words import ( "strings" "sync") func Count(lines []string, workerCount int) map[string]int { if workerCount < 1 { workerCount = 1 } if workerCount > len(lines) && len(lines) > 0 { workerCount = len(lines) } if len(lines) == 0 { return map[string]int{} } partials := make([]map[string]int, workerCount) var workers sync.WaitGroup workers.Add(workerCount) for workerID := 0; workerID < workerCount; workerID++ { start := len(lines) * workerID / workerCount end := len(lines) * (workerID + 1) / workerCount go func(workerID, start, end int) { defer workers.Done() local := make(map[string]int) for _, line := range lines[start:end] { for _, word := range strings.Fields(line) { local[word]++ } } partials[workerID] = local }(workerID, start, end) } workers.Wait() total := make(map[string]int) for _, partial := range partials { for word, count := range partial { total[word] += count } } return total}Each worker owns its local map and one slot in partials. The merge begins only after workers.Wait() returns. There is no mutex on the hot counting path.
The cost is extra memory: the same key may exist in every partial map. The final merge can also become a serial bottleneck. A tree reduction can parallelize the merge when partial results are large.
This pattern works best when the combine operation is associative:
(a + b) + c = a + (b + c)Commutativity is also useful because workers may finish in any order:
a + b = b + aInteger addition, minimum, maximum, set union and many histogram merges have these properties. Operations with order-dependent side effects do not.
How MapReduce grows from this pattern
The local word counter is a small MapReduce job:
input splits │ ▼map workers create local (word, count) pairs │ ▼optional local combine │ ▼shuffle routes the same word to the same reducer │ ▼reducers merge counts by wordThe map phase is worker-local accumulation. The shuffle phase uses the keyed-routing idea from pattern 2. Each reducer becomes the owner for a key range. The reduce phase merges values using an associative operation.
The distributed runtime adds input placement, retry after worker failure, transfer of intermediate data and scheduling near the data. The underlying ownership choices remain familiar.
Database query engines use the same shape for parallel aggregation. Each partition calculates a partial COUNT, SUM or histogram. A coordinator combines those partial values. Metrics systems use it to merge counters from several processes or time windows.
Use local accumulation and merge for finite aggregation when private intermediate state is affordable.
Deriving consistent hashing from keyed routing
Patterns 1 and 2 both started with:
owner := hash(key) % ownerCountModulo routing is simple, deterministic and well distributed while the number of owners stays fixed. The problem appears when membership changes.
If the owner count changes from 8 to 9, the result of % ownerCount changes for many keys. Cached values, sessions or shards must move even though only one owner was added.
Consistent hashing changes the layout. Both keys and owners are placed in one hash space, usually drawn as a ring. A key belongs to the first owner found clockwise from the key's position.
node B ●────────● key-2 ╱ ╲ key-1 ● ● node C ╲ ╱ ●────────● node A key-3Adding a node moves the nearby interval to the new owner. Keys in other intervals remain assigned to their existing owners. Virtual nodes place each physical owner at several points to improve balance and support weights.
Small Go hash ring
package ring import ( "hash/fnv" "sort" "strconv") type Ring struct { points []uint32 owners map[uint32]string} func New() *Ring { return &Ring{owners: make(map[uint32]string)}} func hash(value string) uint32 { h := fnv.New32a() _, _ = h.Write([]byte(value)) return h.Sum32()} func (r *Ring) Add(node string, virtualNodes int) { for replica := 0; replica < virtualNodes; replica++ { point := hash(node + ":" + strconv.Itoa(replica)) r.points = append(r.points, point) r.owners[point] = node } sort.Slice(r.points, func(i, j int) bool { return r.points[i] < r.points[j] })} func (r *Ring) Owner(key string) (string, bool) { if len(r.points) == 0 { return "", false } keyPoint := hash(key) index := sort.Search(len(r.points), func(index int) bool { return r.points[index] >= keyPoint }) if index == len(r.points) { index = 0 // Wrap around the ring. } return r.owners[r.points[index]], true}This example explains routing, but it is not a production ring. A real implementation must handle hash collisions, node removal, weights, replication and concurrent metadata updates.
Consistent hashing solves which owner should receive a key. It does not move the data, wait for in-flight writes, replicate state or decide which copy is authoritative. Those are separate ownership-transfer and consistency problems.
It is also important not to assume that every partitioned system uses a consistent-hash ring. Kafka's producer partitioning and broker partition assignment have their own rules. Adding Kafka partitions can change where keyed records go. A consistent-hash ring is one routing strategy, not a universal description of partitioning.
Rendezvous hashing and jump consistent hash are other ways to reduce movement when the owner set changes. The design question stays the same: how much routing state is acceptable and how much movement can a membership change cause?
One progression from Go maps to distributed systems
The five patterns can be arranged as a design progression.
Step 1: one lock
all goroutines → one mutex → one mapCorrect and simple. Keep it until contention is measured.
Step 2: several lock domains
hash(key) → shard mutex → shard mapMore concurrency for independent keys. The state is still shared.
Step 3: one owner per domain
hash(key) → channel → owner goroutine → private mapThe lock disappears because access is serialized through an owner.
Step 4: owners on different machines
shard metadata → network → process owner → local stateNow messages can be delayed, duplicated or lost. Owners can fail. Routing information can be stale. Replication and recovery become necessary.
Step 5: membership changes
old owners → rebalance or hash-ring change → new ownersThe system needs an ownership-transfer protocol. Routing alone is insufficient.
This is why distributed systems are not simply local concurrency with network calls. The local patterns remain useful, but the network introduces partial failure and uncertain ownership during transitions.
How the larger systems map back to the five patterns
| System or technique | Closest pattern | Shared idea | Important extra concern |
|---|---|---|---|
| In-memory rate limiter | Sharded map + locks | Key routes to a lock domain | Hot keys and atomic multi-key changes |
| Database hash sharding | Sharded map + locks | Shard key routes to an owner | Transactions, replication and resharding |
| Database range sharding | Static ranges | Non-overlapping key ranges | Range splits, hotspots and migration |
| Traditional Kafka consumer group | Keyed worker ownership | Partition has an exclusive consumer owner | Rebalance, offsets and replay |
| Actor system | Keyed worker ownership | Mailbox serializes private state | Actor placement and failure supervision |
| Task broker | Shared worker queue | Next available worker claims a job | Durability, retry and idempotency |
| MapReduce | Local state + merge | Private partial results are reduced | Shuffle, worker retry and data movement |
| Consistent-hash cache | Keyed routing | Stable ownership under membership change | Replication and cache warm-up |
The mapping is not an identity. A production database or Kafka cluster combines several patterns at different layers. A Kafka broker may use queues internally. A stream processor can own partition state and periodically merge metrics. A sharded database can use worker pools inside every shard.
The value of the model is that each layer can be examined with the same questions.
Choosing among the five patterns
Start with the shape of the work.
Is the input finite?
If yes, ask whether item cost is predictable.
- Roughly equal cost and disjoint output: static ranges.
- Uneven cost and independent jobs: shared worker queue.
- Aggregation into a compact result: local state followed by merge.
Is the work keyed?
If yes, ask whether the operation must be synchronous and whether order matters.
- Synchronous reads and writes from arbitrary goroutines: sharded map with locks.
- Continuous stream with per-key ordering: keyed channel with one state owner.
Can membership change?
If the owners are long-lived processes or cache nodes, decide how keys should move. Modulo hashing may be sufficient for a fixed set. Consistent or rendezvous hashing can reduce movement for a changing set. Explicit shard metadata is often better when operators need controlled range placement.
Decision table
| Workload | Pattern | Why |
|---|---|---|
| Fixed batch with equal work | Static slice ranges | Minimal coordination and disjoint writes |
| Fixed or streaming work with variable cost | Shared worker queue | Dynamic load balancing |
| Synchronous keyed state | Sharded mutex map | Immediate operations with bounded lock contention |
| Asynchronous keyed stream | Worker-owned map | Per-key order through exclusive ownership |
| Finite batch aggregation | Local maps, then merge | No locks on the processing path |
Common mistakes
“No mutex” is mistaken for “no synchronization”
A channel send synchronizes with its corresponding receive. A WaitGroup coordinates completion. Ownership transfer also needs synchronization. Removing a map mutex changes the design, but it does not remove coordination.
A channel is placed in front of shared state
If several workers receive from one channel and all mutate the same map, the map is still shared. The channel safely distributes jobs, but it does not protect unrelated state after the receive.
one queue → many workers → one shared map ↑ still needs protectionMore shards are assumed to fix every hotspot
Sharding spreads independent keys. It cannot split one hot key without changing the key model or operation. A celebrity account, popular cache key or single busy Kafka partition can remain a bottleneck.
Ordering is assumed after adding a worker pool
A shared queue gives work to whichever worker is ready. Completion order is nondeterministic. Use sequence numbers and reorder results or route related events to one owner.
Consistent hashing is assumed to migrate state
A ring computes destinations. The system still needs to copy data, coordinate writes during the move and delete the old copy safely.
Atomics are used for multi-field invariants
An atomic counter is good for one independent number:
var accepted atomic.Int64accepted.Add(1)It does not automatically make this invariant safe:
available = capacity - reservedIf several fields must change as one logical operation, use a mutex, a single owner or an immutable snapshot published atomically.
Verification matters more than the diagram
A convincing ownership diagram can still hide an accidental shared write. In Go, run the race detector against realistic concurrent tests:
go test -race ./...Then measure the actual bottleneck. A mutex that looks suspicious may be cheaper than channels and allocation. A sharded design can be slower at low contention. A worker pool can spend more time coordinating than processing tiny jobs.
Useful measurements include:
- Queue depth and time spent waiting to enqueue.
- Lock wait time and critical-section duration.
- Per-shard or per-partition throughput.
- Distribution of job duration rather than only its average.
- Rebalance or state-transfer time.
- Memory held in queued work and partial maps.
The best pattern is the simplest one that preserves the required ownership, ordering and overload behavior at the measured workload.
The compact mental model
Fixed batch, equal work → split rangesVariable independent jobs → shared queueKeyed synchronous state → shard + lockKeyed asynchronous state → hash + channel + ownerBatch totals → local state + mergeChanging keyed owners → stable routing + state transferAnd the two questions remain the fastest way into any design:
How is work distributed?Who owns or protects the state?Once those answers are explicit, the need for a mutex, channel, queue, shard router or merge step becomes much clearer.