procfunc

Services the size of functions.

lesiw.io/proc extends Go’s communication primitives to cluster scale. Procs are context-aware, supervised goroutines. They communicate through typed messages on named channels, and channel messages are stored durably in a broker.

ctx := context.Background()
pool, err := pgxpool.New(ctx, os.Getenv("DB_URL"))
if err != nil {
	log.Fatal(err)
}
ctx, cancel := proc.Context(
	ctx, &procpgx.Broker{Pool: pool},
)
defer cancel()

proc.Go(ctx, serveHTTP, proc.Each("http"))
proc.Go(ctx, sendEmail, proc.Solo("mail"))
log.Fatal(proc.Wait(ctx))
ctx := context.Background()
pool, err := pgxpool.New(
	ctx, os.Getenv("DB_URL"),
)
if err != nil {
	log.Fatal(err)
}
ctx, cancel := proc.Context(
	ctx, &procpgx.Broker{Pool: pool},
)
defer cancel()

proc.Go(ctx, serveHTTP, proc.Each("http"))
proc.Go(ctx, sendEmail, proc.Solo("mail"))
log.Fatal(proc.Wait(ctx))

One binary behind a load balancer. Every peer serves HTTP. Only one sends mail, fed by a durable channel. When a peer dies, its work moves and the messages keep flowing.

01 · proc.Context

A context is a supervised tree.

proc.Context opens a supervised context on the parent ctx. Every proc.Go schedules a proc — a func(context.Context) error — onto it, and proc.Wait blocks until they finish. The tree lives on the ctx, so cancellation, deadlines, and values flow the way Go code already expects.

ctx, cancel := proc.Context(ctx)
defer cancel()
proc.Go(ctx, pollInventory)
proc.Go(ctx, pruneSessions)
proc.Go(ctx, reportMetrics)
return proc.Wait(ctx)

Procs join the tree as they are scheduled. Wait blocks until every proc finishes; cancel tells them all to.

02 · proc.Wait

proc.Wait observes the first failure.

A bare proc.Go is fail-fast. A failure — a non-nil error, or a panic, recovered and wrapped — cancels its siblings through the shared context, and the first failure is what Wait returns once every proc has finished. Errors propagate the errgroup way, and a panic fails the proc.Context, not the program.

proc.Go(ctx, migrate)
proc.Go(ctx, preload)
proc.Go(ctx, inspect) // fails
err := proc.Wait(ctx) // inspect's error

A failure surfaces once, at the root — when everything runs clean, Wait returns nil. Siblings of a failure see ordinary ctx-cancel.

03 · proc.Keep

Keep restarts on error.

proc.Keep lifts a body into a restart loop — the long-lived service shape. Any non-nil return or recovered panic re-invokes the body, under exponential backoff once failures cluster. A clean nil return means the proc is done, and the loop exits: nil is death, on purpose.

proc.Go(ctx, watchQueue, proc.Keep())

The supervisor absorbs the crash and re-invokes the body. Backoff grows while failures cluster and decays when they stop.

04 · proc.Solo

Solo runs exactly one, cluster-wide.

proc.Solo is Keep plus a cluster-wide claim: at most one peer runs the body at a time. The claim rides the same broker the messages do — there is no separate coordination service to operate. When the holding peer dies, its claim lapses and another peer picks the body up.

proc.Go(ctx, deliverMail, proc.Solo("mailer"))
proc.Go(
	ctx, deliverMail, proc.Solo("mailer"),
)

The body's ctx is claim-scoped. Losing the claim cancels it, so a deposed holder winds down instead of racing its successor.

The claim is broker state, not host state. Kill every peer at once and the first one back picks the work up again — scaling to zero is not a special case.

Click a host to kill it. The claim migrates while the ring counts down the reboot; the host rejoins as a standby.

05 · proc.Many

Many is Solo, times N.

proc.Many("resize", 4) claims four slots across the cluster, each one a Solo under the names resize-1 through resize-4. The slots share the cohort resize — a cohort is the group a channel load-balances across, and supervisors set it from their name — so a Recv inside the body splits the channel across the whole pool.

proc.Go(ctx, resizeImages, proc.Many("resize", 4))
proc.Go(
	ctx, resizeImages, proc.Many("resize", 4),
)

proc.Each is the complement: one copy on every peer rather than N across the cluster — the http daemon from the opening example.

Work load-balances across the slots. Kill a host and its slots re-claim on the survivors; placement rebalances when it returns.

06 · proc.Send / proc.Recv

Messages arrive in order, at least once.

Send publishes a typed value to a named channel. Recv yields each claimed message into an ordinary range loop, and the loop's control flow settles it — reaching the end of the body acks — commits the message; break, return, or panic naks — rejects it, and the broker redelivers. Delivery is at-least-once, FIFO within a channel and cohort.

for ctx, o := range proc.Recv[Order](ctx, "orders") {
	process(ctx, o)
}
for ctx, o := range proc.Recv[Order](
	ctx, "orders",
) {
	process(ctx, o)
}

A consumer that dies mid-message loses nothing. The message returns to the channel and is delivered again, in order; an idempotent handler makes the duplicate harmless.

Numbered messages move in order. Crash the consumer mid-message and the same number comes back around.

07 · proc.Chan

One proc, many inboxes.

One goroutine, several typed message streams, handled in one place so the state they touch stays consistent — Go’s own select, with durable settlement on every arm. proc.Chan turns a channel into a plain Go <-chan for select — and hands you the settlement: you call Ack or Nak on every delivery yourself.

proc.Go(ctx, func(ctx context.Context) error {
	select {
	case d := <-proc.Chan[Order](ctx, "orders"):
		add(ctx, d.Value)
		d.Ack(ctx)
	case d := <-proc.Chan[Cancel](ctx, "cancels"):
		drop(ctx, d.Value)
		d.Ack(ctx)
	case <-ctx.Done():
		return nil
	}
	return proc.ErrContinue
}, proc.Solo("inbox"))
proc.Go(ctx, func(
	ctx context.Context,
) error {
	select {
	case d := <-proc.Chan[Order](
		ctx, "orders",
	):
		add(ctx, d.Value)
		d.Ack(ctx)
	case d := <-proc.Chan[Cancel](
		ctx, "cancels",
	):
		drop(ctx, d.Value)
		d.Ack(ctx)
	case <-ctx.Done():
		return nil
	}
	return proc.ErrContinue
}, proc.Solo("inbox"))

Chan is the power-user primitive. Recv acks by control flow and cannot leak a message; with Chan a dropped delivery stays in flight until the scope ends. The channel is never closed — when the context ends it simply goes quiet, so the ctx.Done() arm is the select's one exit. Reach for Chan when a proc needs several channels; prefer Recv everywhere else.

Note the shape: one message per invocation, and Keep owns the loop — returning proc.ErrContinue asks for the next invocation without counting as a failure. The restart reaps every consumer the invocation minted, so the Chan calls can sit inline in the select.

Two channels, one proc. Crash it mid-message and the held delivery returns to the channel it came from.

08 · proc.Query / proc.Serve

RPC through messages.

proc.Serve runs the dispatch loop for a channel — each arriving query invokes the handler, and the reply lands in the requester's inbox. proc.Query sends one query and blocks for the reply. Both ride the same Send and Recv underneath, so there is no second RPC system to operate — and because a query is just a message in a queue, the server does not need to be up the instant it is sent. A restart or a blip delays the reply instead of failing it.

double := proc.ServerFunc[int, int](
	func(_ context.Context, n int) (int, error) {
		return n * 2, nil
	},
)
proc.Go(ctx, func(ctx context.Context) error {
	return proc.Serve(ctx, "double", double)
}, proc.Solo("double"))

reply, ok := proc.Query[int, int](ctx, "double", 21)
if ok {
	fmt.Println(reply) // Output: 42
}
double := proc.ServerFunc[int, int](
	func(_ context.Context, n int) (
		int, error,
	) {
		return n * 2, nil
	},
)
proc.Go(ctx, func(
	ctx context.Context,
) error {
	return proc.Serve(ctx, "double", double)
}, proc.Solo("double"))

reply, ok := proc.Query[int, int](
	ctx, "double", 21,
)
if ok {
	fmt.Println(reply) // Output: 42
}

Serve is at-least-once, and so is the reply. A handler that dies before its ack runs again on redelivery, and a crash mid-reply can deliver the answer twice — so handlers stay idempotent, and requesters treat a duplicate reply as the same news.

Crash the server mid-query and the query comes back around. Everything here is at-least-once — the work, and the reply too.

09 · proc.Go

Durable execution through durable messaging.

Long-running, multi-step work is procs connected by channels. Each step Recvs its input, does one unit of work, and Sends the result onward. Every hop is durable — a step that crashes or deploys mid-message naks it, restarts, and the pipeline resumes exactly where it stopped.

proc.Go(ctx, step1, proc.Solo("step1"))
proc.Go(ctx, step2, proc.Solo("step2"))
proc.Go(ctx, step3, proc.Solo("step3"))

This is durable execution with no separate engine to operate — the broker you already run is the engine. The state of the pipeline is the messages in flight, and the substrate already persists those.

Crash step2 mid-message. The message returns to its channel and continues through the pipeline after the restart.

10 · proc.Load

A cluster-wide cron in one body.

Periodic work in a distributed system asks for three things at once — exactly one runner, durable schedule state, and sane behavior through crashes. Each maps to one primitive. Solo gates the runner. Load and Send keep the last-run timestamp in the substrate. ErrContinue hands the loop to Keep, so the body is one tick.

proc.Go(ctx, func(ctx context.Context) error {
	last, ok := proc.Load[time.Time](
		ctx, "cron/backup/last",
	)
	if !ok {
		return nil
	}
	next := schedule.Next(last)
	select {
	case <-ctx.Done():
		return nil
	case <-time.After(time.Until(next)):
	}
	defer proc.Send(ctx, "cron/backup/last", next)
	return cmp.Or(runBackup(ctx), proc.ErrContinue)
}, proc.Solo("cron/backup"))
proc.Go(ctx, func(
	ctx context.Context,
) error {
	last, ok := proc.Load[time.Time](
		ctx, "cron/backup/last",
	)
	if !ok {
		return nil
	}
	next := schedule.Next(last)
	select {
	case <-ctx.Done():
		return nil
	case <-time.After(time.Until(next)):
	}
	defer proc.Send(
		ctx, "cron/backup/last", next,
	)
	return cmp.Or(
		runBackup(ctx), proc.ErrContinue,
	)
}, proc.Solo("cron/backup"))

The deferred Send records the tick only if the work ran. A crash mid-run leaves the timestamp untouched, so the next holder re-runs the missed tick — at least once, never zero.

Kill the holder mid-run. The claim moves, the new holder reads the last tick, and the missed tick runs again.

11 · proc.Save

A channel is also a value.

Load reads the latest message on a channel; Save appends one — but only if the caller has seen the current head. Together they make every channel a durable KV cell with optimistic concurrency. Two writers cannot silently overwrite each other; the loser learns, re-reads, and retries.

func bump(ctx context.Context) error {
	for {
		n, ok := proc.Load[int](ctx, "counter")
		if !ok {
			return nil
		}
		saved, ok := proc.Save(ctx, "counter", n+1)
		if !ok || saved {
			return nil
		}
	}
}
func bump(ctx context.Context) error {
	for {
		n, ok := proc.Load[int](
			ctx, "counter",
		)
		if !ok {
			return nil
		}
		saved, ok := proc.Save(
			ctx, "counter", n+1,
		)
		if !ok || saved {
			return nil
		}
	}
}

Conflict is a normal-flow signal, not an error. And Load is at-most-once by design — it advances past whatever it reads — which is the other half of the delivery spectrum, there when a problem calls for it.

Both writers load the same value. One commit lands — a new message in the log, a new value right of the divider. The arrowheads are each peer’s high-water mark: peer-1 above the log, peer-2 below. The conflict re-reads and retries; the counter never skips or repeats.

12 · proc.Broker

Distributed in production. Local on your laptop.

The broker is the only thing that changes. With the in-memory broker, every proc — the per-peer daemons, the singletons, the whole pool — runs in one process under go run, including the claims. Swap in a durable broker and the same binary spreads across the cluster by supervisor shape — Each on every peer, Solo on exactly one, Many filling its slots.

var broker proc.Deriver = new(mem.Broker)
if os.Getenv("PGHOST") != "" {
	pool, err := pgxpool.New(ctx, "")
	if err != nil {
		log.Fatal(err)
	}
	broker = &procpgx.Broker{Pool: pool}
}
ctx, cancel := proc.Context(ctx, broker)
var broker proc.Deriver = new(mem.Broker)
if os.Getenv("PGHOST") != "" {
	pool, err := pgxpool.New(ctx, "")
	if err != nil {
		log.Fatal(err)
	}
	broker = &procpgx.Broker{Pool: pool}
}
ctx, cancel := proc.Context(
	ctx, broker,
)

Brokers sit behind proc.Deriver, a four-method interface. Postgres is supported today via lesiw.io/procpgx; NATS JetStream and Redis are under consideration.

The same procs, twice. On the laptop everything shares one process; on the cluster they spread by supervisor shape.

13 · proc.Epoch

New code proves itself before old code is gone.

Every peer stamps its build with an epoch — by default the commit timestamp, read from the binary's VCS metadata, with an override for explicit stamps.

e=$(date -u +%s)
go build -ldflags="-X lesiw.io/proc.Epoch=$e" .
e=$(date -u +%s)
go build \
	-ldflags="-X lesiw.io/proc.Epoch=$e" .

During a rolling deploy the cluster sees both epochs at once. The moment a majority of connected peers reports the new one — membership is connection, so the denominator is whoever is heartbeating — every Solo migrates onto new-epoch peers — the new build takes over the singletons while the old build is still running beside it. A Solo that regresses on the new build fails fast, under supervision, before the rollout finishes.

ctx, cancel := proc.Context(ctx,
	proc.WithTripwire(time.Minute),
	&procpgx.Broker{Pool: pool},
)

WithTripwire arms the window: for its duration after startup, a supervised body's error surfaces through Wait instead of restarting, so a bad rollout halts loudly rather than crash-looping quietly.

Roll a deploy. At two of three peers the majority flips, and the singletons move to the new build.

14 · Partitions

A partitioned peer fences itself.

Every peer heartbeats through the broker. A peer that loses its transport stops hearing back, and past a threshold it assumes the cluster can no longer see it — so it cancels its own procs, before the cluster re-grants its claims. A deposed singleton never races its successor.

Inside the fenced peer, bodies see plain ctx-cancel — the same signal a deploy or a shutdown delivers.

for ctx, job := range proc.Recv[Job](ctx, "jobs") {
	handle(ctx, job)
}
for ctx, job := range proc.Recv[Job](
	ctx, "jobs",
) {
	handle(ctx, job)
}

Code that honors its ctx is already partition-correct; why a context ended is the supervisor's business, not the body's.

Disconnect a host. It fences itself first; only then does the claim move.

15 · One system of record

The broker is the state.

There is no membership protocol beside it — no seed URLs, no gossip, no bootstrap order. A peer that can reach the broker is a member; a peer that cannot, is not. Messages, claims, membership, and schedule state live in one durable system.

Transient failure is the substrate's problem. A flaky connection, a broker restart, a dropped packet — retried internally, invisible above. Your code sees two signals: messages, and cancellation.

Scaling to zero follows from the same fact. Peers hold no state, so an empty cluster is a quiet one; the messages wait in the broker, and the first peer back picks up where everything left off.

Scale to zero mid-queue. The messages wait in the broker; the first hosts back drain them.

Get started

Start on the laptop.

go get lesiw.io/proc

Open a proc.Context with the in-memory broker, schedule bodies with proc.Go, and proc.Wait. When it is time for a cluster, import a durable broker and change that one line.

Coming from Erlang or CSP? The appendix works the classic problems in proc.

Appendix

The classic problems, distributed.

For readers arriving from CSP, Erlang, or Go's own concurrency, the fastest way to calibrate is the classic exercises. Delivery here is at-least-once and FIFO within a channel and cohort, so each solution carries its durability consequence with it.

The column reformatter

Hoare's original exercise reads 80-column cards and reprints their characters as 125-character lines — two processes joined by a stream. One message per character is his framing, not a throughput recommendation. The unpacking side is an ordinary Recv loop feeding a channel.

func unpack(ctx context.Context) error {
	for ctx, card := range proc.Recv[string](ctx, "cards") {
		for _, r := range card {
			proc.Send(ctx, "chars", r)
		}
	}
	return nil
}
func unpack(ctx context.Context) error {
	for ctx, card := range proc.Recv[string](
		ctx, "cards",
	) {
		for _, r := range card {
			proc.Send(ctx, "chars", r)
		}
	}
	return nil
}

The packing side is where durability changes the idiom. A partial line lives in memory, and an acked character never redelivers — so the packer holds every delivery unacked until its line is complete. Chan exists for this settlement-decoupled shape. A crash rebuilds the partial line from redelivery, in order; a finished line may send twice; downstream stays idempotent.

func pack(ctx context.Context) error {
	var line []rune
	var pending []proc.Delivery[rune]
	chars := proc.Chan[rune](ctx, "chars")
	for {
		select {
		case <-ctx.Done():
			return nil
		case d := <-chars:
			line = append(line, d.Value)
			pending = append(pending, d)
			if len(line) < 125 {
				continue
			}
			if !proc.Send(ctx, "lines", string(line)) {
				return nil
			}
			for _, p := range pending {
				p.Ack(ctx)
			}
			line, pending = nil, nil
		}
	}
}

proc.Go(ctx, unpack, proc.Solo("unpack"))
proc.Go(ctx, pack, proc.Solo("pack"))
func pack(ctx context.Context) error {
	var line []rune
	var pending []proc.Delivery[rune]
	chars := proc.Chan[rune](ctx, "chars")
	for {
		select {
		case <-ctx.Done():
			return nil
		case d := <-chars:
			line = append(line, d.Value)
			pending = append(pending, d)
			if len(line) < 125 {
				continue
			}
			ok := proc.Send(
				ctx, "lines", string(line),
			)
			if !ok {
				return nil
			}
			for _, p := range pending {
				p.Ack(ctx)
			}
			line, pending = nil, nil
		}
	}
}

proc.Go(ctx, unpack, proc.Solo("unpack"))
proc.Go(ctx, pack, proc.Solo("pack"))

Fan-out, fan-in

Fan-out is a cohort, not a loop that starts goroutines — Many claims the slots and Recv load-balances the channel across them. Fan-in needs no machinery at all; any number of procs Send to one channel, and a single collector Recvs one ordered stream.

proc.Go(ctx, func(ctx context.Context) error {
	for ctx, j := range proc.Recv[Job](ctx, "jobs") {
		proc.Send(ctx, "results", work(j))
	}
	return nil
}, proc.Many("workers", 8))

proc.Go(ctx, func(ctx context.Context) error {
	for ctx, r := range proc.Recv[Result](ctx, "results") {
		record(ctx, r)
	}
	return nil
}, proc.Solo("collector"))
proc.Go(ctx, func(
	ctx context.Context,
) error {
	for ctx, j := range proc.Recv[Job](
		ctx, "jobs",
	) {
		proc.Send(ctx, "results", work(j))
	}
	return nil
}, proc.Many("workers", 8))

proc.Go(ctx, func(
	ctx context.Context,
) error {
	for ctx, r := range proc.Recv[Result](
		ctx, "results",
	) {
		record(ctx, r)
	}
	return nil
}, proc.Solo("collector"))

The sieve of Eratosthenes

The classic dynamic pipeline — every discovered prime spawns a filter — is also the π-calculus party trick, because channel names are plain values created at runtime and handed to the next proc. Topology is data.

func sieve(in string) func(context.Context) error {
	return func(ctx context.Context) error {
		first, out := 0, ""
		for ctx, n := range proc.Recv[int](ctx, in) {
			if first == 0 {
				first = n
				proc.Send(ctx, "primes", n)
				out = "sieve/" + strconv.Itoa(n)
				proc.Go(ctx, sieve(out))
				continue
			}
			if n%first != 0 {
				proc.Send(ctx, out, n)
			}
		}
		return nil
	}
}
func sieve(
	in string,
) func(context.Context) error {
	return func(ctx context.Context) error {
		first, out := 0, ""
		for ctx, n := range proc.Recv[int](
			ctx, in,
		) {
			if first == 0 {
				first = n
				proc.Send(ctx, "primes", n)
				out = "sieve/" +
					strconv.Itoa(n)
				proc.Go(ctx, sieve(out))
				continue
			}
			if n%first != 0 {
				proc.Send(ctx, out, n)
			}
		}
		return nil
	}
}

This one is a toy — a proc and a channel per prime, best enjoyed on the in-memory broker. The part worth keeping is the mobility.

A cluster-wide rate limiter

The token bucket becomes cluster-wide when the bucket is a channel. One Solo drips tokens; every worker on every host takes one before acting. WithMessageTTL expires unclaimed tokens, so an idle cluster can't bank an unbounded burst.

proc.Go(ctx, func(ctx context.Context) error {
	ctx = proc.WithMessageTTL(time.Second)(ctx)
	for {
		select {
		case <-ctx.Done():
			return nil
		case <-time.After(time.Second / 10):
		}
		proc.Send(ctx, "tokens", struct{}{})
	}
}, proc.Solo("limiter"))
proc.Go(ctx, func(
	ctx context.Context,
) error {
	ctx = proc.WithMessageTTL(
		time.Second,
	)(ctx)
	for {
		select {
		case <-ctx.Done():
			return nil
		case <-time.After(time.Second / 10):
		}
		proc.Send(
			ctx, "tokens", struct{}{},
		)
	}
}, proc.Solo("limiter"))

A consumer inside a Many body takes one token per action, and the shared cohort makes the rate a property of the cluster rather than of any one process.

for ctx := range proc.Recv[struct{}](ctx, "tokens") {
	crawl(ctx)
}
tokens := proc.Recv[struct{}](ctx, "tokens")
for ctx := range tokens {
	crawl(ctx)
}

The differences from in-process CSP are the durability trade, deliberately taken. Delivery is at-least-once rather than exactly-once, so handlers are written idempotent. Ordering is FIFO within a channel and cohort, not global. And messages outlive the processes on both ends, which is the point — a Send is durable the moment it returns, whoever is or isn't running.

The rest of the classics are the tour itself — producer/consumer is at-least-once delivery, the worker pool is Many, request/reply is Query, and the pipeline is durable execution — each running across machines instead of goroutines.

Appendix

Inspired by.

Smalltalk (Kay, 1972). Everything is an object, and objects interact only by sending messages. Decades later Kay insisted the objects were never the point — "the big idea is messaging" — a thread the c2 wiki kept pulling on.

The actor model (Hewitt, Bishop, and Steiger, 1973). Computation as independent entities that hold their own state and interact only through messages to mailboxes.

Communicating Sequential Processes (Hoare, 1978). Programs as processes that share nothing and rendezvous over channels. The exercises in the appendix above are his.

The Bell Labs line (Pike and others, 1985–1996). Newsqueak, Alef, and Limbo made CSP channels a working language feature three times before Go did it a fourth — Russ Cox's history traces the whole line.

Erlang and OTP (Armstrong and others, 1986–1998). Supervision trees, restart strategies, registered processes, and let it crash — the discovery that reliability is a property of the supervisor, not the worker. Keep, Solo, and fail-fast scopes are OTP shapes with Go spelling; Armstrong's thesis is the argument in full.

The π-calculus (Milner, Parrow, and Walker, 1992). Channels as first-class values whose names travel over other channels, so the communication topology itself can move. The sieve above does exactly this.

Go (2009). Goroutines and channels carried the Bell Labs line into the mainstream — "share memory by communicating" — and context, errgroup, and range-over-func iteration gave proc its native vocabulary. proc extends Go's concurrency words across machines rather than inventing parallel ones.

Akka (Bonér, 2009). Erlang's supervision and actors, industrialized on the JVM. Akka's Cluster Singleton is Solo's corporate sibling, with an ops team attached.

Orleans (Bykov and others, 2011). Virtual actors — a named actor conceptually always exists, and the runtime activates it somewhere in the cluster on first use, exactly once. Solo makes the same guarantee with the claim held explicitly on the broker rather than in a runtime's directory.

Durable execution (2010s–2020s). Workflow engines demonstrated the demand for progress that survives crashes and deploys. proc expresses the same property as plain messages on durable channels, with no separate engine to operate.