Accepting a job is a durable commit.

Durable Runner is a background-task runner in Rust. Acknowledged work survives crashes and restarts — at-least-once honestly, every limit bounded.

One runner, reached through the CLI, the HTTP API and the Inspector, calling only registered handlers

  • CLI
  • HTTP API
  • Inspector
  • Allowlisted handlers

What Durable Runner is

  • Accepted means durable

    A run is acknowledged only after its record is durable in the journal on disk.The in-memory index is rebuilt from it at startup — never the authority.

    Journal-first
  • At-least-once, honestly

    Every attempt of a step runs under one business idempotency key.A lost handler reply is reconciled before retrying — external exactly-once is never claimed.

    One key per attempt
  • Every limit enforced

    Queues, payloads, retries, live runs, memory, disk, connections, client addresses.Admission is refused before a reserve runs out — the numbers are configuration, not hopes.

    Bounded
  • The whole runner, visible

    Runs, jobs, submit, cancel, compact, backup and diagnostics — one small UI at /inspector.Served same-origin by the runner; keys stay in sessionStorage.

    Inspector

Evidence

The proof,
not promises

4 of 4

the required failure cases, proven

Killed before acknowledging, killed after it, a handler reply that never arrives, a crash during compaction, a full disk, a second writer — each one mapped to the tests and live drills that cover it.

Measured. The kill -9 drill stops the server mid-work 36 times under load: 25,201 acknowledged submits and 9,119 cancels, zero lost, zero duplicated — a laptop drill, a development observation.

1,000,000

of 1,000,000 submits accepted

Sent, accepted and drained to done — every single one. p50 12.5 ms, p99 41.3 ms; the journal grew to 496 MiB.

Measured. The sustained gate at 100 submits a second for about 2.8 hours on a busy laptop (Apple M2 Pro, median load 3.52) — a development observation, not a capacity figure. The same gate honestly misses on CI: a 4-core runner reaches the live-runs cap at about 85 completions a second, then refuses 503 at_capacity — a kept miss, never rerun until green.

0.4 s / 1.7 s

restart to ready

Process start to answering again, journal replay included: 0.4 s with 100,000 runs queued, 1.7 s with 800,000 — against a 5 s upgrade-unavailability target.

Measured. The restart-ready probe on the same laptop, after the journal became versioned; the unversioned format measured 1.1 s and 8.7 s the same night.

64 · 100/s

per client address

Connections one untrusted address may hold, and requests it may start per second; over the line it gets 429 rate_limited with Retry-After. IPv6 counts per /64, with a /48 aggregate.

Default. Like every limit here, they are flags you set, not promises we make.

/inspector

the runner, inspected

Runs newest-first or oldest-first with cursor paging, job detail, submit, cancel, compact, backup, diagnostics — read and write keys kept separate, stored in sessionStorage.

Same-origin. Served by durable-server itself: no second deploy, no CORS.

The loop

How a job
becomes durable

  1. Submit

    Post a job: an op_key, named steps and a small input — from the CLI or plain curl.

  2. Commit

    The record is appended to the journal and fsynced — the ack only goes out after it is durable.

  3. Dispatch

    Each step calls the HTTP handler it names — and only handlers registered at startup.

  4. Finish or retry

    The outcome is journaled; a retry runs under the same idempotency key, after reconciling what actually happened.

The full execution model lives in concepts.

From source

Quick start,
the real commands

Build it — from source, the only way today

test -d durable-runner || git clone https://github.com/akira-shirase-oss/durable-runner.git
cd durable-runner
cargo build --release --locked -p durable-server -p durable-cli

Run it — one handler, one server

export DURABLE_API_KEY=$(openssl rand -hex 32)
python3 examples/pilot/handler.py 58320 >handler.log 2>&1 &
./target/release/durable-server --journal journal --listen 127.0.0.1:58300 \
  --backup-root backups --handler step=http://127.0.0.1:58320/step >server.log 2>&1 &
export PATH="$PWD/target/release:$PATH"

Submit and inspect — the ack is a durable commit

durable-cli submit --op-key demo/first-job --step greet=step --input '{"name":"ada"}'
durable-cli inspect --run 1

Keys, the health probe, expected output and the HTTP version of each call — walk the full quick start. v1.0.0 — built from source, no published binaries yet.

Work acknowledged stays acknowledged —
crash, restart, deploy.

Read the docs