Benchmarks

Hello-world throughput is not the whole story, but it shows whether the request hot path leaves performance on the table. This page compares tako's two serving models with Axum, Actix Web, and ntex under three loads in the same Linux container.

Three loads

• Loopback. 12 server workers and 12 wrk threads share the CPU quota, and each connection sends one request at a time. Kernel time for the loopback TCP traffic takes about 5 µs of the roughly 6 µs of server CPU per request, so the thread-per-core servers land close together.
• Server-bound. 4 server workers face 16 wrk threads. The server runs out of CPU first, so its cost per request decides the result.
• Pipelined. wrk writes 16 requests at a time on each connection, as in the TechEmpower plaintext test. Each syscall serves up to 16 requests, so the framework's own work dominates.

Results

Requests per second are the median of five 15-second runs. CPU per request is the server's user plus system time divided by the requests it served. Spread is half the range of the five runs, relative to the median. No run recorded a socket error or a non-2xx response.

Against tako-rs 2.3.0 in the same run, each with its default settings, the default server serves 20% more requests at 1,000 connections, 21% more server-bound, and 53% more pipelined; the per-thread server serves 9% more server-bound and 30% more pipelined.

Loopback, 100 connections

Loopback, 1,000 connections

Server-bound, 256 connections

Pipelined, 256 connections

With 16 requests per write, wrk records one latency per batch, so this table leaves latency out. Axum is left out too: axum::serve leaves TCP_NODELAY off, so pipelined responses wait on delayed ACKs, and it served 88,398 requests per second. Turning nodelay on through axum's ListenerExt::tap_io avoids that.

Two serving models

The default server, Server::builder(), runs on Tokio's multi-threaded, work-stealing runtime, the same model as Axum. It serves about 24% more requests than Axum at 1,000 connections and when the server is the bottleneck, because it spends less CPU on each request: 7.6 µs against 10.1 µs at 1,000 connections. At 100 connections the two land within 5% of each other.

Since 2.4 a hello-world request does not allocate, so jemalloc no longer pulls ahead: Tako + jemalloc lands between 3% above and 6% below Tako. It can still pay off for handlers that allocate; see Deployment.

The per-thread feature runs one current-thread Tokio runtime per worker, each with its own SO_REUSEPORT listener. It hands each new connection to the worker with the fewest live connections, and the connection then stays on that worker, so requests never cross threads. Actix Web and ntex use the same shape. Tako per-thread serves about 3% more requests than Actix Web on loopback, 7% more when the server is the bottleneck, and 17% more pipelined.

The Tako per-thread, pinned rows turn on pin_to_core, which pins each worker to a CPU core and is off by default. Pinning gained about 5% pipelined, made no difference server-bound, and cost about 3% on loopback with a longer latency tail, because pinned workers compete with the co-located wrk threads for their cores. Every process pins from core 0, so turn it on only when the server has the machine to itself.

Per-thread workers serve HTTP/1.1; TLS, h2c, and PROXY listeners use the regular server builder. The kernel balances connections across SO_REUSEPORT listeners only on Linux, so measure per-thread throughput there. See Thread-per-core for the entry points.

The Compio flavour (per-thread-compio) is not in the tables: the container blocks io_uring, so its runtime cannot start there.

Methodology

• Servers. Each framework serves GET / → Hello, World! from a minimal release binary (lto = "fat", codegen-units = 1) with its default settings; every response is the same 130 bytes. Versions: tako-rs 2.4.0, axum 0.8.9, actix-web 4.15.0, ntex 3.12.3, tokio 1.53.1.

• Load. wrk runs in the same container and connects over loopback. The loopback and pipelined loads split the 24 vCPU quota into 12 server workers and 12 wrk threads; the server-bound load gives the server 4 workers and wrk 16 threads. For the pipelined load, a Lua script makes wrk write 16 requests at a time:

  init = function(args)
    local batch = {}
    for i = 1, 16 do batch[i] = wrk.format(nil, "/") end
    req = table.concat(batch)
  end
  request = function() return req end

• Runs. Each measurement is a 3-second warm-up followed by a 15-second wrk --latency run. Five rounds rotate the server order, and the tables report medians.

• CPU. The server's user and system time come from /proc/<pid>/stat, read before and after each run.

• jemalloc. The jemalloc feature only re-exports the allocator; the benchmark binary installs it:

  #[global_allocator]
  static GLOBAL: tako::Jemalloc = tako::Jemalloc;

Caveats

Reproducing locally

The default server is examples/hello-world:

cargo run --release --manifest-path examples/hello-world/Cargo.toml
# in another terminal:
wrk -t4 -c100 -d15s --latency http://127.0.0.1:8080/

Save the Lua script above as pipeline.lua to repeat the pipelined load:

wrk -t4 -c256 -d15s -s pipeline.lua http://127.0.0.1:8080/

For the per-thread server, use examples/bench-pt on Linux. It takes the mode, the address, and the worker count:

cargo run --release --manifest-path examples/bench-pt/Cargo.toml -- pt-tokio 127.0.0.1:8080 4

bench-pt counts requests for its load-distribution report, so expect it to land somewhat below the numbers above.