Skip to content

performance.md

Latest commit

 

History

History
1671 lines (1515 loc) · 132 KB

performance.md

File metadata and controls

1671 lines (1515 loc) · 132 KB
Raw

Memory, speed and flash

What each build costs in RAM, instructions and flash, measured by the scripts in bench/. make lint-bench-numbers fails when a figure here disagrees with the CSV the scripts write.

Memory

bench/sram.sh measures every number in this section and writes them to bench/results-sram.csv.

Static working set

The session struct is measured twice, once native on arm64 and once for rv32ic. The byte-count constants do not move, only the pointer fields, so a 32-bit device needs 80 bytes less than the host figure in either device build. TRUST=webpki adds pointer-sized fields to ch_cfg, pointers and size_t lengths, so that build needs 120 bytes less than the host figure on rv32.

what arm64 rv32
ch_tls session struct (includes 622 B TX staging) 1152 1072
receive buffer you provide (2048 shown; floor CH_MIN_RXBUF) 2048 2048
total static working set 3200 3120
ch_tls under KEX=pq (includes 1806 B TX staging) 2336 2256
total static working set, KEX=pq (2048 buffer) 4384 4304
ch_tls under TRUST=webpki (includes 2401 B TX staging) 3072 2952
total static working set, TRUST=webpki (12338 buffer, its floor) 15410 15290
ch_tls under ROLE=server (includes 1221 B TX staging) 1992 1824
total static working set, ROLE=server (2048 buffer) 4040 3872
ch_tls under ROLE=server SUITE=aesgcm 2760 —
total static working set, ROLE=server SUITE=aesgcm (2048 buffer) 4808 —
ch_tls under TRUST=webpki SUITE=aesgcm 3856 —
total static working set, TRUST=webpki SUITE=aesgcm (12338 buffer, its floor) 16194 —
ch_tls under TRUST=webpki, host object 3072 —
total static working set, TRUST=webpki, host object (12338 buffer, its floor) 15410 —
ch_tls under ROLE=server, host object 1992 —
total static working set, ROLE=server, host object (2048 buffer) 4040 —
ch_tls under PROFILE=cnsa2 TRUST=raw-mldsa 2992 —
total static working set, PROFILE=cnsa2 TRUST=raw-mldsa (7843 buffer, its floor) 10835 —
ch_tls under PROFILE=cnsa2 TRUST=ca-mldsa 2992 —
total static working set, PROFILE=cnsa2 TRUST=ca-mldsa (15656 buffer, its floor) 18648 —
ch_tls under PROFILE=cnsa2 ROLE=server (host object) 3160 —
total static working set, PROFILE=cnsa2 ROLE=server (2179 buffer, its floor) 5339 —

Peak stack

The stack peaks are arm64 only. bench/stack.py reads the call graph from the relocations in an arm64 or x86-64 Mach-O object, and bench/sram.sh runs it on an arm64 Mac, so an rv32 peak needs tooling that does not exist yet.

The script computes each entry point's worst case from the object code's call graph. It does not rely on a hand-picked call chain. It compiles the sources the Makefile packages for the build it measures, with that build's defines, and no other source. It treats static functions of one name in two objects as two functions: p256.c and p384.c each define a static point_add, with different frames. It counts a tail call like a call, adding the callee's depth to the caller's whole frame, which can only overstate a peak. A call through a volatile function pointer has no relocation that names its callee. The script reads the callee from the pointer's initializer in the preprocessed source, and it stops on such a pointer that it cannot read. The stack wipes call their frames this way (p256_wide_wipe.c, sha512.c), so each peak above a wipe counts the array the wipe writes. make lint-stack-walk checks that the script follows a tail call at a function's first instruction and a call through a volatile function pointer, keeps two static functions of one name apart, names the calls below that no object defines, and compiles what make packages. The RSA verify holds the deepest frames, so it sets the ch_connect peak in the default build; it runs once per ticket lifetime, and every byte unwinds before ch_connect returns.

A peak leaves out the frames of the calls below, and the script's report names them for each build:

  • C library calls, such as memcpy and strlen;
  • __stack_chk_fail, __memcpy_chk and __memset_chk, the stack protector and the checked memcpy and memset that the host compiler adds by default;
  • the hooks the image defines, such as ch_assert_fail and ch_rand_bytes (docs/porting.md);
  • the caller's send, recv and on_ticket, which the objects call through pointers.

The image's C library and the caller's own functions set those frames.

call arm64
peak stack, ch_connect (RSA-3072 verify) 5008
peak stack, ch_connect (TRUST=raw-ecdsa) 3120
peak stack, ch_connect (PSK) 2784
peak stack, ch_connect (TRUST=ca-rsa / TRUST=ca-ecdsa) 5472 / 3264
peak stack, ch_connect (TRUST=webpki) 16688
peak stack, ch_connect (TRUST=webpki SUITE=aesgcm) 16816
peak stack, ch_connect (KEX=pq) 16016
peak stack, ch_read (worst case: KeyUpdate rekey) 1888
peak stack, ch_write / ch_close 1520 / 896
peak stack, ch_srv_accept (ROLE=server) 12736
peak stack, ch_srv_accept (ROLE=server SUITE=aesgcm) 12896
peak stack, ch_connect (PROFILE=cnsa2 TRUST=raw-mldsa) 19456
peak stack, ch_connect (PROFILE=cnsa2 TRUST=ca-mldsa) 22096
peak stack, ch_srv_accept (PROFILE=cnsa2 ROLE=server) 27968

What the larger builds pay for

KEX=pq. The hybrid build costs more of both:

  • The session struct grows because the ClientHello carries a 1,216-byte key share and is built whole into one staging array.
  • The stack grows because ML-KEM's K-PKE encrypt holds three polynomial vectors and two polynomials: 5,744 bytes in that one frame, against a 2,560-byte budget for every other build (INV-19 carries the per-build numbers). The whole chain peaks at 16,016 bytes, through mlkem_decaps into K-PKE encrypt, the matrix sampler and Keccak, so the hybrid build needs about three times the stack of the classic one rather than the single frame's 5,744.

A device that cannot spare it builds the classic key exchange.

TRUST=webpki. A TRUST=webpki build carries the hybrid in every build, so it pays both costs too, and its handshake state also holds the P-256 scalar and point a retry to secp256r1 draws (docs/decisions.md 63). Its session struct carries a larger TX staging array for the server_name and ALPN extensions and for both key shares, the 1,216-byte hybrid one and a 32-byte x25519 one (docs/decisions.md 51), and for the third group it lists, secp256r1 (63). Its ch_connect peaks at 16,688 bytes, through ML-KEM's decapsulation. With ML-KEM and its copies over the SHA-3 instructions pruned from the call graph (STACK_PRUNE=mlkem_decaps,mlkem_keygen_dk,mlkem_decaps_hw,mlkem_keygen_dk_hw) the deepest chain is the chain walk into a P-384 verify, at 8,304 bytes: this host builds the host object, whose P-384 verifier runs on six 64-bit words (docs/decisions.md 97). A device object holds no copies over the SHA-3 instructions, and with ML-KEM pruned (STACK_MAKE='TRUST=webpki HOST_TARGET=' STACK_PRUNE=mlkem_decaps,mlkem_keygen_dk) its deepest chain is the chain walk into an RSA-4096 verify on 32-bit words, at 7,344 bytes. RSA-4096 is the widest modulus a public root carries.

ROLE=server. A server build pays them as well, because every server holds the hybrid (docs/decisions.md 54). Its ServerHello is built in the clear in the same staging array, and the hybrid one carries a 1,120-byte share, so the array holds 1,216 bytes of message behind the record header. ch_srv_accept peaks at 12,736 bytes in the host object this host builds, through rsa_sign64.c's RSA-PSS signer, which holds a modulus record for each prime, the signature it checks and a table of sixteen powers (docs/decisions.md 95). Its square holds a copy of 2a (106) and calls rsa_mont64_blocks.c's square, which multiplies in blocks of four words under clang for arm64 (118). With that signer pruned from the call graph (STACK_PRUNE=rsa_sign64_pss) the deepest chain is the encapsulation to the client's key into K-PKE encrypt and Keccak, at 10,464 bytes. With the encapsulation pruned as well (STACK_PRUNE=rsa_sign64_pss,srv_kex_share) it is the ECDSA signer on the wide P-256 field, at 6,176 bytes. After each wide operation, p256_wide_wipe_below writes zero over the stack below it, in a frame of 3,120 bytes. A device object holds neither the 64-bit signer nor the wide field: its ch_srv_accept peaks at 10,432 bytes, through the encapsulation (STACK_MAKE='ROLE=server TRUST=none HOST_TARGET='). There the signer's deepest chain is rsa_sign.c's ladder, at 5,360 bytes (STACK_PRUNE=srv_kex_share).

SUITE=aesgcm. bench/sram.sh measures a SUITE=aesgcm build as the host object this host builds for it (docs/decisions.md 89), so its rows are host figures and have no rv32 column. A suite build with AES=extern can run on a device with an AES peripheral (docs/decisions.md 68), and no script here measures one on a device target yet. Its session struct is 768 bytes larger than the same ROLE=server host object without the suite, and 784 larger for TRUST=webpki:

  • each of the two record directions holds its AES key's expanded schedule, 243 bytes, so no record expands the key again (docs/decisions.md 160);

  • the transcript runs a SHA-512 context beside SHA-256's;

  • the traffic secrets take SHA-384's 48 bytes;

  • a webpki hello lists two more suites (docs/decisions.md 58);

  • a webpki client's configuration holds the caller's suite order, a pointer and a count (docs/decisions.md 80).

TRUST=raw-mldsa and TRUST=ca-mldsa. The two ML-DSA-87 trust modes build under PROFILE=cnsa2 alone, on AES=extern (docs/decisions.md 157). bench/sram.sh measures their portable object on this host, so their rows have no rv32 column. The session struct is the same in both, because the blocking driver holds the handshake state on its stack. The receive floor holds one certificate of CH_X509_MAX, 7,808 bytes, in the raw mode and two in the CA mode, and the static working set adds that floor. Each ch_connect peaks through ML-KEM-1024's decapsulation, at 19,456 bytes in the raw mode and 22,096 in the CA mode. The CA mode's handshake state holds the leaf's 2,592-byte key, which is most of the difference. With ML-KEM's two calls pruned (STACK_PRUNE=mlkem_decaps,mlkem_keygen_dk) the deepest chain is the ML-DSA-87 verify, at 13,376 bytes in the raw mode and 16,064 in the CA mode. A device that runs either mode gives ch_connect the larger peak. The profile's 133-byte retry cookie (HSP_COOKIE_MAX, docs/decisions.md 162) adds 8 bytes to each struct with the alignment after it.

PROFILE=cnsa2 ROLE=server. The profile's server is a host object alone (docs/decisions.md 162), so its rows have no rv32 column. bench/sram.sh measures:

  • a session struct of 3,160 bytes, whose TX array holds the 1,664-byte ServerHello with the ML-KEM-1024 ciphertext;
  • a receive floor of 2,179 bytes, the 2,174-byte largest ClientHello of the profile's client behind its record header, and 2,698 bytes over QUIC;
  • a ch_srv_accept peak of 27,968 bytes, through the ML-DSA-87 signer's rejection loop under the CertificateVerify. With the signer pruned (STACK_PRUNE=mldsa87_sign) the peak is ML-KEM-1024's encapsulation, at 12,640 bytes.

The send side needs no floor of its own. The 7.5 kB Certificate and the 4,627-byte CertificateVerify each stream through srv_frag, 512 bytes a record, and only the CertificateVerify's frame holds its whole signature, 5,280 bytes in srv_mldsa_send_certificate_verify.

A host object. On arm64 and x86-64 the Makefile and build.zig build a TRUST=webpki client, ROLE=server and ROLE=both as a host object, whose ch_cfg holds cpu, the caller's description of its CPU (docs/decisions.md 89). The field is 4 bytes, and with the alignment of the pointer after it each session struct is 8 bytes larger than the same build's portable object, which the rows without "host object" measure, and the SUITE=aesgcm rows above are host objects too. Each record direction also holds a copy of cpu, from which a record reads its paths (docs/decisions.md 87 and 89), in the 4 bytes the alignment of its sequence number left unused, so the directions do not grow. A host object runs on those two architectures alone, so these rows are host figures. The stack peaks of TRUST=webpki and ROLE=server are a host object's too: bench/stack.py compiles what make packages for those builds on this host, which holds both multiplies.

The receive buffer

You size the receive buffer. The client advertises a record_size_limit (RFC 8449) of the buffer less 21 bytes: the 5-byte record header and the 16-byte tag. A peer that honors the extension sends no record that the buffer cannot hold. OpenSSL and Go ignore it, so a server on those stacks must cap its own records (examples/server/README.md, "Record size"). A record longer than the buffer fails the session with CH_ECAP. Each trust mode adds one rule:

  • Pinned mode. The server's Certificate message must also fit. A self-signed P-256 certificate needs about 600 bytes and an RSA-3072 one about 1.2 kB, so the 2 kB buffer above covers both.
  • The ca modes. A ca-mode build knows its own worst case and derives the floor for you: CH_MIN_RXBUF becomes 3,112 bytes (RSA) or 1,576 (ECDSA), the largest Certificate message plus the record that completes it. A buffer too small for the largest chain fails at setup rather than mid-handshake.
  • TRUST=webpki. The build derives 12,338 bytes the same way, four certificates at its 3,072-byte cap.
  • The ML-DSA-87 modes. An ML-DSA-87 certificate holds a 2,592-byte key and a 4,627-byte signature, and OpenSSL 3.5 issues one of about 7,500 bytes, so CH_X509_MAX is 7,808 there. TRUST=raw-mldsa derives a floor of 7,843 bytes, one certificate at the cap plus the record that completes it, and TRUST=ca-mldsa 15,656, two certificates (docs/decisions.md 157).

Provisioning buffers

Provisioning with ch_pubkey_from_pem needs three more caller-side buffers. None of them is part of the static working set above, and none is live between calls:

buffer RSA ECDSA
the staged PEM text (CH_PEM_MAX) 3,136 1,600
a scratch array for the decoded certificate (CH_X509_MAX) 1,536 768
the key slot the device already needed (CH_X509_KEY_MAX) 384 64

The scratch must not be cfg.buf while a session is live: ch_read serves unread plaintext out of that buffer across calls. The call itself measures 640 bytes of stack under both TRUST=ca-rsa and TRUST=ca-ecdsa, reported by bench/stack.py beside the other public calls. Its deepest chain is the read of the PEM block's lines.

The three calls of pem_files.h (decision 131) take their rooms from the caller the same way. A host program reads each file into a buffer of its own. The call writes the DER into a second room, and the values point into that room. So the DER room lives as long as the sessions that use it, and the text does not. The caller sizes the rooms, and a call refuses a file that does not fit them. A DER room of pem_len / 4 * 3 bytes holds the DER of any file.

file text, at most DER room entries
chain, ch_chain_from_pem the caller's the caller's 16 bytes a ch_cert, 255 at most
P-256 key, ch_p256_key_from_pem 452 (CH_PEM_KEY_TEXT_MAX) none: 32 for the scalar and 64 for the point none
CA bundle, ch_anchors_from_pem the caller's the caller's 32 bytes a ch_trust_anchor

The sizes of an entry are the arm64 compiler's. Homebrew's ca-certificates bundle is 289,562 bytes of text. It decodes to 205,893 bytes of DER and 192 anchors, 6,144 bytes of entries. The QUIC Interop Runner's chain of nine certificates is 9,661 bytes of DER.

On arm64 bench/stack.py measures 592 bytes of stack for the chain call, 3,760 for the key call and 560 for the bundle call. It reports them for ROLE=server TRUST=none, TRUST=webpki and ROLE=both TRUST=webpki, the host objects this host builds. The chain call's peak is the read of each certificate's outer fields.

The key call's deepest chain ends in a stack wipe. Under CH_CPU_CONSTANT_TIME_MULTIPLY the call computes the point on the wide P-256 field. After each wide operation, p256_wide_wipe_below writes zero over an array of 3,072 bytes (P256_WIDE_BELOW_LEN) in a frame of 3,120. A device object holds no wide field, and its key call peaks at 1,376 bytes (STACK_MAKE='ROLE=server TRUST=none HOST_TARGET='). So a host program gives the key call 3,760 bytes of stack.

Compared with other libraries

wolfSSL needs about 6.2 kB of heap plus buffers, and mbedTLS needs 9 to 15 kB and cannot run without an allocator. Both need a 16 kB record buffer unless the peer supports record_size_limit. docs/landscape.md holds the survey with sources.

Speed and flash

Cost is published as instruction counts, not guessed milliseconds. Every count is a property of what one compiler emits, so each column names its compiler:

  • mips32r2. bench/insn-mips.sh and bench/device-ram.sh cross-compile for mips32r2, the reference target's ISA (32-bit, big-endian, in-order), and measure on that ISA. The column is Alpine's clang 22 at -Os (CLANG_MAJOR in bench/toolchain.env). The millisecond column assumes 500 MHz and one instruction per cycle on that core, which is optimistic for an in-order design, so read it as a lower bound.
  • Cortex-M3. bench/insn-m3.sh measures the same operations as thumbv7m instruction counts on QEMU's Cortex-M3, the core the m3 and freertos CI lanes execute, built with the pinned Arm GNU Toolchain 15.3.rel1 gcc (ARM_GNU_VERSION) at -O2.
  • rv32imac. bench/insn-rv32.sh measures them a third time as rv32imac instruction counts: the 32-bit little-endian build the riscv32 CI lane ships, compiled by the pinned Bootlin gcc 14.3.0 (RV32_TC_VERSION) at -Os and run under qemu-riscv32 user mode.

The flash figures below come from clang 23 at -Os (LLVM_MAJOR in tools/toolchain.env), the clang the codegen lints run.

All three instruction columns run the multiply decomposition firmware ships. Each script also builds the same driver with -DCH_NATIVE_WIDEMUL and records that count beside the shipped one, as the CSV's native_insns column. The decomposition figures under the table are rendered from the two, and make lint-bench-numbers fails when the prose and the CSVs disagree, as it does for every cell of the table.

work mips32r2 insns ms (500 MHz, 1 IPC) Cortex-M3 insns rv32imac insns
AEAD seal, per 1 KB record 82 k 0.16 67 k 107 k
SHA-256, per 1 KB 68 k 0.14 57 k 90 k
x25519 scalar multiply 35.0 M 70 24.4 M 38.2 M
RSA-3072 PSS verify (default) 11.6 M 23 8.4 M 13.2 M
P-256 verify (TRUST=raw-ecdsa) 46.0 M 92 26.7 M 42.6 M
full pinned handshake crypto (default) 82.3 M 165 57.7 M 90.4 M
ML-KEM-768 keygen (KEX=pq) 1.3 M 3 1.0 M 1.2 M
ML-KEM-768 decapsulate (KEX=pq) 1.6 M 3 1.3 M 1.6 M
full hybrid handshake crypto (KEX=pq) 86.5 M 173 61.1 M 94.6 M

The multiply decomposition

The multiply decomposition that keeps secrets off a variable-time umull sets the first and third rows in every instruction column. Measured against the same benchmark with the native multiply instead:

  • on mips32r2, AEAD seal costs 73% more, x25519 22% more, and the pinned handshake 18% more, or 25 ms at 500 MHz;
  • on the Cortex-M3, AEAD seal costs 94% more, x25519 103% more, and the pinned handshake 75% more;
  • on rv32imac, AEAD seal costs 66% more, x25519 76% more, and the pinned handshake 57% more.

SHA-256 and both signature verifies are unchanged, because SHA-256 does not multiply and the verifies read only public bytes.

Flash

Flash is 31.4 kB for the default build (.text + .rodata, -Os), of which the multiply decomposition is 2.2 kB, nearly all of it poly1305's unrolled block: the total (CH_NATIVE_WIDEMUL) row of bench/results-device.csv sizes the same modules over the native multiply.

The TRUST=raw-ecdsa build trades 2.3 kB of RSA for 5.8 kB of P-256 and totals 34.9 kB. Its verify costs 4.0 times the default's on mips32r2, so the 64-byte pin costs both flash and handshake time.

The hybrid key exchange

The hybrid key exchange costs less than its wire size suggests. KEX=pq adds two ML-KEM key expansions and one decapsulation for 4.2 M instructions, 5% over the classic handshake. The key pair lives as a 64-byte seed and is re-expanded rather than stored, which docs/decisions.md 24 explains.

One ML-KEM-768 keygen is an order of magnitude cheaper than one x25519 on this core, because chapulin keeps the 16-bit-word ladder for its machine-checked overflow proof.

Where the time goes

Public-key work dominates. Every handshake runs two x25519 for forward secrecy, whichever mode it is in, so about 140 ms is the recurring floor and 85% of the pinned handshake's crypto. The signature verify is paid only on the first pinned connection; resumptions skip it. Record crypto is about 0.15 ms per kilobyte, which is negligible beside the handshake.

Because a device typically opens one long-lived connection, chapulin keeps the 16-bit-word x25519 as the default and the device path. It does so for its machine-checked overflow proof, and because a 32-bit core has no wider multiply to run a faster field on.

A host object holds a second field for a session whose caller sets CH_CPU_CONSTANT_TIME_MULTIPLY (decisions 52 and 89): five 51-bit words whose products run on the 64x64->128 multiply. On an Apple M1 Pro a scalar multiplication takes 28.5 µs on that field against 886 µs on the 16-word one over the decomposition, and the client side of a pinned RSA-3072 handshake takes 90.8 µs under the value that machine's line states below against 1.90 ms under CH_CPU_PROBED alone (bench/results-primitives-darwin-arm64-clang.csv, bench/notes-primitives.md).

chapulin beside OpenSSL

Camilo set the goal on 2026-10-03: chapulin beats OpenSSL on every primitive TLS runs, in C an auditor can read. This table states the goal as numbers. It has one row per primitive and three cells per machine and compiler: chapulin's time for one operation, OpenSSL's time for it on the same machine in the same run, and the first over the second. A ratio above 1 is a row where chapulin is behind, and it is in bold. Work on a primitive starts from its row here, and the scripts that wrote the row judge the change. The table holds two x86-64 CPUs, one without AVX-512 and one with AVX-512 IFMA. On the second, a host object whose caller sets CH_CPU_AVX512_IFMA verifies RSA on rsa_ifma.c (decision 119) and runs the ChaCha20 keystream on chacha20_avx512.c (decision 121), and with CH_CPU_CONSTANT_TIME_MULTIPLY beside it signs RSA on rsa_ifma_sign.c (decision 120) and runs Poly1305 on poly1305_ifma.c (decision 121). One whose caller sets CH_CPU_AVX512 beside the VAES and AES bits runs each AES-GCM seal and open on a 512-bit kernel (decision 130). Each CPU has two groups of cells: gcc, which CI compiles with, and the pinned clang 23, a newer clang than Zig 0.16's clang 21, which build.zig compiles the object with for a Zig project such as colibri.

one operation M1 Pro, chapulin M1 Pro, OpenSSL ratio x86-64 gcc, chapulin x86-64 gcc, OpenSSL ratio x86-64 clang, chapulin x86-64 clang, OpenSSL ratio x86-64 AVX-512 gcc, chapulin x86-64 AVX-512 gcc, OpenSSL ratio x86-64 AVX-512 clang, chapulin x86-64 AVX-512 clang, OpenSSL ratio
SHA-256, 16 KiB 6.86 µs 6.97 µs 0.98 10.4 µs 10.4 µs 0.99 10.4 µs 10.4 µs 0.99 9.47 µs 9.52 µs 0.99 9.47 µs 9.52 µs 0.99
SHA-256, 64 bytes 72.8 ns 148 ns 0.49 114 ns 193 ns 0.59 126 ns 192 ns 0.66 106 ns 157 ns 0.68 109 ns 156 ns 0.70
SHA-384, 16 KiB 11.7 µs 11.8 µs 0.99 23.1 µs 22.0 µs 1.05 22.9 µs 21.9 µs 1.04 23.1 µs 21.7 µs 1.07 22.6 µs 21.7 µs 1.04
SHA-384, 64 bytes 127 ns 185 ns 0.68 268 ns 333 ns 0.81 218 ns 332 ns 0.66 227 ns 294 ns 0.77 205 ns 294 ns 0.70
SHA3-256, 16 KiB 19.3 µs 18.6 µs 1.04 51.2 µs 38.5 µs 1.33 47.6 µs 38.5 µs 1.23 41.8 µs 31.8 µs 1.31 39.0 µs 31.8 µs 1.23
SHA3-256, 64 bytes 171 ns 341 ns 0.50 458 ns 489 ns 0.94 413 ns 490 ns 0.84 357 ns 393 ns 0.91 334 ns 393 ns 0.85
X25519 key generation 10.7 µs 27.3 µs 0.39 15.2 µs 33.8 µs 0.45 14.1 µs 33.8 µs 0.42 11.5 µs 27.3 µs 0.42 10.8 µs 27.4 µs 0.39
X25519 shared secret 28.5 µs 28.7 µs 0.99 37.5 µs 36.5 µs 1.03 33.5 µs 36.5 µs 0.92 29.3 µs 30.1 µs 0.98 27.1 µs 30.0 µs 0.90
P-256 key generation 10.1 µs 9.68 µs 1.04 19.0 µs 12.4 µs 1.54 13.7 µs 12.3 µs 1.11 14.4 µs 10.5 µs 1.37 10.7 µs 10.5 µs 1.02
P-256 shared secret 53.5 µs 41.5 µs 1.29 93.6 µs 51.7 µs 1.81 70.1 µs 51.7 µs 1.36 71.9 µs 49.6 µs 1.45 53.3 µs 49.6 µs 1.07
ECDSA P-256 sign 14.7 µs 18.3 µs 0.81 28.0 µs 22.1 µs 1.26 20.7 µs 22.2 µs 0.94 22.1 µs 19.8 µs 1.12 16.8 µs 19.8 µs 0.85
ECDSA P-256 verify 55.0 µs 55.0 µs 1.00 96.2 µs 67.7 µs 1.42 71.5 µs 67.8 µs 1.06 73.8 µs 64.2 µs 1.15 55.5 µs 64.2 µs 0.86
ECDSA P-384 verify 275 µs 488 µs 0.56 676 µs 726 µs 0.93 366 µs 726 µs 0.50 493 µs 523 µs 0.94 270 µs 520 µs 0.52
RSA-2048 PSS sign 673 µs 558 µs, PKCS#1 v1.5 sign — 1.38 ms 659 µs, PKCS#1 v1.5 sign — 1.37 ms 660 µs, PKCS#1 v1.5 sign — 268 µs 273 µs, PKCS#1 v1.5 sign — 247 µs 273 µs, PKCS#1 v1.5 sign —
RSA-2048 PSS verify 22.1 µs 14.6 µs, PKCS#1 v1.5 verify — 23.7 µs 18.9 µs, PKCS#1 v1.5 verify — 22.2 µs 18.9 µs, PKCS#1 v1.5 verify — 10.9 µs 16.1 µs, PKCS#1 v1.5 verify — 10.3 µs 16.1 µs, PKCS#1 v1.5 verify —
RSA-2048 PKCS#1 v1.5 verify 18.5 µs 14.6 µs 1.27 20.3 µs 18.9 µs 1.08 18.6 µs 18.9 µs 0.99 7.90 µs 16.1 µs 0.49 7.55 µs 16.1 µs 0.47
RSA-3072 PSS sign 2.02 ms 1.62 ms, PKCS#1 v1.5 sign — 3.89 ms 2.01 ms, PKCS#1 v1.5 sign — 4.30 ms 2.01 ms, PKCS#1 v1.5 sign — 628 µs 735 µs, PKCS#1 v1.5 sign — 591 µs 733 µs, PKCS#1 v1.5 sign —
RSA-3072 PSS verify 42.4 µs 30.9 µs, PKCS#1 v1.5 verify — 42.1 µs 40.1 µs, PKCS#1 v1.5 verify — 40.1 µs 40.1 µs, PKCS#1 v1.5 verify — 17.8 µs 34.7 µs, PKCS#1 v1.5 verify — 17.4 µs 34.6 µs, PKCS#1 v1.5 verify —
RSA-3072 PKCS#1 v1.5 verify 38.0 µs 30.9 µs 1.23 38.0 µs 40.1 µs 0.95 35.6 µs 40.1 µs 0.89 14.1 µs 34.7 µs 0.41 14.2 µs 34.6 µs 0.41
ML-KEM-768 key generation 12.2 µs 22.0 µs 0.55 20.1 µs 36.4 µs 0.55 17.0 µs 36.3 µs 0.47 18.2 µs 31.7 µs 0.57 14.3 µs 31.4 µs 0.45
ML-KEM-768 encapsulation 14.2 µs 13.1 µs 1.08 22.7 µs 21.7 µs 1.05 19.2 µs 21.7 µs 0.88 20.2 µs 17.8 µs 1.14 16.0 µs 17.8 µs 0.90
ML-KEM-768 decapsulation 15.3 µs 18.7 µs 0.82 26.0 µs 33.4 µs 0.78 21.1 µs 33.3 µs 0.63 22.6 µs 27.8 µs 0.81 17.9 µs 27.9 µs 0.64
AES-128-GCM, the key set and one 16 KiB record sealed 2.01 µs 2.02 µs 0.99 2.77 µs 4.05 µs 0.68 2.48 µs 4.05 µs 0.61 942 ns 990 ns 0.95 968 ns 985 ns 0.98
AES-128-GCM, the key set and one 16 KiB record opened 1.99 µs 2.04 µs 0.97 2.75 µs 4.12 µs 0.67 2.58 µs 4.12 µs 0.62 977 ns 1.03 µs 0.95 944 ns 1.04 µs 0.91
AES-256-GCM, the key set and one 16 KiB record sealed 2.35 µs 2.41 µs 0.97 3.13 µs 4.37 µs 0.72 2.81 µs 4.36 µs 0.64 1.14 µs 1.18 µs 0.96 1.17 µs 1.18 µs 0.99
AES-256-GCM, the key set and one 16 KiB record opened 2.33 µs 2.40 µs 0.97 3.08 µs 4.42 µs 0.70 2.93 µs 4.43 µs 0.66 1.14 µs 1.23 µs 0.92 1.15 µs 1.23 µs 0.93
ChaCha20-Poly1305, 16 KiB encrypted and hashed, no tag 10.5 µs 8.95 µs 1.17 9.27 µs 7.47 µs 1.24 8.73 µs 7.47 µs 1.17 3.95 µs 3.93 µs 1.00 3.95 µs 3.95 µs 1.00
ChaCha20-Poly1305, 16 KiB hashed and decrypted, no tag 10.5 µs 9.14 µs 1.15 9.25 µs 7.49 µs 1.24 8.71 µs 7.48 µs 1.16 3.67 µs 3.96 µs 0.93 3.64 µs 3.96 µs 0.92

The machines, as the CSV headers state them:

  • M1 Pro: Apple M1 Pro, Darwin 25.6.0, Apple clang version 21.0.0 (clang-2100.3.34.2), OpenSSL 3.6.4, ch_cfg.cpu 0xe7, and 0x83 for the AES-GCM rows and 0x7 for the ChaCha20-Poly1305 rows, as CH_CPU_CONSTANT_TIME_SHA3 runs AES-GCM's passes with EOR3 (decision 158) and no other hash bit changes a record; one-minute load average 3.15 before the primitives' run and 4.37 after it, and 4.13 and 4.04 around the AEAD rows' run
  • x86-64 gcc: AMD EPYC 7763 64-Core Processor, Linux 7.0.0-1017-azure, gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0, OpenSSL 3.6.4, ch_cfg.cpu 0x43f, and 0x1f for the AEAD rows, which no hash bit changes; one-minute load average 1.13 before the primitives' run and 1.27 after it, and 1.41 and 1.30 around the AEAD rows' run
  • x86-64 clang: AMD EPYC 7763 64-Core Processor, Linux 7.0.0-1017-azure, Ubuntu clang version 23.1.5 (++20261028083854+a738f2737189-1exp120261028203912.92), OpenSSL 3.6.4, ch_cfg.cpu 0x43f, and 0x1f for the AEAD rows, which no hash bit changes; one-minute load average 1.18 before the primitives' run and 1.18 after it, and 1.14 and 1.13 around the AEAD rows' run
  • x86-64 AVX-512 gcc: INTEL(R) XEON(R) PLATINUM 8573C, Linux 7.0.0-1017-azure, gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0, OpenSSL 3.6.4, ch_cfg.cpu 0x73f, and 0x31f for the AEAD rows, which no hash bit changes; one-minute load average 1.18 before the primitives' run and 1.27 after it, and 0.83 and 1.15 around the AEAD rows' run
  • x86-64 AVX-512 clang: INTEL(R) XEON(R) PLATINUM 8573C, Linux 7.0.0-1017-azure, Ubuntu clang version 23.1.5 (++20261028083854+a738f2737189-1exp120261028203912.92), OpenSSL 3.6.4, ch_cfg.cpu 0x73f, and 0x31f for the AEAD rows, which no hash bit changes; one-minute load average 1.06 before the primitives' run and 1.06 after it, and 1.19 and 1.14 around the AEAD rows' run

bench/primitives.sh (make bench-primitives) writes the rows above the AEADs to a CSV for each machine and compiler, such as bench/results-primitives-darwin-arm64-clang.csv, and bench/record.sh writes the six AEAD rows to the record CSVs of the next section. tools/bench_scoreboard.py renders the table and the machines' lines from those files, and make lint-bench-numbers fails when this document disagrees with them.

How the figures are taken:

  • chapulin. Every timed program is a host object, and each row runs under the ch_cfg.cpu value a caller on that CPU states, every bit the CPU has (decision 89). The machine's line above names the value. The CSV also holds each row under ch_cfg.cpu 0x1, CH_CPU_PROBED alone, which a caller that states nothing runs: the 16x16 multiply decomposition and the 16-word X25519 field, which a device object runs too. bench/notes-primitives.md reads those rows and the handshakes'.
  • OpenSSL. openssl speed -mr -seconds 1, on the binary test/e2e.sh takes (bench/openssl.sh). Each machine's line names its version.
  • The clock. openssl speed divides the operations it ran by the user CPU time its process took. bench/primitives.c times each sample on its thread's CPU clock. So neither side counts the time a loaded machine gave to other work. The load in a machine's line can still move both through the caches and the cores they share with that work, and on an M1 Pro through a run that macOS places on efficiency cores, which run the same code in about three times the time. bench/primitives.sh warns of a row whose runs differ by more than half, and the CSV's run_spread_pct states each row's spread. The AEAD rows are bench/record.c's, batches of 1 ms on the wall clock.
  • The runs. Each figure is the median of five runs. One run of bench/primitives.sh is every chapulin row and then one second of each OpenSSL row, so the two sides of a ratio are never more than a run apart.
  • Instructions. On macOS the primitives CSV also holds the instructions each chapulin operation retired, as /usr/bin/time -l counts them for a process. A count does not move with the machine's load, so it shows what a change did to a row where a time cannot.

What a row compares:

  • RSA. openssl speed signs and verifies RSA with PKCS#1 v1.5 padding. chapulin signs PSS alone, so each PSS row shows OpenSSL's PKCS#1 v1.5 time and no ratio. The two PKCS#1 v1.5 verify rows compare one operation: chapulin checks a SHA-256 DigestInfo, and openssl speed a 36-byte value without one.
  • Key generation and encapsulation. chapulin's calls take their random bytes as arguments, so its rows time no draw. OpenSSL's rows draw theirs, and its key generation also builds a key object.
  • ECDSA. chapulin derives its nonce by RFC 6979 and signs a 32-byte digest. openssl speed draws its nonce and signs 20 bytes.
  • AES-GCM. One operation of OpenSSL 3.6's speed -aead sets the key and the IV, hashes 13 bytes of associated data, encrypts 16,384 bytes and computes the tag. chapulin's row is the operation record.c runs for every record: it expands the key, seals 16,385 bytes with the record's 5-byte header as associated data, and wipes the expanded key. On arm64 bench/record.sh times AES-GCM under ch_cfg.cpu 0x3, and the multiply bit changes no AES-GCM path. Where the CPU has FEAT_SHA3 it also times it under 0x83, which runs the copy with EOR3 (decision 158). On the x86-64 CPU without AVX-512 it times AES-GCM under 0x1f. On the one with AVX-512 it times AES-GCM under 0x31f, under which CH_CPU_AVX512 beside the VAES and AES bits runs each seal and open whole on the 512-bit kernel (decision 130), and CH_CPU_AVX512_IFMA changes no AES-GCM path.
  • ChaCha20-Poly1305. One operation of OpenSSL 3.6's speed -aead is one update over 16,384 bytes: ChaCha20 and Poly1305 over them, with no nonce, associated data or tag. chapulin's row is its seal or open of one record less the tag's fixed work, which is the same work on the bytes. The next section has the rows for a whole record, for rec_seal and for each stage.

What the M1 Pro's column shows:

  • The column is one run of 2026-10-28 on the tree of 0b1608d1, beside OpenSSL 3.6.4, the release tools/toolchain.env pins, built as CI builds it (no-shared no-docs). The run of 2026-10-12 timed Homebrew's OpenSSL 3.6.5, and Homebrew's openssl@3 is now 3.5.9. The two OpenSSL builds differ: 3.6.4 took 0.51 to 0.62 of 3.6.5's time for ML-KEM-768, 1.57 times for P-384 verification, and 0.88 to 1.03 times for its other rows. So the ratios of those four rows moved with OpenSSL. Against the run of 2026-10-12, X25519 took 0.32 of its time for key generation and 0.86 for a shared secret, and AES-GCM 0.82 to 0.86. chapulin's other rows took 0.97 to 1.08 times their time, in a run whose one-minute load average reached 9.3.
  • SHA-256 and SHA-384 take 0.98 and 0.99 of OpenSSL's time over 16 KiB, and 0.49 and 0.68 of it over 64 bytes. Under their bits a session's hashes run on the ARMv8 SHA-256 and SHA-512 instructions (decision 93), and so do the HMACs of a server's ECDSA nonce (decision 102). A hash call that takes no ch_cfg.cpu value runs no hash instruction: a certificate's, the RSA signer's and the DRBG's. Its SHA-256 and SHA-3 run portable C, and its SHA-384 the plain C of decision 127, sha512_compress_blocks.c, as does a session's without the SHA-512 bit. The CSV's rows under ch_cfg.cpu 0x1 time that path: 29.3 µs over 16 KiB and 253 ns over 64 bytes, 2.48 and 1.37 times OpenSSL's time, where the portable C took 51.9 µs and 607 ns.
  • The two ECDSA verifiers run on 64-bit words in every session, where the 32-bit words took 23 and 13 times OpenSSL's time. P-384 verifies in 0.56 of OpenSSL 3.6.4's time, and took 0.88 of 3.6.5's: six 64-bit words, coordinates kept in the Montgomery domain and one pass over both scalars' signed digits (decision 97). P-256 verifies in OpenSSL's time, on variable-time Jacobian points in one pass, as P-384's (decision 104), with the field's additions inline (decision 114) and s's inverse by a binary GCD that stops once a is zero (decisions 115 and 116). The wide files' constant-time scalar multiplications took 1.89 times (decision 104), and before decisions 114 to 116 it took 1.29 times.
  • SHA3-256 takes 1.04 times OpenSSL's time over 16 KiB and 0.50 of it over 64 bytes. Under CH_CPU_CONSTANT_TIME_SHA3 an arm64 object that clang compiled runs Keccak on the ARMv8 SHA-3 instructions, as OpenSSL does (decision 99). On the portable code it took 1.58 and 0.89 times, and 3.33 and 1.19 before sha3.c moved eight bytes at a time (decision 98).
  • RSA verifies in 1.27 and 1.23 times OpenSSL's time, on rsa_mont64.c's 64-bit words in every session, with R^2 from a long division of the public modulus and a square of its own (decisions 103 and 106), under clang a step that adds the carry from the word below last (decision 117), and under clang for arm64 multiplications and squares in blocks of four words (decision 118). Before decision 118 it took 1.51 and 1.52 times, before decision 117 1.75 to 1.81 times, and on the 32-bit words 18 to 21 times. openssl speed verifies under a key object that keeps the Montgomery values it computed the first time. A client verifies under each certificate's key once, and with the key built from its bytes before each verification OpenSSL took 26.2 and 47.3 µs (decision 103). Under the multiply bit an RSA-2048 PSS signature takes 673 µs, by the Chinese remainder theorem with a check of every signature, where OpenSSL's PKCS#1 v1.5 signature takes 558 µs. Before decision 118, on rsa_mont64.c's loops alone, it took 813 µs, and the ladder took 37.6 ms (decision 95).
  • P-256's key generation and shared secret take 1.04 and 1.29 times OpenSSL's time, and its signature 0.81 of it. Under the multiply bit they run on four 64-bit words, k·G adds 43 entries of an 86 KiB table of multiples of G (decisions 94 and 109), 41 of them on nine products and two squares (decision 111), the key exchange doubles and adds in Jacobian coordinates (decision 112), the field and the scalar square on ten products (decision 105), the field adds and subtracts inline (decision 114), an inverse runs Pornin's binary GCD (decision 115), and a server's nonce runs on the SHA-256 instructions (decision 102). On the 32-bit words they took 15 to 63 times. Before decisions 114 and 115 they took 1.35, 1.43 and 1.23 times, before decisions 111 and 112 1.57, 1.67 and 1.35, and before decisions 108 and 109 1.99, 1.91 and 1.56. A field multiply in C is what is left, and "Where a server handshake's instructions go" below orders the work.
  • X25519 takes 0.39 of OpenSSL's time for key generation and 0.99 of it for a shared secret. Under the multiply bit a public key adds 64 entries of a 30 KiB table of multiples of edwards25519's base point (decision 148), and the ladder swaps the points once a step (decision 152). AES-GCM takes 0.97 to 0.99 of OpenSSL's time, on the passes of decisions 158 and 161 under 0x83, and ChaCha20-Poly1305 1.15 and 1.17 times.
  • ML-KEM-768 takes 0.55 of OpenSSL 3.6.4's time for key generation, 1.08 times for encapsulation and 0.82 for decapsulation. Against Homebrew's 3.6.5 it took 0.33, 0.58 and 0.40 of its time in the run of 2026-10-12. Its hashes run on the SHA-3 instructions, its matrix sampler squeezes eight groups a call (decision 100), and its NTT and base multiplication run on NEON (decision 101).
  • OpenSSL's ML-KEM-768 rows run on one key object, which holds the matrix OpenSSL expanded when it made the key, and chapulin's start from the key's bytes, as a handshake's do. Decoding the public key first took Homebrew's OpenSSL 3.6.5's encapsulation on this M1 Pro from 23.9 µs to 36.0 µs (decision 100).

What the columns of the x86-64 CPU without AVX-512 show, where they differ from the M1 Pro's:

  • Both compilers' columns are one run of 2026-10-29 on the tree of 38f9a395, on an AMD EPYC 7763, the CPU the runs on the trees of decisions 107, 109, 110, 113, 116, 117, 119, 123, 124, 128 and 130 and on 0b1608d1 drew as well. Its rows run under ch_cfg.cpu 0x43f, which holds CH_CPU_BMI2_ADX. The bench job stops on a CPU with AVX-512, so these columns never run rsa_ifma.c or the 512-bit AES-GCM kernel. The runner's CPU changes from run to run, so a column compares rows of one run with each other, and with an earlier run's only on the same CPU. Against the run of 2026-10-28 on the tree of 0b1608d1, OpenSSL's rows took 1.00 to 1.02 of their time. An X25519 shared secret took 0.94 of its time under gcc and 0.79 under clang, on x25519_adx.c (decisions 163 and 165). SHA-384 over 64 bytes took 1.17 times its time under gcc and 0.96 of it under clang, and no change to its code accounts for that. No other row moved by more than 2%. On the tree of 0b1608d1, against the run on the tree of decision 130, OpenSSL's rows took 0.99 to 1.01 of their time. X25519 took 0.27 of its time for key generation and 0.74 for a shared secret under gcc, and 0.30 and 0.90 under clang (decisions 148 and 152). Under gcc RSA's two signatures took 1.04 and 1.06 times their time, P-256's signature 1.04 times and ML-KEM-768's three rows 0.97, and no other row moved by more than 2%. Under clang SHA-256 and SHA-384 over 64 bytes took 1.07 and 1.10 times their time, and no other row moved by more than 2%. On the tree of decision 130, against the run on the tree of decision 128, under gcc no row of this table moved by more than 2%, and under clang SHA-256 over 64 bytes took 1.04 times its time. On the tree of decision 128, against the run on the tree of decision 124, one change had moved rows of this table: SHA-384 took 0.51 of its time over 16 KiB and 0.57 over 64 bytes under gcc, and 0.47 and 0.39 under clang (decisions 127 and 128, the SHA bullet below). Under gcc three more kinds of row had moved by more than 2%: RSA's two signatures took 0.94 and 0.95 of their time, and ML-KEM-768's three rows and P-384's verification 1.02 to 1.03 times. No change to their code accounts for those. Between the two trees the code those rows run changed in one place: mlk_poly_frommsg reads a message bit through a volatile object (decision 125), and a key generation does not call it. RSA on 64-bit words runs at one of two speeds on this CPU by where the linker places it (the pitfalls table below). Under clang no other row of this table had moved by more than 2.1%. On the tree of decision 124, against the run on the tree of decision 123, two changes had moved rows. Under clang ML-KEM-768's three rows took 0.90 to 0.92 of their time and the ChaCha20-Poly1305 seal 0.97: decision 124 aligned the frames of the two AVX2 kernels they run on. Under both compilers the rows of the hashes, HMAC, HKDF's label expansion, Poly1305 and the ChaCha20-Poly1305 seal took 0.67 to 0.96 of their time, and the 64-byte hashes 0.67 to 0.88, because the bench's sink now consumes one byte of an output where it consumed every byte (the pitfalls table below). On the tree of decision 123, against the run on the tree of decision 119, RSA's verifications and the AES-GCM rows had moved, and under gcc RSA's signatures, as the bullets below state. The RSA handshakes moved with them, and bench/notes-primitives.md reads those. Against the run on the tree of decision 113, decisions 114 to 116 took P-256's key generation, shared secret, signature and verification under gcc from 1.64, 1.97, 1.60 and 1.63 times OpenSSL's time to 1.49, 1.80, 1.21 and 1.41 on the tree of decision 119, and before that decisions 111 to 113 took the first three from 2.08, 2.16 and 1.84 times, and X25519's two rows from 1.82 and 1.68 to 1.60 and 1.49.
  • The clang column is the same object built by the pinned clang 23 on the same CPU in the same job. clang's code is faster on the wide fields' carry chains: P-256 takes 0.72 to 0.75 of gcc's time, which puts its signature at 0.94 of OpenSSL's time and its verification at 1.06 times, and P-384 verifies in 0.54 of gcc's time. It is also faster for ML-KEM-768, 0.81 to 0.85 of gcc's time, and AES-GCM, 0.90 to 0.95, and every ML-KEM-768 and AES-GCM row is ahead of OpenSSL. RSA verifies in 0.91 to 0.94 of gcc's time, both on rsa_avx2.c. RSA-2048 signs in 0.99 of gcc's time and RSA-3072 in 1.11 times it: gcc runs the signer's products in rows of four words on _addcarry_u64 chains, and clang keeps rsa_mont64.c's loops (decision 122). SHA-384 takes 0.99 of gcc's time over 16 KiB and 0.81 of it over 64 bytes, both on sha512_avx2.c.
  • AES-GCM is ahead of OpenSSL. Under the VAES bit the seal and the open run the VAES and VPCLMULQDQ kernels of decision 90, and with the key expansion they take 0.67 to 0.72 of OpenSSL's time under gcc, where they took 0.76 to 0.83 on the tree of decision 119. aes_hw.c now expands a key a round key at a time in vector registers (decision 123): under gcc the AES-128 expansion takes 24 ns where it took 432, and the AES-256 expansion 26 ns where it took 526. ChaCha20-Poly1305 takes 1.24 times OpenSSL's time to seal and to open under gcc, and 1.17 times under clang. Under CH_CPU_AVX2 and the multiply bit its keystream runs eight blocks a pass in 256-bit vectors (decision 90) and its Poly1305 eight blocks at a time in four AVX2 lanes (decision 110). Of the 9.27 µs that sealing a 16 KiB record takes under gcc, less the tag's fixed work, the keystream is 6.51 and Poly1305 2.70, where OpenSSL takes 7.47.
  • ML-KEM-768's key generation, encapsulation and decapsulation take 0.55, 1.05 and 0.78 of OpenSSL's time under gcc, against rows that hold a key's expanded matrix (decision 100). Under CH_CPU_AVX2 its matrix runs on four Keccak states side by side (decision 107), and its NTT and base multiplication run on SSE2 (decision 101).
  • P-384 verifies in 0.93 of OpenSSL's time under gcc and P-256 in 1.42 times, on the 64-bit words of decisions 97 and 96 and, for P-256, the Jacobian points of decision 104 and the inverse of decisions 115 and 116, where the 32-bit words took 8.8 and 30 times.
  • SHA-256 runs in 0.99 of OpenSSL's time over 16 KiB and in 0.59 of it over 64 bytes under gcc, on the SHA extensions (decision 93). SHA-384 takes 1.05 times OpenSSL's time over 16 KiB and 0.81 of it over 64 bytes under gcc, and 1.04 times and 0.66 of it under clang. A session of a host object compresses on sha512_compress_blocks.c, plain C with no bit (decision 127), and one with CH_CPU_AVX2 in a SUITE=aesgcm object on sha512_avx2.c, whose message schedule runs in 256-bit registers, a quarter of a schedule step after each round (decision 128). The table's rows run the second. The first, the CSV's rows under ch_cfg.cpu 0x1, takes 1.42 times OpenSSL's time over 16 KiB and 0.90 of it over 64 bytes under gcc, and 1.44 times and 0.86 of it under clang. On the tree of decision 124, which held no x86-64 SHA-512 code, the portable C took 2.06 and 1.21 times under gcc and 2.20 and 1.57 times under clang (decision 93). SHA3-256 takes 1.33 times and 0.94 of it under gcc, where sha3.c's loops took 7.68 and 5.02 times (decision 98).
  • P-256's key generation and shared secret take 1.54 and 1.81 times OpenSSL's time under gcc, and its signature 1.26 times, on the wide files with the table of decision 109, the additions of decision 111, the Jacobian key exchange of decision 112, the inline additions of decision 114 and the binary GCD of decision 115, where the native copies of the 32-bit files took 21 to 90 times. gcc compiles the wide files' carry steps from the two x86-64 intrinsics (decision 94).
  • RSA verifies in 1.08 and 0.95 times OpenSSL's time under gcc and in 0.99 and 0.89 of it under clang. This CPU has no AVX-512 IFMA, so under CH_CPU_AVX2 the public operation runs in rsa_avx2.c's 28-bit digits, four to a 256-bit register (decision 122). On the tree of decision 119 it ran on rsa_mont64.c's loops and took 1.87 and 1.99 times under gcc (decisions 95 and 117). An RSA-2048 PSS signature takes 1.38 ms under gcc and 1.37 ms under clang, where OpenSSL's PKCS#1 v1.5 signature takes 659 and 660 µs. Under gcc it took 1.35 ms in the run of 2026-10-28, 1.27 ms in the run on the tree of decision 130, 1.35 ms on the tree of decision 124, on the same code, and 1.48 ms on the tree of decision 119. The signer runs on 64-bit words under the multiply bit, and under gcc its products run in rows of four words on _addcarry_u64 chains (decision 122).
  • X25519 takes 0.45 of OpenSSL's time for key generation and 1.03 times for a shared secret on gcc's code, and 0.42 and 0.92 of it on clang's. A public key adds entries of the table of decision 148 on the wide field. Under CH_CPU_BMI2_ADX beside the multiply bit a shared secret runs on x25519_adx.c's four 64-bit words, whose products, sums and differences are MULX, ADCX and ADOX in inline assembly (decision 165). On the wide field, in the run of 2026-10-28, it took 1.09 times OpenSSL's time under gcc and 1.17 times under clang. Both compilers emit the same instructions for the assembly, and the ladder around it is C: clang's code takes 0.89 of gcc's time.

What the AVX-512 machine's columns show:

  • Both compilers' columns are one run of 2026-10-29 on the tree of 38f9a395, on an Intel Xeon Platinum 8573C, which the job drew on its fourth try. Its rows run under ch_cfg.cpu 0x73f: the bits the CPU without AVX-512 states, CH_CPU_AVX512_IFMA and CH_CPU_AVX512. The ChaCha20-Poly1305 rows run under 0x11f and the AES-GCM rows under 0x31f. The run before it, of 2026-10-13 on the tree of decision 130, drew the same model on another host. Against that run OpenSSL's rows took 0.77 to 0.84 of their time under both compilers, and chapulin's primitives under ch_cfg.cpu 0x1 took 0.80 to 0.89 of theirs, so this host is faster. The two runs compare by their ratios to OpenSSL alone. The two x86-64 machines are different CPUs, so both chapulin's and OpenSSL's times change between them. OpenSSL's change most where its source holds AVX-512 code. Under gcc an AES-128-GCM seal takes 990 ns here and 4.05 µs on the CPU without AVX-512, a ChaCha20-Poly1305 seal 3.93 and 7.47 µs, and an RSA-2048 signature 273 and 659 µs.
  • X25519 takes 0.42 of OpenSSL's time for key generation and 0.98 of it for a shared secret under gcc, and 0.39 and 0.90 under clang. In the run of 2026-10-13 it took 1.39 and 1.26 times under gcc, and 1.23 and 1.11 times under clang. A public key now adds entries of the table of decision 148, and the ladder swaps the points once a step (decision 152). Under CH_CPU_BMI2_ADX a shared secret runs on x25519_adx.c's four 64-bit words, whose products are MULX's (decision 165).
  • RSA verifies in 0.49 and 0.41 of OpenSSL's time under gcc and 0.47 and 0.41 under clang. Under CH_CPU_AVX512_IFMA the public operation runs in rsa_ifma.c's 52-bit digits on AVX-512 IFMA (decision 119). OpenSSL 3.6.4 verifies on its MULX Montgomery multiplication, bn_mulx4x_mont in crypto/bn/asm/x86_64-mont.pl.
  • An RSA-2048 PSS signature takes 268 µs under gcc and 247 µs under clang, where OpenSSL's PKCS#1 v1.5 signature takes 273 µs under both. An RSA-3072 one takes 628 and 591 µs, where OpenSSL's takes 735 and 733. Under CH_CPU_AVX512_IFMA and the multiply bit, rsa_sign64.c computes the two exponentiations side by side on rsa_ifma_sign.c and checks the signature on rsa_ifma.c (decision 120). On a CPU with AVX-512 IFMA, OpenSSL 3.6.4 runs the two exponentiations of a 2,048-, 3,072- or 4,096-bit private key on it (ossl_rsaz_mod_exp_avx512_x2, crypto/bn/asm/rsaz-2k-avx512.pl and its 3k and 4k files). On the tree of decision 119 chapulin signed on rsa_sign64.c's 64-bit words under the multiply bit, and on a runner of this model an RSA-2048 signature took 1.24 ms under gcc and 886 µs under clang, where OpenSSL's took 324 and 303 µs.
  • ChaCha20-Poly1305 takes OpenSSL's time to seal to within 1% under both compilers, and 0.93 of it to open under gcc and 0.92 under clang. On the tree of decision 119 it took 2.24 to 2.54 times. Under CH_CPU_AVX512_IFMA the keystream runs sixteen blocks a pass in 512-bit vectors, and beside the multiply bit Poly1305 runs sixteen blocks at a time in eight AVX-512 IFMA lanes (decision 121). Of a 16 KiB record under gcc the keystream takes 2.83 µs and Poly1305 1.05, where the AVX2 kernels take 6.04 and 2.62 under 0x1f. A seal with its tag, the CSV's aead_seal, takes 4.13 µs where its parts sum to 3.88. OpenSSL 3.6.4's source holds AVX-512 paths for both, in crypto/chacha/asm/chacha-x86_64.pl and crypto/poly1305/asm/poly1305-x86_64.pl. On this CPU the whole rows are less steady than their parts: between the five runs before this one, the open row spread by 5.9% under gcc and the seal row by 2.2% under clang, where the keystream's and Poly1305's rows spread by 0.2% at most.
  • AES-GCM takes 0.95 to 0.96 of OpenSSL's time to seal and 0.92 to 0.95 to open under gcc, and 0.98 to 0.99 and 0.91 to 0.93 under clang. Under CH_CPU_AVX512 beside the VAES and AES bits each seal and open runs whole in one call of gcm_avx512_kernel.c: 32 blocks a step in eight 512-bit registers, with GHASH between the AES rounds (decision 130). Under 0x1f, on gcm_vaes.c's loops two blocks to a 256-bit register (decision 90), the same four rows take 1.91 to 2.05 times OpenSSL's time under gcc and 1.89 to 1.97 times under clang, so the kernel takes 0.46 to 0.52 of their time. The key expansion of decision 123 takes 18 and 20 ns under gcc, where it took 410 and 512 ns. OpenSSL 3.6.4's source holds AVX-512 code for AES-GCM, crypto/modes/asm/aes-gcm-avx512.pl.
  • Under clang P-256's signature and verification take 0.85 and 0.86 of OpenSSL's time, its key generation 1.02 times and its shared secret 1.07 times, and P-384 verifies in 0.52 of it. Under gcc P-256 takes 1.12 to 1.45 times and P-384 0.94.
  • SHA-384 takes 1.07 times OpenSSL's time over 16 KiB under gcc and 1.04 times under clang, and 0.77 and 0.70 of it over 64 bytes, on sha512_avx2.c (decision 128). In the run on the tree of decision 124 the portable C took 1.25 to 1.85 times. SHA3-256 over 16 KiB takes 1.31 times under gcc and 1.23 under clang. ML-KEM-768's encapsulation takes 1.14 times OpenSSL's time under gcc and 0.90 of it under clang, and its key generation and decapsulation 0.45 to 0.81 of it.

To record a column again after a change to a primitive:

make bench-primitives              # every row above the AEADs; about nine minutes
make bench-record                  # the six AEAD rows; about a minute
python3 tools/bench_scoreboard.py  # prints the table and the machines' lines to paste here
make lint-bench-numbers

The four x86-64 column groups are bench.yml's record-x86_64 job, a matrix of two jobs. Each job runs both scripts under gcc and under the pinned clang on its own GitHub runner: one on a CPU without AVX-512, and one, with BENCH_MACHINE=avx512, on a CPU with AVX-512 IFMA. A runner's CPU model is drawn, not chosen, so each job first reads /proc/cpuinfo and stops when it drew the wrong kind; gh run rerun <run> --failed draws again. The runners with AVX-512 IFMA are several CPU models, and a record compares with the one before it only on the same model. gh workflow run bench.yml --ref <branch> starts both jobs, and with -f avx512_model=8573C the second also stops unless its model name holds that text. The CSVs the jobs keep in their results-linux-x86_64 and results-linux-x86_64-avx512 artifacts, results-primitives-linux-x86_64[-avx512]-<compiler>.csv and results-record-linux-x86_64[-avx512]-<compiler>.csv for gcc and clang, go into bench/.

Where a record's time goes

bench/record.sh (make bench-record) times the protection of one TLS record and splits it into its stages, for #184 (AES-GCM) and #181 (ChaCha20-Poly1305). It compiles the library sources with the flags make lib uses and the defines of a SUITE=aesgcm host object, once for each ch_cfg.cpu value it times, and every row hands that value to the calls a session hands its own to (decision 89). The values are 0x3, the probe's bit and the AES bit; 0x7, which adds the multiply bit and so the vector Poly1305 (decision 83); on an x86-64 CPU with the instructions, 0x1f, which adds the kernels of decisions 90 and 110; and, on one with AVX-512 IFMA, 0x11f, which adds CH_CPU_AVX512_IFMA and so the ChaCha20-Poly1305 kernels of decision 121; and, on one with AVX-512F, AVX-512BW and AVX-512VL, 0x31f, which adds CH_CPU_AVX512 and so the 512-bit AES-GCM kernel of decision 130, for the AES-GCM rows alone. Under that value the kernel runs each whole AEAD, and the stage rows still time the VAES loops, which it does not call. It times these on the same buffers:

  • rec_seal, which copies the caller's plaintext into the record and seals it in place, as ch_write calls it, and rec_open, which opens a record in place, as ch_read calls it;
  • the AEAD calls record.c makes, with the same buffers;
  • each stage of the AEAD: a library function where the stage has one, and an entry bench/record_stages.h adds where the stage is static; for AES-GCM that includes counter mode and GHASH alone, which the seal and the open no longer run apart over whole passes, beside the loops that run them together;
  • rec_seal and rec_open compiled with stubs in place of the AEAD, which is the record layer's own work.

One stage has no function of its own, so its row is a difference of two timed rows: chacha20_xor_cpu less the keystream's passes gives the ChaCha20 exclusive-or. The passes' row is chacha20_vector.c's, eight blocks a pass on NEON and four on SSE2, the AVX2 kernel's, or under 0x11f the AVX-512 kernel's, with each pass's keystream stored to a buffer. AES-GCM's exclusive-or has no row: gcm_hw.c runs it in the same pass as the AES rounds (below), so its time is inside counter mode's and the loops'.

Each figure is the median of five runs, and each run's figure is the median of 15 batches of at least 1 ms, with every row's batches interleaved. The CSVs hold every row at 1 KiB, 16 KiB and 64 KiB, each with its spread and its batches at the 10th and 90th percentile, and a check line that compares each whole with the sum of its parts.

The machine is an Apple M1 Pro that ran other work: its one-minute load average was 4.0 to 4.1 for the macOS column's run of 2026-10-28, and 6.3 and 7.1 before the VM's runs of 2026-10-04. A Linux CSV's own load average line is the VM's, and its first line holds the host's. So these figures are the filter's, in the terms of the method below. The three columns are three CSVs:

Each row's build column names the ch_cfg.cpu value it ran under:

  • ch_cfg.cpu 0x3 is a host session whose caller set CH_CPU_CONSTANT_TIME_AES: the AES-GCM rows, and ChaCha20-Poly1305 with Poly1305 on the 16x16 decomposition;
  • ch_cfg.cpu 0x7 adds CH_CPU_CONSTANT_TIME_MULTIPLY, and its ChaCha20-Poly1305 rows run the vector Poly1305. The bit changes no AES-GCM path, so the 0x3 AES-GCM rows are that session's too.
  • ch_cfg.cpu 0x83, on an arm64 CPU with FEAT_SHA3, adds CH_CPU_CONSTANT_TIME_SHA3, and its AES-GCM rows run the passes with EOR3 (decision 158). The tables read the 0x3 rows.

Since decision 158 an arm64 host object runs each AES-GCM seal and open whole in gcm_arm64.c, and calls none of gcm_hw.c's loops. So the macOS column has no figure for the two rows of those loops. Its rows of counter mode alone and GHASH alone still time gcm_hw.c's and ghash_hw.c's loops, which the AEAD does not call there. The VM's columns are runs of 2026-10-04, before decision 158.

A host session never runs chacha20.c's portable loop, so no row here times it; bench/aead.sh times it in a device object's sources. The tables this section held before decision 89, when build variables chose the paths, had rows for that loop, and the text below names their figures where it compares against them, with the commit that measured them.

stompy's chapulin compiles with Zig's clang, so the clang columns are the nearest to it. A share in parentheses is the stage's part of rec_seal. The last rows of each table are two other libraries on the same machine: openssl speed -aead, the median of five one-second runs, and Zig 0.16.0's std.crypto, timed as the other rows are, which the VM does not have.

What one operation of openssl speed -aead holds depends on the OpenSSL release, and the macOS column's is 3.6.4 where the VM's is 3.0.13. So each table has a row for each operation, and a column fills the rows its release times (bench/record.sh states where each reading comes from):

  • OpenSSL 3.0 sets the IV, hashes 13 bytes of associated data, encrypts the record and computes the tag, under a key set before the loop. chapulin's row for that operation is the AEAD's seal or open.
  • OpenSSL 3.6 does the same for AES-GCM and sets the key as well, for every operation. chapulin's row for that operation is the AEAD under a key expanded for the record, which is what record.c's seal_aes_gcm and open_aes_gcm run: rec_dir keeps the key's bytes and no expanded key, so every record pays the expansion.
  • For ChaCha20-Poly1305, OpenSSL 3.6 runs one update over the record's bytes, with no nonce, associated data or tag. chapulin's row for that operation is the seal or the open less its tag's fixed work.
AES-128-GCM, one 16 KiB record, µs macOS, Apple clang 21 Linux VM, clang 18 Linux VM, gcc 13
rec_seal 2.3 3.1 3.2
rec_seal without its AEAD 0.2 (10%) 0.6 (19%) 0.7 (21%)
the AEAD under a key expanded for the record, as seal_aes_gcm runs it 2.1 2.8 2.8
the AEAD's seal, gcm_traffic_seal 2.1 (89%) 2.6 (83%) 2.5 (79%)
counter mode and GHASH in one loop, the seal's whole passes — 2.5 (80%) 2.5 (77%)
counter mode alone, in place 1.5 1.7 1.7
GHASH alone, over the ciphertext 1.0 1.1 1.4
rec_open 2.1 2.8 2.8
the AEAD under a key expanded for the record, as open_aes_gcm runs it 2.1 2.8 2.8
the AEAD's open, gcm_traffic_open 2.1 2.6 2.5
GHASH and counter mode in one loop, the open's whole passes — 2.5 2.4
OpenSSL, the key set and one record sealed 2.0 — —
OpenSSL, the key set and one record opened 2.0 — —
OpenSSL, one record sealed under a key set before — 2.8 2.8
OpenSSL, one record opened under a key set before — 2.9 2.9
Zig std.crypto, one record sealed 4.3 — —
AES-256-GCM, one 16 KiB record, µs macOS, Apple clang 21 Linux VM, clang 18 Linux VM, gcc 13
rec_seal 2.7 3.6 3.7
the AEAD under a key expanded for the record, as seal_aes_gcm runs it 2.5 3.2 3.3
the AEAD's seal, gcm_traffic_seal 2.5 (91%) 3.0 (82%) 3.0 (80%)
counter mode and GHASH in one loop, the seal's whole passes — 2.9 (80%) 3.0 (80%)
rec_open 2.5 3.3 3.3
the AEAD under a key expanded for the record, as open_aes_gcm runs it 2.5 3.3 3.3
the AEAD's open, gcm_traffic_open 2.5 3.0 3.0
OpenSSL, the key set and one record sealed 2.4 — —
OpenSSL, the key set and one record opened 2.4 — —
OpenSSL, one record sealed under a key set before — 3.1 3.1
OpenSSL, one record opened under a key set before — 3.2 3.1
Zig std.crypto, one record sealed 5.4 — —
ChaCha20-Poly1305, one 16 KiB record, µs macOS, Apple clang 21 Linux VM, clang 18 Linux VM, gcc 13
rec_seal, ch_cfg.cpu 0x3 40.7 37.1 54.9
rec_seal without its AEAD 0.2 (1%) 0.4 (1%) 0.4 (1%)
ChaCha20 in place 8.2 (20%) 8.1 (22%) 9.0 (16%)
Poly1305 over the ciphertext, on the 16x16 decomposition 32.0 (79%) 28.1 (76%) 45.6 (83%)
rec_open, ch_cfg.cpu 0x3 40.5 36.7 54.6
rec_seal, ch_cfg.cpu 0x7 10.9 11.6 12.3
the AEAD's seal, aead_seal_cpu, ch_cfg.cpu 0x7 10.6 (98%) 11.3 (97%) 11.8 (97%)
the seal less its tag's fixed work, ch_cfg.cpu 0x7 10.5 11.1 11.7
ChaCha20 in place, ch_cfg.cpu 0x7 7.9 (73%) 8.2 (70%) 8.9 (73%)
Poly1305 over the ciphertext, the vector path 2.5 (23%) 2.9 (25%) 2.8 (22%)
rec_open, ch_cfg.cpu 0x7 10.6 11.2 11.9
the AEAD's open, aead_open_cpu, ch_cfg.cpu 0x7 10.6 11.2 11.9
the open less its tag's fixed work, ch_cfg.cpu 0x7 10.5 11.0 11.7
OpenSSL, one record sealed — 10.0 10.0
OpenSSL, one record opened — 10.0 10.0
OpenSSL, ChaCha20 and Poly1305 over the record's bytes, sealing 8.9 — —
OpenSSL, ChaCha20 and Poly1305 over the record's bytes, opening 9.1 — —
Zig std.crypto, one record sealed 38.0 — —
Work a record pays whatever its size, and a 1 KiB record, ns macOS, Apple clang 21 Linux VM, clang 18 Linux VM, gcc 13
AES-128 key expansion, aes_traffic_key_init 10 205 283
AES-256 key expansion 11 273 330
the wipe of the expanded key 14 4 5
the GCM tag's fixed work 42 56 55
the Poly1305 tag's fixed work, on the 16x16 decomposition 190 176 252
the Poly1305 tag's fixed work, ch_cfg.cpu 0x7 154 146 148
AES-128-GCM rec_seal, 1 KiB 237 491 584
rec_seal without its AEAD, 1 KiB 32 (14%) 224 (46%) 314 (54%)

The stages add up: every whole is within 17% of the sum of its parts. Every gap of 5% or more is at 16 bytes or 1 KiB. On the M1 the parts of a tag, GCM's or Poly1305's, sum to as much as 16% less than the tag, and the parts of a 16-byte ChaCha20-Poly1305 AEAD to as much as 9% less. On the Linux VM the parts of an AES-GCM AEAD sum to as much as 8% less than the AEAD, and the parts of the GCM tag to as much as 8% more than the tag. What the numbers show for AES-GCM:

  • Counter mode runs a record's whole blocks through gcm_hw.c, eight blocks a pass. Each round key is loaded once and runs on the eight counter blocks of the pass, the eight keystream blocks are exclusive-ored into the data in vector registers, and the state that holds a pass's keystream is wiped once per call. The last partial block takes the one-block cipher. Counter mode alone, gcm_counter_blocks_hw, takes 1.5 to 1.7 µs of a 16 KiB AES-128 record, where the AES calls and the exclusive-or in place took 16.6 to 20.1 µs at 044a49c.
  • gcc 13 at -O2 needed #pragma GCC unroll 8 for that. Without it, gcc left each loop over a pass's eight blocks rolled and kept the eight states in memory, and counter mode took 16.5 µs on gcc against 1.8 µs on clang (the pitfalls below).
  • GHASH multiplies eight blocks a pass by the powers H^8 to H of the hash subkey, which each call computes first, and reduces once per pass. Since 88c64be every value stays in a vector register from the load to the reduction: three carry-less products a block in Karatsuba's form, and a reduction by 0xc200000000000000 on the same instruction (ghash_vector.h). GHASH alone takes 1.1 to 1.4 µs of a 16 KiB record, where the one-block loop took 9.2 to 10.4 at 044a49c and the eight-block loop that took its sums out of the vector registers for each reduction took 2.2 to 2.8 at 225a890.
  • clang's GHASH is no longer the slower one: 1.1 µs against gcc's 1.4. clang loaded each power of H through two general registers; the powers now come from ghash_state in one vector load each, a volatile read that also keeps the compilers from copying them to stack slots the wipe does not clear (INV-17).
  • The seal runs its whole passes through gcm_seal_passes_hw (c91e2f3). Each pass computes its keystream, writes its ciphertext and then hashes the pass before, whose GHASH waits on no AES round of this one, so the core runs one pass's AES rounds beside the other's carry-less multiplies. On the VM the loop takes 2.5 µs of a 16 KiB AES-128 record, where counter mode and GHASH alone take 2.8 to 3.1 one after the other.
  • The open runs the same loop the other way round, gcm_open_passes_hw (01ba8fc): iteration p hashes pass p and decrypts pass p - 1, and the tag is compared once the plaintext is written, which a mismatch then wipes (decision 85). On the VM the loop takes 2.4 to 2.5 µs of a 16 KiB AES-128 record.
  • rec_seal copies the caller's plaintext with one memmove. gcc 13 did not vectorize the byte loop that did it before, and ran it one byte at a time: 5.9 µs, three fifths of gcc's AES-GCM seal once counter mode and GHASH were fast. The record layer's own work is now 0.2 to 0.7 µs of a 16 KiB AES-GCM record and 0.3 to 0.4 µs of a ChaCha20-Poly1305 one on both compilers.
  • Each GHASH call wipes the part of its state it wrote: H, the powers it computed and a pass's sums. The tag's fixed work, one GHASH call over the associated data and one multiply for the block of lengths, each of which computes one power, takes 42 to 56 ns, where it took 150 to 169 while ct_wipe stored one byte at a time (decision 91) and 240 to 300 at 225a890; a 1 KiB seal takes 237 to 584 ns. At 1 KiB the key expansion makes most of the record layer's share. The wipe of the expanded key that record.c runs for every record takes 4 to 14 ns of it, where it took 91 to 93.
  • Measured one commit at a time, the three commits for #184 took the 16 KiB seal from 26.8, 27.6 and 36.3 µs at 044a49c to 11.5, 12.9 and 17.6 µs with counter mode, to 4.8, 5.4 and 10.0 µs with GHASH, and to 4.9, 5.3 and 4.9 µs with the memmove. Three more, in paired runs of the four trees, took it from 4.8, 5.2 and 5.0 µs at 225a890 to 3.5, 3.8 and 4.0 µs with GHASH in vector registers (88c64be) and to 3.0, 3.5 and 3.5 µs with the one-pass seal (c91e2f3); the one-pass open (01ba8fc) took the 16 KiB open from 3.3, 3.3 and 3.6 µs to 2.9, 3.2 and 3.2 µs. In paired runs of a104a3e and the tree after it, the memset wipe (decision 91) took the 16 KiB seal from 3.1, 3.6 and 3.7 µs to 2.7, 3.5 and 3.3 µs, and the 16 KiB open from 3.0, 3.4 and 3.3 µs to 2.6, 3.0 and 2.9 µs.
  • The seal now takes 2.3, 3.1 and 3.2 µs, and its AEAD alone 2.1, 2.6 and 2.5 µs. The open takes 2.1, 2.8 and 2.8 µs, and its AEAD alone 2.1, 2.6 and 2.5 µs. Against 044a49c the seal takes 0.09, 0.11 and 0.09 of its time, and on the VM the open 0.13 and 0.09.
  • Against OpenSSL on the same machine, operation for operation: on the VM, where OpenSSL 3.0.13 seals under a key set before the loop, the AEAD's seal takes 0.91 and 0.92 times its time, and the open 0.91 and 0.87 times. On macOS, where OpenSSL 3.6.4 sets the key for every operation, the AEAD under a key expanded for the record takes 2.1 µs to seal and to open under 0x3, 1.05 and 1.04 times OpenSSL's 2.0, and 0.99 and 0.97 of it under 0x83 (the table beside OpenSSL above). Of chapulin's time, aes_traffic_key_init takes 10 ns, and 11 ns for AES-256, where it took 202 and 255 ns before decision 158. Over 16 bytes the seal takes 88 ns, key and tag included, where OpenSSL's whole operation takes 161 ns.

For ChaCha20-Poly1305:

  • A session without the multiply bit, ch_cfg.cpu 0x3, runs Poly1305 on the 16x16 decomposition ct.h builds, and there Poly1305 is the largest stage: 76% to 83% of the record. colibri's sessions are such sessions until it sets the bit.
  • ChaCha20 runs the keystream and the exclusive-or in 8.1 to 9.0 µs on the vector path, on both compilers. chacha20.c's portable loop, which a device object runs, took 34 to 35 µs here at a104a3e, when a build without CHACHA=vector ran it: 25.2 to 29.0 for the block function and 6.3 to 8.9 for its exclusive-or, which ran one byte at a time in place. Since 97826ba a NEON pass computes two groups of four blocks side by side and XORs the keystream into the data from the registers that computed it (decision 86); a pass of one group took 13.6 to 14.6 µs. On clang the exclusive-or adds 0.4 to 0.6 µs to the keystream alone, whose row stores each pass's keystream to a buffer; under gcc 13 that row takes 0.4 to 0.5 µs longer than the whole, so the difference measures nothing there.
  • With the multiply bit, ch_cfg.cpu 0x7, a session runs the vector Poly1305 too (decision 83), which takes Poly1305 over the ciphertext from 28.1 to 45.6 µs on the decomposition to 2.5 to 2.9 µs. rec_seal then takes 10.9 to 12.3 µs, 22% to 31% of the 0x3 record's time, and a 1 KiB one 31% to 42%. ChaCha20 is 70% to 73% of the record, and Poly1305 22% to 25%. rec_open takes 10.6 to 11.9 µs.
  • Against OpenSSL on the same machine, operation for operation: on the VM, where OpenSSL 3.0.13 seals a whole record, the AEAD's seal takes 1.13 and 1.18 times its time and the open 1.12 and 1.19 times, and rec_seal and rec_open 1.16 and 1.22, and 1.12 and 1.19. On macOS, where OpenSSL 3.6.4 runs ChaCha20 and Poly1305 over the bytes and nothing else, the seal and the open less their tag's fixed work take 10.5 µs, 1.18 and 1.15 times OpenSSL's 8.9 and 9.1.
  • A one-pass seal and open did not pay. In a scratch timing loop on the three columns' compilers, a seal that runs ChaCha20 over a chunk and then Poly1305 over the ciphertext it just wrote, and an open that hashes each chunk and then decrypts it, took no less time than two passes at any chunk from 512 bytes to 16 KiB. A 16 KiB record stays in the M1 Pro's 128 KiB L1 data cache between the passes, so the second pass reads it from that cache too, and every chunk calls the vector Poly1305 again, which computes and wipes the powers of r on each call: in chunks of 1 KiB, the seal took 13.8 µs on macOS where two passes took 11.8. With the powers computed once a record, every chunk took from 2% less to 3% more than two passes. So the seal keeps its two passes, and the open still compares the tag before it decrypts, which decision 85 allows.

The split named four candidates for #184, in the order of the share of the record each one addressed:

  1. Several counter blocks per iteration. The AES stage was a third or more of the seal, and its time went to the work done for each block: the per-block wipe, the calls and the round-key loads. gcm_counter_blocks_hw does that work once per pass or once per call (above).
  2. GHASH over several blocks against precomputed powers of H. It was four fifths of the seal on clang after 1, one dependent multiply per block; ghash_hw.c now runs eight blocks per reduction (above).
  3. A word-wide exclusive-or. gcm_counter_blocks_hw runs the exclusive-or of whole blocks in vector registers, so the byte loop runs only on a record's last partial block and in the builds that run AES on the table or a peripheral, none of which this split times; no word-wide loop was written. Beside it, rec_seal's plaintext copy, which gcc ran one byte at a time, is one memmove (above).
  4. One pass for the keystream and the hash. GHASH's reduction outside the vector registers and clang's loads of the powers through general registers fell on the seal and the open alike, so 88c64be moved both into vector registers first. Then gcm_seal_passes_hw (c91e2f3) and gcm_open_passes_hw (01ba8fc) ran counter mode and GHASH in one loop, which hides most of the shorter one: 2.2 to 2.6 µs of a 16 KiB AES-128 record, where the two alone take 2.8 to 3.1. The open's loop needed the open to decrypt before it compares the tag, and to wipe the plaintext when the tag does not match (decision 85).

For #181, the numbers supported the vector path for ChaCha20 first, because the block function and the in-place exclusive-or were half of the record on clang with Poly1305 on the decomposition and more than two thirds of it on the native multiply, and decision 82 added it. Its two paired runs on each platform agreed: the vector rec_seal took 0.71, 0.68 and 0.78 of the portable one's time in the three columns in the second, and 0.78, 0.70 and 0.78 in the first. In the last runs that timed both, at a104a3e with decision 86's two groups a pass, it took 0.62, 0.60 and 0.68. Most of what remained of that record was Poly1305 on the multiply decomposition: the same loop on the native multiply ran in about a third of the time. A vector Poly1305 needs a widening multiply too, so it needs the same statement about that multiply's timing, and decision 83 added one under it: CHACHA=vector with WIDEMUL=native, where Poly1305 was most of what the vector ChaCha20 left. Paired runs agree on each platform, two in the Linux VM and three on macOS, where the second ran at a load average of 44 to 78. In the last pair on each, that build's rec_seal took 0.66, 0.66 and 0.69 of its time before the change in the three columns; the earlier pairs gave 0.66 and 0.66 on macOS, 0.57 on the VM's clang and 0.65 on gcc 13. Those pairs came before each call of the vector Poly1305 began to end with a wipe of the powers of r it computed, one ct_wipe of 208 bytes on NEON and 352 on SSE2. The runs the tables now hold measure the code with that wipe, and the 128-byte threshold still rests on decision 83's scratch timing. Under ch_cfg.cpu 0x7 the vector Poly1305 takes 2.5 and 2.8 µs on macOS and gcc 13, as in the pairs, and 2.9 µs on the VM's clang. On that compiler the figure moves with where the linker places the function. It took 4.7 µs in the bench build the tables held at d10b6e2, and 3.0 µs in one whose aes_hw.c differed in one function; that build with -falign-functions=64 took 2.8 µs (the pitfalls below). It took 3.0 µs in the runs at 97826ba, 4.8 µs again at a104a3e, before decision 91 and after it alike, and 4.6 µs in this tree's bench an hour before the tables' run, when bench/record_rows.c lacked its two seal_aes_gcm rows.

ChaCha20 was then the larger stage of that build, 80% of its record, and decision 86 runs two groups of four blocks a pass on NEON, because one group left the vector units waiting on each quarter round's chain of operations. In paired runs, that build's 16 KiB rec_seal took 0.68, 0.70 and 0.69 of its time before the change in the three columns, and 0.67, 0.64 and 0.67 in the first pair. The two builds that ran the portable ChaCha20 then did not move beyond the spread between runs: 68.1, 62.0 and 80.5 µs on the decomposition where the tables held 67.5, 60.8 and 79.0 at 9287540, and 46.3, 44.9 and 47.8 on the native multiply where they held 46.3, 44.0 and 47.0. OpenSSL's ChaCha20 alone took 7.0 µs over 16 KiB on macOS in three one-second runs of openssl speed -evp chacha20, against that build's 8.5, and its whole seal took 9.4 against 11.8.

No x86-64 machine that runs nothing else has timed the vector paths' SSE2 arms or the AES-NI arm of gcm_hw.c. bench.yml's record-x86_64 job times them on a shared GitHub runner, once with gcc and once with clang, and on a runner whose CPU has AVX2, VAES and VPCLMULQDQ it adds the ch_cfg.cpu 0x1f rows for the x86-64 kernels of decision 90, which records the figures they had when their rows were marked AVX2+VAES. On a runner whose CPU has AVX-512 IFMA it also adds the 0x11f ChaCha20-Poly1305 rows for the kernels of decision 121, and on one with AVX-512F, AVX-512BW and AVX-512VL the 0x31f AES-GCM rows for the kernel of decision 130. The table beside OpenSSL above takes its x86-64 AEAD rows from that job's CSVs, such as bench/results-record-linux-x86_64-gcc.csv. A column in this section's tables needs make bench-record on an x86-64 host that runs nothing else, once with gcc and once with clang, and tools/bench_record.py then takes the two files as columns. An emulated x86-64, such as an OrbStack amd64 container, runs translated code, so its times say nothing about those arms.

arm64 AES-GCM beside OpenSSL

Decision 158 set the goal on 2026-10-25: on arm64, AES-GCM takes no more time than OpenSSL 3.6.4, for both key sizes, seal and open. This section holds the numbers on a Neoverse N2, the CPU of GitHub's ubuntu-24.04-arm runner. make bench-record ran on it under gcc 13 and the pinned clang 23, beside the pinned OpenSSL 3.6.4. The "161" rows time the passes of decision 161, and their CSVs are bench/results-record-linux-arm64-n2-gcc.csv and bench/results-record-linux-arm64-n2-clang.csv. The "before" run timed the tree of 54a05fd1, and the "after" run the tree of decision 158, on the same runner type. Their CSVs are not in the tree.

Each cell is chapulin's time for one record over OpenSSL's in the same run: seal_aes_gcm or open_aes_gcm, which expand the key, run the AEAD with 5 bytes of associated data and wipe the key, against one operation of openssl speed -aead, which sets the key and runs the AEAD with 13 bytes of associated data. 0x3 is a session with the AES bit, and 0x83 adds CH_CPU_CONSTANT_TIME_SHA3, which runs gcm_arm64_eor3.c's copy with EOR3. A ratio above 1 is in bold.

N2, seal / open 16 B 1 KiB 16 KiB
AES-128, clang, before 1.68 / 1.29 1.44 / 1.29 1.18 / 1.15
AES-128, clang, after, 0x3 0.72 / 0.58 0.97 / 0.85 1.02 / 0.99
AES-128, clang, after, 0x83 0.75 / 0.59 0.94 / 0.82 0.98 / 0.94
AES-128, clang, 161, 0x3 0.72 / 0.57 0.96 / 0.85 1.00 / 0.97
AES-128, clang, 161, 0x83 0.74 / 0.59 0.94 / 0.82 0.97 / 0.94
AES-128, gcc, before 2.74 / 2.15 1.97 / 1.76 1.27 / 1.25
AES-128, gcc, after, 0x3 0.73 / 0.60 0.97 / 0.87 1.02 / 0.99
AES-128, gcc, after, 0x83 0.74 / 0.60 0.94 / 0.85 0.99 / 0.96
AES-128, gcc, 161, 0x3 0.72 / 0.59 0.96 / 0.86 1.00 / 0.97
AES-128, gcc, 161, 0x83 0.73 / 0.61 0.94 / 0.84 0.97 / 0.93
AES-256, clang, before 2.04 / 1.59 1.56 / 1.41 1.17 / 1.17
AES-256, clang, after, 0x3 0.72 / 0.59 0.97 / 0.88 1.01 / 1.02
AES-256, clang, after, 0x83 0.75 / 0.61 0.95 / 0.86 0.98 / 0.97
AES-256, clang, 161, 0x3 0.72 / 0.58 0.96 / 0.87 1.00 / 1.01
AES-256, clang, 161, 0x83 0.74 / 0.61 0.94 / 0.86 0.96 / 0.96
AES-256, gcc, before 3.12 / 2.48 2.05 / 1.89 1.24 / 1.26
AES-256, gcc, after, 0x3 0.75 / 0.63 0.99 / 0.90 1.01 / 1.01
AES-256, gcc, after, 0x83 0.76 / 0.63 0.96 / 0.88 0.98 / 0.99
AES-256, gcc, 161, 0x3 0.74 / 0.61 0.97 / 0.90 0.99 / 1.00
AES-256, gcc, 161, 0x83 0.75 / 0.63 0.95 / 0.87 0.98 / 0.99

The times behind the "after" rows, in ns, under clang and 0x83: AES-128 seals 16 bytes in 137, 1 KiB in 384 and 16 KiB in 3,635, where OpenSSL takes 184, 410 and 3,717. AES-256 seals them in 142, 430 and 4,237, where OpenSSL takes 190, 452 and 4,342.

What the numbers show:

  • The 16-byte and 1 KiB records gained the most. Before, the key expansion took 194 ns for AES-128 and 335 ns for AES-256, and it now takes 25 ns. The steps around the passes took about 100 ns of a 1 KiB record, and the whole AEAD of gcm_arm64.c removed most of them.
  • The 16 KiB records gained from the passes in assembly: 3.67 µs for AES-128's passes, where the compilers' code took 4.11 µs.
  • Without the SHA-3 bit the 16 KiB cells take 0.99 to 1.02 of OpenSSL's time. Two exclusive-ors take the place of each EOR3, and the two vector pipes run the extra instructions.
  • Two builds of the same tree differ by up to 3% in a 16 KiB cell, as the code's place in memory moves. The margins at 16 KiB under 0x83 are 1% to 6%.

The M1 Pro ran make bench-record BENCH_MACHINE=m1 at a load average of 3.6 to 4.3, into bench/results-record-darwin-arm64-m1-clang.csv, on the passes of decision 161. Homebrew holds OpenSSL 3.5.9 on this machine, not 3.6, so the OpenSSL figure is one operation of 3.5.9's speed -aead. The record tables above keep the run of 2026-10-09 against OpenSSL 3.6.5, in which a 16 KiB record took 1.13 to 1.15 times OpenSSL's time.

M1 Pro, Apple clang 21, seal / open 16 B 1 KiB 16 KiB
AES-128, 0x3 0.63 / 0.55 0.93 / 0.82 1.04 / 1.04
AES-128, 0x83 0.64 / 0.56 0.90 / 0.79 1.00 / 0.97
AES-256, 0x3 0.62 / 0.53 0.92 / 0.82 1.01 / 1.00
AES-256, 0x83 0.63 / 0.54 0.89 / 0.80 0.97 / 0.96

What the numbers show:

  • Under 0x83 no cell takes more time than OpenSSL's. Before decision 161 the 16 KiB cells took 1.00 to 1.03 of it, and under 0x3 1.06 to 1.10. The rows of this run spread by 6%. In runs that set the code before and after decision 161 in turn, a 16 KiB record took 4.4% to 7.7% less time under 0x83 and 2.6% to 4.5% less under 0x3.

  • The M1 decodes the passes' instructions slower than its four vector pipes run them. Decision 161 took 23 instructions out of each AES-128 pass. A first step that took 9 out gained about one cycle a pass.

  • OpenSSL 3.5.9 runs its unroll8_eor3 kernel on this CPU, so the 0x83 row compares two kernels that sum three values with EOR3.

  • The 0x3 rows compare passes without EOR3 against OpenSSL's kernel with EOR3. On a CPU without FEAT_SHA3, OpenSSL runs its kernel of four blocks a pass, which has no EOR3. OPENSSL_armcap=0x67d clears OpenSSL's SHA-3 and EOR3 bits from this CPU's 0x987d, and OpenSSL then runs that kernel. On 2026-10-29, at a load average of 3.9 to 5.7, four rounds timed the bench binary under 0x3 and speed -aead in turn, at 16 KiB. The medians:

    M1 Pro, 16 KiB, ns 0x3 OpenSSL, 0x67d OpenSSL, 0x987d 0x3 / 0x67d
    AES-128 seal 2,209 2,648 2,110 0.83
    AES-128 open 2,214 2,785 2,149 0.80
    AES-256 seal 2,578 3,059 2,525 0.84
    AES-256 open 2,544 2,996 2,514 0.85

    So a session without the SHA-3 bit takes 0.80 to 0.85 of the time OpenSSL takes without EOR3. Each M1 has FEAT_SHA3, so a caller on this CPU can state 0x83.

  • On the N2 decision 161 took up to 2.6% off the 16 KiB cells under both compilers. The 16-byte and 1 KiB cells took from 6.4% less to 1.1% more time. Its rows are the "161" rows of the N2 table.

Where a server handshake's instructions go

#188 measured colibri's server at 55.4 M user instructions per TLS connection on a Neoverse-N2, where h2o and nginx over OpenSSL 3.0.13 took 2.1 and 2.4 M. This section counts the server's side of that connection, splits it into stages and sets OpenSSL beside it. The connection is the one h2load makes in colibri's bench: TLS 1.3 with TLS_AES_256_GCM_SHA384 over X25519, a P-256 ECDSA leaf and the P-256 root that signed it, ALPN h2 and no ticket key. After the handshake the client sends one 100-byte request record, the server answers with one 256-byte record, and each side sends close_notify.

The section holds two counts. bench/handshake-n2.sh made the first on 2026-10-25, over ct_wipe's word stores (docs/decisions.md 144), the fixes of entries 145 to 147 and 149, the key generation of entry 148 and the ladder of entry 152. make lint-bench-numbers holds its three tables to bench/results-handshake-n2.csv (tools/bench_handshake.py). The script counted the tree at d0731465 on 2026-10-21 too, and "What changed since 2026-10-21" below gives those figures, and "What decision 148 changed" the count of the same day before entry 148, and "What decision 152 changed" the count before entry 152. The second count is the count of 2026-10-01, at c798fb8. No script made it, and this section keeps it as the record of that date.

The count of 2026-10-25

The script counts this way:

  • The machine is GitHub's ubuntu-24.04-arm runner, a Neoverse-N2 (CPU part 0xd49 in /proc/cpuinfo) with 4 cores, under Linux 6.17. A workflow on a scratch branch built the chapulin servers on an x86-64 runner with the pinned Zig, and ran the script on the arm64 runner.
  • The object is the TCP object colibri builds: RAND=session TRANSPORT=tcp-nonblocking ROLE=both TRUST=webpki EXPORTER=on SUITE=aesgcm TX_RECORD=16384 KEYLOG=off, from colibri's build/modules.zig. build.zig builds it with Zig 0.16.0 for aarch64-linux-gnu, once for -Dcpu=neoverse_n2 and once for the base armv8-a. colibri pins chapulin at 50d7d17b, 21 commits before d0731465.
  • bench/handshake_server.c calls the object as colibri's server does. It reads the chain and the key from PEM files, offers ALPN h2 and http/1.1, and holds a cookie key and no ticket key. Each session states one ch_cfg.cpu value. 0xe7 is what a caller on a Neoverse-N2 states: the AES, multiply, SHA-256, SHA-512 and SHA-3 bits. 0x07 is what colibri states today, the AES and multiply bits (its src/tls/config.zig, cpu_of). 0x03 is the AES bit alone.
  • bench/handshake_openssl_server.c serves the same connection over OpenSSL 3.6.4, the release tools/toolchain.env pins, linked statically, and over the runner's OpenSSL 3.0.13 (the libssl3 package), shared and static. It holds the same chain and key, and no session tickets and no session cache. bench/handshake_client.c, over OpenSSL 3.0.13, offers what h2load 1.59.0 offers: the one suite, the groups X25519, P-256, P-384 and P-521 with an X25519 share, and ALPN h2.
  • Each figure counts the server's user-mode instructions in the second connection of a run. The server reads the perf_events counter of its own process when the connection starts, when the handshake is complete and when the socket is closed. Each such figure is the median of five runs, and the five runs of a figure spread by 0.02% at most. perf stat -e instructions:u over the whole process, a run of two connections less a run of one, gives the third column.
  • QEMU 8.2.2 in user mode, with -cpu neoverse-n2, runs each server one instruction per translation block. bench/handshake_stages.py rebuilds the call stack from that trace and the binary's bl and blr addresses, and charges each instruction to the outermost root function it runs under. A tail call gives its frame to the function it calls. Each handshake of the trace is within 0.09% of the perf_events count of the same server.
  • The nonce row holds chapulin's RFC 6979 generator: its SHA-256 calls, and the functions of p256_sign.c whose names start with generator_. On 2026-10-21 the generator's own functions fell in the row of the rest.
  • The row of hashes holds three uses of SHA-2 in chapulin: the transcript hashes, the SHA-256 of the CertificateVerify signed content, and the ClientHello digest. That digest is client_hello.frozen, the SHA-256 that srv_parse_client_hello computes over the fields of each ClientHello a HelloRetryRequest must not change. For OpenSSL the row holds its EVP_Digest calls.
  • Valgrind 3.22's callgrind decodes neither SVE nor the SHA-512 instructions. So it cannot run the Neoverse-N2 build, whose code holds SVE instructions, or a session that states CH_CPU_CONSTANT_TIME_SHA512. It runs the base build under 0x07 and 0x03, OpenSSL 3.6.4 and the shared OpenSSL 3.0.13. It gives the inclusive cost of the functions under X25519, the signature and the two parts of the connection.
server side of the second connection on the Neoverse-N2, M instructions handshake, perf_events request, response and close_notify, perf_events whole connection, perf stat handshake, callgrind handshake against OpenSSL 3.6.4
chapulin, Neoverse-N2 build, 0xe7 0.821 0.015 0.836 — 0.61
chapulin, Neoverse-N2 build, 0x07 1.233 0.015 1.248 — 0.92
chapulin, Neoverse-N2 build, 0x03 43.599 0.015 43.614 — 32.40
chapulin, base armv8-a build, 0xe7 0.968 0.015 0.983 — 0.72
chapulin, base armv8-a build, 0x07 1.388 0.015 1.402 1.402 1.03
chapulin, base armv8-a build, 0x03 37.779 0.015 37.793 37.793 28.08
OpenSSL 3.6.4, static 1.346 0.038 1.385 1.605 1.00
OpenSSL 3.0.13, shared 1.877 0.047 1.926 2.143 1.40
OpenSSL 3.0.13, static 1.873 0.046 1.922 — 1.39
server side of the second connection, M instructions (QEMU's trace) Neoverse-N2 build, 0xe7 Neoverse-N2 build, 0x07 Neoverse-N2 build, 0x03 base armv8-a build, 0xe7 OpenSSL 3.6.4, static OpenSSL 3.0.13, static
X25519 key generation 0.119 0.119 13.404 0.135 0.227 0.423
X25519 shared secret 0.345 0.345 13.404 0.447 0.380 0.469
CertificateVerify: the P-256 ECDSA signature 0.165 0.325 16.345 0.172 0.197 0.249
of which k·G 0.103 0.103 14.447 0.104 0.069 0.070
of which the nonce 0.023 0.182 0.182 0.028 0.011 0.021
of which the rest 0.039 0.039 1.716 0.040 0.117 0.159
HKDF and HMAC of the key schedule 0.094 0.284 0.284 0.093 0.309 0.459
hashes of the transcript, the signed content and the ClientHello 0.015 0.076 0.076 0.015 0.018 0.016
certificate, other messages and their records 0.084 0.084 0.084 0.107 0.215 0.257
the handshake 0.821 1.233 43.598 0.968 1.347 1.874
one request's records and the two close_notify 0.015 0.015 0.015 0.015 0.038 0.046
of the handshake, the image's own memset 0.042 0.040 0.040 0.070 0.000 0.000
of the handshake, the C library 0.000 0.000 0.000 0.000 0.162 0.233
base armv8-a build, M instructions 0x07, callgrind 0x07, QEMU's trace 0x03, callgrind 0x03, QEMU's trace
X25519 key generation 0.135 0.135 10.581 10.581
X25519 shared secret 0.448 0.447 10.581 10.581
CertificateVerify: the P-256 ECDSA signature 0.335 0.335 16.147 16.147
of which k·G 0.104 0.104 14.261 14.261
the handshake 1.402 1.388 37.793 37.779
one request's records and the two close_notify 0.015 0.015 0.015 0.015
of the handshake, the image's own memset 0.068 0.067 0.068 0.067

What the count of 2026-10-25 shows:

  • chapulin runs fewer instructions than OpenSSL 3.6.4 on this connection when the session states every bit the Neoverse-N2 has. Under 0xe7 the Neoverse-N2 build's handshake takes 821,090 instructions, 0.61 of OpenSSL 3.6.4's 1,345,612 and 0.44 of OpenSSL 3.0.13's 1,877,140. The whole connection takes 0.836 M against 1.385 M and 1.926 M.
  • Under 0x07, the value colibri states today, chapulin runs 0.92 of OpenSSL 3.6.4's handshake and 0.66 of OpenSSL 3.0.13's. The three SHA bits remove 411,735 instructions. The key schedule, the transcript hashes, the signed content's digest, the ClientHello digest and the nonce's SHA-256 calls then run on the SHA instructions. Only the caller can state those bits, so this change is colibri's.
  • Under 0xe7 every stage costs fewer instructions than OpenSSL 3.6.4's. The X25519 shared secret costs 0.345 M against 0.380 M, on the ladder of decision 152. X25519 key generation costs 0.119 M against 0.227 M, since both compute the public key from a table of multiples of the base point (decision 148). The row of hashes now costs 0.015 M against 0.018 M, because the ClientHello digest runs on the SHA-256 instructions (decision 149). The signature, the key schedule, the hashes, the messages and the request cost less than OpenSSL's.
  • 42,214 of the 821,109 instructions of the trace, 5%, run in the image's own memset. Zig 0.16 links memset from its compiler_rt into each image it links, and that memset writes one byte at a time. ct_wipe calls it no longer (decision 144), so these are the calls that clear a structure before its first use. The OpenSSL servers, which gcc links, call the C library's memset, and their row of the C library holds it.
  • The Neoverse-N2 build costs fewer instructions than the base build on the paths of a host session: 0.821 M against 0.968 M under 0xe7. Under 0x03 it costs more, 43.60 M against 37.78 M, because x25519.c's mul holds SVE instructions, as on 2026-10-01.
  • callgrind agrees with the trace on the base build. Under 0x07 X25519 key generation and k·G are the same to the instruction, the shared secret differs by 257 instructions and the signature by 25. callgrind's handshake under 0x07 is 1.1% larger than the trace's, and 294 instructions of that difference are in memset. Under valgrind OpenSSL finds no SHA-512 instructions and runs SHA-384 in scalar code, so callgrind counts its handshakes 19% and 14% above perf_events.

What changed since 2026-10-21. The script counted d0731465 that day, before docs/decisions.md 144 to 147 and 149, on the same runner type under Linux 7.0:

Neoverse-N2 build, M instructions 0xe7, 2026-10-21 0xe7, 2026-10-25 before entry 148 0x07, 2026-10-21 0x07, 2026-10-25 before entry 148
the handshake, perf_events 1.307 1.144 1.830 1.555
the handshake against OpenSSL 3.6.4 0.97 0.85 1.36 1.16
CertificateVerify: the P-256 ECDSA signature 0.220 0.165 0.416 0.324
HKDF and HMAC of the key schedule 0.158 0.094 0.381 0.284
hashes of the transcript, the signed content and the ClientHello 0.046 0.015 0.150 0.076
of the handshake, the image's own memset 0.207 0.042 0.221 0.040
  • Two kinds of change are in the new column. ct_wipe stores words and calls no memset (decision 144), which every stage that wipes a secret gains from. The fixes of entries 145 to 147 and 149 remove hash work: the signed content's digest and the ClientHello digest run on the session's path, the SHA-256 transcript stops at the ServerHello, and the nonce keeps its HMAC midstates for each K.
  • A count of the fixes alone, on 2026-10-24 over 773ac1da, put the handshake at 1,281,526 instructions under 0xe7 and 1,707,844 under 0x07. So the fixes removed 25,951 and 121,800 instructions, and ct_wipe's word stores most of the rest.
  • The nonce row now holds the generator's out-of-line functions as well as its SHA-256 calls, so compare the signature row across the two dates and not its parts.

What decision 148 changed. The script counted 5f2424a6 on 2026-10-25, before entry 148, on the same runner type under the same Linux, and the tables above count the tree with it:

M instructions, server side of the second connection before entry 148 with entry 148 OpenSSL 3.6.4
X25519 key generation, Neoverse-N2 build, 0xe7 0.393 0.132 0.227
X25519 key generation, base armv8-a build, 0xe7 0.457 0.150 0.227
the handshake, Neoverse-N2 build, 0xe7, perf_events 1.144 0.883 1.346
the handshake, Neoverse-N2 build, 0x07, perf_events 1.555 1.295 1.346
the handshake, base armv8-a build, 0xe7, perf_events 1.300 0.991 1.346
  • A session that states its multiply computes the public key from a table of multiples of edwards25519's base point in place of the ladder on u = 9 (decision 148). The key generation takes 132,403 instructions where it took 393,095, 0.58 of OpenSSL 3.6.4's 227,086, and the handshake loses 260,419 instructions under 0xe7. The shared secret keeps the ladder. Under 0x03 nothing moves: that session runs the 16-word field's ladder for both.

What decision 152 changed. The script counted f96c5192 on 2026-10-25, on the same runner type under the same Linux, and the tables above hold that count. The column before entry 152 is the count of the tree with entry 148:

M instructions, server side of the second connection before entry 152 with entry 152 OpenSSL 3.6.4
X25519 shared secret, Neoverse-N2 build, 0xe7 0.393 0.345 0.380
X25519 key generation, Neoverse-N2 build, 0xe7 0.132 0.119 0.227
X25519 shared secret, base armv8-a build, 0xe7 0.455 0.447 0.380
the handshake, Neoverse-N2 build, 0xe7, perf_events 0.883 0.821 1.346
the handshake, Neoverse-N2 build, 0x07, perf_events 1.295 1.233 1.346
the handshake, base armv8-a build, 0xe7, perf_events 0.991 0.968 1.346

  • The wide field's ladder swaps the points once a step, carries columns 1 to 4 in 64-bit words, runs add, sub and cswap without loops, and runs its products in place (decision 152). The shared secret takes 345,032 instructions where it took 393,447, 0.91 of OpenSSL 3.6.4's 380,382. Key generation shares the field and takes 118,825 where it took 132,403.

  • The base armv8-a build gains less: 455,006 to 447,353. On that target the products in place cost more than they save, and the other three changes more than make up for it.

  • A QEMU trace of one shared secret, split by the source function of each instruction, gives the N2 build's parts. Code in place moves some loads and moves to the line of the product they feed, so the products' line grows although their count does not change:

    Neoverse-N2 build, instructions of one shared secret before entry 152 with entry 152
    ct_mul128, the 64x64->128 products 106,140 118,142
    carry_columns and column_carry 102,427 82,169
    mul and sqr: the sums of products and the multiples of 19 113,753 107,253
    cswap 31,110 11,527
    add and sub 28,560 19,384
    step, the ladder's loop, sqr_times and invert 10,164 5,250
    the wipes, pack, unpack and the entry 1,303 1,317
    the whole shared secret 393,457 345,042

    By call site, the 255 steps take 315,953 instructions where they took 362,100, and the inversion 22,960 where it took 25,521.

  • On the M1 Pro, Apple clang 21 at -O2, proc_pid_rusage counts 372,726 instructions for a shared secret before entry 152 and 340,978 with it, and 129,818 and 119,682 for key generation. Decision 152 gives the x86-64 counts under gcc 13 and two clangs.

What decision 153 adds. The server's ECDSA signature now takes a hedged nonce, and the signer checks that k·G is on the curve and that s·k = z + r·d before a signature leaves it. The tables above count the tree before that entry, and the next run of bench/handshake-n2.sh on the arm64 runner records the N2 figures. On the M1 Pro, Apple clang 21 at -O2, proc_pid_rusage over 2,000 calls of a program that signs with the RFC 6979 key gives these counts of one signature:

M1 Pro, instructions of one P-256 signature before entry 153 RFC 6979 with both checks hedged with both checks
host object, 0xe7 172,683 179,127 179,944
host object, 0x05 332,936 339,208 352,127
host object, 0x01 16,288,929 16,314,992 16,332,321
device object, WIDEMUL=native 5,491,941 5,501,123 5,512,328
device object, the 16x16 decomposition 16,290,539 16,319,254 16,327,462
  • The two checks add about 6,400 instructions on the wide field, which a host session that states its multiply runs, and 9,000 to 29,000 on the 32-bit field: two squares and a product for the curve equation, two scalar products for s·k and r·d. A count moves by up to 0.05% from one run to the next on a loaded machine, so the 16 M rows carry that much noise.
  • The hedged nonce adds four SHA-256 compressions: steps d and f each hash 224 bytes where they hashed 97. That is about 800 instructions on the SHA-256 instructions and 13,000 on the portable code.
  • Under 0xe7 a signature costs 1.04 times what it cost before. On the Neoverse-N2 count above, the handshake under 0xe7 takes 0.821 M with a 0.165 M signature, so the change adds about 0.007 M, under 1% of the handshake, if the N2 pays what the M1 pays.
  • The bench's p256_sign row now times p256_sign_hedged_cpu, the server's path, over the first hedged vector's noise. Its CSVs keep the earlier entry's figures until the next run records them.

The count of 2026-10-01

The figures were taken this way:

  • The object is the issue's: c798fb8 with RAND=session TRANSPORT=tcp-nonblocking ROLE=both TRUST=webpki EXPORTER=on SUITE=aesgcm AES=runtime CHACHA=vector TX_RECORD=16384 KEYLOG=off and CH_NATIVE_AES, built by build.zig with Zig 0.16.0 for aarch64-linux-gnu. colibri builds for the CPU it runs on, so the N2 columns pass -Dcpu=neoverse_n2, and one column builds for the base armv8-a instead.
  • A test server calls the object as colibri's server does (ch_srv_record_init, ch_srv_record_in, ch_read, ch_write, ch_close). An OpenSSL 3.0.13 client offers what h2load 1.59.0 offers there: the one suite, h2load's default groups X25519, P-256, P-384 and P-521 with an X25519 share, and ALPN h2. Both run in an arm64 Ubuntu 24.04 container in OrbStack on an Apple M1 Pro.
  • QEMU 8.2.2 in user mode, with -cpu neoverse-n2, runs the server one instruction per translation block, as bench/insn-*.sh count. A scratch tool rebuilds the call stack from that trace and the binary's bl and blr addresses, and charges each instruction to the outermost crypto function it runs under, and inside p256_sign to the part of the signature it runs in: the nonce generator, p256_point_base_mul, p256_point_affine_x or the scalar arithmetic. Valgrind 3.22's callgrind, which cannot run the N2 build's SVE instructions, counted the base build's handshake at 42,370,882 instructions, and QEMU at 42,370,683.
  • Each figure counts the user-mode instructions of the server process in the second connection of a run. The first takes about 3,000 more, and under callgrind a third matched the second to within 100.
  • The OpenSSL column serves the same connection with OpenSSL 3.0.13, the same chain and key and no session tickets, under callgrind. Its two key rows come from separate calls of OpenSSL's public API, 20 of each: EVP_PKEY_keygen and EVP_PKEY_derive for X25519, and EVP_PKEY_sign over a SHA-256 digest for P-256. Under valgrind, OpenSSL finds no SHA-512 instructions, so its SHA-384 runs the scalar code, 0.27 M of its handshake. A Neoverse-N2 has those instructions, and OpenSSL uses them there.
  • No script wrote these figures, so make lint-bench-numbers checks none of them.
server side of one connection, M instructions c798fb8 for Neoverse-N2 for the base armv8-a N2, WIDEMUL=native N2, WIDEMUL=native X25519=wide base, Zig's ReleaseSafe flags OpenSSL 3.0.13
X25519 key share: key generation and shared secret 33.83 25.48 4.32 0.79 67.42 0.89
CertificateVerify: P-256 ECDSA signature 16.39 16.21 5.51 5.51 24.17 0.21
of which k·G, the 256-round ladder 14.45 14.26 4.82 4.82 21.43 —
of which the two inversions, the scalar arithmetic and the DER 1.75 1.74 0.50 0.50 2.43 —
of which the RFC 6979 nonce 0.19 0.19 0.19 0.19 0.29 —
transcript hashes and key schedule 0.59 0.62 0.59 0.59 0.88 —
certificate, other messages and their records 0.06 0.07 0.06 0.06 0.18 —
the handshake 50.87 42.37 10.48 6.95 92.65 2.10
one request's records and the two close_notify 0.020 0.025 0.020 0.020 0.075 0.045

What the numbers show:

  • Two public-key operations make up 98% to 99% of the handshake on the decomposed multiply and 91% to 94% on the native one. In the issue's object X25519 takes 66.5% and the signature 32.2%. Each X25519 scalar multiplication costs half of its row, because key generation runs the same ladder as the shared secret.
  • The multiply decomposition is most of the gap. WIDEMUL=native alone takes the N2 handshake from 50.87 M to 10.48 M, and X25519=wide then takes it to 6.95 M. X25519 then costs 0.79 M, against OpenSSL's 0.89 M. In 0.2.0 a host session gets both from CH_CPU_CONSTANT_TIME_MULTIPLY (decision 89). At the commit that moved the wide field under that bit, the same object as a host object, built for the base armv8-a with no AES value, takes 7.06 M when its ch_cfg.cpu holds the AES bit and the multiply bit, and 42.30 M with the AES bit alone: callgrind as above, 7,061,840 and 42,302,196 instructions in the second connection. At a104a3e the default object took 42.37 M on the base armv8-a, as at c798fb8, and a WIDEMUL=runtime object answered CH_WIDEMUL_CONSTANT_TIME took 12.08 M, 0.02 M above WIDEMUL=native's 12.07 M. That object ran the 16-word field on the native multiply, which no object holds now.
  • The CPU target costs instructions only on the decomposed multiply. Built for neoverse_n2, x25519.c's mul holds SVE instructions that clang 21 emits for that CPU, and it runs 5,452 instructions a call where the base build's runs 4,079: 8.39 M more over the 6,114 calls of a handshake. On the native multiply the N2 build is the cheaper one: X25519 costs 4.32 M against the base build's 5.86 M.
  • The signature remains. It costs 5.51 M on the native multiply, 26 times OpenSSL's 0.21 M. p256_point_base_mul runs a 256-round Montgomery ladder with two complete additions a round on 32-bit words, and keeps no table of multiples of G (p256_point.h). OpenSSL 3.0's arm64 build signs with the ecp_nistz256 assembly, which reads a precomputed table of multiples of G and computes on 64-bit words. Items 2 and 3 below have since changed both for a host session that states its multiply, and this table's objects were not counted again.
  • The rest is small. The transcript and key schedule take 0.59 M: HKDF and its HMAC-SHA384 take 0.44 M, the SHA-384 transcript 0.07 M, and the SHA-256 transcript, which transcript.h runs beside SHA-384, 0.08 M. Messages and their records take 0.06 M.
  • colibri's ReleaseSafe does not change the object. build.zig compiles it at ReleaseFast with sanitize_c, the stack protector and the stack check off, whatever mode the dependent builds in, and --verbose-cc shows -O2 with no -fsanitize. Zig 0.16 compiles C for ReleaseSafe with -O2 -fsanitize=undefined -fsanitize-trap=undefined -fstack-protector-strong -D_FORTIFY_SOURCE=2, and the same sources with those flags take 2.2 times the instructions.
  • This count accounts for 50.89 M of the 55.41 M user instructions per connection the issue measured. The other 4.5 M are outside it: colibri's own code, which this harness does not run, and any difference between the CPU features Zig detects on the runner and its neoverse_n2 model. colibri's cleartext connection costs 0.22 M.

The order of the work, by the instructions each item removes from a handshake:

  1. A host session that states its multiply, in 0.2.0, removes 43.9 M of the N2 object's 50.87 M. That statement is the caller's (decision 89), so this item is colibri's.
  2. k·G from a precomputed table of multiples of G, read by a full scan with mask selection, in place of the ladder: done, for a host session that states its multiply (decision 94). Eight odd multiples for each of 64 four-bit windows take 32 KiB of affine points, and k·G is 64 mixed additions. Decision 109 widened the windows to six bits: 32 odd multiples for each of 43 windows take 86 KiB, k·G is 43 mixed additions, and a key generation on the M1 Pro took 14.8 µs where it took 18.7. On the M1 Pro under Apple clang 21 a signature went from 1.88 M instructions to 0.53 M and a key generation from 1.58 M to 0.23 M, where OpenSSL 3.6.5 takes 0.18 M and 0.17 M. A session without the bit keeps the ladder: the table's additions are the wide field's. The key exchange multiplies the peer's point, which no table holds, by four-bit windows over eight multiples of it: 0.92 M instructions where the ladder on the wide field took 1.58 M and OpenSSL takes 0.48 M. In bench/primitives.sh's pinned ECDSA P-256 handshake on the M1 Pro, with this item and the next, the server's side takes 162 µs where it took 784 µs, and both ends retire 12.24 M instructions where they retired 17.23 M. The client's side takes 1.41 ms, of which p256.c's verification on 32-bit words is 1.30 ms.
  3. A 64-bit-word P-256 field and scalar under the multiply bit, as x25519_wide.c is for X25519: done (decision 94). On the M1 Pro under Apple clang 21, with the ladder unchanged, a signature through p256_sign went from 5.52 M instructions to 1.88 M and a key exchange through p256_ecdh from 5.04 M to 1.58 M, where OpenSSL 3.6.5 takes 0.18 M and 0.48 M. The Lean spec computes modulo p and n and states no word, so it serves both copies, and bin/diff_p256_wide runs each against it.
  4. SHA-256 on the ARMv8 and x86-64 SHA instructions and SHA-512 on the ARMv8.2 ones, behind new ch_cfg.cpu bits. Hashing takes 0.78 M of the 6.95 M: the transcript and key schedule's 0.59 M, the nonce's 0.19 M and the signed content's 0.01 M. CBMC cannot read the intrinsics, so equivalence tests against the portable code hold them, as they hold the AES instructions. Since decision 94 the RFC 6979 nonce is the largest part of a signature: 0.24 M of its 0.53 M instructions on the M1 Pro, where OpenSSL's whole signature takes 0.18 M.
  5. X25519 key generation from a table of multiples of the base point, as BoringSSL computes it: done, for a host session that states its multiply (decision 148). A clamped scalar is 64 signed four-bit digits, and the key generation adds one entry of a 30 KiB table of multiples of edwards25519's base point for each, doubles four times, and maps the sum to its u. On a Neoverse-N2, bench/handshake-n2.sh counts a key generation at 132,403 instructions where the ladder took 393,095, and OpenSSL 3.6.4's at 227,086; the handshake under 0xe7 takes 0.883 M where it took 1.144 M ("What decision 148 changed", above). The table sits inside the L1 data cache of every core here, which decision 109 found to matter on the M1 Pro. The shared secret keeps the ladder.
  6. A server whose suite hashes with SHA-384 stops the SHA-256 transcript, 0.08 M.

The method behind these numbers

Every performance change follows pepegrillo's docs/performance.md: measure the gap, attribute the cost, choose the lever, build for the hardware, prove it, land it. This document is chapulin's appendix to it, and two things set this tree apart from the others that share the method: what binds is SRAM and flash before instructions, and constant time is a correctness property, never a cost to trade (ct.[ch]).

  • The judge is the device builds, through the scripts in bench/ that write the figures above; a laptop's speed is a filter that orders candidates and lands in no document. A figure here is measured, never estimated, and a change to the code re-measures it in the same commit (make lint-bench-numbers).
  • make check builds every program those scripts build and runs none of them (test/script-builds.sh), so a source list that misses a source the code calls fails check rather than the next measurement. The instruction-count scripts take their sources and defines from the Makefile's INSN_SRCS and INSN_DEF, and the host builds take AES_HW_SRCS (decision 88).
  • The record split above prints a filter's figures, as an exception to the rule above: a SUITE=aesgcm host object runs on hosts alone, so no device build can judge it, and #184 and #181 asked for the order the split gives. It orders candidates. A change it points to still needs a judge's runs on a host that runs nothing else, and this document names no such host yet.
  • The table beside OpenSSL prints a filter's figures too, for the same reason: every row of it runs in a host object. It states each machine's load beside its figures and takes both sides of a ratio in one run, on CPU time for every row above the AEADs, and its CSV holds each chapulin row's instruction count where macOS gives one. It ranks the work by ratio. A change that claims a row records the row again with both scripts, and reports the instructions of the rows it moved.
  • The units of work are a handshake, a record, a byte of SRAM and a byte of flash.
  • A change stays when it lowers SRAM, flash or instructions past the noise on the device builds and raises none of them past it; a change that trades constant time for any of the three is refused, whatever it saves.
  • The pitfalls it has paid for, as symptom, cause and rule, are the table below.
symptom cause rule
The 16-byte wipe after each AES block took 7.5 µs of a 16 KiB AES-GCM record, more than the AES rounds (044a49c). ct_wipe stores one byte per iteration through a volatile pointer and is a call, and aes_hw.c ran it after every block. Wipe a buffer once, after the loop that fills it, never once per block.
gcc 13's AES-GCM counter mode took 16.5 µs of a 16 KiB record where clang's took 1.8. gcc at -O2 leaves a loop of eight iterations over vector states rolled and keeps the states in memory, so every round loads and stores all eight. A loop over the states a pass keeps in registers carries #pragma GCC unroll, and every change is measured under gcc as well as clang.
clang's GHASH took 2.6 and 2.8 µs of a 16 KiB record where gcc's took 2.2, on the same core. clang loads each lane of a vector that PMULL reads from memory as a 64-bit word in a general register and moves it back into a vector register, two moves for each operand. Read a PMULL loop's disassembly under both compilers, and look for fmov and dup from general registers inside it.
The vector Poly1305 took 4.7 µs of a 16 KiB record in one bench build under the VM's clang 18, and 3.0 µs in a build whose only difference was a function in aes_hw.c. Where the linker places the function: the first build with -falign-functions=64 took 2.8 µs. When a row moves on a path the change did not touch, rebuild with functions aligned before reading the move as the change's.
bin/ghash_equiv_test found powers of H in the stack below a GHASH call after the call had wiped the state that held them (88c64be). Holding the eight powers in registers across a pass, clang and gcc copied some to stack slots of their own, which the wipe of the state does not clear. Read a value the wipe must clear from the wiped state, through a volatile lvalue, each time it is used, and run the stack check after every change to a loop that holds one.
gcc 13's one-pass seal loop took 3.6 µs of a 16 KiB record where it had taken 2.7, once the open called the same exclusive-or. gcc at -O2 kept one out-of-line copy of a helper that two pass loops call. For the AES rounds, whose keys are bytes and so may alias the states, such a copy stored all eight states every round. A helper that works on the states a pass keeps in registers is static inline, and a pass loop's disassembly under gcc holds no call.
gcc 13's one-pass open loop took 3.5 µs of a 16 KiB record where its seal loop took 2.6. With the first pass's hash before the loop, gcc counted the low byte of each counter in a register of its own and built each counter word with bfi, where the seal's loop reverses one word with rev. Count bfi and rev in a pass loop's counter code under gcc, and when gcc builds the counters from bytes, change the loop's shape rather than the arithmetic.
Under clang 18, bin/ghash_equiv_test found powers of H in the stack below the one-pass open, and it passed under Apple clang and gcc. With a pass's hash and its AES rounds in one block, clang 18 moved the powers' volatile reads up among the rounds and ran out of registers. Put a pass loop's hash and its rounds in separate if blocks, and run the stack check under clang 18 as well as the compilers make check runs.
On the EPYC 7763 under gcc 13, the server side of a pinned RSA-2048 handshake took 2.07 ms where it had taken 1.88, after a change to sha3.c, which that handshake never calls. The signature's own row, in another program, stayed at 1.74 ms. RSA on 64-bit words runs at one of two speeds on that CPU, by where the linker places it. With -falign-functions=64 the signature's row took 1.93 ms and the RSA-2048 verify's 65.8 µs against 59.0, and the handshake's stayed at 2.07 ms. A build with functions aligned does not cure this one: it shows the slower speed. Read a move of about a tenth on a path the change did not touch against that build's figure, and when they agree, record it as placement and leave the change alone.
An eight-block ChaCha20 pass took 36.6 µs over 16 KiB under gcc 13 and 8.4 under clang 18, in a scratch timing loop. gcc 13 at -O2 kept one out-of-line copy of a static inline function that ran a quarter round on both groups of a pass, called eight times a double round, so every call stored the 32 state vectors to memory and loaded them back. Build a pass's rounds from a helper small enough that gcc inlines it, such as one quarter round on one group, and read the round loop's disassembly under gcc for a call.
On the CPU clock, an AES-GCM seal through OpenSSL took 5.9 µs for some minutes on the M1 Pro, and 2.1 µs before and after. macOS placed the process on efficiency cores while other work held the performance cores, and a CPU clock counts that time as it counts any other. Under taskpolicy -b, which runs a process on those cores, the same seal took 7.2 µs. Read a row's spread between runs before its median. A run on efficiency cores moves the spread past 100%, and bench/primitives.sh warns past 50%.
The AES-GCM AEAD read as 1.05 times OpenSSL's seal on macOS, and under a key expanded for the record, which is the operation OpenSSL timed, as 1.13 times. OpenSSL 3.6's speed -aead sets the key for every operation, where 3.0's sets it before the loop, and for ChaCha20-Poly1305 3.6 runs one update with no nonce and no tag. Before a ratio against another library's benchmark, read what one of its operations holds in the release that ran, and time a 16-byte operation to confirm it. bench/record.sh names each OpenSSL row after the stage that times the same operation.
gcc 13 for arm64 wrote 13 vector registers to the stack in sha3_hw.c's permutation, and three a round with four lanes held in a struct the function wiped, where clang 18, 21 and 23 wrote none. A round of Keccak-f[1600] keeps 32 values at once, the 25 lanes and seven more, and arm64 has 32 vector registers. A compiler that needs a 33rd writes a lane to a slot it picks, and no wipe written in C clears one. Before a path that keeps every register live holds secrets, count its stores of vector registers to the stack under each compiler that builds it. Decision 99 builds this one under clang alone.
Apple clang 21 and clang 23 compiled a second copy of an earlier form of sha3_hw.c's permutation for the arguments one caller passed, and the second copy wrote five or six lanes to the stack where the first wrote none. A compiler picks the registers of each copy of a function anew. Call a function whose register use a check depends on through a volatile function pointer, so the one copy compiled is the copy checked (decision 99).
With four lanes of the Keccak state kept in memory to free four registers, clang took 30% longer over 16 KiB, and 2.5% longer once the round computed their row first. The next round's column parities read those lanes first, so each read waited for the store the round before had just made. A value a loop keeps in memory to free a register is written early in an iteration and read late.
Under gcc 13 on x86-64, an ML-KEM-768 encapsulation spent 49,591 of its 515,432 instructions in sha3.c's block_bytes, which writes out squeezed bytes. gcc at -O2 kept lane_to_bytes, a loop over a lane's eight bytes, as a loop: a shift and a one-byte store each iteration. With each byte named, it compiles the eight stores to one 8-byte store. Name each byte of a lane's store, as lane_from_bytes names each byte of its load, and count a byte loop's stores in gcc's disassembly.
The Jacobian key exchange's multiplication took 58.2 µs on the M1 Pro where a scratch program's, on the same products and sums, took 54.0 (decision 112). The doubling computed Z3 = 2 Y1 Z1 last, as the Explicit-Formulas Database writes it, and the next doubling squares Z first, so each of 253 doublings waited for the product and the sum that ended the one before. Every field operation is a call, and the core starts a call's work before the call before it ends only where it reads nothing that call writes. In a chain of point formulas, compute first what the next formula reads first, and time the chain, never one formula alone.
On the EPYC 7763 under gcc 13 the 64-byte SHA-384 row read 500 ns where the call took 408, and under clang 113 ns over its call (run 38140976845). bench_consume added every byte of the digest into a volatile after each call: a load, an add and a store for each byte. The hash, MAC and tag rows consumed their whole output, the public-key rows one byte, and openssl speed has no such loop. A row's sink consumes one byte of the output. Before a short row is set beside another library's, time the call with the bench's own loop and without it.
Under clang 23, rec_seal of a 16 KiB ChaCha20-Poly1305 record under ch_cfg.cpu 0x1f read 14.42 µs on one EPYC 7763 runner and 9.36 µs on another, with 0.1% between the five runs of each, while aead_seal read 9.11 µs on both (runs 38097442274 and 38100535828). clang did not align the frame of chacha20_avx2.c's pass to 32 bytes, and the round loop stores and reloads six state words in 32-byte stack slots. Linux places each process's stack at a random address, in steps of 16 bytes. At 6 of the 256 positions within a page one of those slots lay across the page boundary, and the exclusive-or of 16 KiB took 1.25 to 1.84 times as long. At every other position 16 past a multiple of 32, three of the slots lay across a cache line, and it took 1.04 times as long. Each row calls the kernel at its own stack depth, so one row of one process read slow, and that process's five runs agreed. Over 100 processes on five runners, 30 read at least one of the bench's six rows on that kernel more than 5% over its median. gcc 13 aligns the frame. keccak_avx2.c's permutation had the same frame under clang: a call took 441 ns aligned, 675 ns at 16 past a multiple of 32, and up to 1,228 ns (runs 38153456033 and 38154870948). Both functions now carry the attribute that aligns the frame (decision 124): the exclusive-or takes 6,231 to 6,350 ns and the permutation 440 to 442 ns at every one of 273 stack positions, and over 80 processes on two runners none read one of the six rows more than 3% over its median (run 38159239411). A bench process has one stack address, and the spread between its runs does not see it. When bench/record.sh warns that a record row differs from the sum of its parts, or a row moves on one runner alone, time the binary in several new processes, or under setarch -R at several sizes of the environment, before reading the move. Under clang, read the prologue of a function that keeps 256-bit values in stack slots inside a loop for and rsp, as make lint-frame-align does for those two files and for sha512_avx2.c (INV-49).
rec_seal_without_aead read the copy of a 16 KiB plaintext as 257 ns in one record on the EPYC 7763 and as 684 ns in another, and in a scratch program whose plaintext and record buffers started at the same offset within a page the copy took 5.4 µs. glibc 2.39 on that CPU copies more than 2,112 bytes with rep movsb. With the destination 1 to 31 bytes past the source's offset within a page, 16 KiB takes 5.3 to 5.6 µs in every process, against about 0.25 µs at the other offsets: a record's body starts 5 bytes past the record, so a plaintext and a record at the same offset are in that range. With the bench's buffers, 61 bytes apart, the copy takes 0.25 µs unless a source page and the destination page its bytes go to have physical frame numbers equal modulo 16. It then takes 0.44 to 1.3 µs: in 243 of 900 processes on three runners, each of which had such a pair, and in none of the 657 that had none. Under GLIBC_TUNABLES=glibc.cpu.x86_rep_movsb_threshold=16777216, which keeps glibc off rep movsb, the copy takes at most 0.37 µs at any offset, and 0.16 to 0.18 µs in each of 300 processes with the bench's buffers. An EPYC 9V45 showed neither effect (run 38154870948). bench_state now starts its plaintext on a page boundary and its record buffer 2,048 bytes into a page, so the copy's destination is 2,053 bytes past its source's offset, and the copy took 250 ns in each of 600 processes on two runners (run 38159239411). Keep a timed copy's destination more than 31 bytes from its source's offset within a page, and read rec_seal_without_aead beside rec_seal before reading a move of up to 1 µs in a 16 KiB row as the AEAD's.
On the EPYC 7763 under clang 23, sha384_of_cpu of 64 bytes under CH_CPU_AVX2 read 230 ns in a program linked with sha512_avx2.c after decision 128's quarter steps and 205 ns in one linked with the file before them, on each of 11 runners, while the kernel alone over one block read 182 ns and 186 (run 38182747184). That call runs at one of two speeds on that CPU by where the kernel's code starts within a 64-byte line. Linked with 0, 16, 32 and 48 bytes of padding before the kernel's object, the kernel after the change read 230, 230, 203 and 232 ns and the one before it 205, 234, 235 and 235, on each of 13 runners. The 16 KiB row and the kernel alone over one block did not move with the padding, and under gcc 13 one padding of the four read 230 ns where the other three read 220 to 224 (run 38184809651). What in the CPU makes the two speeds was not found. A bench program links a kernel at one address. Before a move of a tenth in a row of one block is read as a change's, link the kernel at each of the four offsets a function takes within a 64-byte line and read the row at all four.
Under qemu-x86_64 8.2.2, sha512_avx2_compress_blocks as gcc 13.3 compiles it took SIGSEGV on one block that ends at the last mapped byte before an unmapped page, and passed as clang 18 compiles it. gcc loads each four words of a block as a 16-byte VMOVDQU and a VINSERTI128 whose second operand is 16 bytes of memory, and that qemu reads past those 16 bytes: the same VINSERTI128 on the last 16 mapped bytes took the signal in a probe of its own, where a 16-byte VMOVDQU of them did not, and a CPU reads 16. clang loads a block in four 32-byte VMOVDQU. Read a fault under qemu-user at an instruction with a memory operand against that operand's size before reading it as the code's. A gcc host object that runs under that qemu with CH_CPU_AVX2 needs 16 mapped bytes after a message, and what a kernel reads past its input is measured on a CPU.