Build Conscrypt with ECH, and test it on the JVM and on Android - #33
Merged
Conversation
`network:echTest` cannot pass on the JVM, and saying so is less useful than saying why. Three things are in the way and only one of them is OkHttp's: no published TLS stack a JVM can load will encrypt a client hello; `ConscryptPlatform.configureTlsExtensions` takes an `echConfigList` and ignores it; and Conscrypt's OpenJDK `Platform` throws away the retry configs a server offers when it rejects a stale one. `conscrypt/` closes the first. Conscrypt's `google3-export` branch has `Conscrypt.setEchConfigList(SSLSocket, byte[])`, which is in no release, so `build-conscrypt.sh` builds it against a pinned BoringSSL and the `conscrypt` workflow caches the jars as a release keyed by both shas. Suites download that rather than rebuilding: the build is fifteen minutes and its output changes only when `pinned.properties` does. `network:echConscryptTest` runs on it. `EchClientHelloTest` reads the client hello off a local socket that answers nothing and asserts the name is not in the bytes — no DNS, no server, no internet, so it is the one ECH result that is never about somebody else's uptime. `EchConscryptTest` is `EchTest`'s cases against the public servers, with the two things the JVM lacks supplied from outside OkHttp. The gap between it and `echTest` is then the second item above and nothing else. Neither claims OkHttp does ECH on the JVM: the suite makes the `setEchConfigList` call itself, from a socket factory, precisely because OkHttp doesn't. The third item has no counterpart at all, which is why the stale-config case doesn't cross over. `android-ech` gains `PublicEncryptedClientHelloTest`, the same public-server cases on the API 37 emulator, so the JVM and Android rows of the status page are the same assertions against the same servers with only the TLS stack differing. It runs first and does not fail the job, for the reason nothing in `network` gates; the fixture suite after it still does. All of `conscrypt/` should be deleted the day Conscrypt ships ECH. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CqiK79k9uoWsn2AzgXpHMA
… run that produced nothing The first PR run installed the test APK while the emulator's package service was still coming up, got `Broken pipe`, and reported zero tests for `PublicEncryptedClientHelloTest`. Zero tests is the worst failure mode available here: it isn't an ECH result, and on the status page it doesn't look like one either. Two changes, because the race has two ends. `wait_for_package_service` asks `pm` to answer before the first suite runs — `sys.boot_completed` is already true when the install fails, so it is the wrong thing to wait on. And a run that fails without writing any results is retried once, which covers the same failure wherever it turns up and costs nothing when it doesn't. Also drop the "fifteen minutes" figure for the Conscrypt build; the workflow does it in about six. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CqiK79k9uoWsn2AzgXpHMA
…ostname
With the install race fixed, the public suite ran for the first time and all
six cases failed identically, none of them for a reason involving ECH:
java.lang.IllegalStateException: Unable to load PublicSuffixDatabase.list
at okhttp3.internal.publicsuffix.AssetPublicSuffixList.listSource
at okhttp3.dnsoverhttps.DnsOverHttps$Companion.isPrivateHost
On Android the public suffix list is an asset rather than a jar resource, and
`assets/PublicSuffixDatabase.list` is shipped by `okhttp-android` — which this
module didn't depend on, because nothing in the source names it. `DnsOverHttps`
consults the list on every query to decide whether a host is private, so the
suite failed before it opened a socket.
Verified by assembling the test APK and finding the asset in it. What the six
cases actually say about ECH on API 37 is now a question CI can answer, which
it could not before.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CqiK79k9uoWsn2AzgXpHMA
yschimke
pushed a commit
that referenced
this pull request
Aug 14, 2026
The retry added in #33 never fired on the failure it was written for. It asked whether the results directory was absent; a run whose APK install never happened still leaves that directory behind, empty. So the run on the merge commit reported `Starting 0 tests`, took the non-fatal path, and the job went green with the public ECH suite having said nothing at all — which is what the retry existed to prevent. Ask about test cases instead. Checked against all four states: no directory, empty directory, XML with zero cases (the observed failure), and real results — only the last declines to retry. Also drop `wait_for_package_service`. #31 landed a real readiness gate a few commits later, and it is strictly stronger than this one on every axis: it waits on the settings provider and external storage as well as the package service, it proves the path with a live call, and it fails the run instead of shrugging when the device never arrives. Two waits where the weaker one runs second is just a claim that the first didn't work. This does not yet say whether ECH works on API 37. No run has executed those six cases: the first two never installed, and the third failed on the public suffix asset. It says that a run which doesn't execute them will stop looking like one that did. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CqiK79k9uoWsn2AzgXpHMA
yschimke
added a commit
that referenced
this pull request
Aug 14, 2026
…ory (#35) The retry added in #33 never fired on the failure it was written for. It asked whether the results directory was absent; a run whose APK install never happened still leaves that directory behind, empty. So the run on the merge commit reported `Starting 0 tests`, took the non-fatal path, and the job went green with the public ECH suite having said nothing at all — which is what the retry existed to prevent. Ask about test cases instead. Checked against all four states: no directory, empty directory, XML with zero cases (the observed failure), and real results — only the last declines to retry. Also drop `wait_for_package_service`. #31 landed a real readiness gate a few commits later, and it is strictly stronger than this one on every axis: it waits on the settings provider and external storage as well as the package service, it proves the path with a live call, and it fails the run instead of shrugging when the device never arrives. Two waits where the weaker one runs second is just a claim that the first didn't work. This does not yet say whether ECH works on API 37. No run has executed those six cases: the first two never installed, and the third failed on the public suffix asset. It says that a run which doesn't execute them will stop looking like one that did. Claude-Session: https://claude.ai/code/session_01CqiK79k9uoWsn2AzgXpHMA Co-authored-by: Claude <noreply@anthropic.com>
yschimke
pushed a commit
that referenced
this pull request
Aug 14, 2026
The six public ECH cases executed for the first time on e0c7c1d, and all six failed the same way: java.lang.IllegalStateException: Unable to load PublicSuffixDatabase.list Caused by: java.io.IOException: Platform applicationContext not initialized. Startup Initializer possibly disabled, call OkHttp.initialize before test. So #33's diagnosis was wrong in mechanism. The asset was missing and adding `okhttp-android` did ship it, but that was only half: the list is read through a `Context` that nothing here ever supplied. `DnsOverHttps` asks `isPrivateHost` about the name before opening anything, so all six died there, upstream of any TLS — the runs said nothing about ECH. `okhttp3.internal.platform.PlatformInitializer` is declared in the merged androidTest manifest via androidx Startup, and it still didn't run. Rather than work out why an initializer inside an instrumentation APK stays dormant, set the context outright, in the runner, where it covers every suite and happens once. Verified as far as it can be without an emulator: the APK assembles and the merged manifest names the new runner — <instrumentation android:name="okhttp.testbed.android.ech.EchTestRunner" Not claimed: that this makes the public suite pass. It removes the failure that stood in front of ECH; what those six cases then say about API 37 is the result we have never had. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CqiK79k9uoWsn2AzgXpHMA
yschimke
added a commit
that referenced
this pull request
Aug 14, 2026
The six public ECH cases executed for the first time on e0c7c1d, and all six failed the same way: java.lang.IllegalStateException: Unable to load PublicSuffixDatabase.list Caused by: java.io.IOException: Platform applicationContext not initialized. Startup Initializer possibly disabled, call OkHttp.initialize before test. So #33's diagnosis was wrong in mechanism. The asset was missing and adding `okhttp-android` did ship it, but that was only half: the list is read through a `Context` that nothing here ever supplied. `DnsOverHttps` asks `isPrivateHost` about the name before opening anything, so all six died there, upstream of any TLS — the runs said nothing about ECH. `okhttp3.internal.platform.PlatformInitializer` is declared in the merged androidTest manifest via androidx Startup, and it still didn't run. Rather than work out why an initializer inside an instrumentation APK stays dormant, set the context outright, in the runner, where it covers every suite and happens once. Verified as far as it can be without an emulator: the APK assembles and the merged manifest names the new runner — <instrumentation android:name="okhttp.testbed.android.ech.EchTestRunner" Not claimed: that this makes the public suite pass. It removes the failure that stood in front of ECH; what those six cases then say about API 37 is the result we have never had. Claude-Session: https://claude.ai/code/session_01CqiK79k9uoWsn2AzgXpHMA Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
network:echTestcannot pass on the JVM. Saying so is less useful than saying why, and the answer turns out to be smaller than the red result suggests.What's actually in the way
google3-exportConscryptPlatform.configureTlsExtensionsechConfigListand ignores itPlatform.wrapEchRejectedExceptionOnly the middle row is OkHttp's, and it is one call next to the ALPN one —
Android10Platformis the model, and it's what lysine-dev/okhttp#9559 does.conscrypt/Conscrypt.setEchConfigList(SSLSocket, byte[])is public API ongoogle3-exportand in no release. (google/conscrypt#1406 is still open; what landed on that branch is Google's internal version, exported by Copybara.)build-conscrypt.shbuilds it against a pinned BoringSSL. The build takes several minutes (about six on CI) and its output changes only whenpinned.propertiesdoes, so it doesn't run per checkin: theconscryptworkflow builds it and publishes the jars as a release tagged with both shas, and the suites download that. Bumping a pin invalidates the cache by construction — the tag no longer exists, so the next run builds and publishes it. A pull request touching the pin gets the build checked and nothing published.network/build.gradle.ktspicks upconscrypt/build/dist/conscrypt-openjdk-*.jarif it's there and leaves the suite out of the source set if it isn't, so neither the fetch nor the build is on anyone's critical path.network:echConscryptTestTwo suites, both gated on that jar.
EchClientHelloTestconnects to a local socket that accepts and says nothing, reads the client hello, and asserts the name is not in the bytes. No DNS, no server, no internet — it's the one ECH result that is never about somebody else's uptime. Verified locally:encrypted_client_hellopresent, outer SNI is the public name, inner name absent from the cleartext.EchConscryptTestisEchTest's cases against the public servers with the two things the JVM lacks supplied from outside OkHttp: this Conscrypt, and a network security policy saying ECH is allowed (on Android that comes fromnetwork_security_config.xml; the JVM default is a value Conscrypt reads as "no"). The gap between it andechTestis then the second table row and nothing else.Neither claims OkHttp does ECH on the JVM. The suite makes the
setEchConfigListcall itself, from a socket factory, precisely because OkHttp doesn't. The third table row has no counterpart at all — on OpenJDK a rejected config throws its retry configs away and nothing public exposes them, sostale.tls-ech.devcan be detected there and not recovered from, whatever OkHttp does.Android
PublicEncryptedClientHelloTestrunsEchTest's public-server cases on the API 37 emulator, so the JVM and Android rows of the status page become the same assertions against the same servers with only the TLS stack differing. It runs first and does not fail the job, for the reason nothing innetworkgates; the fixture suite after it still does.run-ech-test.shmoves each run's XML aside so both are uploaded.Two things had to be fixed before that suite could say anything, both found by reading a green job rather than trusting it:
Broken pipe, which is the worst failure available here — not an ECH result, and not visibly distinguishable from a pass on the status page.run-ech-test.shnow waits forpmto answer before the first suite (sys.boot_completedis already true when the install fails, so it's the wrong thing to wait on) and retries any run that writes no results.Unable to load PublicSuffixDatabase.list. On Android that list is an asset, shipped byokhttp-android, which this module didn't depend on because nothing in its source names it — andDnsOverHttpsconsults it on every query to decide whether a host is private. So the suite failed before opening a socket. Fixed by adding the dependency; verified by findingassets/PublicSuffixDatabase.listin the assembled test APK.Verified
d65d6b6built against BoringSSL3c31f33, x86-64 and aarch64.network:echConscryptTest→EchClientHelloTestpasses.android-ech:compileDebugAndroidTestKotlinpasses; the androidTest APK packages the public suffix asset.The public-server assertions could not run where this was developed — outbound TLS is intercepted there, so the preflight skips them. What the six Android cases say about ECH on API 37 is a question the next CI run answers; the two runs so far never got far enough to ask it.
All of
conscrypt/should be deleted the day Conscrypt ships ECH.🤖 Generated with Claude Code
https://claude.ai/code/session_01CqiK79k9uoWsn2AzgXpHMA
Generated by Claude Code