fix(docker): make auth bootstrap safe for mounted and upgraded configs by Adarsh-Me · Pull Request #3192 · apache/hugegraph

Adarsh-Me · 2026-09-03T15:50:23Z

What is changing

Closes #3133 (the parts still open on master 3681148, since #3119 landed the rest).

Properties rewriting now implements the Java grammar. docker-entrypoint.sh previously rewrote rest-server.properties / hugegraph.properties with grep/sed, which disagrees with HugeConfig on mounted or upgraded configs: backslash-escaped keys, :/whitespace separators, line continuations, and duplicate definitions are all parsed differently. The property logic moves to a new props.awk loaded by the entrypoint, which implements the java.util.Properties line grammar (comments, both separators, continuations, backslash escapes, first-definition-wins duplicates) and rewrites the first definition in place while keeping every untouched line byte-for-byte.

PASSWORD no longer appears in ps output. The old sed rewrite interpolated the encoded value into sed's command line, so when a key already existed (mounted or persisted config) the password was visible in ps. Values now travel through an environment variable into awk, never through argv.

enable-auth.sh appends are per-file guarded. The old script appended authentication definitions whenever conf-bak/ was absent. On a config it did not write, that created duplicate definitions that the properties parser (first definition wins) and snakeyaml (last definition wins) resolved in opposite directions — Gremlin and REST could land on different authenticators with no error from either. Now each append runs only when its file lacks the definition (or still has it commented out), re-runs are idempotent, custom gremlin.graph factories are preserved, and the authenticator class can be overridden via AUTHENTICATOR_CLASS.

The entrypoint aligns both sides before enabling auth. If the yaml declares an authenticator but the properties file does not (or vice versa), the entrypoint propagates it to the other side instead of letting the default StandardAuthenticator split the pair. When both sides name genuinely different authenticators, it logs a WARN and leaves both untouched instead of silently splitting them.

Implementation notes

I chose a shell/awk implementation of the properties grammar over the issue's proposed Java ConfigTool CLI: it delivers the same parser-agreement contract with a much smaller footprint and no new build artifact. Happy to rework toward the ConfigTool if reviewers prefer that direction.
props.awk ships in both server images (Dockerfile COPY) and in the test sandbox; CI already runs the unit suite via docker-build-ci.yml.

How was this tested

docker/test/test-docker-entrypoint.sh extended with cases for: escaped-key definitions rewritten in place, continuation lines consumed with the key they belong to, get-mode separator/continuation/duplicate semantics, and appends when the key only exists commented out. All pass.
docker-entrypoint-test.sh full harness passes end-to-end (secret round-trips incl. backslash/space/trailing-space secrets, enable-auth call counting unchanged).
enable-auth.sh manually exercised against: fresh default config, idempotent re-run, mounted config with custom authenticator (no duplicates), and custom gremlin.graph factory (preserved).

Code Review Handbook

props.awk is the core: block model (comment lines and logical entries), first-definition-wins, raw-value round-trip (get returns the on-disk escaped form so feeding it back into set is byte-exact).
The value of get is intentionally not unescaped: the entrypoint re-writes secrets it just read, and unescape-then-re-encode would double-escape backslashes (caught by the complex-secret round-trip in docker-entrypoint-test.sh).
The escaped-key unit case is the reported auth\.admin_pa scenario: the old grep could not match it, so the append created a duplicate and HugeConfig silently kept pa.

The entrypoint's grep/sed property rewriting disagrees with HugeConfig on mounted or upgraded configs: escaped keys, ':'/whitespace separators, line continuations, and duplicate definitions are all read differently, so a mounted config could end up with two logical definitions of one key. Property reading/writing now goes through props.awk, which implements the java.util.Properties grammar (comments, both separators, continuations, backslash escapes, first-definition-wins duplicates) and keeps every untouched line byte-for-byte. Values travel through environment variables instead of command arguments, so a PASSWORD no longer shows up in 'ps' output when a key is rewritten in place. enable-auth.sh appended authentication definitions whenever conf-bak/ was absent, which on a mounted config created duplicate definitions that the properties parser (first definition wins) and the yaml parser (last definition wins) resolved in opposite directions -- Gremlin and REST could land on different authenticators with no error from either. Its appends are now guarded per file, only an absent or still commented-out definition triggers an append, re-runs are idempotent, and the authenticator class is overridable through AUTHENTICATOR_CLASS. The entrypoint aligns both sides before calling it: it copies a yaml authenticator into rest-server.properties, or exports the REST one for the yaml append, and warns without touching anything when the two name genuinely different authenticators. The unit test suite covers escaped keys, continuations, get-mode semantics, and comment-guarded appends; the entrypoint harness now ships props.awk into its sandbox, and both server Dockerfiles COPY it next to the entrypoint. Fixes apache#3133

bitflicker64

Blocking: yes. Summary: props.awk is careful work and both shell suites this PR touches pass under the image's own mawk, but the new alignment layer does not hold on the mounted configs the PR targets. The yaml authenticator is copied into rest-server.properties without unquoting, a flow-style authentication: block reads as absent so REST silently falls back to the default, enable-auth.sh's guards accept only the key= spelling so duplicates are still appended, and the narrowed gremlin.graph guard no longer converts a CRLF config the old sed did convert. Two doc fixes as well. Evidence: measured at 698b0c3 in ubuntu:22.04 (GNU grep 3.7, GNU sed, mawk 1.3.4), the same toolchain eclipse-temurin:11-jre-jammy ships; test/test-docker-entrypoint.sh and docker-entrypoint-test.sh both exit 0 there; each finding below quotes the config the run produced. All seven workflow runs on this head are action_required and the combined status is pending with zero statuses, so there is no CI evidence for the image builds.

imbajin

Blocking: yes. Summary: The new properties parser still mishandles valid indented keys, so mounted authentication values can remain stale after an environment override. Evidence: reproduced at the exact head with the PR helper; existing current-head comments cover other findings and are not duplicated.

codecov · 2026-09-04T15:33:37Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 37.80%. Comparing base (3681148) to head (698b0c3).

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #3192      +/-   ##
============================================
+ Coverage     37.77%   37.80%   +0.02%     
- Complexity     6560     6567       +7     
============================================
  Files           800      800              
  Lines         68960    68960              
  Branches       9166     9166              
============================================
+ Hits          26052    26069      +17     
+ Misses        39841    39825      -16     
+ Partials       3067     3066       -1

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Review follow-ups on the props.awk bootstrap: the yaml authenticator scalar now goes through a snakeyaml-shaped cleanup (inline comments, quotes and padding stripped) instead of only cutting at the first comma or colon; a flow mapping on the authentication line itself is read, and an authentication block without a readable authenticator takes the WARN branch instead of the both-empty default. props.awk strips leading whitespace before the key the way java.util.Properties does, so an indented key is rewritten in place rather than duplicated. enable-auth.sh's append guards now accept the ':', bare-whitespace and backslash-escaped spellings with [[:blank:]] classes (the '[ \t]' bracket matched space, backslash and the letter t), and the gremlin.graph flip embeds the carriage return as a byte because GNU grep reads \r in a pattern as the letter r, which made the anchored guard drop mounted CRLF configs. Test docs name the environment variables and the function count they rely on, and new regression tests cover indented keys, yaml scalar cleanup, flow mappings and the block-without-authenticator WARN.

bitflicker64

Blocking: no. Summary: The fresh-default path works and is idempotent at this head (one auth.authenticator, one authentication: block, gremlin.graph flipped to HugeFactoryAuthProxy, stable across three re-runs), and the fixes for the earlier review round all check out. Four gaps remain between what the new code promises for mounted or upgraded configs and what it does, three of them around CRLF and separator spellings that java.util.Properties accepts. Evidence: entrypoint helpers extracted the way test/test-docker-entrypoint.sh does, run against verbatim head copies of props.awk and bin/enable-auth.sh with GNU sed; props.awk output compared byte for byte against java.util.Properties.load on the same files; the gremlin.graph guard and its sed run against eight legal spellings. Not covered: no Docker daemon on this host, so the images were not built and the runtime mawk path was not exercised. props.awk uses only POSIX awk features and awk is already a dependency of the shipped bin/*.sh, but CI's docker-build-ci.yml run of the suite remains the authority there.

bitflicker64 · 2026-09-12T07:01:14Z

+# logical entry, spanning exactly the physical lines it occupies.
+function props_load(file,    raw, nl, next_raw, start, logical) {
+    NLINES = 0
+    while ((getline raw < file) > 0) {


⚠️ getline here keeps the \r of a CRLF line, so props.awk parses differently from the grammar it implements. java.util.Properties treats \r\n as the line terminator and drops it.

# rest-server.properties saved with CRLF props.awk get auth.authenticator -> org.apache.hugegraph.auth.StandardAuthenticator\r java.util.Properties.load -> org.apache.hugegraph.auth.StandardAuthenticator

Continuations break too, because trailing_backslashes (line 96) sees the \r, not the backslash, as the last character:

# pd.peers=a,\<CR><LF> b<CR><LF> props.awk get pd.peers -> a,\<CR> java.util.Properties -> a,b

Downstream, align_auth_config compares this CR-bearing value against get_yaml_authenticator, whose scalar() strips CR, so a REST and a Gremlin side naming the same class log WARN: REST and Gremlin name different authenticators with two strings that print identically, and neither side is aligned. Mounted CRLF configs are in scope: the enable-auth.sh hunk in this PR embeds a CR byte for exactly that case.

Requested change: strip one trailing \r while assembling logical in props_load, after this getline and inside the continuation loop, rather than from RAW[], which props_set replays byte for byte. A CRLF case in test/test-docker-entrypoint.sh would pin both halves.

Fixed in 5f50511. props_load now strips one trailing CR while assembling logical lines (after each line and inside the continuation loop) so CRLF parses like java.util.Properties; RAW[] is untouched so rewrites replay untouched lines byte-for-byte. Pinned with a CRLF regression case in test-docker-entrypoint.sh (value, continuation, CR preservation); suite passes.

bitflicker64 · 2026-09-12T07:01:14Z

+    for (b = 1; b <= NBLOCK; b++) {
+        if (BDROP[b]) continue
+        if (b == first) {
+            printf "%s=%s\n", key, enc_val > file


⚠️ This rewrite is destructive and non-atomic, a step back from the sed -i it replaces, which writes a temp file and renames.

props_load's while ((getline raw < file) > 0) (line 142) treats getline's -1, an unreadable file, the same as EOF. NLINES stays 0, props_set reaches the first == 0 append with nothing to replay, and the config is gone:

$ printf 'a=1\nb=2\nauth.token_secret=keepme\n' > w.properties && chmod 222 w.properties $ PROPS_MODE=set PROPS_KEY=x PROPS_VALUE_ENCODED=1 PROPS_FILE=w.properties awk -f props.awk /dev/null; echo $? 0 $ cat w.properties x=1

Separately, awk's > truncates on this first write, before the preserved lines are replayed, so a kill or ENOSPC mid-rewrite leaves a mounted rest-server.properties truncated. conf-bak/ is no help: it is written later, by enable-auth.sh.

Requested change: die() in props_load when getline returns -1, and have props_set write to a sibling temp file, close() it, then rename it over the original with the paths shell-quoted. Note the die() also rejects a genuinely missing file, since awk cannot tell missing from unreadable; nothing in the entrypoint or the suite depends on that append path.

Fixed in 5f50511. props_load dies when getline returns -1 (unreadable file no longer truncates to an append), and props_set writes a sibling temp file, closes it, then renames over the original with shell-quoted paths. Verified: unreadable file errors with the original intact, no .tmp leftovers; suite passes.

bitflicker64 · 2026-09-12T07:01:14Z

+# misses a mounted CRLF config and the factory is never wrapped for auth
+# although both servers already believe authentication is on.
+CR=$'\r'
+if grep -Eq "^gremlin\\.graph[[:blank:]]*=org\\.apache\\.hugegraph\\.HugeFactory[[:blank:]]*${CR}?\$" "${CONF}/graphs/${GRAPH_CONF}"; then


⚠️ This guard is stricter than the sed it guards, so part of that sed's own pattern can never fire. The guard needs HugeFactory immediately after graph[[:blank:]]*=, while the replacement on line 82 allows [[:blank:]]* there:

gremlin.graph=org.apache.hugegraph.HugeFactory guard=MATCH gremlin.graph = org.apache.hugegraph.HugeFactory guard=miss, the sed would have rewritten it gremlin.graph= org.apache.hugegraph.HugeFactory guard=miss, the sed would have rewritten it gremlin.graph=org.apache.hugegraph.HugeFactory guard=miss

End to end at this head with gremlin.graph = org.apache.hugegraph.HugeFactory in a mounted hugegraph.properties: REST and the yaml both come out on StandardAuthenticator while gremlin.graph stays on the unwrapped HugeFactory. java.util.Properties reads that spelling identically to the unspaced one. The guard shape came from the CRLF thread on the previous head, and the auth.authenticator and auth.graph_store guards above were widened to [:=], bare whitespace and escaped keys in that same round; this one was not.

Requested change: widen the guard and the sed together, since relaxing only the guard just lets the sed no-op. Verified against all four spellings above plus :, bare-whitespace and escaped-key forms, and it still skips an already-proxied line, a commented line, and preserves a trailing CR:

if grep -Eq "^[[:blank:]]*gremlin[\\\\]?\\.graph[[:blank:]]*([:=]|[[:blank:]])[[:blank:]]*org\\.apache\\.hugegraph\\.HugeFactory[[:blank:]]*${CR}?\$" "${CONF}/graphs/${GRAPH_CONF}"; then sed -i -E "s#^([[:blank:]]*gremlin[\\\\]?\\.graph[[:blank:]]*([:=]|[[:blank:]])[[:blank:]]*)org\\.apache\\.hugegraph\\.HugeFactory#\\1org.apache.hugegraph.auth.HugeFactoryAuthProxy#" "${CONF}/graphs/${GRAPH_CONF}" fi

Fixed in 5f50511. Guard and sed widened together: leading blanks, escaped dot, colon/equals/bare-whitespace separators, optional CR via the $CR byte variable. Verified against 8 spellings plus CRLF (all rewrite), already-proxied and commented lines (both skipped), CR preserved by the prefix-only sed. Suite passes.

bitflicker64 · 2026-09-12T07:01:14Z

+            "without a readable authenticator; leaving both sides untouched"
+        return
+    fi
+    if [[ -n "${rest_auth}" && -n "${yaml_auth}" && "${rest_auth}" != "${yaml_auth}" ]]; then


🧹 These two values are decoded differently, so identical configurations can read as a mismatch. props.awk returns the on-disk escaped form on purpose ("The value of get is intentionally not unescaped"), while get_yaml_authenticator's scalar() returns what snakeyaml decodes:

rest-server.properties: auth.authenticator=org.apache.hugegraph.auth\.StandardAuthenticator gremlin-server.yaml: authenticator: org.apache.hugegraph.auth.StandardAuthenticator WARN: REST and Gremlin name different authenticators ('org.apache.hugegraph.auth\.StandardAuthenticator' vs 'org.apache.hugegraph.auth.StandardAuthenticator'); leaving both untouched

java.util.Properties reads both spellings as the same class, so the WARN is spurious and the alignment this function exists for is skipped.

Line 173 is the mirror of it: set_prop_encoded "auth.authenticator" "${yaml_auth}" hands a snakeyaml-decoded scalar to the setter that skips encode_prop_value. Harmless for a bare class name, wrong for any scalar carrying a backslash or leading space.

Requested change: unescape before comparing and before the export below, either through a get_decoded mode in props.awk (keeping the raw mode for the secret round trip that needs it) or an unescape here, and use set_prop rather than set_prop_encoded on line 173.

Fixed in 5f50511. align_auth_config now reads the authenticator via a get-decoded mode (new get_prop wrapper) that unescapes like java.util.Properties before comparing with the snakeyaml scalar, so an escaped properties value and a plain yaml scalar no longer WARN; the yaml-to-properties write on line 173 goes through the encoding setter set_prop. Raw mode kept for the token-secret round trip. Covered by an escaped-authenticator regression case; suite passes.

Address review 5185689081 on the auth bootstrap alignment: - props.awk: strip one trailing CR while assembling logical lines so CRLF configs parse like java.util.Properties, without touching the RAW bytes replayed on rewrite; add get-decoded mode. - props.awk: die when getline fails and rewrite atomically through a sibling temp file renamed over the original. - enable-auth.sh: widen the gremlin.graph guard and flip together for colon, equals, bare-whitespace, leading-blank and escaped-dot spellings with optional CR, still skipping proxied/commented lines. - docker-entrypoint.sh: compare the unescaped authenticator with the yaml scalar and write the yaml side through the encoding setter. Add CRLF plus escaped-authenticator regression cases to test-docker-entrypoint.sh.

Adarsh-Me · 2026-09-12T09:08:40Z

All 4 inline threads of review 5185689081 addressed in 5f50511 (branch fix-entrypoint-auth-bootstrap): (A) CRLF stripped while assembling logical lines, RAW replay untouched + regression test; (B) die on getline -1 plus atomic sibling-temp rewrite with quoted rename; (C) gremlin.graph guard and sed widened together (colon/equals/bare-whitespace, leading blank, escaped dot, optional CR), proxied/commented still skipped, CR preserved; (D) decoded authenticator comparison via new get-decoded mode with encoding setter on write. Verification: bash test-docker-entrypoint.sh passes (exit 0), bash -n clean on all three shell files, guard/sed exercised against 8 spellings + CRLF/proxied/commented cases. shellcheck not installed on this host, so that step was skipped. No Docker daemon here, so image build/mawk paths remain for CI (docker-build-ci.yml).

This was referenced Sep 3, 2026

[Bug] Docker entrypoint auth bootstrap is unsafe for mounted and upgraded configs #3133

Open

feat(helm): add HStore deployment chart #3132

Draft

bitflicker64 reviewed Sep 3, 2026

View reviewed changes

imbajin reviewed Sep 3, 2026

View reviewed changes

Comment thread hugegraph-server/hugegraph-dist/docker/props.awk

bitflicker64 reviewed Sep 12, 2026

View reviewed changes

Uh oh!

Conversation

Adarsh-Me commented Sep 3, 2026

What is changing

Implementation notes

How was this tested

Code Review Handbook

Uh oh!

bitflicker64 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

imbajin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov Bot commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

bitflicker64 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Adarsh-Me commented Sep 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov Bot commented Sep 4, 2026 •

edited

Loading