Skip to content

Fix control-plane join failing with empty certificate key#57

Merged
alicefr merged 2 commits into
mainfrom
investigating-ha-proxy-failure
Jun 9, 2026
Merged

Fix control-plane join failing with empty certificate key#57
alicefr merged 2 commits into
mainfrom
investigating-ha-proxy-failure

Conversation

@alicefr

@alicefr alicefr commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

The upload-certs command was piped through 2>/dev/null | tail -1, which suppressed errors and masked the exit code. When kubeadm failed, we got an empty string instead of an error.

Remove the pipe and stderr suppression so failures propagate properly. Extract the certificate key by matching the 64-char hex string from the output instead of blindly taking the last line.

Summary by Sourcery

Ensure control-plane join correctly obtains the kubeadm certificate key by using the full upload-certs output and parsing the hex key instead of relying on a piped tail command.

Bug Fixes:

  • Fix control-plane joins failing due to empty certificate keys when kubeadm upload-certs reports an error that was previously suppressed.

Enhancements:

  • Improve robustness of certificate key extraction by scanning upload-certs output for a 64-character hexadecimal key and surfacing detailed errors when not found.

The upload-certs command was piped through `2>/dev/null | tail -1`,
which suppressed errors and masked the exit code. When kubeadm failed,
we got an empty string instead of an error.

Remove the pipe and stderr suppression so failures propagate properly.
Extract the certificate key by matching the 64-char hex string from
the output instead of blindly taking the last line.

Assisted-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Jun 9, 2026

Copy link
Copy Markdown

Reviewer's Guide

Adjusts the control-plane join flow to preserve kubeadm error propagation and robustly extract the certificate key from upload-certs output, avoiding failures due to empty keys.

Sequence diagram for updated control-plane join certificate key extraction

sequenceDiagram
    participant Cluster
    participant cpSSHClient
    participant kubeadm

    Cluster->>Cluster: generateJoinCommand(ctx, cpSSHClient, true)
    Cluster->>cpSSHClient: Exec(ctx, sudo kubeadm init phase upload-certs --upload-certs)
    cpSSHClient->>kubeadm: run upload-certs
    kubeadm-->>cpSSHClient: certKeyOutput (logs on stderr, key on stdout)
    cpSSHClient-->>Cluster: certKeyOutput

    loop for each line in certKeyOutput
        Cluster->>Cluster: strings.TrimSpace(line)
        alt len(line) == 64 and isHex(line)
            Cluster->>Cluster: set certificateKey
        end
    end

    alt certificateKey == ""
        Cluster->>Cluster: fmt.Errorf(certificate key not found in upload-certs output)
        Cluster-->>Cluster: generateJoinCommand returns error
    else certificateKey != ""
        Cluster->>Cluster: logger.Infof(Certificate key: %s, certificateKey)
        Cluster-->>Cluster: generateJoinCommand returns join command
    end
Loading

File-Level Changes

Change Details Files
Stop suppressing kubeadm upload-certs errors and parsing the certificate key via a fragile tail-based pipeline; instead, run the command directly and parse a 64-character hex key from its output.
  • Remove the shell pipeline `2>/dev/null
tail -1from the kubeadm upload-certs invocation so stderr is not discarded and the exit code is preserved.</li><li>Capture the full stdout ofkubeadm init phase upload-certs --upload-certs` and scan each line for a 64-character hex string, treating that as the certificate key.
  • Improve error handling by failing with a descriptive error message that includes the full upload-certs output when no valid certificate key is found.
  • Introduce a small utility to validate hexadecimal strings used when extracting the certificate key.
    • Add an isHex helper function that returns true only if a string consists solely of 0–9, a–f, or A–F characters.
    • Use the isHex helper when scanning upload-certs output to avoid misinterpreting unrelated log lines as the certificate key.
    internal/cluster/join.go

    Tips and commands

    Interacting with Sourcery

    • Trigger a new review: Comment @sourcery-ai review on the pull request.
    • Continue discussions: Reply directly to Sourcery's review comments.
    • Generate a GitHub issue from a review comment: Ask Sourcery to create an
      issue from a review comment by replying to it. You can also reply to a
      review comment with @sourcery-ai issue to create an issue from it.
    • Generate a pull request title: Write @sourcery-ai anywhere in the pull
      request title to generate a title at any time. You can also comment
      @sourcery-ai title on the pull request to (re-)generate the title at any time.
    • Generate a pull request summary: Write @sourcery-ai summary anywhere in
      the pull request body to generate a PR summary at any time exactly where you
      want it. You can also comment @sourcery-ai summary on the pull request to
      (re-)generate the summary at any time.
    • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
      request to (re-)generate the reviewer's guide at any time.
    • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
      pull request to resolve all Sourcery comments. Useful if you've already
      addressed all the comments and don't want to see them anymore.
    • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
      request to dismiss all existing Sourcery reviews. Especially useful if you
      want to start fresh with a new review - don't forget to comment
      @sourcery-ai review to trigger a new review!

    Customizing Your Experience

    Access your dashboard to:

    • Enable or disable review features such as the Sourcery-generated pull request
      summary, the reviewer's guide, and others.
    • Change the review language.
    • Add, remove or edit custom review instructions.
    • Adjust other review settings.

    Getting Help

    @sourcery-ai sourcery-ai Bot left a comment

    Copy link
    Copy Markdown

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Hey - I've found 1 issue, and left some high level feedback:

    • When scanning for the certificate key, consider breaking out of the loop as soon as a valid 64-char hex string is found instead of overwriting certificateKey on each match, to avoid unintentionally preferring a later hex-looking token if kubeadm ever prints more than one.
    Prompt for AI Agents
    Please address the comments from this code review:
    
    ## Overall Comments
    - When scanning for the certificate key, consider breaking out of the loop as soon as a valid 64-char hex string is found instead of overwriting `certificateKey` on each match, to avoid unintentionally preferring a later hex-looking token if kubeadm ever prints more than one.
    
    ## Individual Comments
    
    ### Comment 1
    <location path="internal/cluster/join.go" line_range="167-168" />
    <code_context>
    +				certificateKey = line
    +			}
    +		}
     		if certificateKey == "" {
    -			return "", fmt.Errorf("certificate key is empty")
    +			return "", fmt.Errorf("certificate key not found in upload-certs output: %s", certKeyOutput)
     		}
    
    </code_context>
    <issue_to_address>
    **🚨 issue (security):** Avoid returning the full upload-certs output (including the key) in the error message
    
    Including `certKeyOutput` here will log and propagate the certificate key, which is sensitive and should never be exposed. Instead, either omit the raw output from the error or redact the key (e.g., log only a short, non-sensitive snippet or use a structured message without the full value).
    </issue_to_address>

    Sourcery is free for open source - if you like our reviews please consider sharing them ✨
    Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

    Comment thread internal/cluster/join.go
    Comment on lines 167 to +168
    if certificateKey == "" {
    return "", fmt.Errorf("certificate key is empty")
    return "", fmt.Errorf("certificate key not found in upload-certs output: %s", certKeyOutput)

    Copy link
    Copy Markdown

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    🚨 issue (security): Avoid returning the full upload-certs output (including the key) in the error message

    Including certKeyOutput here will log and propagate the certificate key, which is sensitive and should never be exposed. Instead, either omit the raw output from the error or redact the key (e.g., log only a short, non-sensitive snippet or use a structured message without the full value).

    Without --config, kubeadm upload-certs auto-discovers the API server
    and connects to the container's podman bridge IP instead of the VM
    cluster IP, causing a certificate validation error (x509: certificate
    is valid for 10.0.0.x, not 10.89.x.x).
    
    Pass --config /etc/kubernetes/kubeadm-config.yaml so kubeadm uses the
    correct controlPlaneEndpoint from the init config.
    
    Assisted-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
    @alicefr alicefr merged commit de1c8d9 into main Jun 9, 2026
    6 checks passed
    @alicefr alicefr deleted the investigating-ha-proxy-failure branch June 11, 2026 08:35
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Labels

    None yet

    Projects

    None yet

    Development

    Successfully merging this pull request may close these issues.

    1 participant