OpenSearch Operator

Description

The Charmed OpenSearch Operator deploys and operates the OpenSearch software on both machine (VMs/LXD) and Kubernetes clusters.

This operator provides an OpenSearch cluster, with:

• TLS (for the HTTP and Transport layers)
• Automated node discovery
• Observability
• Backup / Restore
• Safe horizontal scale-down/up
• Large deployments

This repository contains two charms, each in its own folder:

• machine/ — the machine charm (opensearch), which installs and manages OpenSearch from the OpenSearch Snap on VMs and machine clusters.
• kubernetes/ — the Kubernetes charm (opensearch-k8s), which deploys and manages OpenSearch as a container workload on Kubernetes.

Both charms are Python projects that provide lifecycle management and handle events (install, start, etc).

Usage

Machine charm (opensearch)

Bootstrap a lxd controller to juju and create a model:

juju add-model opensearch

Configure the system settings required by OpenSearch, we'll do that by creating and setting a cloudinit-userdata.yaml file on the model. As well as setting some kernel settings on the host machine.

On the host machine, create a sysctl configuration file and apply it:

sudo tee /etc/sysctl.d/opensearch.conf <<EOF
vm.swappiness = 0
vm.max_map_count = 262144
EOF

sudo sysctl -p /etc/sysctl.d/opensearch.conf

Create a cloud-init user-data file and set it on the model:

cat <<EOF > cloudinit-userdata.yaml
cloudinit-userdata: |
  postruncmd:
    - echo 'vm.max_map_count=262144' >> /etc/sysctl.conf
    - echo 'vm.swappiness=0' >> /etc/sysctl.conf
    - echo 'fs.file-max=1048576' >> /etc/sysctl.conf
    - sysctl -p
EOF

juju model-config --file=./cloudinit-userdata.yaml

To deploy a single unit of OpenSearch using its default configuration:

juju deploy opensearch --channel=2/edge

Kubernetes charm (opensearch-k8s)

Bootstrap a Kubernetes controller to juju and create a model:

juju add-model opensearch

OpenSearch benefits from certain kernel settings being applied on the underlying Kubernetes hosts (the nodes where the workload pods are scheduled). These are not required, but are recommended for production setups. Apply the following on each node:

sudo tee -a /etc/sysctl.conf > /dev/null <<EOT
vm.max_map_count=262144
vm.swappiness=0
fs.file-max=1048576
EOT

sudo sysctl -p

Note that net.ipv4.tcp_retries2 is scoped to the pod's network namespace rather than the host, so it cannot be set on the node. To configure it, deploy the data-platform-k8s-mutator admission webhook before OpenSearch, which sets this sysctl value on newly created workload pods. It is optional but recommended for production setups; see the Kubernetes deployment guide for the setup steps.

To deploy a single unit of OpenSearch using its default configuration:

juju deploy opensearch-k8s opensearch --channel=2/edge --trust

Relations / Integrations

The relevant provided relations of Charmed OpenSearch are shown below. Both deployment examples name the Juju application opensearch; on Kubernetes, the charm itself is named opensearch-k8s.

Client interface

To connect to the Charmed OpenSearch Operator and exchange data, relate to the opensearch-client endpoint:

juju deploy data-integrator --channel=latest/stable
juju integrate opensearch data-integrator

Large deployments

Charmed OpenSearch also allows to form large clusters or join an existing deployment, through the relations:

• peer-cluster
• peer-cluster-orchestrator

juju integrate main:peer-cluster-orchestrator data-hot:peer-cluster

TLS

The Charmed OpenSearch Operator also supports TLS encryption as a first class citizen, on both the HTTP and Transport layers. TLS is enabled by default and is a requirement for the charm to start.

The charm relies on the tls-certificates interface.

Example with Self-signed certificates

# Deploy the self-signed TLS Certificates Operator.
juju deploy self-signed-certificates --channel=latest/stable

# Add the necessary configurations for TLS.
juju config \
    self-signed-certificates \
    ca-common-name="Test CA" \
    certificate-validity=365 \
    root-ca-validity=365
    
# Enable TLS via relation.
juju integrate self-signed-certificates opensearch

# Disable TLS by removing relation.
juju remove-relation opensearch self-signed-certificates

Note: The TLS settings shown here are for self-signed-certificates, which are not recommended for production clusters. The Self Signed Certificates Operator offers a variety of configuration options. Read more on the TLS Certificates Operator here.

Security

For the security features overview, see the security hardening guide in the Charmed OpenSearch documentation.

Security issues in the Charmed OpenSearch Operator can be reported through LaunchPad. Please do not file GitHub issues about security issues. See SECURITY.md for the full security policy.

Contributing

The documentation for Charmed OpenSearch is available at canonical.com/data/opensearch/docs.

See the contributor's guide in the documentation for how to report issues, build and test the charm, and contribute code and documentation.

Format documentation Markdown

The documentation sources in docs/ are formatted with mdformat (with the MyST plugin). The formatter runs in its own virtual environment, installed from a hash-pinned lock file, so no global packages are needed.

cd docs
make mdformat        # format all tracked documentation Markdown files in place
make mdformat-check  # report unformatted files without changing them

To format automatically on each commit, install the pre-commit hook from the repository root:

python3 -m pip install pre-commit
pre-commit install

License

The Charmed OpenSearch Operator is free software, distributed under the Apache Software License, version 2.0. See LICENSE for more information.