Add Your Own SSL Certificates to Open Distro for Elasticsearch
Open Distro for Elasticsearch’s security plugin comes with authentication and access control out of the box. To make it easy to get started, the binary distributions contain passwords and SSL certificates that let you try out the plugin. Before adding any of your private data, you need to change the default passwords and certificates. In a prior post, we showed how you can change your admin password in Open Distro for Elasticsearch. In this post we cover changing your SSL certificates.
To change your SSL certificates, you’ll copy the certificate files into the distribution and modify your
elasticsearch.yml to use them. I’ll cover changing certificates for Elasticsearch’s node-to-node communication, REST APIs, and Kibana’s back-end communication to Elasticsearch. I’ll cover both the RPM and Docker distributions of Open Distro for Elasticsearch.
Before you can change the certificates, you’ll need to generate (or have) the following
.pem files for the certificate and key:
- Elasticsearch admin
- Elasticsearch node
- Kibana node
- Certificate authority
If you want to support SSL connections to Kibana, you need to add a certificate to Kibana as well. You can use the Elasticsearch node certificate and key files for Kibana, or use separate certificates.
There are many ways that you can create the CA and certificates. You might have a certificate authority (CA) that can issue certificates in your organization. If so, use that. If you don’t have access to your own CA, you can use the demo files that ship with Open Distro for Elasticsearch. Or you can use OpenSSL, create a CA, and then create and sign certificates with your CA. In this post, I describe copying the demo files and also creating a CA and certificates with OpenSSL.
First, make a directory to hold the various assets you’re building:
Using the demo
Download and install the Open Distro for Elasticsearch RPM, or run Open Distro for Elasticsearch in Docker (see Get Up and Running with Open Distro for Elasticsearch for instructions on how to run Docker locally). The demo
.pem files are located in different directories, depending on the distribution you’re running:
root-ca.pem to the
If you’re running Docker, use:
to cat the files to your machine. Replace
<container ID> with the ID from one of your Elasticsearch containers. Replace
filename2.pem with the above files.
If you’re running the RPM, you can simply
cp the files to the
Creating a New Certificate Authority (CA), Node, and Admin Certificates
If you want to create a CA and new certificates instead, you use OpenSSL to create a local, self-signed Certificate Authority (CA). You also create server and admin certificates. Then, use your CA to sign the certificates.
To install OpenSSL, run the below commands. You can find the latest version on the OpenSSL website:
First, create a private key for the CA:
$ openssl genrsa -out MyRootCA.key 2048 Generating RSA private key, 2048 bit long modulus ................+++ ...............................+++ e is 65537 (0x10001)
Create the CA and enter the Organization details:
$ openssl req -x509 -new -key MyRootCA.key -sha256 -out MyRootCA.pem You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ---- Country Name (2 letter code) [AU]:GB State or Province Name (full name) [Some-State]: Locality Name (eg, city) :London Organization Name (eg, company) [Internet Widgits Pty Ltd]:Example Corp Organizational Unit Name (eg, section) : Common Name (e.g. server FQDN or YOUR name) :Example Corp CA Root Email Address :
For the server and admin certificates, create keys, a certificate signing request (CSR) and
a certificate signed by the CA. In the below example, I walk through the commands
for one server — “odfe-node1”. You need to repeat this process for odfe-node2,
the admin certificate, and the kibana certificate:
IMPORTANT: Convert these to PKCS#5 v1.5 to work correctly with the JDK. Output from
this command will be used in all the config files.
Create the CSR and enter the organization and server details:
$ openssl req -new -key odfe-node1.key -out odfe-node1.csr You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ---- Country Name (2 letter code) [AU]:GB State or Province Name (full name) [Some-State]: Locality Name (eg, city) :London Organization Name (eg, company) [Internet Widgits Pty Ltd]:Example Corp Organizational Unit Name (eg, section) : Common Name (e.g. server FQDN or YOUR name) :odfe-node1.example.com Email Address : Please enter the following 'extra' attributes to be sent with your certificate request A challenge password : An optional company name :
Use the CSR to generate the signed Certificate:
$ openssl x509 -req -in odfe-node1.csr -CA MyRootCA.pem -CAkey MyRootCA.key -CAcreateserial -out odfe-node1.pem -sha256 Signature ok subject=/C=GB/ST=Some-State/L=London/O=Example Corp/CN=odfe-node1.example.com Getting CA Private Key
Edit elasticsearch.yml to Add Your Certificates
Now you need to use the certificates you created or copied to
setup-ssl. Whether you are running the .rpm distribution of Open Distro for Elasticsearch or the Docker distribution, you’ll edit
elasticsearch.yml to add the certificate information. This will enable Open Distro for Elasticsearch’s security plugin to accept SSL requests, as well as enable node-to-node SSL communication. Create a copy of
elasticsearch.yml in your
setup-ssl directory. You can find
elasticsearch.yml in the same directory as the
Open your local copy of
elasticsearch.yml with your favorite editor. You’ll see a block of settings that begins with:
######## Start OpenDistro for Elasticsearch Security Demo Configuration ######## # WARNING: revise all the lines below before you go into production opendistro_security.ssl.transport.pemcert_filepath: esnode.pem opendistro_security.ssl.transport.pemkey_filepath: esnode-key.pem opendistro_security.ssl.transport.pemtrustedcas_filepath: root-ca.pem ...
The opendistro_security.ssl.transport.* settings enable SSL transport between nodes. The opendistro_security.ssl.http.* enable SSL for REST requests to the cluster. You need to replace the values for these variables with your own certificate files.
Make sure to remove the entry:
to use your certificates instead of the demo certificates.
You can also further improve security by adding Distinguished Names (DN) verification settings as below. The Security plugin supports wildcards and regular expressions:
opendistro_security.nodes_dn: - 'CN=node2.example.com,OU=SSL,O=Example Corp,L=London,C=GB' - 'CN=*.example.com,OU=SSL,O=Example Corp,L=London,C=GB' - 'CN=odfe-cluster*'- '/CN=.*regex/'
If you are running the .rpm distribution, copy your certificates and elasticsearch.yml to the
/etc/elasticsearch/config directory. Change the file names to match the names of your certificate files.
For container deployments, override the files in the container with the your local files by modifying docker-compose.yml. Open this file in your editor and locate the volumes section for both the odfe-node1 and odfe-node2 services. Add additional lines to these sections that map your local files onto the container’s file system. When you’re done, it should look like this:
version: '3' services: odfe-node1: image: amazon/opendistro-for-elasticsearch:0.7.0 container_name: odfe-node1 environment: - cluster.name=odfe-cluster - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping - "ES_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM ulimits: memlock: soft: -1 hard: -1 volumes: - odfe-data1:/usr/share/elasticsearch/data - ./MyRootCA.pem:/usr/share/elasticsearch/config/MyRootCA.pem - ./odfe-node1.pem:/usr/share/elasticsearch/config/odfe-node1.pem - ./odfe-node1.key:/usr/share/elasticsearch/config/odfe-node1.key - ./node1-elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml ports: - 9200:9200 - 9600:9600 # required for Performance Analyzer networks: - odfe-net odfe-node2: image: amazon/opendistro-for-elasticsearch:0.7.0 container_name: odfe-node2 environment: - cluster.name=odfe-cluster - bootstrap.memory_lock=true - "ES_JAVA_OPTS=-Xms512m -Xmx512m" - discovery.zen.ping.unicast.hosts=odfe-node1 ulimits: memlock: soft: -1 hard: -1 volumes: - odfe-data2:/usr/share/elasticsearch/data - ./MyRootCA.pem:/usr/share/elasticsearch/config/MyRootCA.pem - ./odfe-node2.pem:/usr/share/elasticsearch/config/odfe-node2.pem - ./odfe-node2.key:/usr/share/elasticsearch/config/odfe-node2.key - ./node2-elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml networks: - odfe-net .....
Encrypting Access to Kibana
You enable TLS/SSL encryption between the browser and Kibana server by setting the below
server.ssl options in
kibana.yml. The location depends on the distribution you’re running:
server.ssl.enabled: true server.ssl.key: <full path to your key file> server.ssl.certificate: <full path to your certificate>
If you are running the .rpm distribution, copy your certificates to the
/etc/kibana/ directory and update ssl settings in
For container deployments, update the Kibana section in docker-compose.yml file by adding file mappings in
volumes section and SERVER_SSL options in the
environment section and save the file.
..... kibana: image: amazon/opendistro-for-elasticsearch-kibana:0.7.0 container_name: odfe-kibana ports: - 5601:5601 expose: - "5601" environment: ELASTICSEARCH_URL: https://odfe-node1:9200 SERVER_SSL_ENABLED: "true" SERVER_SSL_KEY: /usr/share/kibana/config/odfe-node2.key SERVER_SSL_CERTIFICATE: /usr/share/kibana/config/odfe-node2.pem volumes: - ./MyRootCA.pem:/usr/share/kibana/config/MyRootCA.pem - ./odfe-node2.pem:/usr/share/kibana/config/odfe-node2.pem - ./odfe-node2.key:/usr/share/kibana/config/odfe-node2.key networks: - odfe-net .....
Restart Your World
Now you need to restart Elasticsearch. In order to remove the demo certificates from the security plugin’s Elasticsearch index, you need to remove the existing volumes. From the directory that contains your docker-compose.yml, issue the following commands:
NOTE! the following commands will erase all data that you have in Elasticsearch!
docker-compose down -v docker-compose up
You should be able to browse to
https://<localhost or FQDN of kibana>:5601/. You might need to sign out of Kibana’s UI to remove any browser-cached certificates before you can log in.
To suppress security warnings in the browser, you can use its settings panel to add the self-signed MyRootCA certificate to your Trusted Certificate Authorities.
You have now made your Open Distro for Elasticsearch cluster even more secure by adding your own SSL certificates. Your certificates cover (optionally) communication from your browser to Kibana, communication to your Elasticsearch endpoint, and intra-cluster communication between nodes.
from AWS Open Source Blog