Introduction
The ELK (Elasticsearch, Logstash, Kibana) stack, also known as the Elastic stack, runs on various setups and operating systems. A simple way to try out, install and test the ELK stack is to run it on Docker.
This tutorial outlines two ways to install the ELK stack on Docker.
Prerequisites
- Docker installed and configured.
- Docker Compose installed and configured (recommended).
- Git installed to clone the ELK Docker repository.
- Access to a web browser to view the Kibana dashboard.
- Access to the internet to pull plugins.
- Command-line interface/terminal access with sudo privileges.
Installing ELK on Docker
There are two ways to install ELK on Docker:
1. Pull an automatically built image from the Docker registry.
2. Build the image from source files.
Before starting either method, the ELK stack requires a change on the mmap counts to not run out of virtual memory during installation and use.
To change the value on Linux, run the following:
sudo sysctl -w vm.max_map_count=262144
The mmap counts help store and access index data. If not set, the installation comes to a halt due to the lower default OS value.
Pulling the Image
To pull the ELK image from the Docker registry, open the terminal and run:
sudo docker pull sebp/elk
Use tags to specify a specific version of Elasticsearch, Kibana, and Logtash:
sudo docker pull sebp/elk:<tag>
The command pulls the latest version of the ELK stack using the default latest
tag. The complete list is available at Docker Hub.
Building the Image from Source Files
There are two ways to build the image from a source file. Use either vanilla Docker or Docker compose. In both cases, clone the Git repository and enter the directory:
git clone https://github.com/spujadas/elk-docker.git
cd elk-docker
Use either docker build
or docker-compose
to build the image.
Using Docker Build
To build the Docker image with the docker build
command, run:
sudo docker build -t elk-docker .
This option does not require Docker compose. The better option is to use Docker compose to ensure an isolated and functional environment.
Using Docker Compose
The Git repository comes with the YAML configuration file for setup with Docker compose.
1. Open the docker-compose.yml file in a text editor, such as Nano:
sudo nano docker-compose.yml
2. Delete everything from the compose file and add the following:
services:
elk:
build: .
ports:
- "5601:5601"
- "9200:9200"
- "5044:5044"
The ports help run the container at a later step.
Press CTRL+X, then Y, and Enter to save the file and exit the editor.
3. Build the image with:
sudo docker-compose build elk
If using a different repository, exchange the repository/image name appropriately. Wait for the build to complete before continuing.
Installing Plugins
ELK provides various plugins to enrich the system with additional features and libraries. When running ELK on Docker, use the Dockerfile to install plugins and build the image to run the installation.
Open the Dockerfile located in the repository directory:
sudo nano Dockerfile
Below are examples for installing Elasticsearch, Logstash, and Kibana plugins through Docker. The plugin install process is similar for all three.
Elasticsearch
To install an Elasticsearch plugin, do the following:
1. Add the following at the end of the Dockerfile:
FROM sebp/elk
ENV ES_HOME /opt/elasticsearch
WORKDIR ${ES_HOME}
RUN yes | CONF_DIR=/etc/elasticsearch gosu elasticsearch bin/elasticsearch-plugin
install -b <plugin name or link>
Exchange <plugin name or link>
for the plugin of your choice.
The code sets the working directory, configuration directory location, and searches for the plugin.
The install script located in bin/elasticsearch-plugin runs the installation. When installing Elasticsearch plugins, the command requires the elasticsearch
user while gosu
elevates the privileges.
2. Save the Dockerfile and close the editor.
3. Build the image using either docker build
or docker-compose
.
The image build runs the Dockerfile commands and executes the installation.
Logstash
Follow the steps below to install Logstash plugins.
1. Add the following code to the Dockerfile:
FROM sebp/elk
WORKDIR ${LOGSTASH_HOME}
RUN gosu logstash bin/logstash-plugin install <plugin name>
The Logstash plugins do not require a configuration directory. Install directly using the install script located in bin/logstash-plugin and replace <plugin name or link>
with the plugin’s accurate information.
2. Save the contents and close the Dockerfile.
3. Run the build to install the plugin.
The installation information and resulting output show up at the end of the build log.
Kibana
To install Kibana plugins, do the following:
1. Insert the following code at the end of the Dockerfile:
FROM sebp/elk
WORKDIR ${KIBANA_HOME}
RUN gosu kibana bin/kibana-plugin install <plugin name or link>
The process is the same as with Logstash plugins. Exchange <plugin name or link>
for an actual plugin name or link.
2. Save the file and close.
3. Build the Docker image and check the output for the installation results.
The command automatically searches for the plugin and installs it with the kibana-plugin script.
Running the ELK Container
There are two ways to run the ELK container:
1. From the image through the docker run
command.
2. Using Docker compose.
Below are the commands and explanations for both cases.
From the Image via Command
To start the whole ELK stack container via docker run
, use the following:
sudo docker run -p 5601:5601 -p 9200:9200 -p 5044:5044 -it --name elk sebp/elk
The command publishes the following ports:
5601
serves the Kibana web interface.9200
for Elasticsearch JSON interface.5044
for Logstash Beats interface.
The three ports are necessary for the stack to work correctly. Additionally, the following ports are exposed but not published:
9300
for the transport interface of Elasticsearch (expose with-p 9300:9300
).9600
for the Logstash monitoring API (expose with-p 9600:9600
).
Access the Kibana web interface with:
http://<host>:5601
Replace <host>
with the hostname or IP of Docker’s host. If running a local version, use localhost.
Using Docker Compose
To run the ELK container with Docker compose, use:
sudo docker-compose up elk
The port mapping is in the docker-compose.yml file. Access Kibana from the web browser with:
http://<host>:5601
Use localhost if running Docker locally.
Running Individual Services
The following environment variables allow running ELK services individually instead of the whole stack:
ELASTICSEARCH_START
LOGSTASH_START
KIBANA_START
Setting the variables to anything other than one (1
) does not start the service.
For example, to start Elasticsearch without Logstash and Kibana, use:
sudo docker run -p 5601:5601 -p 9200:9200 -p 5044:5044 -it -e LOGSTASH_START=0 -e KIBANA_START=0 --name elk sebp/elk
Check that Elasticsearch is running with a curl request:
curl localhost:9200
The Kibana dashboard page (localhost:5601
) does not display because the service is not running.
Conclusion
After following the steps from this guide, you’ve installed and run the ELK stack on Docker!
Next, check out how to deploy Elasticsearch on Kubernetes.
Đăng ký liền tay Nhận Ngay Bài Mới
Subscribe ngay
Cám ơn bạn đã đăng ký !
Lỗi đăng ký !
Add Comment