Elastic Search, Logstash and Kibana
Overview
Authors: Samuel Ambroj Pérez, Kajorn Pathomkeerati, [1] [2]
Introduction to Elasticsearch and Logstash
Introduction to Kibana
Installation of ELK in one single machine (Debian 8)
Installation of Elasticsearch (Debian 8)
Connect to the first machine (on the left side) that has been provided to you:
ssh gks@141.52.X.X
This machine is going to be used for a while, so do not connect yet to the second VM.
The gks user has sudo rights because is included in the sudoers file, so from your gks user, execute:
sudo -i bash
Update and upgrade all the packages:
# apt-get update # apt-get upgrade
Change the timezone (optional):
# dpkg-reconfigure tzdata
Installation of aptitude, curl, openjdk (open source java) and chkconfig:
# apt install -y aptitude curl openjdk-7-jdk chkconfig
Download and install the Public Signing Key:
# wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
Save the repository definition:
# echo "deb http://packages.elastic.co/elasticsearch/1.7/debian stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-1.7.list
Update in order to make the repository ready to use and install Elasticsearch:
# aptitude update && aptitude install elasticsearch
Start Elasticsearch:
# /etc/init.d/elasticsearch start
Check the status (two options):
# systemctl status elasticsearch # /etc/init.d/elasticsearch status
Modify chkconfig in order to start ES when booting:
# chkconfig elasticsearch on
Elasticsearch is now installed. Congratulations! Let's continue with the installation of Logstash and finally Kibana.
Installation of Logstash (Debian 8)
Download the .deb file:
# wget -q https://download.elastic.co/logstash/logstash/packages/debian/logstash_1.5.4-1_all.deb
Install it:
# dpkg -i logstash_1.5.4-1_all.deb
It would be enough, but when preparing the tutorial we saw the following WARNING:
WARN -- Concurrent: [DEPRECATED] Java 7 is deprecated, please use Java 8. Java 7 support is only best effort, it may not work. It will be removed in next release (1.0).
So, we install the Oracle java, version 8.
Installation of Oracle Java 8
Download the tarball:
# wget --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u60-b27/jdk-8u60-linux-x64.tar.gz
Create the folder /opt/jdk and move the tarball there:
# mkdir /opt/jdk # mv jdk-8u60-linux-x64.tar.gz /opt/jdk/
Extract the tarball there:
# cd /opt/jdk # tar -xzvf jdk-8u60-linux-x64.tar.gz
Update alternatives:
# update-alternatives --install /usr/bin/java java /opt/jdk/jdk1.8.0_60/bin/java 100 # update-alternatives --install /usr/bin/javac javac /opt/jdk/jdk1.8.0_60/bin/javac 100
Display the priorities and the version of Java:
# update-alternatives --display java # java -version
java version "1.7.0_79" OpenJDK Runtime Environment (IcedTea 2.5.6) (7u79-2.5.6-1~deb8u1) OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode)
It is not pointing to the Oracle version, so we increase the value from 100 to 10000:
# update-alternatives --install /usr/bin/java java /opt/jdk/jdk1.8.0_60/bin/java 10000 # update-alternatives --install /usr/bin/javac javac /opt/jdk/jdk1.8.0_60/bin/javac 10000 # java -version
java version "1.8.0_60" Java(TM) SE Runtime Environment (build 1.8.0_60-b27) Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
Installation of Kibana 4
Download Kibana4:
# wget https://download.elastic.co/kibana/kibana/kibana-4.1.1-linux-x64.tar.gz
Extract the file:
# tar xvf kibana-4.1.1-linux-x64.tar.gz
Move it and change to a shorter name:
# mv kibana-4.1.1-linux-x64/ /opt/ # mv /opt/kibana-4.1.1-linux-x64/ /opt/kibana4
Launch Kibana:
# cd /opt/kibana4/ # ./bin/kibana > /dev/null &
Check that Kibana is working in a browser:
http://141.52.X.X:5601
The access is not secured. In order to make it more secure we are going to install a reverse nginx proxy in the next section.