ELK快速搭建教程

一、 Elasticsearch安裝

環(huán)境準(zhǔn)備
系統(tǒng)： Centos7
防火墻： 關(guān)閉
Sellinux: 關(guān)閉
JDK 1.8（備注至少是1.8版本的JDK）


image.png

1、下載

https://www.elastic.co/downloads/elasticsearch

image.png

[elk@localhost elasticsearch]$ pwd
/usr/local/src/elasticsearch
[elk@localhost elasticsearch]$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.4.2.tar.gz
--2018-10-05 23:26:43-- https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.4.2.tar.gz
Resolving artifacts.elastic.co (artifacts.elastic.co)... 54.225.221.128, 54.225.214.74, 184.73.245.233, ...
Connecting to artifacts.elastic.co (artifacts.elastic.co)|54.225.221.128|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 97914519 (93M) [application/x-gzip]
Saving to: ‘elasticsearch-6.4.2.tar.gz’
100%[====================================================================================================================================================================================================================================>] 97,914,519 2.55MB/s in 24s   
2018-10-05 23:27:09 (3.84 MB/s) - ‘elasticsearch-6.4.2.tar.gz’ saved [97914519/97914519]
[elk@localhost elasticsearch]$ ll
total 95620
-rw-rw-r--. 1 elk elk 97914519 Oct 2 06:58 elasticsearch-6.4.2.tar.gz

2、解壓

[elk@localhost elasticsearch]$ tar -zxvf elasticsearch-6.4.2.tar.gz

3、配置elasticsearch.yml配置文件

在config目錄下的elasticsearch.yml文件中你可以設(shè)置es服務(wù)的端口號以及網(wǎng)絡(luò)ip等


image.png

4、啟動es

切換到bin目錄下

[elk@localhost bin]$ ./elasticsearch

啟動的過程可能有點緩慢，請耐心等待
啟動報錯：
問題一：max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
解決方案：
切換到root用戶
編輯 /etc/sysctl.conf，追加以下內(nèi)容：
vm.max_map_count=655360
保存后，執(zhí)行：
sysctl -p

問題二：ERROR: bootstrap checks failed
max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]
max number of threads [1024] for user [lishang] likely too low, increase to at least [2048]
解決：切換到root用戶，編輯limits.conf 添加類似如下內(nèi)容
vi /etc/security/limits.conf
添加如下內(nèi)容:

* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096

問題三：max number of threads [1024] for user [lish] likely too low, increase to at least [2048]
解決：切換到root用戶，進入limits.d目錄下修改配置文件。
vi /etc/security/limits.d/90-nproc.conf
修改如下內(nèi)容：

* soft nproc 1024

修改為

* soft nproc 2048

重新啟動，成功。


image.png

5、驗證es是否啟動成功

[elk@localhost ~]$ curl localhost:9200
{
    "name": "bWZVh7O",
    "cluster_name": "elasticsearch",
    "cluster_uuid": "9Rvwbh7bQs6SsRALTwefNQ",
    "version": {
        "number": "6.4.2",
        "build_flavor": "default",
        "build_type": "tar",
        "build_hash": "04711c2",
        "build_date": "2018-09-26T13:34:09.098244Z",
        "build_snapshot": false,
        "lucene_version": "7.4.0",
        "minimum_wire_compatibility_version": "5.6.0",
        "minimum_index_compatibility_version": "5.0.0"
    },
    "tagline": "You Know, for Search"
}

或者在瀏覽器輸入：


image.png

二、 Logstash的安裝

1、下載

[elk@localhost elasticsearch]$ wget https://artifacts.elastic.co/downloads/logstash/logstash-6.4.2.tar.gz
--2018-10-05 23:41:42-- https://artifacts.elastic.co/downloads/logstash/logstash-6.4.2.tar.gz
Resolving artifacts.elastic.co (artifacts.elastic.co)... 54.235.82.130, 107.21.237.188, 23.21.67.46, ...
Connecting to artifacts.elastic.co (artifacts.elastic.co)|54.235.82.130|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 153924169 (147M) [application/x-gzip]
Saving to: ‘logstash-6.4.2.tar.gz’
100%[====================================================================================================================================================================================================================================>] 153,924,169 2.27MB/s in 56s   
2018-10-05 23:42:40 (2.62 MB/s) - ‘logstash-6.4.2.tar.gz’ saved [153924169/153924169]
[elk@localhost elasticsearch]$ ll
total 245940
drwxr-xr-x. 9 elk elk 155 Oct 5 23:33 elasticsearch-6.4.2
-rw-rw-r--. 1 elk elk 97914519 Oct 2 06:58 elasticsearch-6.4.2.tar.gz
-rw-rw-r--. 1 elk elk 153924169 Oct 2 06:59 logstash-6.4.2.tar.gz
[elk@localhost elasticsearch]$

2、解壓

[elk@localhost elasticsearch]$ tar -zxvf logstash-6.4.2.tar.gz

3、配置文件

在config目錄下的logstash.yml我這里也都采用的是默認(rèn)的,大家可以根據(jù)自己的需求自行設(shè)置

[elk@localhost config]$ vim logstash.yml

4、設(shè)置tomcat的serer.xml

[elk@localhost conf]$ pwd
/usr/local/src/tomcat/apache-tomcat-8.5.34/conf
[elk@localhost conf]$ vim server.xml

將最后的pattern后面的參數(shù)更改如下

pattern="%h %l %u %t "%r" %s %b %D "%{Referer}i" "%{User-Agent}i""
image.png

5、創(chuàng)建root-tomcat.conf文件

[elk@localhost logstash-6.4.2]$ pwd
/usr/local/src/elasticsearch/logstash-6.4.2
[elk@localhost logstash-6.4.2]$ vim root-tomcat.conf

新增內(nèi)容

input {
    file {  
        path => "/usr/local/src/tomcat/apache-tomcat-8.5.34/logs/localhost_access_log*.txt"  
        sincedb_path => "/usr/local/src/elasticsearch/logstash-6.4.2/config/sincedb_apache_access_log.txt"
        type => "apache_access_log"
        add_field => {"tomcatip" => "192.168.1.23"}
    }
}
filter{
    if [type] == "apache_access_log" {
        grok{
            match => { "message" => "%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] \"(?:%{WORD:verb} %{URIPATHPARAM:request}(?: HTTP/%{NUMBER:httpversion})?|-)\" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{NUMBER:responsetime} \"(?:%{URI:referrer}|-)\" %{QS:agent}" }
        }
        ruby {
            code => "event.timestamp.time.localtime"
        }
        date{
            match => [ "timestamp", "dd/MM/yyyy:HH:mm:ss Z" ]
            target => ["writetime"]
        }
        mutate {
            convert => { 
                "response" => "integer" 
                "bytes" => "integer" 
                "responsetime" => "integer" 
            }
        }
    }
    
}
output {
    if [type] == "apache_access_log" {
        elasticsearch {
            hosts => ["192.168.1.23:9200"]
            index => "logstash-apacheaccesslog-%{+YYYY.MM.dd}"
        }
    }
}

6、啟動logstash

[elk@localhost logstash-6.4.2]$ ./bin/logstash -f root-tomcat.conf
image.png

三、 安裝kibana

1、下載

[elk@localhost elasticsearch]$ wget https://artifacts.elastic.co/downloads/kibana/kibana-6.4.2-linux-x86_64.tar.gz
--2018-10-06 00:21:55-- https://artifacts.elastic.co/downloads/kibana/kibana-6.4.2-linux-x86_64.tar.gz
Resolving artifacts.elastic.co (artifacts.elastic.co)... 54.225.221.128, 107.21.237.188, 107.21.202.15, ...
Connecting to artifacts.elastic.co (artifacts.elastic.co)|54.225.221.128|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 188077286 (179M) [application/x-gzip]
Saving to: ‘kibana-6.4.2-linux-x86_64.tar.gz’
100%[====================================================================================================================================================================================================================================>] 188,077,286 4.75MB/s in 41s   
2018-10-06 00:22:39 (4.34 MB/s) - ‘kibana-6.4.2-linux-x86_64.tar.gz’ saved [188077286/188077286]
[elk@localhost elasticsearch]$ ll
total 429616
drwxr-xr-x. 9 elk elk 155 Oct 5 23:33 elasticsearch-6.4.2
-rw-rw-r--. 1 elk elk 97914519 Oct 2 06:58 elasticsearch-6.4.2.tar.gz
-rw-rw-r--. 1 elk elk 188077286 Oct 2 06:58 kibana-6.4.2-linux-x86_64.tar.gz
drwxrwxr-x. 13 elk elk 4096 Oct 6 00:15 logstash-6.4.2
-rw-rw-r--. 1 elk elk 153924169 Oct 2 06:59 logstash-6.4.2.tar.gz

2、解壓

[elk@localhost elasticsearch]$ tar -zxvf kibana-6.4.2-linux-x86_64.tar.gz

3、修改配置文件

[elk@localhost kibana-6.4.2-linux-x86_64]$ cd config/
[elk@localhost config]$ vim kibana.yml
image.png

4、啟動

[elk@localhost kibana-6.4.2-linux-x86_64]$ ./bin/kibana

通過提示，無任何錯誤表示啟動完成。此時通過訪問http://192.168.1.23:5601即可；
在訪問這個頁面之前別忘記啟動tomcat

四、 成功的喜悅

如下圖，大家可以開始elk之旅了


image.png
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時請結(jié)合常識與多方信息審慎甄別。
平臺聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點，簡書系信息發(fā)布平臺，僅提供信息存儲服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容