配置nginx-module-vts模塊
收集nginx指標(biāo)的較多采用的有nginx-vts模塊，prometheus-lua兩種方式進(jìn)行采集，本文采用nginx-vts模塊方式進(jìn)行數(shù)據(jù)收集。
nginx配置
查看是否安裝了nginx-vts模塊
$ /usr/local/nginx/sbin/nginx -V
nginx version: nginx/1.10.0
...
configure arguments: --prefix=/usr/local/nginx --user=nginx --group=nginx --with-stream --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module
安裝nginx-vts模塊
$ cd /root/
$ git clone https://github.com/vozlt/nginx-module-vts
$ cd /usr/local/src/nginx-1.10.0
$ ./configure --add-module=/root/nginx-module-vts --prefix=/usr/local/nginx --user=nginx --group=nginx --with-stream --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module
$ make
# 使用新編譯的nginx執(zhí)行文件替換掉舊的
$ mv /usr/local/nginx/sbin/nginx /usr/local/nginx/sbin/nginx.bak.20191211
$ cp objs/nginx /usr/local/nginx/sbin/
# 生成新的nginx.pid文件
$ kill -USR2 `cat /usr/local/nginx/logs/nginx.pid`
# 退出舊版本低nginx
$ /usr/local/nginx/sbin/nginx.bak.20191211 -s stop
# 查看最新的編譯參數(shù)
$ /usr/local/nginx/sbin/nginx -V
nginx version: nginx/1.10.0
...
configure arguments: --add-module=/root/nginx-module-vts --prefix=/usr/local/nginx --user=nginx --group=nginx --with-stream --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module
添加nginx配置文件,暴露監(jiān)控指標(biāo)
$ /usr/local/nginx/conf/nginx.conf
http {
...
vhost_traffic_status_zone;
...
}
$ vim /usr/local/nginx/conf/vhost/nginx-vts-status.conf
server {
listen 8088;
location /status {
vhost_traffic_status_display;
vhost_traffic_status_display_format html;
}
}
#需要重啟模塊才能生效
$ /usr/local/nginx/sbin/nginx -s stop
$ /usr/local/nginx/sbin/nginx
然后訪問ip:8088/status,可以看到下面的數(shù)據(jù)
image
指標(biāo)的數(shù)據(jù)類型轉(zhuǎn)換
要將獲取到的數(shù)據(jù)接入prometheus還需要將數(shù)據(jù)轉(zhuǎn)為metrics類型的數(shù)據(jù)，vts模塊中直接提供了/status/format/prometheus 接口，訪問IP:8088/status/format/prometheus 即可看到轉(zhuǎn)換后的數(shù)據(jù)。
image
部署nginx-vts-exporter
默認(rèn)端口為 :9913/metrics
$ wget http://download.baiyongjie.com/linux/prometheus/nginx-vts-exporter-0.10.3.linux-amd64.tar.gz
$ tar zxvf nginx-vts-exporter-0.10.3.linux-amd64.tar.gz
$ cp nginx-vts-exporter-0.10.3.linux-amd64/nginx-vts-exporter /usr/local/node_exporter/
$ mv nginx-vts-exporter-0.10.3.linux-amd64/ /usr/local/src/
$ cd /tmp/; nohup /usr/local/node_exporter/nginx-vts-exporter -nginx.scrape_uri=http://127.0.0.1:8088/status/format/json &> /dev/null &
$ netstat -nptl|grep 9913
tcp6 0 0 :::9913 :::* LISTEN 27435/nginx-vts-exp
$ curl -s 127.0.0.1:9913/metrics|tail
nginx_upstream_responseMsec{backend="100.115.173.4:80",upstream="::nogroups"} 0
nginx_upstream_responseMsec{backend="100.115.173.5:80",upstream="::nogroups"} 0
nginx_upstream_responseMsec{backend="100.115.173.6:80",upstream="::nogroups"} 0
nginx_upstream_responseMsec{backend="100.115.173.7:80",upstream="::nogroups"} 0
nginx_upstream_responseMsec{backend="100.115.173.8:80",upstream="::nogroups"} 0
nginx_upstream_responseMsec{backend="100.115.173.9:80",upstream="::nogroups"} 0
nginx_upstream_responseMsec{backend="127.0.0.1:9000",upstream="::nogroups"} 0
# HELP nginx_vts_exporter_build_info A metric with a constant '1' value labeled by version, revision, branch, and goversion from which nginx_vts_exporter was built.
# TYPE nginx_vts_exporter_build_info gauge
nginx_vts_exporter_build_info{branch="HEAD",goversion="go1.10",revision="8aa2881c7050d9b28f2312d7ce99d93458611d04",version="0.10.3"} 1
配置prometheus
$ vim /usr/local/prometheus/prometheus.yml
...
- job_name: 'nginx-vts'
static_configs:
- targets:
- 172.31.217.169:9913
$ kill -hup `ps -ef |grep prometheus|grep -v grep|awk '{print $2}'`
查看Prometheus是否采集到的數(shù)據(jù)
image
配置grafana
- 導(dǎo)入模板: 2949
image
安裝GeoIP模塊
$ yum -y install epel-release geoip-devel
$ cd /usr/local/src/nginx-1.10.0
$ ./configure --add-module=/root/nginx-module-vts --prefix=/usr/local/nginx --user=nginx --group=nginx --with-stream --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_geoip_module
$ make
...平滑升級(jí)參考上面安裝nginx-vts模塊
# 查看最新的編譯參數(shù)
$ /usr/local/nginx/sbin/nginx -V
nginx version: nginx/1.10.0
...
configure arguments: --add-module=/root/nginx-module-vts --prefix=/usr/local/nginx --user=nginx --group=nginx --with-stream --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_geoip_module
$ ldd /usr/local/nginx/sbin/nginx |grep libGeoIP
libGeoIP.so.1 => /lib64/libGeoIP.so.1 (0x00007f49d503d000)
# 引用地址庫
$ vim /usr/local/nginx/conf/nginx.conf
http {
...
vhost_traffic_status_zone;
geoip_country /usr/share/GeoIP/GeoIP.dat;
...
}
# 重啟nginx
$ /usr/local/nginx/sbin/nginx -s stop
$ /usr/local/nginx/sbin/nginx
通過VTS自定義key
通過以上部署只能拿到默認(rèn)的指標(biāo)，生產(chǎn)中可能還會(huì)需要監(jiān)控uri的請(qǐng)求量，監(jiān)控IP訪問情況（同一個(gè)IP出現(xiàn)大量訪問時(shí)可能被攻擊），獲取不同agent請(qǐng)求量用于分析等，通過vts模塊的vhost_traffic_status_filter_by_set_key功能可以自定義需要獲取的指標(biāo)。此處的指標(biāo)需要加到對(duì)應(yīng)的server配置中
# 添加自定義配置
$ vim /usr/local/nginx/conf/vhost/baiyongjie.com.conf
server {
listen 443 ssl;
server_name baiyongjie.com blog.baiyongjie.com;
...
vhost_traffic_status_filter_by_set_key $uri uri::$server_name; #每個(gè)uri訪問量
vhost_traffic_status_filter_by_set_key $geoip_country_code country::$server_name; #不同國家/區(qū)域請(qǐng)求量
vhost_traffic_status_filter_by_set_key $status $server_name; #http code統(tǒng)計(jì)
vhost_traffic_status_filter_by_set_key $upstream_addr upstream::backend; #后端轉(zhuǎn)發(fā)統(tǒng)計(jì)
vhost_traffic_status_filter_by_set_key $remote_port client::ports::$server_name; #請(qǐng)求端口統(tǒng)計(jì)
vhost_traffic_status_filter_by_set_key $remote_addr client::addr::$server_name; #請(qǐng)求IP統(tǒng)計(jì)
location ~ ^/storage/(.+)/.*$ {
set $volume $1;
vhost_traffic_status_filter_by_set_key $volume storage::$server_name; #請(qǐng)求路徑統(tǒng)計(jì)
}
...
}
再次訪問status接口, 可以看到多了很多自定義的監(jiān)控項(xiàng)
image
到Prometheus查詢,可以看到新的字段已經(jīng)可以查詢到了
image
參考:
