prometheus nginx monitoring example

This memo about how to install Nginx lua module for Prometheus – “Prometheus metric library for Nginx

cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)

After installation you should see this output from http://youhostname:9145

# HELP nginx_http_request_duration_seconds HTTP request latency
# TYPE nginx_http_request_duration_seconds histogram
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="00.005"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="00.010"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="00.020"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="00.030"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="00.050"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="00.075"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="00.100"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="00.200"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="00.300"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="00.400"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="00.500"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="00.750"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="01.000"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="01.500"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="02.000"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="03.000"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="04.000"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="05.000"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="10.000"} 10
nginx_http_request_duration_seconds_bucket{host="alexlinux.com",le="+Inf"} 10
nginx_http_request_duration_seconds_count{host="alexlinux.com"} 10
nginx_http_request_duration_seconds_sum{host="alexlinux.com"} 0
# HELP nginx_http_requests_total Number of HTTP requests
# TYPE nginx_http_requests_total counter
nginx_http_requests_total{host="alexlinux.com",status="200"} 10
# HELP nginx_metric_errors_total Number of nginx-lua-prometheus errors
# TYPE nginx_metric_errors_total counter
nginx_metric_errors_total 33

Installation steps:

yum install pcre-devel luajit-devel -y

wget https://github.com/simpl/ngx_devel_kit/archive/v0.3.0.tar.gz
tar -xvzf v0.3.0.tar.gz

wget https://github.com/openresty/lua-nginx-module/archive/v0.10.7.tar.gz
tar -xvzf v0.10.7.tar.gz

wget 'http://nginx.org/download/nginx-1.11.2.tar.gz'
tar -xvzf nginx-1.11.2.tar.gz
cd nginx-1.11.2
./configure --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log  --pid-path=/var/run/nginx.pid  --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp  --http-proxy-temp-path=/var/cache/nginx/proxy_temp  --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp  --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp  --user=nginx  --group=nginx   --with-ld-opt="-Wl,-rpath,/usr/local/lib"  --add-module=/root/ngx_devel_kit-0.3.0 --add-module=/root/lua-nginx-module-0.10.7  --without-http_gzip_module
make -j 2 && make install  
mkdir -p /var/cache/nginx/
useradd nginx
mkdir -p /etc/nginx/nginx-lua-prometheus
cd /etc/nginx/nginx-lua-prometheus
wget https://raw.githubusercontent.com/knyar/nginx-lua-prometheus/master/prometheus.lua
vim /etc/nginx/nginx.conf

add inside “http” block

lua_shared_dict prometheus_metrics 10M;
lua_package_path "/etc/nginx/nginx-lua-prometheus/?.lua";
init_by_lua '
  prometheus = require("prometheus").init("prometheus_metrics")
  metric_requests = prometheus:counter(
    "nginx_http_requests_total", "Number of HTTP requests", {"host", "status"})
  metric_latency = prometheus:histogram(
    "nginx_http_request_duration_seconds", "HTTP request latency", {"host"})
  metric_connections = prometheus:gauge(
    "nginx_http_connections", "Number of HTTP connections", {"state"})
';
log_by_lua '
  local host = ngx.var.host:gsub("^www.", "")
  metric_requests:inc(1, {host, ngx.var.status})
  metric_latency:observe(ngx.now() - ngx.req.start_time(), {host})
';

and this one

server {
  listen 9145;
  allow 192.168.0.0/16;
  location /metrics {
    content_by_lua '
      metric_connections:set(ngx.var.connections_reading, {"reading"})
      metric_connections:set(ngx.var.connections_waiting, {"waiting"})
      metric_connections:set(ngx.var.connections_writing, {"writing"})
      prometheus:collect()
    ';
  }
}

And finally start Nginx:

/usr/sbin/nginx -g "daemon off;"

P.S. source RPM here

2 thoughts on “prometheus nginx monitoring example

  1. oldspl

    I have this issue when I tried to start the nginx
    nginx: [emerg] unknown directive “lua_shared_dict” in /etc/nginx/nginx.conf:11

    In “http” block?

    Reply

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>