update readme and add files

new snippets, docker-compose, fail2ban, filebeat time spent: 9.67h
2022-03-31 23:37:11 +03:00 · 2020-07-09 19:26:57 +03:00
parent 0e5181fde2
commit 81463e0505
10 changed files with 385 additions and 2 deletions
--- a/README.md
+++ b/README.md
@@ -1,7 +1,93 @@
 ## Nginx common useful configuration

+Nginx configs. Not the most powerful, productive or the best one. Just useful configs, which I would like to see in default nginx packages out of the box 😆
+Bonus: fail2ban, filebeat and docker-compose configs for nginx :)
+
+Warning: I'm living in Belarus - country between EU and Russia. [And today we are fighting for our freedom against 'the last dictator of the Europe'](https://www.euronews.com/2020/06/25/belarus-is-no-longer-scared-of-lukashenko-europe-s-last-dictator-will-fall-view).
+So I can't guarantee that I'll be able to maintain this repo scrupulously this summer. Sorry, guys
+
+**Motivation**: I have been using nginx for last 4 years at least, and I configured it really for hundreds setups of 30+ companies and startups: sites, apps, websockets, proxies, load balancing, from few up to 1k rps, etc... And I'm a little bit disappointed by [the official nginx wiki](https://www.nginx.com/resources/wiki/).
+The last drop was this [blog post in the official blog](https://www.nginx.com/blog/help-the-world-by-healing-your-nginx-configuration/):
+this post doesn't provide a complete solution, half of these tips can be included into nginx configs or snippets by default,
+and some of the other tips, such as disabling access logging, in my opinion are the bad practice 😆
+
+At the same time there are a lot good documentation and best practices:
+[nginx docs](https://nginx.org/en/docs/),
+[digitalocean config generator](https://www.digitalocean.com/community/tools/nginx),
+[mozilla ssl best practices](https://ssl-config.mozilla.org/#server=nginx&version=1.17.7&config=intermediate&openssl=1.1.1d&guideline=5.4),
+etc... And here I'm trying to put together all good patterns and knowledges so anyone will be able to copy this configs and get good nginx setup out of the box :)
+
+There is also interesting [openbridge nginx](https://github.com/openbridge/nginx) docker image,
+but I haven't checked it properly yet, their configs require addition nginx modules and setup
+and it can't be just copied to usual nginx. However, you can use it with docker.
+Also I don't agree with nginx microcache for every site, see known traps.
+
 Time track:
- Filipp Frizzy 23.67h
+- [Filipp Frizzy](https://github.com/Friz-zy/) 33.34h
+
+### Configs
+
+#### Main configs
+Almost all sections moved from main `nginx.conf` into `conf.d` directory:
+
+* `basic.conf`  
+Basic settings, security, mime types, charset, index, timeouts, open file cache, etc...
+* `cache.conf`  
+Fastcgi, Proxy and Uwsgi cache setup, see known traps before using ;)
+* `gzip.conf`  
+Gzip and gzip static
+* `log_format.conf`  
+Extended log formats
+* `real_ip.conf`  
+Allow X-Forwarded-For header from local networks and [cloudflare](https://www.cloudflare.com/)
+* `request_id.conf`  
+Add X-Request-ID header into each request for tracing and debugging
+* `ssl.conf`  
+SSL best practice from [mozilla](https://ssl-config.mozilla.org/#server=nginx&version=1.17.7&config=intermediate&openssl=1.1.1d&guideline=5.4)
+
+#### Snippets
+Templates and includes. You can also use [config generator](https://www.digitalocean.com/community/tools/nginx) from digitalocean :)
+
+* corps.conf.j2  
+Template of corps politic for multiple subdomains setup
+* default.conf  
+Example of default config with nginx_status, let's encrypt check and redirect to https
+* fastcgi.conf  
+Include for php locations: fastcgi parameters, timeouts and cache example
+* headers.conf  
+Include with all headers, see known traps
+* protected_locations.conf  
+Include with protected locations with 'deny all'
+* proxy.conf  
+Include for proxy locations: proxy headers, parameters, timeouts and cache example
+* referer.conf.j2  
+Template of referer protection for cases when you concurents use your fail2ban protection against you, see known traps
+* site.conf.j2  
+Template of common site configuration
+* static_location.conf  
+Include with location for static files
+
+#### Docker-compose
+`docker-compose.yml` example for nginx
+
+#### Fail2ban
+You can use fail2ban for banning some bots even behind load balancer.
+`nginx-deny` action will add `deny <ip>;` into `/etc/nginx/conf.d/banned.conf` and reload nginx.
+
+Warning: your evil competitors can use your protection like fail2ban against you, check known traps ;)
+
+Files for copying:
+```
+fail2ban/jail.local => /etc/fail2ban/jail.local
+fail2ban/action-nginx-deny.conf => /etc/fail2ban/action.d/nginx-deny.conf
+fail2ban/filter-magento.conf => /etc/fail2ban/filter.d/nginx-magento.conf
+fail2ban/filter-wordpress.conf => /etc/fail2ban/filter.d/nginx-wordpress.conf
+fail2ban/filter-nginx-noscript.conf => /etc/fail2ban/filter.d/nginx-noscript.conf
+```
+
+#### Filebeat
+Filebeat by default can't parse extended nginx access log formats, so you should override ingest json:
+Copy `filebeat/nginx_access_ingest.json` to `/usr/share/filebeat/module/nginx/access/ingest/default.json`

 ### Known traps

@@ -34,7 +120,7 @@ in both locations Nginx will cache every response.
 So if your site has some login functionality or shopping cart or whatever, 
 it will be mixed and most of clients will get response with content of some other clients.

-In this configuration I suggest caches only as a auxiliary tool for caching common non 200 status responses:
+In this configuration I suggest caches only as an additional tool for caching common non 200 status responses:
 ```
 fastcgi_cache_valid 499 500 502 503 504 521 522 523 524 3s; # circuit breaker
 fastcgi_cache_valid 404 15m; # cache Not Found for decrease loading to backend
@@ -97,6 +183,21 @@ into sections under HTTP one.
 include /etc/nginx/snippets/headers.conf
 ```

+#### Fail2ban and any other protection can be used against you
+
+Not only that incorrectly configured protection will block valid users,
+even right configured protection like fail2ban, especially with `botsearch-common` filter,
+can be used for attack to you. For example, you competitors can add to their sites something like
+```
+<img src="https://{{ your site }}/admin/1.jpg">
+<img src="https://{{ your site }}/phpmyadmin/1.jpg">
+<img src="https://{{ your site }}/roundcube/1.jpg">
+```
+
+Then valid user after visit to the their site will be automatically blocked on your site 😆
+You can fight with this practice using `http_referer`, see `snippets/referer.conf.j2` template ;)
+Warning: I have not tested this code yet
+
 ### Nginx build info

 #### Docker
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -0,0 +1,29 @@
+version: '3'
+
+services:
+  nginx:
+      image: nginx
+      volumes:
+        # you can copy this configs into /etc/nginx and mount whole dir
+        - nginx.conf:/etc/nginx/nginx.conf
+        - conf.d:/etc/nginx/conf.d
+        - snippets:/etc/nginx/snippets
+        #- /etc/nginx:/etc/nginx
+        #- /etc/letsencrypt:/etc/letsencrypt
+        - /etc/ssl:/etc/ssl
+        #- /var/log/nginx:/var/log/nginx
+        - /var/www:/var/www
+      ports:
+        - 80:80
+        - 443:443
+      command: nginx -g 'daemon off; master_process on;'
+      ulimits:
+        nproc: 65535
+        nofile:
+          soft: 100000
+          hard: 100000
+      restart: unless-stopped
+      logging:
+        driver: journald
+        options:
+          tag: "docker/nginx/{{.Name}}"
--- a/fail2ban/action-nginx-deny.conf
+++ b/fail2ban/action-nginx-deny.conf
@@ -0,0 +1,12 @@
+# Fail2Ban action configuration file for nginx deny configuration
+
+[Definition]
+
+actionstart =
+actionstop =
+actioncheck =
+actionban = echo "deny <ip>;" >> /etc/nginx/conf.d/banned.conf && /usr/sbin/service nginx reload
+actionunban = sed --in-place '/deny <ip>;/d' /etc/nginx/conf.d/banned.conf && /usr/sbin/service nginx reload
+
+[Init]
+
--- a/fail2ban/filter-magento.conf
+++ b/fail2ban/filter-magento.conf
@@ -0,0 +1,5 @@
+# https://magento.stackexchange.com/questions/110299/the-code-provided-by-magento-against-admin-password-guessing-to-protect-magent
+[Definition]
+# Use this for "soft" bad behaviour, as the source will only be banned after multiple retries.
+failregex = ^<HOST> .+"POST \S+(/downloader/|/downloader/index.php\?A=loggedin|/admin/index/|/admin/)\s
+ignoreregex =
--- a/fail2ban/filter-nginx-noscript.conf
+++ b/fail2ban/filter-nginx-noscript.conf
@@ -0,0 +1,10 @@
+#
+# Block IPs trying to execute scripts such as .php, .pl, .exe and other funny scripts.
+#
+# Matches e.g.
+# 192.168.1.1 - - "GET /something.php
+#
+[Definition]
+# failregex = ^<HOST> -.*GET.*(\.php|\.asp|\.exe|\.pl|\.cgi|\scgi)
+failregex = ^<HOST> -.*GET.*(\.asp|\.exe|\.pl|\.cgi|\scgi)
+ignoreregex =
--- a/fail2ban/filter-wordpress.conf
+++ b/fail2ban/filter-wordpress.conf
@@ -0,0 +1,6 @@
+# https://blog.rimuhosting.com/2016/11/02/using-fail2ban-on-wordpress-wp-login-php-and-xmlrpc-php/
+[Definition]
+failregex = ^<HOST> .* "POST .*wp-login.php
+            ^<HOST> .* "POST .*xmlrpc.php
+ignoreregex =
+
--- a/fail2ban/jail.local
+++ b/fail2ban/jail.local
@@ -0,0 +1,76 @@
+[DEFAULT]
+
+#
+# MISCELLANEOUS OPTIONS
+#
+
+# "ignoreip" can be an IP address, a CIDR mask or a DNS host. Fail2ban will not
+# ban a host which matches an address in this list. Several addresses can be
+# defined using space (and/or comma) separator.
+ignoreip = 127.0.0.0/8 ::1/128 10.0.0.0/8 100.64.0.0/10 172.16.0.0/12 192.168.0.0/16 fc00::/7
+
+# External command that will take an tagged arguments to ignore, e.g. <ip>,
+# and return true if the IP is to be ignored. False otherwise.
+#
+# ignorecommand = /path/to/command <ip>
+ignorecommand =
+
+# "bantime" is the number of seconds that a host is banned.
+bantime  = 600
+
+# A host is banned if it has generated "maxretry" during the last "findtime"
+# seconds.
+findtime  = 600
+
+# "maxretry" is the number of failures before a host get banned.
+maxretry = 10
+
+
+
+#
+# JAILS
+#
+
+[sshd]
+enabled = true
+banaction = iptables-multiport
+bantime  = 3600
+maxretry = 5
+
+[nginx-magento]
+enabled = true
+banaction = nginx-deny
+filter = nginx-magento
+logpath = /var/log/nginx/*access*.log
+
+[nginx-wordpress]
+enabled = true
+banaction = nginx-deny
+filter = nginx-wordpress
+logpath = /var/log/nginx/*access*.log
+
+[nginx-noscript]
+enabled = true
+banaction = nginx-deny
+filter = nginx-noscript
+logpath = /var/log/nginx/*access*.log
+
+[nginx-limit-req]
+enabled = true
+banaction = nginx-deny
+filter = nginx-limit-req
+logpath = /var/log/nginx/*error*.log
+
+[nginx-http-auth]
+enabled = true
+banaction = nginx-deny
+filter = nginx-http-auth
+logpath = /var/log/nginx/*error*.log
+
+[botsearch-common]
+enabled = false
+banaction = nginx-deny
+filter = botsearch-common
+logpath = /var/log/nginx/*error*.log
+        /var/log/nginx/*access*.log
+
--- a/filebeat/nginx_access_ingest.json
+++ b/filebeat/nginx_access_ingest.json
@@ -0,0 +1,131 @@
+{
+    "description": "Pipeline for parsing Nginx access logs. Requires the geoip and user_agent plugins.",
+    "processors": [
+        {
+            "grok": {
+                "field": "message",
+                "patterns": [
+                    "\"?(?:%{IP_LIST:nginx.access.remote_ip_list}|%{DATA:source.address}) - %{DATA:user.name} \\[%{HTTPDATE:nginx.access.time}\\] \"%{DATA:nginx.access.info}\" %{NUMBER:http.response.status_code:long} %{NUMBER:http.response.body.bytes:long} \"%{DATA:http.request.referrer}\" \"%{DATA:user_agent.original}\"\\s?(?<message_postfix>.*)"
+                ],
+                "pattern_definitions": {
+                    "IP_LIST": "%{IP}(\"?,?\\s*%{IP})*"
+                },
+                "ignore_missing": true
+            }
+        },
+        {
+            "grok": {
+                "field": "nginx.access.info",
+                "patterns": [
+                    "%{WORD:http.request.method} %{DATA:url.original} HTTP/%{NUMBER:http.version}",
+                    ""
+                ],
+                "ignore_missing": true
+            }
+        },
+        {
+            "remove": {
+                "field": "nginx.access.info"
+            }
+        },
+        {
+            "split": {
+                "field": "nginx.access.remote_ip_list",
+                "separator": "\"?,?\\s+",
+                "ignore_missing": true
+            }
+        },
+        {
+            "split": {
+                "field": "nginx.access.origin",
+                "separator": "\"?,?\\s+",
+                "ignore_missing": true
+            }
+        },
+        {
+            "set": {
+                "field": "source.ip",
+                "value": ""
+            }
+        },
+        {
+            "script": {
+                "lang": "painless",
+                "source": "boolean isPrivate(def dot, def ip) { try { StringTokenizer tok = new StringTokenizer(ip, dot); int firstByte = Integer.parseInt(tok.nextToken());       int secondByte = Integer.parseInt(tok.nextToken());       if (firstByte == 10) {         return true;       }       if (firstByte == 192 && secondByte == 168) {         return true;       }       if (firstByte == 172 && secondByte >= 16 && secondByte <= 31) {         return true;       }       if (firstByte == 127) {         return true;       }       return false;     } catch (Exception e) {       return false;     }   }   try {    ctx.source.ip = null;    if (ctx.nginx.access.remote_ip_list == null) { return; }    def found = false;    for (def item : ctx.nginx.access.remote_ip_list) {        if (!isPrivate(params.dot, item)) {            ctx.source.ip = item;            found = true;            break;        }    }    if (!found) {     ctx.source.ip = ctx.nginx.access.remote_ip_list[0];   }} catch (Exception e) { ctx.source.ip = null; }",
+                "params": {
+                    "dot": "."
+                }
+            }
+        },
+        {
+            "remove": {
+                "field": "source.ip",
+                "if": "ctx.source.ip == null"
+            }
+        },
+        {
+            "convert": {
+                "field": "source.ip",
+                "target_field": "source.address",
+                "type": "string",
+                "ignore_missing": true
+            }
+        },
+        {
+            "rename": {
+                "field": "@timestamp",
+                "target_field": "event.created"
+            }
+        },
+        {
+            "date": {
+                "field": "nginx.access.time",
+                "target_field": "@timestamp",
+                "formats": [
+                    "dd/MMM/yyyy:H:m:s Z"
+                ],
+                {< if .convert_timezone >}"timezone": "{{ event.timezone }}",{< end >}
+                "ignore_failure": true
+            }
+        },
+        {
+            "remove": {
+                "field": "nginx.access.time"
+            }
+        },
+        {
+            "user_agent": {
+                "field": "user_agent.original"
+            }
+        },
+        {
+            "geoip": {
+                "field": "source.ip",
+                "target_field": "source.geo",
+                "ignore_missing": true
+            }
+        },
+        {
+            "kv": {
+                "field": "message_postfix",
+                "field_split": " ",
+                "value_split": "=",
+                "prefix": "http.request.",
+                "ignore_missing": true
+            }
+        },
+        {
+            "remove": {
+                "field": "message_postfix"
+            }
+        }
+    ],
+    "on_failure": [
+        {
+            "set": {
+                "field": "error.message",
+                "value": "{{ _ingest.on_failure_message }}"
+            }
+        }
+    ]
+}
--- a/snippets/referer.conf.j2
+++ b/snippets/referer.conf.j2
@@ -0,0 +1,7 @@
+# http://nginx.org/en/docs/http/ngx_http_referer_module.html
+
+valid_referers none blocked server_names *.{{ item.domain }};
+
+if ($invalid_referer) {
+    return 307 $http_referer;
+}
--- a/snippets/site.conf.j2
+++ b/snippets/site.conf.j2
@@ -38,6 +38,12 @@ server {
    # ssl_certificate         /etc/ssl/certs/{{ item.domain }}.crt;
    # ssl_certificate_key     /etc/ssl/private/{{ item.domain }}.key;

+    # corps hack
+    # include /etc/nginx/snippets/corps.conf
+
+    # referer protection
+    # include /etc/nginx/snippets/referer.conf
+
    # location ~* \.(js|css|png|jpg|jpeg|gif|ico|swf|eot|ttf|otf|woff|woff2)$
    include /etc/nginx/snippets/static_location.conf