Nginx网站使用CDN获取攻击用户真实IP和fail2ban自动禁止的方法

nginx网站在使用cdn后,用户访问就变成了“浏览器 --> DNS解析 --> CDN节点 --> WEB数据处理 --> 浏览器展示”这种形式。对于web服务器nginx来说,直连过来的IP就都是cdn,所以前几篇使用iptables的防御方法就已经不适用这种场景。那么如果碰到大规模的攻击,首先需要获取到攻击用户的真实IP,然互再自动禁止。

一、获取真实IP

在nginx的http模块中添加如下内容。

获取用户真实IP,并赋值给变量$clientRealIP

map $http_x_forwarded_for $clientRealIp {

    ""      $remote_addr;
    ~^(?P<firstAddr>[0-9\.]+),?.*$  $firstAddr;

}
$clientRealIP 就是用户真实IP。

二、Nginx禁止控制

获取到用户的真实IP后,我们可以使用nginx的deny或者return指令最指定IP做禁止访问的控制。

可以使用张戈博主写的脚本,内容如下:

!/bin/bash

Deny Real IP for Nginx; Author: Jager mailto:ge@zhangge.net

For more information please visit https://zhangge.net/5096.html

-----------------------------------------------------------------

Copyright ©2016 zhangge.net. All rights reserved.

NGINX_BIN=/usr/local/nginx/sbin/nginx
DENY_CONF=/usr/local/nginx/conf/deny_ip.conf

COLOR_RED=$( echo -e "\e[31;49m" )
COLOR_GREEN=$( echo -e "\e[32;49m" )
COLOR_RESET=$( echo -e "\e[0m" )

rep_info() { echo;echo -e "${COLOR_GREEN}$*${COLOR_RESET}";echo; }
rep_error(){ echo;echo -e "${COLOR_RED}$*${COLOR_RESET}";echo;exit 1; }

show_help()
{
printf "

Deny Real IP for Nginx; Author: Jager mailto:ge@zhangge.net

For more information please visit https://zhangge.net/5096.html

-----------------------------------------------------------------

Copyright ©2016 zhangge.net. All rights reserved.

Usage: $0 [OPTIONS]

OPTIONS:
-h | --help : Show help of this script
-a | --add : Add a deny ip to nginx, for example: ./$0 -a 192.168.1.1
-c | --create : Create deny config file($DENY_CONF) for Nginx
-d | --del : Delete a ip from deny list, for example: ./$0 -d 192.168.1.1
-s | --show : Show current deny list

"
}

reload_nginx()
{
$NGINX_BIN -t >/dev/null 2>&1 && \
$NGINX_BIN -s reload && \
return 0
}

show_list()
{
awk -F '["){|]' '/if/ {for(i=2;i<=NF;i++) if ($i!="") printf $i"\n"}' $DENY_CONF
}

pre_check()
{
test -f $NGINX_BIN || rep_error "$NGINX_BIN not found,Plz check and edit."
test -f $DENY_CONF || rep_error "$DENY_CONF not found,Plz check and edit."
MATCH_COUNT=$(show_list | grep -w $1 | wc -l)
return $MATCH_COUNT
}

create_rule()
{
test -f $DENY_CONF && \
rep_error "$DENY_CONF already exist!."
cat >$DENY_CONF<<EOF
if ($clientRealIp ~* "8.8.8.8") {
#add_header Content-Type text/plain;
#echo "son of a bitch,you mother fucker,go fuck yourself!";
return 403;
break;
}
EOF
test -f $DENY_CONF && \
rep_info "$DENY_CONF create success!" && \
cat $DENY_CONF && \
exit 0

rep_error "$DENY_CONF create failed!" && \
exit 1

}

add_ip()
{
pre_check $1
if [[ $? -eq 0 ]];then
sed -i "s/\")/|$1&/g" $DENY_CONF && \
reload_nginx && \
rep_info "add $1 to deny_list success." || \
rep_error "add $1 to deny_list failed."
else
rep_error "$1 has been in deny list!"
exit
fi
}

del_ip()
{
pre_check $1
if [[ $? -ne 0 ]];then
sed -ie "s/(|$1\|$1|)//g" $DENY_CONF && \
reload_nginx && \
rep_info "del $1 from deny_list success." || \
rep_error "del $1 from deny_list failed."
else
rep_error "$1 not found in deny list!"
exit
fi
}

case $1 in
"-s"|"--show" )
show_list
exit
;;
"-h"|"--help" )
show_help
exit
;;
"-c"|"--create" )
create_rule
;;
esac

while [ $2 ];do
case $1 in
"-a"|"--add" )
add_ip $2;
;;
"-d"|"--del" )
del_ip $2
;;

  • )
    show_help
    ;;
    esac
    exit

done
show_help
使用方法:
1、根据实际情况修改第9、10行 Nginx 二进制文件及其deny配置文件路径
2、然后将此脚本保存为 deny_ctrl.sh 上传到服务器任意目录,比如放到 /root
3、给脚本赋予可执行权限:chmod +x deny_ctrl.sh 即可使用

初次使用,先执行 ./deny_ctrl.sh -c 创建一下 Nginx 相关配置文件,完成后会生成/usr/local/nginx/conf/deny_ip.conf文件。

生成这个文件之后,编辑网站对应的配置文件,比如 www.sijitao.net.conf,在 server {} 模块内部插入 include deny_ip.conf; 即可。

脚本常用命令如下:

添加禁止IP

/root/deny_ctrl.sh -a 1.2.3.4

删除禁止IP

/root/deny_ctrl.sh -d 1.2.3.4
三、集成Fail2ban

到这里我们已经可以手动把某些IP禁止了,如何自动禁止删除IP博主推荐使用fail2ban。fail2ban的安装和部署规则可以参考之前的几篇文章。接下来以这篇文章 “使用Fail2ban禁止垃圾采集爬虫,保护Nginx服务器。”为例,对加cdn的nginx做禁止配置。

1、创建nginx-deny动作

vi /etc/fail2ban/action.d/nginx-deny.conf

内容如下

[Definition]

actionstart =

actionstop =

actioncheck =

actionban = /root/deny_ctrl.sh -a

actionunban = /root/deny_ctrl.sh -d

[Init]
2、创建filter规则

和上面那篇文章一样,filter规则类似如下。

vi /etc/fail2ban/filter.d/nginx-badbots.conf

内容如下

[Definition]

badbotscustom = EmailCollector|WebEMailExtrac|TrackBack/1.02|sogou music spider

badbots = -|Atomic_Email_Hunter/4.0|atSpider/1.0|autoemailspider|bwh3_user_agent|China Local Browse 2.6|ContactBot/0.2|ContentSmartz|DataCha0s/2.0|DBrowse 1.4b|DB
rowse 1.4d|Demo Bot DOT 16b|Demo Bot Z 16b|DSurf15a 01|DSurf15a 71|DSurf15a 81|DSurf15a VA|EBrowse 1.4b|Educate Search VxB|EmailSiphon|EmailSpider|EmailWolf 1.00|ESu
rf15a 15|ExtractorPro|Franklin Locator 1.8|FSurf15a 01|Full Web Bot 0416B|Full Web Bot 0516B|Full Web Bot 2816B|Guestbook Auto Submitter|Industry Program 1.0.x|ISC S
ystems iRc Search 2.1|IUPUI Research Bot v 1.9a|LARBIN-EXPERIMENTAL (efp@gmx.net)|LetsCrawl.com/1.0 +http\://letscrawl.com/|Lincoln State Web Browser|LMQueueBo
t/0.2|LWP\:\:Simple/5.803|Mac Finder 1.0.xx|MFC Foundation Class Library 4.0|Microsoft URL Control - 6.00.8xxx|Missauga Locate 1.0.0|Missigua Locator 1.9|Miss
ouri College Browse|Mizzu Labs 2.2|Mo College 1.9|MVAClient|Mozilla/2.0 (compatible; NEWT ActiveX; Win32)|Mozilla/3.0 (compatible; Indy Library)|Mozilla/3.0 (
compatible; scan4mail (advanced version) http\://www.peterspages.net/?scan4mail)|Mozilla/4.0 (compatible; Advanced Email Extractor v2.xx)|Mozilla/4.0 (compat
ible; Iplexx Spider/1.0 http\://www.iplexx.at)|Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt; DTS Agent|Mozilla/4.0 efp@gmx.net|Mozilla/5.0 (Version\
: xxxx Type\:xx)|NameOfAgent (CMS Spider)|NASA Search 1.0|Nsauditor/1.x|PBrowse 1.4b|PEval 1.4b|Poirot|Port Huron Labs|Production Bot 0116B|Production Bot 2016B|
Production Bot DOT 3016B|Program Shareware 1.0.2|PSurf15a 11|PSurf15a 51|PSurf15a VA|psycheclone|RSurf15a 41|RSurf15a 51|RSurf15a 81|searchbot admin@google.com|Shabl
astBot 1.0|snap.com beta crawler v0|Snapbot/1.0|Snapbot/1.0 (Snap Shots, +http\://www.snap.com)|sogou develop spider|Sogou Orion spider/3.0(+http\://www
.sogou.com/docs/help/webmasters.htm#07)|sogou spider|Sogou web spider/3.0(+http\://www.sogou.com/docs/help/webmasters.htm#07)|sohu agent|SSurf15a 11 |TSurf15
a 11|Under the Rainbow 2.2|User-Agent\: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)|VadixBot|WebVulnCrawl.unknown/1.0 libwww-perl/5.803|Wells Search II|
WEP Search 00|Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1)|Apache-HttpClient/UNAVAILABLE (java 1.4)

failregex = ^. -."(GET|POST|HEAD).HTTP." \d+ \d+ ".*" "(?:%(badbots)s|%(badbotscustom)s)" "(-|)"$

ignoreregex =
3、创建jail规则

这里我还是在iptables把这些IP禁止了,所以配置了两个。

vi /etc/fail2ban/jail.d/nginx.local

内容如下

[nginx-badbots]

enabled = true
port = http,https
filter = nginx-badbots
logpath = /var/logs/nginx/www.sijitao.net.log
maxretry = 3

[nginx-deny]

enabled = true
filter = nginx-badbots
action = nginx-deny
logpath = /var/logs/nginx/www.sijitao.net.log
maxretry = 3
4、重启fail2ban

总结下,在nginx上使用deny规则可以减少服务器上的压力,但是日志和连接还是会有。所以更加好的办法是使用有ban规则和api接口的cdn厂商,这样再结合fail2ban的action规则就可以从源头上禁止了。