Created
June 18, 2012 23:58
-
-
Save supairish/2951524 to your computer and use it in GitHub Desktop.
Nginx - how to limit requests by User Agent
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
http { | |
map $http_user_agent $limit_bots { | |
default ''; | |
~*(google|bing|yandex|msnbot) $binary_remote_addr; | |
} | |
limit_req_zone $limit_bots zone=bots:10m rate=1r/m; | |
server { | |
location / { | |
limit_req zone=bots burst=5 nodelay; | |
} | |
} | |
} |
One way of doing IP address and http_user_agent could be to put $binary_remote_addr and $http_user_agent into 1 variable for example "10.1.1.10 google" and use that as the key
hope this helps
this is what I did, and it works. had to define a $geo_whitelist variable FIRST, and then reuse that variable in the map {} for user agent (as the default value)
# ref: http://serverfault.com/questions/177461/how-to-rate-limit-in-nginx-but-including-excluding-certain-ip-addresses
# whitelisted IP ranges - will not have limits applied
geo $geo_whitelist {
default 0;
1.2.3.4 1;
2.3.4.5/24 1;
}
# whitelisted user agents - will not have limits applied
map $http_user_agent $whitelist {
default $geo_whitelist;
~*(google) 1;
}
# if whitelist is 0, put the binary IP address in $limit so the rate limiting has something to use
map $whitelist $limit {
0 $binary_remote_addr;
1 "";
}
limit_req_zone $limit zone=perip:30m rate=1r/s;
thanks mike503 this help a lot :D
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This sounds promising for what I'm looking to do - it appears as if you will map the value of $limit_bots based on the value of the $http_user_agent. That is if the $http_user_agent is google, bing, yandex, or msnbot set the value of $limit_bots to be the binary of the IP address. What if, though, you want different rate limiting that takes into account both the IP address and the $http_user_agent?