Sam 7b562d2f46 FEATURE: much improved and simplified crawler detection
- phase one does it match 'trident|webkit|gecko|chrome|safari|msie|opera'
    yes- well it is possibly a browser

- phase two does it match 'rss|bot|spider|crawler|facebook|archive|wayback|ping|monitor'
    probably a crawler then

Based off: https://gist.github.com/SamSaffron/6cfad7ea3e6df321ffb7a84f93720a53
2018-01-16 15:41:45 +11:00
..
2017-07-28 10:20:09 +09:00
2017-10-09 10:27:52 +11:00
2017-07-28 10:20:09 +09:00
2017-07-28 10:20:09 +09:00
2017-07-28 10:20:09 +09:00
2017-12-23 14:42:40 +05:30
2017-12-05 07:04:41 +11:00