Sam 7b562d2f46 FEATURE: much improved and simplified crawler detection
- phase one does it match 'trident|webkit|gecko|chrome|safari|msie|opera'
    yes- well it is possibly a browser

- phase two does it match 'rss|bot|spider|crawler|facebook|archive|wayback|ping|monitor'
    probably a crawler then

Based off: https://gist.github.com/SamSaffron/6cfad7ea3e6df321ffb7a84f93720a53
2018-01-16 15:41:45 +11:00
..
2018-01-05 11:01:23 +01:00
2017-11-10 14:10:27 -05:00
2017-07-28 10:20:09 +09:00
2018-01-14 20:01:45 +01:00
2017-12-23 01:56:33 +01:00
2017-07-28 10:20:09 +09:00
2016-02-25 12:13:59 +10:00
2017-09-08 14:07:24 +08:00
2017-07-28 10:20:09 +09:00
2016-07-15 11:36:06 +08:00
2017-07-28 10:20:09 +09:00
2017-07-03 15:26:57 -04:00
2017-11-02 12:33:35 +01:00
2017-12-13 17:36:36 +11:00
2017-07-28 10:20:09 +09:00
2018-01-03 16:54:55 -05:00
2017-07-28 10:20:09 +09:00