Sam d84256a876 FEATURE: add Noindex to robots.txt for disallowed routes ... This strips pages out of indexes that should not exist see: https://meta.discourse.org/t/pages-listed-in-the-robots-txt-are-crawled-and-indexed-by-google/100309/11?u=sam		2018-11-02 16:39:47 +11:00
..
index.erb	FEATURE: add Noindex to robots.txt for disallowed routes	2018-11-02 16:39:47 +11:00
no_index.erb	prefix the robots.txt rules with the directory when using subfolder	2018-04-11 22:05:02 +02:00