discourse

Author	SHA1	Message	Date
Tobias Eigen	709bed5d1c	removed broken link and comments from no_index.erb (#25648 ) We intend to move guidance about how to configure robots.txt to the admin interface and a documentation topic on meta.	2024-02-14 12:09:24 +08:00
Daniel Waterworth	721ee36425	Replace `base_uri` with `base_path` (#10879 ) DEV: Replace instances of Discourse.base_uri with Discourse.base_path This is clearer because the base_uri is actually just a path prefix. This continues the work started in `555f467`.	2020-10-09 12:51:24 +01:00
Sam Saffron	bb4e8899c4	FEATURE: let Google index pages so it can remove them Google insists on indexing pages so it can figure out if they can be removed from the index. see: https://support.google.com/webmasters/answer/6332384?hl=en This change ensures the we have special behavior for Googlebot where we allow indexing, but block the actual indexing via X-Robots-Tag	2020-05-11 12:15:18 +10:00
Régis Hanol	df7970a6f6	prefix the robots.txt rules with the directory when using subfolder	2018-04-11 22:05:02 +02:00
Guo Xiang Tan	77d4c4d8dc	Fix all the errors to get our tests green on Rails 5.1.	2017-09-25 13:48:58 +08:00
Sam Saffron	c50a9e4d01	added support for disabling indexing by google using SiteSetting.allow_index_in_robots_txt = false	2013-02-11 11:02:57 +11:00