Joshua Rosenfeld b52143feff

FIX: Remove paths from robots.txt in favor of noindex header ...

Google no longer supports the use of robots.txt to block indexing.
See https://support.google.com/webmasters/answer/6062608 and
https://support.google.com/webmasters/answer/93710

Previous commits have added the `noindex` header to appropriate pages,
now we need to remove the paths from robots.txt so the pages can be
crawled.

Follow up to:
13f229808a22db9e1032832a313ab701b66614c8
b6765aac4b532c026418a7ffd9effd0741ab8a37
676be3a853454a33cf627c3d570feb37d3bb0bfd
07b728c5e557c9aae91c51f3eaac5c32d479f2a2
c94e6a9a66757ea48d99e3ee8d880523871cb6f4

2020-06-25 13:55:06 -04:00

..

assets

FIX: ensures moderation history is accessible from topic/post admin menu (#10118)

2020-06-24 10:49:47 +02:00

controllers

FIX: Remove paths from robots.txt in favor of noindex header

2020-06-25 13:55:06 -04:00

helpers

New bootstrap.json endpoint for starting up Discourse

2020-06-03 14:45:23 -04:00

jobs

DEV: improve verbose mode for reindexer

2020-06-24 17:29:45 +10:00

mailers

FIX: Use correct URL for unsubscribe (#10077)

2020-06-24 09:31:20 +02:00

models

FIX: properly invalidate inline oneboxes when rebaking

2020-06-24 11:54:54 +02:00

serializers

FIX: skip category notification_level unless scoped

2020-06-24 17:08:12 +10:00

services

FEATURE: allow disabling of extra term injection in search

2020-06-25 13:36:52 +10:00

views

FEATURE: allows to have header/footer in publshed pages (#10067)

2020-06-19 09:51:03 +02:00