Per https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers
> GoogleOther is the generic crawler that may be used by various product teams for fetching publicly accessible content from sites. For example, it may be used for one-off crawls for internal research and development.
This commit will ensure it's served the crawler view, and included in crawler metrics
This commit changes request method for "categories/search" from GET to
POST to make sure that long filters can be passed to the server. For
example, category selectors with many categories are setting the full
list of selected category IDs to ensure these are filtered out from the
list of choices. This can result in a long URL that exceeds the maximum
length.
The watched word group's create, update and delete action logs were missing the translations. This PR will add those strings and will use the group key instead of watched word key where needed.
This commit switches `DiscourseIpInfo.mmdb_download` to use the
permalinks supplied by MaxMind to download the MaxMind databases as
specified in
https://dev.maxmind.com/geoip/updating-databases#directly-downloading-databases
which states:
```
To directly download databases, follow these steps:
1. In the "Download Links" column, click "Get Permalink(s)" for the desired database.
2. Copy the permalink(s) provided in the modal window.
3. Provide your account ID and your license key using Basic Authentication to authenticate.
```
Previously we are downloading from `https://download.maxmind.com/app/geoip_download` but this is not
documented anyway on MaxMind's docs so this URL can in theory break
in the future without warning. Therefore, we are taking a proactive
approach to download the databases from MaxMind the recommended way
instead of relying on a hidden URL. This old way of downloading the
databases with only a license key will be deprecated in 3.3 and be
removed in 3.4.
This keeps coming up in user testing as something
we want to get rid of. The `navigation_menu` setting
has been set to sidebar by default for some time now,
and we are rolling out admin sidebar widely. It just
doesn't make sense to let people turn this off in
the first step of the wizard -- we _want_ people to
use the sidebar.
At the moment, there is no way to create a group of related watched words together. If a user needed a set of words to be created together, they'll have to create them individually one at a time.
This change attempts to allow related watched words to be created as a group. The idea here is to have a list of words be tied together via a common `WatchedWordGroup` record. Given a list of words, a `WatchedWordGroup` record is created and assigned to each `WatchedWord` record. The existing WatchedWord creation behaviour remains largely unchanged.
Co-authored-by: Selase Krakani <skrakani@gmail.com>
Co-authored-by: Martin Brennan <martin@discourse.org>
This commit introduces a few changes as a result of
customer issues with finding why a topic was relisted.
In one case, if a user edited the OP of a topic that was
unlisted and hidden because of too many flags, the topic
would get relisted by directly changing topic.visible,
instead of going via TopicStatusUpdater.
To improve tracking we:
* Introduce a visibility_reason_id to topic which functions
in a similar way to hidden_reason_id on post, this column is
set from the various places we change topic visibility
* Fix Post#unhide! which was directly modifying topic.visible,
instead we use TopicStatusUpdater which sets visibility_reason_id
and also makes a small action post
* Show the reason topic visibility changed when hovering the
unlisted icon in topic status on topic titles
Our 'page_view_crawler' / 'page_view_anon' metrics are based purely on the User Agent sent by clients. This means that 'badly behaved' bots which are imitating real user agents are counted towards 'anon' page views.
This commit introduces a new method of tracking visitors. When an initial HTML request is made, we assume it is a 'non-browser' request (i.e. a bot). Then, once the JS application has booted, we notify the server to count it as a 'browser' request. This reliance on a JavaScript-capable browser matches up more closely to dedicated analytics systems like Google Analytics.
Existing data collection and graphs are unchanged. Data collected via the new technique is available in a new 'experimental' report.
Made another effort to improve the description of the personal_message_enabled _groups site setting, to clarify who can already create and reply to messages no matter which groups are specified in this setting.
This service-worker caching functionality was disabled by default in 1c58395bca, and the setting to re-enable was marked as experimental. Now we are dropping all the related logic.
- Run the CSP-nonce-related middlewares on the generated response
- Fix the readonly mode checking to avoid empty strings being passed (the `check_readonly_mode` before_action will not execute in the case of these re-dispatched exceptions)
- Move the BlockRequestsMiddleware cookie-setting to the middleware, so that it is included even for unusual HTML responses like these exceptions
This commit adds the most common AI bot crawlers seen
on our hosting (claudebot, gptbot, anthropic-ai, brightbot)
to our `slow_down_crawler_user_agents` and `crawler_user_agents`
site settings by default.
This means these AI bots will be rate limited by default instead
of site admins having to remember to do it for themselves.
LinkedIn has grandfathered its old OAuth2 provider. This can only be used by existing apps. New apps have to use the new OIDC provider.
This PR adds a linkedin_oidc provider to core. This will exist alongside the discourse-linkedin-auth plugin, which will be kept for those still using the deprecated provider.
When the user sees no results in their admin sidebar query,
we are adding two additional links:
* "Search site settings" - Navigates to the site settings page
with the filter prefilled in the search
* "Admin user list" - Navigates to the user list with the filter
prefilled in the username search
This will bridge the gap until we have a full admin-wide search.
Also make admin site setting search param refresh on filter changes
---------
Co-authored-by: Jarek Radosz <jradosz@gmail.com>
This commit ensures that additional keywords for admin
sidebar links (which are also stored in the admin sidebar state
manager) are translated with I18n, which was discussed
in https://meta.discourse.org/t/introducing-experimental-admin-sidebar-navigation/289281/58?u=martin
This also changes the admin sidebar state manager keywords to
not be a TrackedObject -- this is not necessary as keywords are
only set once, and it was causing rendering issues because
the keywords were being set at the same time they were read.
Finally this adds a "theme" keyword to the "Components" link
because we often refer to components as Theme Components
Co-authored-by: Jarek Radosz <jradosz@gmail.com>
Automatically add `moderators` and `admins` auto groups to specific site settings.
In the new group-based permissions systems, we just want to check the user’s groups since it more accurately reflects reality
Affected settings:
- tag_topic_allowed_groups
- create_tag_allowed_groups
- send_email_messages_allowed_groups
- personal_message_enabled_groups
- here_mention_allowed_groups
- approve_unless_allowed_groups
- approve_new_topics_unless_allowed_groups
- skip_review_media_groups
- email_in_allowed_groups
- create_topic_allowed_groups
- edit_wiki_post_allowed_groups
- edit_post_allowed_groups
- self_wiki_allowed_groups
- flag_post_allowed_groups
- post_links_allowed_groups
- embedded_media_post_allowed_groups
- profile_background_allowed_groups
- user_card_background_allowed_groups
- invite_allowed_groups
- ignore_allowed_groups
- user_api_key_allowed_groups
To add a components link to the sidebar refactoring was required to create unique URLs for themes and components. Before the query param was used. After changes, we have two URLs `/admin/customize/themes` and `/admin/customize/components`.
This is to enable :array type attributes for Contract
attributes in services, this is a followup to the move
of services from chat to core here:
cab178a405
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
The bulk actions menu for topics has multiple options to work
with tags on topics (append, replace, remove). Our tagging system
along with categories allows for some complicated tag restrictions
to be applied via tag groups. This was a problem for the topic bulk
actions because you couldn't append restricted tags to topics.
This commit allows restricted tags to be used in bulk tagging actions
as long as all selected topics are for a sole category. The category
information will be shown in the modal, and the category ID is used
for the tag search.
This will automatically enable the glimmer header when all installed themes/plugins are ready. This replaces the old group-based site setting.
In 'auto' mode, we check for calls to deprecated APIs (e.g. decorateWidget) which affect the old header. If any are present, we stick to the old header implementation and print a message to the console alongside the normal deprecation messages.
To override this automatic behavior, a new `glimmer_header_mode` site setting can be set to 'disabled' or 'enabled'.
This change also means that our test suite is running with the glimmer header. This unveiled a couple of small issues (e.g. some incorrect `aria-*` and `alt` text) which are now fixed. A number of selectors had to be updated to ensure the tests were clicking the actual `<button>` elements rather than the surrounding `<li>` elements.
Display additional confirmation when:
- The public section is going to be updated;
- The public section is going to be deleted;
- The public section is going to be marked as private.
It's mostly fine to use the plural form instead of writing something like "topic(s)" when one or more topics could be meant, but the actual count is not known.
This also removes some unused strings from the locale files.
This commmit removes the unused `/u/:username/preferences/categories`
route which was merged into the `/u/:username/preferences/tracking`
route in 2fc2d7d828.
Adds the new quick menu for bookmarking. When you bookmark
a post (chat message behaviour will come later) we show this new quick
menu and bookmark the item straight away.
You can then choose a reminder quick option, or choose Custom... to open
the old modal. If you click on an existing bookmark, we show the same quick menu
but with Edit and Delete options.
A later PR will introduce a new bookmark modal, but for now we
are using the old modal for Edit and Custom... options.
To remove the Getting Started button manually, you have to disable bootstrap mode by setting bootstrap_mode_min_users to 0. I clarified this in the description for the setting.
- Add a "Skip tips" button to first notification tip
- Add a "Skip tips" button to the admin guide tip
- Fixes the timeline tip showing when no timeline was present
- Fixes post menu tip showing when no "..." button is present
- Adds system tests
- Marks each tip as seen as soon as it is displayed so that refreshing,
clicking outside, etc. won't show it again
- Change just above means we no longer need a MessageBus track
Co-authored-by: Bianca Nenciu <nbianca@users.noreply.github.com>
When a user is manually deactivated, they should not be deleted by our background job that purges inactive users.
In addition, site settings keywords should accept an array of keywords.
In this PR, all references in the UI to the word "`upgrade`" are changed to "`update`". This is to differentiate the update process in self-hosted sites from the plan "upgrade" process in hosted sites.
Follow-up to the PR: https://github.com/discourse/docker_manager/pull/208
Why this change?
This allows downloading the MaxMind databases from a mirror in cases
where downloading directly from MaxMind's API endpoint is problematic
due to API limits.
* DEV: Various bulk-select dropdown tweaks
- Setting is no longer hidden
- descriptions have been moved to the modal
- Removed ... from one of the dropdown titles
This adds a hidden site setting of `skip_email_bulk_invites`
If set to `true`, the `BulkInvite` job will pass the value to `Invite`, meaning the generated invite wont trigger an email notification being sent to the newly invited user.
(This is useful if you want to manage the sending of the invite emails outside of Discourse.)
The personal message enabled groups site setting is overridden by the group interaction settings for specifying who is allowed to write to groups. This was not clearly explained in the description here and I think I fixed it!
Why this change?
This is a follow-up to 86b2e3a.
Basically, we want to allow people to select more than 1 group as well.
What does this change do?
1. Change `type: group` to `type: groups` and support `min` and `max`
validations for `type: groups`.
2. Fix the `<SchemaThemeSetting::Types::Groups>` component to support the
`min` and `max` validations and switch it to use the `<GroupChooser>` component
instead of the `<ComboBoxComponent>` component which previously only supported
selecting a single group.
It's not really intentional to have regular admins change
this in all but pathological cases. It deletes all notifications
over this threshold for users without warning. If admins
really want to turn this on, they can do it via the app.yml file
Why this change?
This is a follow up to 897be75941.
When updating `net-smtp` from `0.4.x` to `0.5.x`, our test suite passed
but the error `ArgumentError: SMTP-AUTH requested but missing user name`
was being thrown in production leading to emails being failed to send
out via SMTP.
This commit adds a test to ensure that our production SMTP settings will
at least attemp to connect to an SMTP server.
When a topic fails to be created due to the user not having permission to add tags to the topic, the error message that you get is `There was an error tagging the topic` which is very generic and doesn't explain where/what the problem is. This commit adds a clearer error message for this scenario.
This commit removes the 'experimental_preconnect_link_header' site setting, and the 'preload_link_header' site setting, and introduces two new global settings: early_hint_header_mode and early_hint_header_name.
We don't actually send 103 Early Hint responses from Discourse. However, upstream proxies can be configured to cache a response header from the app and use that to send an Early Hint response to future clients.
- `early_hint_header_mode` specifies the mode for the early hint header. Can be nil (disabled), "preconnect" (lists just CDN domains) or "preload" (lists all assets).
- `early_hint_header_name` specifies which header name to use for the early hint. Defaults to "Link", but can be changed to support different proxy mechanisms.
Why this change?
This is a follow-up to 86b2e3aa3e.
Basically, we want to allow people to select more than 1 category as well.
What does this change do?
1. Change `type: category` to `type: categories` and support `min` and `max`
validations for `type: categories`.
2. Fix the `<SchemaThemeSetting::Types::Categories>` component to support the
`min` and `max` validations and switch it to use the `<CategorySelector>` component
instead of the `<CategoryChooser>` component which only supports selecting one category.
This user-agent is sent when URLs are inspected via the UI of Google's search console. It makes sense for us to serve it the same content as other bots, including GoogleBot.
When crawlers visit a post-specific URL like `/t/-/{topic-id}/{post-number}`, we use the canonical to direct them to the appropriate crawler-optimised paginated view (e.g. `?page=3`).
However, analysis of google results shows that the post-specific URLs are still being included in the index. Google doesn't tell us exactly why this is happening. However, as a general rule, 'A large portion of the duplicate page's content should be present on the canonical version'.
In our previous implementation, this wasn't 100% true all the time. That's because a request for a post-specific URL would include posts 'surrounding' that post, and won't exactly conform to the page boundaries which are used in the canonical version of the page. Essentially: in some cases, the content of the post-specific pages would include many posts which were not present on the canonical paginated version.
This commit aims to resolve that problem by simplifying the implementation. Instead of rendering posts surrounding the target post_number, we will only render the target post, and include a link to 'show post in topic'. With this new implementation, 100% of the post-specific page content will be present on the canonical paginated version, which will hopefully mean google reduces their indexing of the non-canonical post-specific pages.
Why this change?
Previously, we were preloading the necessary metadata for
`adminCustomizeThemes.show.schema` route in the
`adminCustomizeThemes.show` route. This is wasteful because we're
loading data upfront when the objects setting editor may not be used.
This change also lays the ground work for a future commit where we need
to be shipping down additional metadata which may further add to the
payload.
Why this change?
When a property of `type: tags` is required, we should be displaying the
"at least 1 tag is required" validation error message when there are no
tags selected in the `TagChooser` compoment. However, we were passing
`this.min` as the `count` attribute when generating the translation
string which is incorrect as `this.min` is not always set.
To improve performance, we omit the basic-HTML version of pages when users are logged in, or when they are using a modern mobile device. This can be confusing when analysing the SEO of sites, so this commit adds a short static message when content is omitted.
Why this change?
While working on the tag selector for the theme object editor, I
realised that there is an extremely high possibility that users might want to select
more than one tag. By supporting the ability to select more than one
tag, it also means that we get support for a single tag for free as
well.
What does this change do?
1. Change `type: tag` to `type: tags` and support `min` and `max`
validations for `type: tags`.
2. Fix the `<SchemaThemeSetting::Types::Tags>` component to support the
`min` and `max` validations
Why this change?
Fortunately or unfortunately in Discourse core, we mainly use `Tag#name`
to look up tags and not its id. This assumption is built into the
frontend as well so we need to use the tag's name instead of the id
here.
Previously, we had an instant redirect back to the homepage, and clicking avatars would do nothing. This made things feel 'broken' for anon when 'hide_user_profiles_from_public' was enabled.
This commit does a few things to resolve this:
1. Improve our 'exception' system for routes so that developers can deliberately trigger it without an ajax error
2. Improve 'exception' system so that the browser URL bar is updated correctly, and the 'back' button works as expected
3. Replace the redirect-to-home with an 'access denied' error page, with specific copy for 'You must log in to view user profiles'
4. Update user-card logic to display this new page instead of doing nothing on click
Why this change?
This is a continuation of 8de869630f.
In our schema, we support the `min` and `max` validation
rules like so:
```
some_objects_setting
type: objects
schema:
name: some_object
properties:
id:
type: integer
validations:
min: 5
max: 10
```
While the validations used to validate the objects on the server side,
we should also add client side validation for better UX.
Why this change?
In our schema, we support the `min_length` and `max_length` validation
rules like so:
```
some_objects_setting
type: objects
schema:
name: some_object
properties:
title:
type: string
validations:
min_length: 1
max_length: 10
```
While the validations used to validate the objects on the server side,
we should also add client side validation for better UX.
Currently, a new sidebar link for what's new and reports is going to the main dashboard page and activates the proper tab.
It might be problematic, especially, when the instance has a lot of problems. In that case, it would be difficult for admin to find reports or what’s new which is rendered at the bottom of the page.
Therefore separate pages for reports and what's new were created.
Reports were moved to a component that is shared between a separate page and the dashboard.
Why this change?
In https://web.dev/articles/preconnect-and-dns-prefetch, it describes
how hinting to the browser to preconnect to domains which we will
eventually use the connection for can help improve the time it takes to
load a page.
We are putting this behind an experimental flag so that we can test and
profile this in a production environment.
What does this change introduce?
Introduce a hidden experimental `experimental_preconnect_link_header`
site setting which when enabled will add the `preconnect` and
`dns-prefetch` resource hints to the response headers for full page load
requests.
Why this change?
This is a first pass at styling the editor for creating/editing/updating
an objects typed theme setting. Only the desktop view is being
considered at the current moment.
The objects typed theme setting is still behind a feature flag at this moment so there is no need for us to get the styling perfect. The purpose of this PR is to get us to a state which we can quickly iterate with a designer on.
This commit operates at three levels of abstraction:
1. We want to prevent user history rows from being unbounded in size.
This commit adds rails validations to limit the sizes of columns on
user_histories,
2. However, we don't want to prevent certain actions from being
completed if these columns are too long. In those cases, we truncate
the values that are given and store the truncated versions,
3. For endpoints that perform staff actions, we can further control
what is permitted by explicitly validating the params that are given
before attempting the action,
SaraDev
David Taylor
Bianca Nenciu
Vinoth Kannan
Neil Lalonde
Alan Guo Xiang Tan
David McClure
Jan Cernik
Martin Brennan
Discourse Translator Bot
Krzysztof Kotlarek
Charley Sheets
Joffrey JAFFEUX
Tobias Eigen
Ted Johansson
Natalie Tay
Blake Erickson
Ella E
Gerhard Schlager
Penar Musaraj
Kris
GuteLaune
jbrw
Daniel Waterworth
Angus McLeod
Osama Sayegh
Sam
Gabriel Grubba