Commit Graph

673 Commits

Author SHA1 Message Date
Daniel Waterworth 47709c6d49
DEV: Reinstate --single-branch when cloning themes (#19026) 2022-11-15 13:23:57 -06:00
Blake Erickson 7be53b1588
FEATURE: Default Composer Category Site Setting (#18967)
* FEATURE: Default Composer Category Site Setting

- Create the default_composer_category site setting
- Replace general_category_id logic for auto selecting the composer
  category
- Prevent Uncategorized from being selected if not allowed
- Add default_composer_category option to seeded categories
- Create a migration to populate the default_composer_category site
  setting if there is a general_category_id populated
- Added some tests

* Add missing translation for the new site setting

* fix some js tests

* Just check that the header value is null
2022-11-14 11:09:57 -07:00
Selase Krakani 0b367216ae
FIX: Ensure moderators_manage_categories_and_groups is respected (#18884)
Currently, moderators are able to set primary group for users
irrespective of the of the `moderators_manage_categories_and_groups` site
setting value.

This change updates Guardian implementation to honour it.
2022-11-11 11:06:05 +00:00
Jarek Radosz 3e0196cbbf
FIX: Shrinking images where smaller image upload exists (#18965) 2022-11-10 12:43:56 +01:00
David Taylor 8700c5ee6b
PERF: Make stylesheet hashes consistent between deploys (#18909)
Previously the stylesheet cachebusting hash was based on the maximum mtime of files. This works well in development and during in-container updates (e.g. via docker_manager). However, when a fresh docker image is created for each deploy, the file mtimes will change even if the contents has not.

This commit changes the production logic to calculate the cachebuster from the filenames and contents of the relevant assets. This should be consistent across deploys, thereby improving cache hits and improving page load times.
2022-11-07 16:13:35 +00:00
David Taylor f30f9ec5d9
PERF: Update `s3:expire_missing_assets` to delete in batches (#18908)
Some sites may have thousands of stale assets - deleting them one-by-one is very slow.

Followup to e8570b5cc9
2022-11-07 12:53:14 +00:00
Martin Brennan d6bd4ad7ee
FIX: Make can_send_private_messages not reliant on system user (#18812)
Since the system user is a regular user, it can have its
`allow_private_messages` user option turned off, which
with our current `can_send_private_message?(Discourse.system_user)`
check inside the CurrentUserSerializer, will prevent any
user from sending messages in the UI if the system user is not
accepting PMs.

This commit adds a new `can_send_private_messages?` method to
the Guardian, which can be used in serializers and not depend
on the system user. When the user actually sends a message
we still rely on the old `can_send_private_message?(target)`
call to see if they are allowed to send the message to the target.
The new method is just to say they can "generally" send
private messages.
2022-11-07 09:11:18 +10:00
Jarek Radosz c32fe340f0
DEV: Fix mocha deprecations (#18828)
It now supports strict keyword argument matching by default.
2022-11-02 10:47:59 +01:00
David Taylor 68b4fe4cf8
SECURITY: Expand and improve SSRF Protections (#18815)
See https://github.com/discourse/discourse/security/advisories/GHSA-rcc5-28r3-23rr

Co-authored-by: OsamaSayegh <asooomaasoooma90@gmail.com>
Co-authored-by: Daniel Waterworth <me@danielwaterworth.com>
2022-11-01 16:33:17 +00:00
Alan Guo Xiang Tan a473e352de
DEV: Introduce TopicGuardian#can_see_topic_ids method (#18692)
Before this commit, there was no way for us to efficiently check an
array of topics for which a user can see. Therefore, this commit
introduces the `TopicGuardian#can_see_topic_ids` method which accepts an
array of `Topic#id`s and filters out the ids which the user is not
allowed to see. The `TopicGuardian#can_see_topic_ids` method is meant to
maintain feature parity with `TopicGuardian#can_see_topic?` at all
times so a consistency check has been added in our tests to ensure that
`TopicGuardian#can_see_topic_ids` returns the same result as
`TopicGuardian#can_see_topic?`. In the near future, the plan is for us
to switch to `TopicGuardian#can_see_topic_ids` completely but I'm not
doing that in this commit as we have to be careful with the performance
impact of such a change.

This method is currently not being used in the current commit but will
be relied on in a subsequent commit.
2022-10-27 06:13:21 +08:00
Alan Guo Xiang Tan 305b7c8fae
DEV: Update rubocop (#18754) 2022-10-26 09:05:15 +08:00
Jarek Radosz df56ab172a
DEV: Remove remaining hardcoded ids (#18735) 2022-10-25 15:29:09 +08:00
Alan Guo Xiang Tan 0ffd408674
DEV: Remove harded id when fabricating in tests (#18729)
Hardcoding ids always lead to sadness for our test suite
2022-10-25 06:18:40 +08:00
Bianca Nenciu 266e165885
FIX: Use only first line from commit message (#18724)
Linking a commit from a GitHub pull request included the complete commit
message, instead of just the first line. The rest of the commit message
will be added to the body of the Onebox.
2022-10-24 22:26:48 +03:00
Jan Cernik 25fbbf4020
UX: Stop falling back to topic image on embeds (#18720)
If linked post (not OP) has no image, it won't fall back to the topic
image anymore.
2022-10-24 14:54:02 -03:00
Alan Guo Xiang Tan c752d68737
DEV: Fabricate instead of just building topic, post and user in tests (#18698)
Building does not persist the object in the database which is
unrealistic since we're mostly dealing with persisted objects in
production.

In theory, this will result our test suite taking longer to run since we
now have to write to the database. However, I don't expect the increase
to be significant and it is actually no different than us adding new
tests which fabricates more objects.
2022-10-24 05:51:41 +08:00
David Taylor cc439cf412
DEV: Apply theme template transforms to colocated components (#18704) 2022-10-21 19:05:34 +01:00
dependabot[bot] 5827d1eecc
Build(deps): Bump @babel/standalone from 7.19.3 to 7.19.6 in /app/assets/javascripts (#18689)
Bumps [@babel/standalone](https://github.com/babel/babel/tree/HEAD/packages/babel-standalone) from 7.19.3 to 7.19.6.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.19.6/packages/babel-standalone)

---
updated-dependencies:
- dependency-name: "@babel/standalone"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Tweak whitespace expectations in tests

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: David Taylor <david@taylorhq.com>
2022-10-21 14:48:27 +01:00
Alan Guo Xiang Tan 5fbd16926b
DEV: Add missing test case for `TopicGuardian#can_see_topic?` (#18694)
Staged users are allowed to view topics they created in a read restricted category
when category has `Category#email_in` and
`Category#email_in_allow_strangers` configured.
2022-10-21 16:40:46 +08:00
Bianca Nenciu 6b788d7329
FIX: Skip all post validations if necessary (#18625)
When PostRevisor is called with 'skip_validations: true' it can save
the post twice and one of the calls passes the correct 'validate: false'
argument, but the other one does not.
2022-10-19 18:54:32 +03:00
David Taylor c185043590
FIX: Ensure minification does not break colocated connectors (#18664)
The filenames (minus the extensions) were being used as keys in a hash to pass to Terser, which meant that colocated connector files would overwrite each other. This commit moves the un-colocating earlier in the pipeline so that the fixed filenames are passed to Terser.

Followup to be3d6a56ce
2022-10-19 10:49:01 +01:00
David Taylor be3d6a56ce
DEV: Introduce minification and source maps for Theme JS (#18646)
Theme javascript is now minified using Terser, just like our core/plugin JS bundles. This reduces the amount of data sent over the network.

This commit also introduces sourcemaps for theme JS. Browser developer tools will now be able show each source file separately when browsing, and also in backtraces.

For theme test JS, the sourcemap is inlined for simplicity. Network load is not a concern for tests.
2022-10-18 18:20:10 +01:00
David Taylor cb87067c77 DEV: Introduce support for template colocation in themes 2022-10-18 10:10:49 +01:00
David Taylor 65a5c84a92 DEV: perform theme extra_js compilation all together
Previously, compiling theme 'extra_js' was done with a number of steps. Each theme_field would be compiled into its own value_baked column, and then the JavascriptCache content would be built by concatenating all of those compiled values.

This commit streamlines things by removing the value_baked step. The raw value of all extra_js theme_fields are passed directly to the ThemeJavascriptCompiler, and then the result is stored in the JavascriptCache.

In itself, this commit should not cause any behavior change. It is designed to open the door to more advanced compilation features which have interdependencies between different source files (e.g. template colocation, sourcemaps).
2022-10-18 10:10:49 +01:00
Alan Guo Xiang Tan d1304ecb46
DEV: Move sidebar site settings to the sidebar category (#18596)
Sidebar category is only shown in the admin panel when `enable_experimental_sidebar_hamburger` site setting is enabled
2022-10-17 10:33:15 +08:00
David Taylor e0a6d12c55
Use service account credentials for fetching google hd groups (#18329)
The previous implementation would attempt to fetch groups using the end-user's Google auth token. This only worked for admin accounts, or users with 'delegated' access to the `admin.directory.group.readonly` API.

This commit changes the approach to use a single 'service account' for fetching the groups. This removes the need to add permissions to all regular user accounts. I'll be updating the [meta docs](https://meta.discourse.org/t/226850) with instructions on setting up the service account.

This is technically a breaking change in behavior, but the existing implementation was marked experimental, and is currently unusable in production google workspace environments.
2022-10-13 16:04:42 +01:00
Vinoth Kannan 7f167de10b
FIX: remove `nil` items before sorting the sha1 string array. (#18571)
Previously, when the array had both nil and string values it returned the error "comparison of NilClass with String failed". Now I added the `.compact` method to prevent this issue as per @martin-brennan's suggestion https://github.com/discourse/discourse/pull/18431#discussion_r984204788
2022-10-13 02:43:56 +05:30
Bianca Nenciu 33087f0bdf
A11Y: Set role=presentation if alt attr is missing (#18546)
This applies to all images posted that do not have a user description.
2022-10-12 14:07:37 +03:00
Rafael dos Santos Silva 6888eb5c2d
Revert "Revert "FEATURE: Preload resources via link header (#18475)" (#18511)" (#18531)
* Revert "Revert "FEATURE: Preload resources via link header (#18475)" (#18511)"

This reverts commit 95a57f7e0c.

* put behind feature flag

* env -> global setting

* declare global setting

* forgot one spot
2022-10-11 20:11:44 -03:00
Jacob Michalskie 36f7fbebdc
DEV: Allow ManagedAuthenticator classes to match by username (#18517) 2022-10-11 11:25:13 +01:00
Martin Brennan cab173d3b5
DEV: Add specs for hashtags related code (#18526)
Adding a couple of small specs to code around hashtags
for tags and categories since I will be working in this
area soon and these were not covered.
2022-10-11 09:04:51 +10:00
Blake Erickson e320bbe513
FEATURE: Adds seeded default categories to the sidebar (#18512) 2022-10-07 16:02:03 -06:00
Rafael dos Santos Silva 95a57f7e0c
Revert "FEATURE: Preload resources via link header (#18475)" (#18511)
This reverts commit 2d1dbc6f96.

We need to increase nginx proxy buffer to land this.
2022-10-07 15:08:40 -03:00
Rafael dos Santos Silva 2d1dbc6f96
FEATURE: Preload resources via link header (#18475)
Experiment moving from preload tags in the document head to preload information the the response headers.

While this is a minor improvement in most browsers (headers are parsed before the response body), this allows smart proxies like Cloudflare to "learn" from those headers and build HTTP 103 Early Hints for subsequent requests to the same URI, which will allow the user agent to download and parse our JS/CSS while we are waiting for the server to generate and stream the HTML response.

Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
2022-10-07 13:19:50 -03:00
Krzysztof Kotlarek e6b5b6eae3
DEV: update to patched discourse-seed-fu (#18493)
Original seed-fu was cloned and patched with David's fix 576b69a368
2022-10-07 09:16:04 +11:00
Bianca Nenciu 73e9875a1d
FEATURE: Handle oneboxes for complex GitHub URLs (#18474)
GitHub PR URLs can link to a commit of the PR, a comment or a review
discussion.
2022-10-06 20:26:04 +03:00
Alan Guo Xiang Tan f3392a5a81
DEV: Avoid configuring Rails configuration in tests (#18499) 2022-10-06 15:45:19 +08:00
Arpit Jalan 8ae1edeb79
FIX: do not prefill default site title value on wizard introduction step (#18496) 2022-10-06 12:02:48 +05:30
Martin Brennan 6d7abc1c85
FIX: Make sure first admin users are added to auto groups (#18494)
When a user with an email matching those inside the
DISCOURSE_DEVELOPER_EMAILS env var log in, we make
them into admin users if they are not already. This
is used when setting up the first admin user for
self-hosters, since the discourse-setup script sets
the provided admin emails into DISCOURSE_DEVELOPER_EMAILS.

The issue being fixed here is that the new admins were
not being automatically added to the staff and admins
automatic groups, which was causing issues with the site
settings that are group_list based that don't have an explicit
staff override. All we need to do is refresh the automatic
staff, admin groups when admin is granted for the user.
2022-10-06 15:16:38 +10:00
Martin Brennan f5194aadd3
DEV: Remove usages of enable_personal_messages (#18437)
cf. e62e93f83a

This PR also makes it so `bot` (negative ID) and `system` users are always allowed
to send PMs, since the old conditional was just based on `enable_personal_messages`
2022-10-05 10:50:20 +10:00
Osama Sayegh 2d391565e4
FIX: Skip quality title validations for static topics when edited by admin (#18468)
Static topics are the seeded topics that are automatically created for every Discourse instance to hold the content for the FAQ, ToS and Privacy pages. These topics are allowed to bypass the minimum title length checks when they're edited by admins:

ba27ee1637/app/assets/javascripts/discourse/app/models/composer.js (L487-L496)

However, on the server-side, the "quality title" validations aren't skipped for static topics and that can cause confusion for admins when they change the title of a static topic to something that's short enough to fail the quality title validations. This commit ignores all quality title validations on static topics when they're edited by admins.

Internal topic: t/75745.
2022-10-04 21:55:21 +03:00
Selase Krakani 0c38757250
FIX: Revert recursively tag lookup with missing ancestor tags (#18439)
This reverts commit 049f8569d8.

To be revisited with a more comprehensive solution covering parent
selection when multiple parents exist.
2022-09-30 08:28:09 +00:00
Vinoth Kannan 6ebd2cecda
FIX: missing theme upload should not break precompile process. (#18431)
Previously, if an active default theme's upload record went missing then it will break the site and cause downtime.
2022-09-30 10:48:26 +05:30
Bianca Nenciu f60e6837c6
FEATURE: Add setting to always confirm old email (#18417)
By default, only staff members have to confirm their old email when
changing it. This commit adds a site setting that when enabled will
always ask the user to confirm old email.
2022-09-30 00:49:17 +03:00
Jarek Radosz b27d5626d2
SECURITY: Prevent arbitrary file write when decompressing files (#18421)
* SECURITY: Prevent arbitrary file write when decompressing files
* FIX: Allow decompressing files into symlinked directories

Co-authored-by: OsamaSayegh <asooomaasoooma90@gmail.com>
Co-authored-by: Gerhard Schlager <gerhard.schlager@discourse.org>
2022-09-29 20:00:38 +02:00
Martin Brennan 8ebd5edd1e
DEV: Rename secure_media to secure_uploads (#18376)
This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality.

This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site.

Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing.

This also keeps compatibility with the `secure-media-uploads` path, and changes new
secure URLs to be `secure-uploads`.

Deprecated settings:

* secure_media -> secure_uploads
* secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails
* secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 09:24:33 +10:00
Arpit Jalan 2ee721f8aa
FEATURE: add composer warning when user haven't been seen in a long time (#18340)
* FEATURE: add composer warning when user haven't been seen in a long time

When a user creates a PM and adds a recipient that hasn't been seen in a
long time then we'll now show a warning in composer indicating that the
user hasn't been seen in a long time.
2022-09-27 22:06:40 +05:30
Selase Krakani 049f8569d8
FIX: Recursively tag topics with missing ancestor tags (#18344)
* FIX: Recursively tag topics with missing ancestor tags

Given only a child tag, walk up the ancestry chain, get all of it's
ancestors for use in tagging a topic

* FIX: Ensure only one parent tag is returned for topic tagging

Current implementation selects and return first parent tag if child tag
has multiple parents.

This change updates recursive parent tag implementation to only return
parent tags via only one ancestry line.

* DEV: Add test case for tag cycles

Given we aren't performing a strict graph traversal to get a tag's
parent, cycles do not have any effect on the tags returned for topic
tagging.
2022-09-27 12:04:16 +00:00
Vinoth Kannan 076abe46fa
FEATURE: new site setting to set locale from cookie for anonymous users. (#18377)
This new hidden default-disabled site setting `set_locale_from_cookie` will set locale from anonymous user's cookie value.
2022-09-27 14:26:06 +05:30
Martin Brennan 7152345ee7
FIX: list_suggested_for conditional for personal_message_enabled_groups (#18373)
Follow-up to e62e93f83a,
misplaced a bracket and changed the meaning of the conditional.
2022-09-27 16:54:44 +10:00
Martin Brennan e62e93f83a
FEATURE: Introduce personal_message_enabled_groups setting (#18042)
This will replace `enable_personal_messages` and
`min_trust_to_send_messages`, this commit introduces
the setting `personal_message_enabled_groups`
and uses it in all places that `enable_personal_messages`
and `min_trust_to_send_messages` currently apply.

A migration is included to set `personal_message_enabled_groups`
based on the following rules:

* If `enable_personal_messages` was false, then set
  `personal_message_enabled_groups` to `3`, which is
  the staff auto group
* If `min_trust_to_send_messages` is not default (1)
  and the above condition is false, then set the
  `personal_message_enabled_groups` setting to
  the appropriate auto group based on the trust level
* Otherwise just set `personal_message_enabled_groups` to
  11 which is the TL1 auto group

After follow-up PRs to plugins using these old settings, we will be
able to drop the old settings from core, in the meantime I've added
 DEPRECATED notices to their descriptions and added them
to the deprecated site settings list.

This commit also introduces a `_map` shortcut method definition
for all `group_list` site settings, e.g. `SiteSetting.personal_message_enabled_groups`
also has `SiteSetting.personal_message_enabled_groups_map` available,
which automatically splits the setting by `|` and converts it into
an array of integers.
2022-09-26 13:58:40 +10:00
Martin Brennan e3d495850d
FEATURE: Overhaul email threading (#17996)
See https://meta.discourse.org/t/discourse-email-messages-are-incorrectly-threaded/233499
for thorough reasoning.

This commit changes how we generate Message-IDs and do email
threading for emails sent from Discourse. The main changes are
as follows:

* Introduce an outbound_message_id column on Post that
  is either a) filled with a Discourse-generated Message-ID
  the first time that post is used for an outbound email
  or b) filled with an original Message-ID from an external
  mail client or service if the post was created from an
  incoming email.
* Change Discourse-generated Message-IDs to be more consistent
  and static, in the format `discourse/post/:post_id@:host`
* Do not send References or In-Reply-To headers for emails sent
  for the OP of topics.
* Make sure that In-Reply-To is filled with either a) the OP's
  Message-ID if the post is not a direct reply or b) the parent
  post's Message-ID
* Make sure that In-Reply-To has all referenced post's Message-IDs
* Make sure that References is filled with a chain of Message-IDs
  from the OP down to the parent post of the new post.

We also are keeping X-Discourse-Post-Id and X-Discourse-Topic-Id,
headers that we previously removed, for easier visual debugging
of outbound emails.

Finally, we backfill the `outbound_message_id` for posts that have
a linked `IncomingEmail` record, using the `message_id` of that record.
We do not need to do that for posts that don't have an incoming email
since they are backfilled at runtime if `outbound_message_id` is missing.
2022-09-26 09:14:24 +10:00
Loïc Guitaut 26fe047724 DEV: Use AR enums in reviewables related code
This is the first patch of many to replace our custom enums in Ruby by
the ones provided by `ActiveRecord`.
2022-09-22 14:44:27 +02:00
David Taylor d0243f741e
UX: Use dominant color as image loading placeholder (#18248)
We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images.

This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly.

Dominant color will be calculated:
1. When a new upload is created
2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored
3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job)

Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 10:28:17 +01:00
Gerhard Schlager f91426a7fa
FIX: Empty inline BBCodes were broken (#18276)
Upgrading to Markdown.it v13 broke empty inline BBCodes. This works around the problem by adding an empty token before a closing token if the previous token was a BBCode token.

It also removes the unused `jump` attribute which was removed in Markdown.it v12.3
2022-09-20 09:50:22 +10:00
Jarek Radosz 08e63ddab2
DEV: Fix spec file name (#18227)
Match the impl file name
2022-09-12 14:03:23 +02:00
David Taylor 7e74dd0afe
DEV: Use DiscourseJsProcessor for theme template compilation (#18135)
Previously we were relying on a highly-customized version of the unmaintained Barber gem for theme template compilation. This commit switches us to use our own DiscourseJsProcessor, which makes use of more modern patterns and will be easier to maintain going forward.

In summary:
- Refactors DiscourseJsProcessor to move multiline JS heredocs into a companion `discourse-js-processor.js` file
- Use MiniRacer's `.call` method to avoid manually escaping JS strings
- Move Theme template AST transformers into DiscourseJsProcessor, and formalise interface for extending RawHandlebars AST transformations
- Update Ember template compilation to use a babel-based approach, just like Ember CLI. This gives each template its own ES6 module rather than directly assigning `Ember.TEMPLATES` values
- Improve testing of template compilation (and move some tests from `theme_javascript_compiler_spec.rb` to `discourse_js_processor_spec.rb`
2022-09-01 11:50:46 +01:00
Arpit Jalan 3aaf4dcfd0
FIX: hide welcome topic banner as soon as the welcome topic is edited (#18125)
* FIX: hide welcome topic banner as soon as the welcome topic is edited

This commit adds a message bus listener on client to hide the welcome
topic banner as soon as the welcome topic is edited.

* update test

* only subscribe when show_welcome_topic_banner is true

* Do not lookup for messageBus service if it's not required

* Remove unneeded code

* Cache result for Site.show_welcome_topic_banner

* Update tests per latest changes

* Changes per PR review
2022-09-01 09:44:21 +05:30
Jarek Radosz 8fa9f0cf92
DEV: Fix a flaky spec (#18146)
In some cases the topic of the fabricated post can be titled "This is a test topic 777" which matches the search query "#777"
2022-08-31 20:52:57 +02:00
David Taylor 7eaa4f6ad3
DEV: Add test for EmberCli#ember_version (#18140)
Followup to 6570fed175
2022-08-30 20:33:08 +01:00
Blake Erickson 00539307e1
FIX: Only seed general category on new sites (#18130)
* FIX: Only seed general category on new sites

If the site already has human users (users with an id > 0) don't seed
the categories.

Follow up to: a6ad74c759

* use human_users scope
2022-08-29 18:23:14 -06:00
Martin Brennan b58d168f05
FIX: Do not save default auto_delete_preference for bookmark (#18119)
We don't want to save the auto_delete_preference for bookmarks to the
user options if it was passed through as nil from the frontend,
this leads to confusion for the end user since they did not explicitly set it.

It's fine to create the bookmark with the default of "never" if no
auto_delete_preference is provided since it applies only to the
single bookmark, not future bookmarks.
2022-08-30 09:21:41 +10:00
Blake Erickson eba8b8d34d
FEATURE: Replace Lounge with General Category (#18097)
- Seed the General category so that the general chat channel will have
  a home
- Do not seed the Lounge category anymore
- Move the "Welcome to Site" topic to the General category
2022-08-29 13:05:41 -06:00
David Taylor e16c8ea2e7
DEV: Support inline-hbs compilation in themes (#18112)
This commit makes a number of improvements to the DiscourseJsProcessor:

1. Remove dependence on the out-of-date Ember template compiler from the ember-rails gem; switch to modern template compiler
2. Refactor to make use of a proper module system with `define`/`require`
3. Introduce `babel-plugin-ember-template-compilation` to enable inline hbs compilation

The `mini-loader` is upgraded to support relative lookup and `require.has`, so that these new JS packages work correctly.
2022-08-29 19:53:42 +01:00
Bianca Nenciu 33ea8b4756
FIX: Skip topic allowed user for small actions (#18075)
Topic allowed user records were created for small actions, which lead to
the system user being invited in many private topics when the user
removed themselves or if a group was invited but some members already
had access.

This commits skips creating topic allowed user. They are already skipped
for the whisper posts.
2022-08-29 13:01:16 +03:00
Arpit Jalan c4bb15441d
FIX: reload styling changes for wizard styling step (#18121) 2022-08-29 15:10:47 +05:30
Alan Guo Xiang Tan acb6222caf
Revert "DEV: Upgrade Redis to 4.8" (#18099)
This reverts commit 0b5a0fd857.

Pending release of Sidekiq 6.5.6 so that we don't get depreciation
warnings
2022-08-26 15:16:50 +10:00
Alan Guo Xiang Tan 0b5a0fd857 DEV: Upgrade Redis to 4.8 2022-08-25 11:11:08 +08:00
David Taylor e7a84948b9
FIX: Avoid duplicate topic-list requests (#18073)
When preloading topic_list data we were giving it a 'preload key' which was loosely based on the parameters of the list. However, it did not include all parameters, and mismatches between client/server-side logic would cause the preloaded data to be ignored.

This commit simplifies things by using a single key for all topic_list preloading. This works on the assumption that "The first topic_list the JS app will load is the one which was preloaded". That assumption also existed to some extent in the old design, so we don't expect any regressions here.
2022-08-24 11:54:01 +01:00
Gerhard Schlager 9ff13cee14
FIX: Backup/Restore didn't use correct Redis namespace in multisite (#18060)
In a multisite Discourse reported that no backup is running after 60 seconds because the Redis key expired. Also, the thread that listens for a shutdown signal stopped running immediately because it didn't detect a running operation.
2022-08-24 01:43:42 +02:00
Sam df04462475
FIX: ignore malformed HTML for title extraction (#18040)
Certain HTML can be rejected by nokogumbo, specifically cases where there
are enormous amounts of attributes

This ensures that malformed HTML is simply skipped instead of leaking out
an exception and terminating downstream processes.
2022-08-23 15:03:57 +10:00
Bianca Nenciu ec8306835d
FIX: Improvements to like button for archived topics (#17951)
* FIX: Do not allow to remove like if topic is archived

* FIX: Always show like button

The like button used to be hidden if the topic was archived and it had
no likes. This commit changes that to always show the like button, but
with a not-allowed cursor if the topic is archived.
2022-08-22 14:58:02 +03:00
Jarek Radosz aa59229c80
DEV: Fix busted twitter onebox spec (#18009)
Added in #17519. Was causing spec failures.
2022-08-20 11:19:19 +02:00
jbrw 73b2522261
FIX: Allow `match_all_tags` to be passed as a URL param (#17972)
`TopicQueryParams` allows for `match_all_tags` to be passed as a query parameter. `TagsController` forces the value to be true.

This change allows a value to be passed, and only sets it to true if no value has been set. It then uses `ActiveModel::Type::Boolean.new.cast` to compare the value.
2022-08-19 15:41:56 -04:00
Bianca Nenciu 707034bc75
FIX: Limit new and existent staged users for email topics (#17970)
The maximum_staged_users_per_email site setting controls how many
staged users will be invited to the topic created from an incoming
email. Previously, it counted only the new staged users.
2022-08-18 18:19:20 +03:00
Bianca Nenciu b082f459c9
FEATURE: Limit maximum recipients for group emails (#17971)
New maximum_recipients_per_new_group_email site setting can be used to
prevent spam group emails with many recipients.
2022-08-18 18:18:58 +03:00
Bianca Nenciu 626d50c15c
FIX: Disable Twitter onebox without API support (#17519)
Twitter removed OpenGraph tags from their pages. We can no longer
extract all the information (for example, the quoted tweet) we need
to render Oneboxes without using their API.
2022-08-17 18:32:48 +03:00
Arpit Jalan 3a21618e4e
FEATURE: allow wizard checkbox field to be disabled (#17916)
* FEATURE: allow wizard checkbox field to be disabled

* Changes per review feedback
2022-08-15 05:52:07 +05:30
Bianca Nenciu c789c689c2
FIX: Remove dead and large images from oneboxes (#17868)
Dead and large images are replaced with a placeholder, either a broken
chain icon or a short text. This commit no longer applies this
transformation for images inside Oneboxes, but removes them instead.
2022-08-11 19:09:48 +03:00
Loïc Guitaut afe7785141 FIX: Swallow SSL errors when generating oneboxes 2022-08-09 16:54:45 +02:00
David Taylor 3c81683955 DEV: Rename `UriHelper.escape_uri` to `.normalized_encode`
This is a much better description of its function. It performs idempotent normalization of a URL. If consumers truly need to `encode` a URL (including double-encoding of existing encoded entities), they can use the existing `.encode` method.
2022-08-09 11:55:25 +01:00
Sam Saffron f0a0252526 FIX: broken onebox images due to url normalization bugs
normalized_encode in addressable has a number of issues, including https://github.com/sporkmonger/addressable/issues/472

To temporaily work around those issues for the majority of cases, we try parsing with `::URI`. If that fails (e.g. due to non-ascii characters) then we will fall back to addressable.

Hopefully we can simplify this back to `Addressable::URI.normalized_encode` in the future.

This commit also adds support for unicode domain names and emoji domain names with escape_uri.

This removes an unneeded hack checking for pre-signed urls, which are now handled by the general case due to starting off valid and only being minimally normalized. Previous test case continues to pass.

UrlHelper.s3_presigned_url? which was somewhat wide was removed.
2022-08-09 11:55:25 +01:00
Loïc Guitaut 00b3f0e2c4 DEV: Make the first argument to the top-level describe a constant in specs 2022-08-08 18:07:49 +02:00
Loïc Guitaut 3eaac56797 DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
Daniel Waterworth 83d3543e33
DEV: Guardians aren't active record objects, so shouldn't use fab! (#17789) 2022-08-03 19:17:40 -05:00
Martin Brennan 7cab189b1e
FIX: Handle actor not having preferences in UserCommScreener (#17790)
Followup to d66115d918

* Makes sure the `actor_preferences` all initialize with an empty array instead of nil if there are no preferences e.g. the actor is not ignoring anyone
* If the actor has disabled all PMs make `actor_disallowing_pms?` always return true
2022-08-04 10:16:54 +10:00
Sam 28968d9977
FIX: don't memoize site setting in guardian (#17788)
* FIX: don't memoize site setting in guardian

Memoizing site settings can make tests more fragile and harder to debug


Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2022-08-04 10:07:12 +10:00
Martin Brennan d66115d918
DEV: UserCommScreener fine-grained actor improvements (#17737)
This commit introduces several fine-grained methods
to UserCommScreener which can be used to show the actor
who they are ignoring/muting/blocking DMs from in order
to prevent them initiating conversation with those users
or to display relevant information in the UI to the
actor.

This will be used in a companion PR in discourse-chat,
and is a follow up to 74584ff3ca

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
Co-authored-by: Osama Sayegh <asooomaasoooma90@gmail.com>
2022-08-04 09:06:51 +10:00
Sam ff53f2c7bc
FIX: scope memoization of category moderation per category (#17786) 2022-08-04 08:43:19 +10:00
Sam 3b42e69174
FIX: avoid usage of dig when looking for job class (#17772)
`{a: "a"}.dig(:a, :b)` will result in an exception, since ruby assumes that `"a"` will be another hash it can look up the `:b` key on.
2022-08-03 14:28:46 +10:00
Sam bfe502012d
FEATURE: track stats around failing scheduled jobs (#17769)
* FEATURE: track stats around failing scheduled jobs

Discourse.job_exception_stats can now be used to gather stats around how
many regular scheduled jobs failed in the current process.

This will be consumed by the Prometheus plugin and potentially other
monitoring plugins.
2022-08-03 12:53:26 +10:00
Arpit Jalan 5cbf0255a6
FIX: do not overwrite top_menu site setting in wizard styling step (#17743) 2022-08-02 19:29:34 +05:30
Selase Krakani 862007fb18
FEATURE: Add support for case-sensitive Watched Words (#17445)
* FEATURE: Add case-sensitivity flag to watched_words

Currently, all watched words are matched case-insensitively. This flag
allows a watched word to be flagged for case-sensitive matching.
To allow allow for backwards compatibility the flag is set to false by
default.

* FEATURE: Support case-sensitive creation of Watched Words via API

Extend admin creation and upload of Watched Words to support case
sensitive flag. This lays the ground work for supporting
case-insensitive matching of Watched Words.

Support for an extra column has also been introduced for the Watched
Words upload CSV file. The new column structure is as follows:

 word,replacement,case_sentive

* FEATURE: Enable case-sensitive matching of Watched Words

WordWatcher's word_matcher_regexp now returns a list of regular
expressions instead of one case-insensitive regular expression.

With the ability to flag a Watched Word as case-sensitive, an action
can have words of both sensitivities.This makes the use of the global
Regexp::IGNORECASE flag added to all words problematic.

To get around platform limitations around the use of subexpression level
switches/flags, a list of regular expressions is returned instead, one for each
case sensitivity.

Word matching has also been updated to use this list of regular expressions
instead of one.

* FEATURE: Use case-sensitive regular expressions for Watched Words

Update Watched Words regular expressions matching and processing to handle
the extra metadata which comes along with the introduction of
case-sensitive Watched Words.

This allows case-sensitive Watched Words to matched as such.

* DEV: Simplify type casting of case-sensitive flag from uploads

Use builtin semantics instead of a custom method for converting
string case flags in uploaded Watched Words to boolean.

* UX: Add case-sensitivity details to Admin Watched Words UI

Update Watched Word form to include a toggle for case-sensitivity.
This also adds support for, case-sensitive testing and matching of  Watched Word
in the admin UI.

* DEV: Code improvements from review feedback

 - Extract watched word regex creation out to a utility function
 - Make JS array presence check more explicit and readable

* DEV: Extract Watched Word regex creation to utility function

Clean-up work from review feedback. Reduce code duplication.

* DEV: Rename word_matcher_regexp to word_matcher_regexp_list

Since a list is returned now instead of a single regular expression,
change `word_matcher_regexp` to `word_matcher_regexp_list` to better communicate
this change.

* DEV:  Incorporate WordWatcher updates from upstream

Resolve conflicts and ensure apply_to_text does not remove non-word characters in matches
that aren't at the beginning of the line.
2022-08-02 10:06:03 +02:00
Martin Brennan f4b45df83f
FIX: UserCommScreener filter acting user ID from target user IDs (#17702)
Fixes edge case from fa5f3e228c.
In case the acting user is sent in with the target_user_ids,
we do not need to load those preferences, because even if the
acting user is preventing PMs or muting etc they need to always be able to
send themselves messages.
2022-07-28 13:04:24 +10:00
Phil Pirozhkov 493d437e79
Add RSpec 4 compatibility (#17652)
* Remove outdated option

04078317ba

* Use the non-globally exposed RSpec syntax

https://github.com/rspec/rspec-core/pull/2803

* Use the non-globally exposed RSpec syntax, cont

https://github.com/rspec/rspec-core/pull/2803

* Comply to strict predicate matchers

See:
 - https://github.com/rspec/rspec-expectations/pull/1195
 - https://github.com/rspec/rspec-expectations/pull/1196
 - https://github.com/rspec/rspec-expectations/pull/1277
2022-07-28 10:27:38 +08:00
Loïc Guitaut 296aad430a DEV: Use `describe` for methods in specs 2022-07-27 16:35:27 +02:00
Loïc Guitaut c9d22b643f DEV: Fix flaky `FinalDestination` specs 2022-07-27 14:52:55 +02:00
Bianca Nenciu bc476978e8
FIX: Support for group everyone in tag setting (#17669)
The "everyone" group is an automatic group and GroupUser records do not
exist for it. This commit allows all users if the group everyone is one
of the groups in the setting "pm_tags_allowed_for_groups".
2022-07-27 15:44:41 +03:00
Blake Erickson 8b08b9a763
FIX: Rejected emails should not be cleaned up before their logs (#17648)
* FIX: Rejected emails should not be cleaned up before their logs

If we delete the rejected emails before we delete their associated logs
we will receive 404 errors trying to inspect an email message for that
log.

* don't add a blank line

* test for max value as well

* pr cleanup and add migration

* Fix failing test
2022-07-27 07:28:44 +01:00
Arpit Jalan 10f200a5d3
FEATURE: revamped wizard (#17477)
* FEATURE: revamped wizard

* UX: Wizard redesign (#17381)

* UX: Step 1-2

* swap out images

* UX: Finalize all steps

* UX: mobile

* UX: Fix test

* more test

* DEV: remove unneeded wizard components

* DEV: fix wizard tests

* DEV: update rails tests for new wizard

* Remove empty hbs files that were created because of rebase

* Fixes for rebase

* Fix wizard image link

* More rebase fixes

* Fix rails tests

* FIX: Update preview for new color schemes: (#17481)

* UX: make layout more responsive, update images

* fix typo

* DEV: move discourse logo svg to template only component

* DEV: formatting improvements

* Remove unneeded files

* Add tests for privacy step

* Fix banner image height for step "ready"

Co-authored-by: Jordan Vidrine <30537603+jordanvidrine@users.noreply.github.com>
Co-authored-by: awesomerobot <kris.aubuchon@discourse.org>
2022-07-27 06:53:01 +05:30
Loïc Guitaut 91b6b5eee7 DEV: Don’t use `change { … }.by(0)` in specs 2022-07-26 10:34:15 +02:00
Jarek Radosz 59686c1cb8
DEV: Fix a flaky FinalDestination spec (#17655) 2022-07-26 01:28:53 +02:00
Loïc Guitaut db9245d188
DEV: Fix `FinalDestination` specs (#17642) 2022-07-25 11:38:54 +01:00
Loïc Guitaut 1f5682b7d7 FIX: Don’t raise an error on onebox timeouts
Currently when generating oneboxes if the connection timeouts and we’re
using the `FinalDestination#get` method, then it raises an exception.

We already catch this exception when using the
`FinalDestination#resolve` method so this patch just applies the same
logic to `FinalDestination#get`.
2022-07-25 10:41:46 +02:00
Jarek Radosz 02a96a53fb
DEV: Be specific about time units (#17629)
All other tests that are setting grade_period use either unitless `0`, `1.minute` or `5.minutes` so it wasn't clear if `5` was meant to be seconds (it was)
2022-07-25 02:32:15 +02:00
Jarek Radosz 6cae26c499
DEV: Always `await` for `publishToMessageBus` (#17626)
(+ typos)
2022-07-24 19:55:58 +02:00
Osama Sayegh 306dca3a48
DEV: Add a helper method to clear every possible theme cache (#17585)
Our theme system is very complex and it can take a while to figure out how to invalidate the various types of caches that are used throughout the theme system. So, having a single helper method that invalidates everything can be useful in emergency situations where there is no time to read through the code and figure out how to clear the various caches.

Internal ticket: t64732.
2022-07-22 09:46:52 +03:00
Vinoth Kannan eef58f510c
FEATURE: users with no posts shouldn't able to edit username after the allowed period. (#17583)
We are no longer going to let users change their username after the allowed `username_change_period` when there are no posts created by the user.
2022-07-21 04:46:15 +05:30
Bianca Nenciu f75a99e932
FIX: Notify tag watchers when publishing topic (#17576)
When a topic was published from a shared draft and it had tags, the
users watching the tags were not notified. The problem was that the
topics are usually created in a secret category and publishing it just
moves an existent topic to the target category, without making any
changes to the tags.
2022-07-20 19:07:18 +03:00
Bianca Nenciu e7f04a8674
FIX: Use URI#merge to merge base and relative URLs (#17454)
The old implementation did not handle all cases, such as the case when
`src` is a relative URL that starts with `..`.
2022-07-18 14:17:54 +03:00
David Taylor fab1c00c8f
DEV: Drop the deprecated `themeSettings.blah` syntax (#17394)
This syntax has been printing deprecation messages since 880311dd4d
2022-07-18 10:10:23 +01:00
Bianca Nenciu e8d802eb86
FIX: Check if tags edit notifications are disabled (#17499)
Tag edit notifications are either created by PostActionNotifier or
PostRevisor. PostActionNotifier already checks if the site setting is
enabled. but PostRevisor scheduled a NotifyTagChange job without
checking disable_tags_edit_notifications.
2022-07-15 16:14:46 +10:00
Martin Brennan 098ab29d41
FEATURE: Add plugin API to register About stat group (#17442)
This commit introduces a new plugin API to register
a group of stats that will be included in about.json
and also conditionally in the site about UI at /about.

The usage is like this:

```ruby
register_about_stat_group("chat_messages", show_in_ui: true) do
  {
    last_day: 1,
    "7_days" => 10,
    "30_days" => 100,
    count: 1000,
    previous_30_days: 120
  }
end
```

In reality the stats will be generated any way the implementer
chooses within the plugin. The `last_day`, `7_days`, `30_days,` and `count`
keys must be present but apart from that additional stats may be added.
Only those core 4 stat keys will be shown in the UI, but everything will be shown
in about.json.

The stat group name is used to prefix the stats in about.json like so:

```json
"chat_messages_last_day": 2322,
"chat_messages_7_days": 2322,
"chat_messages_30_days": 2322,
"chat_messages_count": 2322,
```

The `show_in_ui` option (default false) is used to determine whether the
group of stats is shown on the site About page in the Site Statistics
table. Some stats may be needed purely for reporting purposes and thus
do not need to be shown in the UI to admins/users. An extension to the Site
serializer, `displayed_about_plugin_stat_groups`, has been added so this
can be inspected on the client-side.
2022-07-15 13:16:00 +10:00
Martin Brennan 6b2ea1b47b
FIX: Change UserCommScreener to use user_ids (#17489)
It makes more sense to use user_ids for the UserCommScreener
introduced in fa5f3e228c since
in most cases the ID will be available, not the username. This
was discovered while starting work on a plugin that will
use this. In the cases where only usernames are available
the extra query is negligble.
2022-07-14 15:23:09 +10:00
Martin Brennan fa5f3e228c
DEV: Refactor user mute/ignore/disallow PM checks into central class (#17366)
The idea behind this refactor is to centralise all of the user ignoring / muting / disallow PM checks in a single place, so they can be used consistently in core as well as for plugins like chat, while improving the main bulk of the checks to run in a single fast non-AR query.

Also fixed up the invite error when someone is muting/ignoring the user that is trying to invite them to the topic.
2022-07-13 13:58:01 +10:00
Jarek Radosz 87353faac6
DEV: Implement distributed mutex in lua (#16228)
The rationale behind this was mostly to stop using `redis.synchronize` (now removed in redis gem 4.6)
2022-07-11 14:16:37 +02:00
Jordan Vidrine fec3df60cb
DEV: Add more color schemes (#17398) 2022-07-08 16:46:32 -05:00
David Taylor 8e0b1ee74d
DEV: Future-proof ThemeJavascriptCompiler spec (#17393)
Previously the spec was hardcoded to expect a certain compiled result. The precise result will change as we update to more recent Ember versions.

Instead, we can compile via the special theme compiler, and then compile our expected result via the normal compiler. If they match, then things are working as intended. This technique should be safe across template-compiler upgrades.
2022-07-08 16:28:46 +01:00
Rafael dos Santos Silva 75e40baa64
FIX: min/max username length limits weren't validated (#17382)
* FIX: min/max username length limits weren't validated

The custom validators introduced in e0d7cda made so we ignored the mix
and max values set on site_settings.yml. That change allowed admins to
set values outside of the range defined on the yaml file.

Related to https://meta.discourse.org/t/group-names-with-more-than-60-characters-broken/232115?u=falco

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2022-07-08 12:00:47 -03:00
Gerhard Schlager 8bdbefe0e0
FIX: Logout could fail due to cached user (#17325)
Logging out failed when the current user was cached by an instance of `Auth::DefaultCurrentUserProvider` and `#log_off_user` was called on a different instance of that class.

Co-authored-by: Sam <sam.saffron@gmail.com>
2022-07-04 17:01:19 +02:00
Jarek Radosz c5f0aa2f32
DEV: Remove handlebars from the old package.json (#17319) 2022-07-04 15:05:03 +02:00
Gerhard Schlager caa0247f5c
FIX: Incorrect `currentUser` could be cached for requests with API key (#17279)
This happened when a middleware accessed the `currentUser` before a controller had a chance to populate the `action_dispatch.request.path_parameters` env variable. In that case Discourse would always cache `nil` as `currentUser`.
2022-07-01 10:18:24 +02:00
Krzysztof Kotlarek 09932738e5
FEATURE: whispers available for groups (#17170)
Before, whispers were only available for staff members.

Config has been changed to allow to configure privileged groups with access to whispers. Post migration was added to move from the old setting into the new one.

I considered having a boolean column `whisperer` on user model similar to `admin/moderator` for performance reason. Finally, I decided to keep looking for groups as queries are only done for current user and didn't notice any N+1 queries.
2022-06-30 10:18:12 +10:00
Sam 6ecfdc8f55
FIX: keep unique post checks separate for PMs vs topics (#17272)
This allows for people to use PMs for drafting and then post them on topics.

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2022-06-29 15:35:07 +10:00
Gerhard Schlager 9d870f151c
FIX: Uploading large files (> 5GB) failed when `enable_direct_s3_uploads` is enabled (#16724)
Larger files require a multipart copy.
2022-06-28 21:30:00 +02:00
Gerhard Schlager 0bcc478635 DEV: Run some specs with fake S3 implementation instead of stubs 2022-06-28 21:27:52 +02:00
Sérgio Saquetim 5840fb5c62
UX: Update topics stats automatically (#17135)
Updates automatically data on the stats section of the topic.

It will update automatically the following information: likes, replies and last reply (timestamp and user)
2022-06-27 18:21:05 -03:00
Rafael dos Santos Silva f56c44d1c7
FEATURE: Validate tags in WatchedWords (#17254)
* FEATURE: Validate tags in WatchedWords

We didn't validate watched words automatic tagging, so it was possible
for an admin to created watched words with an empty tag list which would
result in an exception when users tried to create a new topic that
matched the misconfigured watched word.

Bug report: https://meta.discourse.org/t/lib-topic-creator-fails-when-the-word-math-appears-in-the-topic-title-or-text/231018?u=falco
2022-06-27 16:16:33 -03:00
Penar Musaraj 3baefa25b5
FIX: Use first supported type item when JSON-LD returns array (#17217) 2022-06-23 13:02:01 -04:00
Joe e82a2ce9ae
UX: Introduces a splash screen behind a hidden site setting (#17094)
This PR introduces a new hidden site setting that allows admins to display a splash screen while site assets load.

The splash screen can be enabled via the `splash_screen` hidden site setting.

This is what the splash screen currently looks like

5ceb72f085.mp4

Once site assets load, the splash screen is automatically removed.

To control the loading text that shows in the splash screen, you can change the preloader_text translation string in admin > customize > text
2022-06-22 04:35:46 +08:00
Michael Brown beda4708f6 FIX: 99999 is not a valid TCP port
Future versions of redis will validate this port number causing the tests
relying on this to fail with:

```
Redis::CommandError:
ERR Invalid master port
```

Also change from an IPv4 address that might feasibly be in use to an IPv6
random ULA address that almost *certainly* won't be.
2022-06-21 14:55:19 -04:00
Roman Rizzi e0ba35350e
FEATURE: Custom unsubscribe options (#17090)
With this change, plugins can create custom unsubscribe keys, extend the unsubscribe view with custom preferences, and decide how they are updated.
2022-06-21 15:49:47 -03:00
Alan Guo Xiang Tan 3562c598ef
DEV: Upgrade Sidekiq to 6.5 (#17142) 2022-06-21 09:23:36 +08:00
Osama Sayegh 5176c689e9
UX: Change wording for 'regular' categories to 'normal' (#17134)
At some point in the past we decided to rename the 'regular' notification state of topics/categories to 'normal'. However, some UI copy was missed when the initial renaming was done so this commit changes the spots that were missed to the new name.
2022-06-20 06:49:33 +03:00
Jarek Radosz fddd6fd5e0
DEV: Fix an rspec warning (#17123)
```
WARNING: Using `expect { }.not_to raise_error(SpecificErrorClass)` risks false positives, since literally any other error would cause the expectation to pass, including those raised by Ruby (e.g. `NoMethodError`, `NameError` and `ArgumentError`), meaning the code you are intending to test may not even get reached. Instead consider using `expect { }.not_to raise_error` or `expect { }.to raise_error(DifferentSpecificErrorClass)`. This message can be suppressed by setting: `RSpec::Expectations.configuration.on_potential_false_positives = :nothing`. Called from /var/www/discourse/spec/lib/retrieve_title_spec.rb:155:in `block (3 levels) in <main>'.
```
2022-06-17 13:45:16 +02:00
David Taylor e0ff367b49
DEV: Stop logging error response body in FileHelper (#17120)
This doesn't cope well with gzipped, binary, or large responses. Ideally we would teach FinalDestination to safely retrieve and decode some of the response body. But for now, let's remove the broken implementation.
2022-06-17 11:50:30 +01:00
Alan Guo Xiang Tan f618fdf17f
Revert "DEV: Centralize user updates to a single MessageBus channel. (#17058)" (#17115)
This reverts commit 94c3bbc2d1.

At this current point in time, we do not have enough data on whether
this centralisation is the trade-offs of coupling features into a single
channel.
2022-06-17 12:24:15 +08:00
Jarek Radosz f723b4c322
FIX: Handle sites with more than 1 JSON-LD element (#17095)
A followup to #17007
2022-06-15 02:55:55 +02:00
Martin Brennan 4d3c1ceb44
FEATURE: Log the SMTP response in EmailLog (#17056)
When sending emails with delivery_method_options -> return_response
set to true, the SMTP sending code inside Mail will return the SMTP
response when calling deliver! for mail within the app. This commit
ensures that Email::Sender captures this response if it is returned
and stores it against the EmailLog created for the sent email.

A follow up PR will make this visible within the admin email UI.
2022-06-15 10:28:30 +10:00
sansnumero f0c6dd5682
Add support for JSON LD in Onebox (#17007)
* FIX: Fix a bug that is accessing the values in a hash wrongly and write tests

I decided to write tests in order to be confident in my refactor that's in the next commit.
Meanwhile I have discovered a potential bug. The `title_attr` key was accessed as a string,
but all the keys are actually symbols so it was never evaluated to be true.

irb(main):025:0> d = {key: 'value'}
=> {:key=>"value"}
irb(main):026:0> d['key']
=> nil
irb(main):027:0> d[:key]
=> "value"

* DEV: Extract methods for readability

I will be adding a new method following the conventions in place for adding a new normalizer. And this will make the readability of the `raw` block even more difficult; so I am extracting self contained private methods beforehand.

* FEATURE: Parse JSON-LD and introduce Movie object

JSON LD data is very easily transferable to Ruby objects because they contain types. If these types are mapped to Ruby objects, it is also better to make all the parsed data very explicit and easily extendable.

JSON-LD has many more standardized item types, with a full list here: https://schema.org/docs/full.html
However in order to decrease the scope, I only adapted the movie type.

* DEV: Change inheritance between normalizers

Normalizers are not supposed to have an inheritance relationships amongst each other. They are all normalizers, but all normalizing separate protocols. This is why I chose to extract a parent class and relieve Open Graph off that responsibility. Removing the parent class altogether could also a possibility, but I am keeping the scope limited to having a more accurate representation of the normalizers while making it easier to add a new one.

* Lint changes

* Bring back the Oembed OpenGraph inheritance

There is one test that caught that this inheritance was necessary. I still think modelling wise this inheritance shouldn't exist, but this can be tackled separately.

* Return empty hash if the json received is invalid

Before this change if there was a parsing error with JSON it would throw an exception. The goal of this commit is to rescue that exception and then log a warning. I chose to use Discourse's logger wrapper `warn_exception` to have the backtrace and not just used Rails logger. I considered raising an `InvalidParameters` error however if the JSON here is invalid it should not block showing of the Onebox, so logging is enough.

* Prep to support more JSONLD schema types with case

* Extract mustache template object created from JSONLD
2022-06-13 17:32:34 +02:00
Alan Guo Xiang Tan 94c3bbc2d1
DEV: Centralize user updates to a single MessageBus channel. (#17058)
Introduces an interface to publish user updates on the server side and
helps to reduce the growing number of subscriptions on the client side.
2022-06-13 14:27:43 +08:00
Sérgio Saquetim 300f835703
DEV: Supress logs when RetrieveTitle.crawl fails with Net::ReadTimeout errors (#16971)
This PR changes the rescue block to rescue only Net::TimeoutError exceptions and removes the log line to prevent clutter the logs with errors that are ignored. Other errors can bubble up because they're errors we probably want to know about
2022-06-09 16:30:22 -03:00
Bianca Nenciu 9db8f00b3d
FEATURE: Create upload_references table (#16146)
This table holds associations between uploads and other models. This can be used to prevent removing uploads that are still in use.

* DEV: Create upload_references
* DEV: Use UploadReference instead of PostUpload
* DEV: Use UploadReference for SiteSetting
* DEV: Use UploadReference for Badge
* DEV: Use UploadReference for Category
* DEV: Use UploadReference for CustomEmoji
* DEV: Use UploadReference for Group
* DEV: Use UploadReference for ThemeField
* DEV: Use UploadReference for ThemeSetting
* DEV: Use UploadReference for User
* DEV: Use UploadReference for UserAvatar
* DEV: Use UploadReference for UserExport
* DEV: Use UploadReference for UserProfile
* DEV: Add method to extract uploads from raw text
* DEV: Use UploadReference for Draft
* DEV: Use UploadReference for ReviewableQueuedPost
* DEV: Use UploadReference for UserProfile's bio_raw
* DEV: Do not copy user uploads to upload references
* DEV: Copy post uploads again after deploy
* DEV: Use created_at and updated_at from uploads table
* FIX: Check if upload site setting is empty
* DEV: Copy user uploads to upload references
* DEV: Make upload extraction less strict
2022-06-09 09:24:30 +10:00
Loïc Guitaut fff3ff11c0 FIX: Make disabling TLS in mail possible again
Following the Rails 7 upgrade, the `DISCOURSE_SMTP_ENABLE_START_TLS`
setting doesn’t work anymore. This is because Rails upgraded the
`net-smtp` gem to the 0.3.1 version which enables `starttls` by default.
The `mail` gem doesn’t support this new behavior yet and doesn’t know
how to disable TLS. This should be fixed in an upcoming release.

Meanwhile applying this patch allows us to get back the previous
behavior which is expected by many.
2022-06-08 14:04:05 +02:00
Mayfield 99b0578b4c
FIX: escape youtube title when constructing onebox preview html (#16999) 2022-06-08 13:42:37 +08:00
Alan Guo Xiang Tan 82ac698d4f
FIX: Missing tracked sub category topics from tracked topic list (#17034)
Follow-up to 7ae647d092
2022-06-08 10:45:59 +08:00
Jarek Radosz 1a5dbbf430
FIX: Correctly handle invalid auth cookies (#16995)
Previously it would blow up on invalid utf byte sequences. This was a source of spec flakiness.
2022-06-07 13:00:25 +02:00
Alan Guo Xiang Tan 7da074d591
DEV: Implement "My Posts" section link for experimental sidebar (#17008) 2022-06-07 10:52:54 +08:00
Blake Erickson 35e17ce115
FIX: Email Send post has already been taken error (#16992)
* FIX: Email Send post has already been taken error

Adding a failing test first before coming up with a good solution.

Related: 357011eb3b

The above commit changed

```
PostReplyKey.find_or_create_by_safe!
```

to

```
PostReplyKey.create_or_find_by!
```

But I don't think it is working as a 1-1 replacement because of the
`Validation failed: Post has already been taken` error we are receiving
with this change. Also we need to make sure we don't re-introduce any
concurrency issues.

Reported: https://meta.discourse.org/t/224706/13

* Remove rails unique constraint and rely on db index

I believe this is what is causing `create_or_find_by!` to fail. Because
we have a unique constraint in the db I think we can remove this rails
unique constraint?

* clean up spec wording
2022-06-06 13:13:26 -06:00
Martin Brennan f94682e2c4
FIX: Do not use SVGs for twitter:image metadata (#16973)
Twitter does not allow SVGs to be used for twitter:image
metadata (see https://developer.twitter.com/en/docs/twitter-for-websites/cards/overview/markup)
so we should fall back to the site logo if the image option
provided to `crawlable_meta_data` or SiteSetting.site_twitter_summary_large_image_url
is an SVG, and do not add the meta tag for twitter:image at all
if the site logo is an SVG.
2022-06-03 09:02:57 +10:00