Commit Graph

2490 Commits

Author SHA1 Message Date
Sam Saffron 38e7b1a049
FIX: when destroying uploads clear card and profile background
There is an fk to user_profile that can make destroying uploads fail
if they happen to be set as user profile.

This ensures we clear this information when destroying uploads.

There are more relationships, but this makes some more progress.
2020-08-18 10:55:16 +10:00
Vinoth Kannan 476d26159a
FEATURE: add new user option `skip_new_user_tips`. (#10437)
And add new site setting `default_other_skip_new_user_tips` in user preferences category.
2020-08-14 19:10:56 +05:30
Blake Erickson c2f3c0dc44 FIX: generate_topic_thumbnails job infinitely running for corrupted images
It's possible that the original topic image is broken in some form, so
we shouldn't try and generate a topic thumbnail for it. The fix will
prevent the generate_topic_thumbnails job being enqueued every time the
topic is viewed.
2020-08-13 17:08:32 -06:00
Neil Lalonde d65a839577
FEATURE: allow group membership to unmute categories and tags
For sites that are configured to mute some or all categories and tags
for users by default, groups can now be configured to set members'
notification level to normal from the group manage UI.
2020-08-13 17:20:53 -04:00
Krzysztof Kotlarek 1f7f8ee72c
FIX: try to make topic_tracking_state_spec stable (#10419)
Sometime parallel spec if failing with error:
```
NoMethodError:
       undefined method `data' for nil:NilClass
     # ./spec/models/topic_tracking_state_spec.rb:339:in `block (4 levels) in <main>'
```

I have a theory that it might be related to instance variables in before block
2020-08-12 15:56:06 +10:00
Blake Erickson 2032c11f78
FIX: Return 422 when creating topics with tags w/out permission (#10400)
The UI prevents users from trying to create tags on topics when they
don't have permission, but if you are trying to add tags to a topic via
the API and you don't have permission before this change it would
silently succeed in creating the topic, but it wouldn't have any tags.

Now a 422 error will be returned with an error message when trying to
create a topic with tags when tagging is disabled or you don't have
enough trust level to add tags to a topic.

Bug report: https://meta.discourse.org/t/-/70525/14
2020-08-10 16:14:15 -06:00
David Taylor fe7a7ecf6c
FIX: Include secure media URLs when linking post uploads (#10404)
Normally, secure media urls are linked like `/secure-media-uploads/...`. In this case, uploads were already being linked correctly.

But sometimes (e.g. when pulling hotlinked onebox images) secure media is referenced with a full domain name (`//example.com/secure-media-uploads`). This commit ensures that those uploads are also linked correctly.
2020-08-10 17:59:29 +01:00
jbrw 3593e582a3
FIX - limit number of embedded media items in a post (#10391)
* FIX - limit number of embedded media items in a post

* Add renamed settings to DeprecatedSettings
2020-08-07 12:08:59 -04:00
Neil Lalonde 1ca81fbb95
FEATURE: set notification levels when added to a group (#10378)
* FEATURE: set notification levels when added to a group

This feature allows admins and group owners to define default
category and tag tracking levels that will be applied to user
preferences automatically at the time when users are added to the
group. Users are free to change those preferences afterwards.
When removed from a group, the user's notification preferences aren't
changed.
2020-08-06 12:27:27 -04:00
Martin Brennan 1f7b44de67
DEV: Remove flaky secure optimized image spec 2020-08-04 16:29:41 +10:00
Martin Brennan 5a3494b1e1
FIX: IMAP archive fix and group list mailbox code unification (#10355)
* Fixed an issue I introduced in the last PR where I am just archiving everything regardless of whether it is actually archived in Discourse man_facepalming
* Refactor group list_mailboxes IMAP code to use providers, add specs, and add provider code to get the correct prodivder
2020-08-04 14:19:57 +10:00
Gerhard Schlager 957e851ffe Revert "FIX: Regularly reset unknown extension of uploads"
This reverts commit cc7b24b88b as it shouldn't be needed anymore for new uploads.
2020-08-03 13:37:32 +02:00
Sam Saffron 3f054d3350
DEV: merge 2 specs
No need to run 2 specs here when 1 will do
2020-08-03 16:53:37 +10:00
Blake Erickson 8de635fe92
DEV: Show message when cannot invite user to PM (#10336)
* DEV: Show message when cannot invite user to PM

When inviting a user to a PM return a message that says, "Sorry, this
user can't be invited." if they have been muted or are not in a users
allowed pm users list.

* Minor refactor & improved some text
2020-07-31 09:52:19 -06:00
Guo Xiang Tan 4f8262e0d5
FIX: Cooked snippet of raw in `Topic.similar_to`.
If we don't cook the raw, we end up trying to match uncooked raw against
`TopicSearchData#search_data` which consists of cooked raw.
2020-07-28 15:20:18 +08:00
Guo Xiang Tan 4b21b5aac1
FIX: Handle case where `Post#raw` is blank in `Topic.similar_to`. 2020-07-28 13:23:53 +08:00
Guo Xiang Tan 597d542c33
FIX: Improve `Topic.similar_to` with better `Topic#title` matches.
This changes PG text search to only match the given title against
lexemes that are formed from the title. Likewise, the given raw will
only be matched against lexemes that are formed from the post's raw.
2020-07-28 12:00:27 +08:00
Krzysztof Kotlarek e0d9232259
FIX: use allowlist and blocklist terminology (#10209)
This is a PR of the renaming whitelist to allowlist and blacklist to the blocklist.
2020-07-27 10:23:54 +10:00
Guo Xiang Tan c6202af005
Update rubocop to 2.3.1. 2020-07-24 17:19:21 +08:00
Guo Xiang Tan b979579c1b
DEV: Refactor draft attributes for `CategoryList` and `TopicList`.
Avoid repeating the same logic in a bunch of places which will allow us
to make changes to the draft attributes easier in the future.
2020-07-24 10:11:30 +08:00
jbrw 0ed784b4fc
FEATURE: Create logs for Group Moderator changes (#10271) 2020-07-21 14:29:02 -04:00
Blake Erickson cf16173eca Remove extra blank line 2020-07-16 14:57:29 -06:00
Blake Erickson cc8540701e FIX: Be sure to use same units when comparing thumbnail size 2020-07-16 14:30:29 -06:00
Roman Rizzi f13ec11c64
FEATURE: Add scopes to API keys (#9844)
* Added scopes UI

* Create scopes when creating a new API key

* Show scopes on the API key show route

* Apply scopes on API requests

* Extend scopes from plugins

* Add missing scopes. A mapping can be associated with multiple controller actions

* Only send scopes if the use global key option is disabled. Use the discourse plugin registry to add new scopes

* Add not null validations and index for api_key_id

* Annotate model

* DEV: Move default mappings to ApiKeyScope

* Remove unused attribute and improve UI for existing keys

* Support multiple parameters separated by a comma
2020-07-16 15:51:24 -03:00
Vinoth Kannan 3252cb847c FIX: : trigger `user_updated` event only if email changed after user creation.
Follow-up to 1460d7957c
2020-07-16 18:21:30 +05:30
Vinoth Kannan 1460d7957c FEATURE: trigger `user_updated` event if email changed.
It will trigger `user_updated` Discourse event if a user_email record is created or updated or destroyed.
2020-07-16 14:20:41 +05:30
Blake Erickson 1e9ce51151
FIX: Prevent thumbnail gen if image too large (#10247)
It's possible through an import or other means to have images larger
than the current max allowed image size in the db.

If this happens the thumbnail generation job will keep running
indefinitely trying to download a new copy of the original but
discarding it because it is larger than the max_file_size eventually
causing this error

`Job exception: undefined method `path' for nil:NilClass`

because the newly downloaded image is now nil.

This fix stops the enqueuing of the `GenerateTopicThumbnails` job for
all images that happen to be larger than the max image size.
2020-07-16 11:15:53 +10:00
jbrw 06073fe8c6
FEATURE: Allow group moderators to close/archive topics
* FEATURE: Allow group moderators to close/archive topics
2020-07-14 12:36:19 -04:00
Bianca Nenciu 4492718864
FIX: Skip whisper posts when updating topic like count (#10157) 2020-07-13 16:30:00 +10:00
Arpit Jalan 78beb4368a FIX: 'resend all invite' button was not working as expected 2020-07-08 15:52:44 +05:30
Blake Erickson 2e13ed061d FIX: Generate Topic Thumbnails nil to Array error
If any value, including nil, is passed in as an argument the default
won't be set, so we need to handle when a non-Array value is passed in
to the `generate_thumbnails!` method.
2020-07-06 15:35:33 -06:00
Jarek Radosz 64ce12a758
FIX: `OptimizedImage#filesize` (#10095)
`OptimizedImage#filesize` calls `Discourse.store.download` with an OptimizedImage as an argument. It would in turn attempt to call `#original_filename` and `#secure?` on that object. Both would fail as these methods do not exist on OptimizedImage, only on Upload. We didn't know about these issues because:
1. `#calculate_filesize` is not called often, because the filesize is saved on OptimizedImage creation, so it's used mostly for manual filesize recalculation
2. we were using `rescue nil` which swallows all errors
2020-07-06 17:01:29 +02:00
Régis Hanol 7109d94ee7 FIX: properly invalidate inline oneboxes when rebaking
When rebaking a post we were invalidating _regular_ oneboxes but not inline oneboxes.

DEV: also renamed 'InlineOneboxer.purge' to 'InlineOneboxer.invalidate' to keep
the API consistent with 'Oneboxer.invalidate'
2020-06-24 11:54:54 +02:00
Sam Saffron 2987901043
FIX: skip category notification_level unless scoped
#b19dcac2 improved the serializer so it sends default notification
levels to users to work around cases where a category edit would
would result in clients being left with invalid notification state

Unfortunately this did not address the root issue.

When we edit categories we publish state to multiple users this
means that the serializer is executed unscoped with no user.

The client already handles this case per:

dcad720a4c/app/assets/javascripts/discourse/app/models/site.js (L119-L119)

If a property is not shipped to it, it will leave it alone on the
existing category.


This fix ensures that these wide category info updates do not
include notification state to avoid corruption of local state.
2020-06-24 17:08:12 +10:00
Bianca Nenciu 68f767a557
FEATURE: Check if selectable avatars exist before enabling them (#10032) 2020-06-22 16:58:26 +03:00
Jarek Radosz 58a5293d9b
FIX: Delete PostUploads on Post deletion (#10090)
Related uploads will then be removed by Jobs::CleanUpUploads
2020-06-19 17:45:08 +02:00
Vinoth Kannan ae47bcb269
DEV: move user references deletion code to `before_destroy`. (#10085)
Moving the `delete_source_user_references` method code from user merger service to user model.
2020-06-18 17:42:39 -04:00
Cassius d13d5e2b99
FIX: Allow multiple hyphens in host (#10078)
* FIX: Allow multiple hyphens in host

* FIX: added spec for multiple hyphens
2020-06-18 13:58:47 -04:00
Bernhard Suttner e31471585a
DEV: allow to have duplicate topic titles if categegory is different (#10034)
Co-authored-by: Robin Ward <robin.ward@gmail.com>

Co-authored-by: Robin Ward <robin.ward@gmail.com>
2020-06-18 11:19:47 -04:00
Dan Ungureanu d21a08c284
DEV: Deprecate Category#url_with_id in favor of Category#url (#9972) 2020-06-18 11:32:14 +03:00
David Taylor 6caad5c083
FIX: Do not send staff welcome message if user already has role 2020-06-17 12:12:55 +01:00
Jarek Radosz 669c940ec3 Revert "DEV: Remove the remaining ENV["TRAVIS"] usage (#10041)"
This reverts commit 78aff841e3.

See https://review.discourse.org/t/dev-remove-the-remaining-env-travis-usage-10041/12737/4?u=cvx
2020-06-16 19:42:00 +02:00
Jarek Radosz 78aff841e3
DEV: Remove the remaining ENV["TRAVIS"] usage (#10041) 2020-06-16 17:41:15 +02:00
Arpit Jalan a94387c088 FIX: check for inviter group permissions at the time of redeeming invite 2020-06-15 14:44:35 +05:30
Guo Xiang Tan 58e52c0e4f
DEV: Use rails_failover gem for ActiveRecord and Redis failover handling 2020-06-15 15:47:07 +08:00
Sam Saffron a26b490047
FIX: present correct new/unread counts when filtered by tag
Previously we would incorrectly ignore tags.

This ensures tracking state is properly shipped to client if
show_filter_by_tag is enabled.
2020-06-11 16:47:57 +10:00
Dan Ungureanu 3a7ca97c36
FIX: Use include-subcategories filter in report export (#10007)
Some filters were renamed and the conversion of the filter names and arguments
was removed.
2020-06-10 18:57:39 +03:00
Arpit Jalan 3094459cd9
FEATURE: multiple use invite links (#9813) 2020-06-09 20:49:32 +05:30
Gerhard Schlager 8c6a42c589 FIX: Redirects containing Unicode usernames didn't work 2020-06-08 10:26:29 +02:00
Guo Xiang Tan 062db10c52
FIX: `EmailValidator` needs to validate format of email. 2020-06-03 10:34:37 +08:00
Penar Musaraj fa6aa7f627
FIX: Digest frequency issue on user creation
If `default email digest frequency` was set to "Never", users would get
a `digest_after_minutes` set to `nil` which triggered this error
in the logs if/when the site eventually changed that setting and
enabled digests:

```
NoMethodError (undefined method `>=' for nil:NilClass)
/var/www/discourse/app/mailers/user_notifications.rb:227:in `digest'
```
2020-06-01 17:39:16 -04:00
Jarek Radosz 00aab49829
FIX: Restore the ability to rebake posts in deleted topics (#9951)
Lost in #9852
2020-06-01 07:04:16 +02:00
David Taylor d29d69e10d
FIX: Invalidate database theme cache when hostname changes (#9908)
Hostname can vary per-site on a multisite cluster, so this change requires converting the compiler_version from a constant into a class method which is evaluated at runtime. The value is stored in the theme DistributedCache, so performance impact should be negligible.
2020-05-29 13:04:51 +01:00
Sam Saffron 2acec4370b
FIX: muted tags removed topics with no tags from counts
We previously did not account for completely untagged topics when
looking at muted tags, this caused new/unread counts to be off if

1. You had muted tags
2. You had an unread/new topic
3. This topic had no tags
2020-05-29 12:59:47 +10:00
Sam Saffron 136835370c
FEATURE: optionally allow tags in topic tracking state
This feature allows certain plugins to output tag information
to topic tracking state, this allows per tag stats which can be
used by sidebars and other plugins that need per tag stats

Not enabled by default cause this would add cost to a critical
query
2020-05-29 12:59:47 +10:00
Roman Rizzi a41476800b
FIX: Don't raise an exception if a topic cannot be retrieved (#9906) 2020-05-28 11:59:20 -03:00
Roman Rizzi 461df7d050
UX: Rename Priority to score for sorting. (#9846) 2020-05-27 12:50:28 -03:00
Blake Erickson da839e6d26 SECURITY: Use FinalDestination for topic embeds 2020-05-27 09:26:09 -06:00
Arpit Jalan 5462fe9462
FIX: do not allow tag with name 'none' (#9867)
https://meta.discourse.org/t/none-tag-is-uneditable/152003
2020-05-26 08:15:45 +05:30
Sam Saffron fc97f7e0e7
FIX: properly ban non human users from draft system
Previously we had a partial fix in place where non human users
were not allowed draft sequences, this left edges around where non
human users asked for drafts yet had none.

For example system could already have a few drafts in place.

This also removes and extensibility point we added that is not in use
2020-05-26 10:07:09 +10:00
Gerhard Schlager 631024ae5d FEATURE: Permalinks for tags 2020-05-25 14:51:01 +02:00
Sam Saffron c011b817b2
FIX: do not fallback to flair_url column
flair_url is not loaded by our query anymore so avoid falling back
to the old value
2020-05-25 16:25:42 +10:00
Gerhard Schlager 13d5ccedf5 FIX: Destroying a user failed when it had title
...because updating the user caused a validation of the primary email and the UserEmail records are already deleted at that time.
2020-05-24 22:14:50 +02:00
Michael Brown d9a02d1336
Revert "Revert "Merge branch 'master' of https://github.com/discourse/discourse""
This reverts commit 20780a1eee.

* SECURITY: re-adds accidentally reverted commit:
  03d26cd6: ensure embed_url contains valid http(s) uri
* when the merge commit e62a85cf was reverted, git chose the 2660c2e2 parent to land on
  instead of the 03d26cd6 parent (which contains security fixes)
2020-05-23 00:56:13 -04:00
Jeff Atwood 20780a1eee Revert "Merge branch 'master' of https://github.com/discourse/discourse"
This reverts commit e62a85cf6f, reversing
changes made to 2660c2e21d.
2020-05-22 20:25:56 -07:00
Blake Erickson 03d26cd6f0 SECURITY: ensure embed_url contains valid http(s) uri 2020-05-22 14:54:56 -06:00
Roman Rizzi 671f882fa3
FIX: We don't create a Post object if the queued post gets rejected. We need to count review items directly. (#9856) 2020-05-22 11:50:28 -03:00
Martin Brennan f9d55b4941
FEATURE: Update the topic excerpt when the OP is rebaked (#9852)
* We now have a site setting "topic_excerpt_maxlength" that is used when the OP is created or revised to generate a topic excerpt.
* However, posts created before this setting was introduced cannot benefit from this change unless they are revised, and if the topic excerpt length setting is changed that situation is also not covererd.
* This PR makes a change to rebake! to update the topic excerpt IF the post is the OP.
2020-05-22 13:04:15 +10:00
Martin Brennan df68d11c38
FEATURE: Add topic excerpt max length site setting (#9847)
Adds a new topic_excerpt_maxlength site setting.

* When topic excerpt is requested for a post, use the new topic_excerpt_maxlength site setting to limit the size of the excerpt
* Remove code for getting/setting Post.excerpt_size as it is not used anywhere
2020-05-21 13:19:48 +10:00
Guo Xiang Tan 96c02caba7
DEV: Change use of Redis `flushall` to `flushdb`.
FLUSHALL removes all keys from all databases. Instead we only want to
remove keys from the current Redis database.
2020-05-19 10:20:00 +08:00
Gerhard Schlager 6d5e9db883 FIX: Restoring backup didn't clear cached translation overrides 2020-05-18 18:51:51 +02:00
Gerhard Schlager 27c611b1d1 FIX: Reverting multiple translation overrides didn't clear cache 2020-05-18 18:51:29 +02:00
Guo Xiang Tan d01c336899
DEV: Clean up some Redis leaks in test env. 2020-05-18 17:27:37 +08:00
Krzysztof Kotlarek ea63fa7de7
FIX: extract reset_last_seen_cache! in user_spec (#9808)
Helper method to avoid code duplication
2020-05-18 09:13:37 +10:00
Guo Xiang Tan 165fa13c9d
FIX: Always return 0 for non-human user current DraftSequence. 2020-05-15 10:40:36 +08:00
Krzysztof Kotlarek 65a3fdbc57
FIX: randomly failing user_spec (#9754)
* FIX: randomly falling user_spec

When we evaluate `update_last_seen!` we relay on Redis to not run that code too often

https://github.com/discourse/discourse/blob/master/app/models/user.rb#L753

The problem is that not all specs which are running `update_last_seen!` are not cleaning after themselves

For examples specs in that block https://github.com/discourse/discourse/blob/master/spec/models/user_spec.rb#L901

So it can be replicated when you run a few times
`bundle exec rspec  ./spec/models/user_spec.rb -e "should not update the first seen value if it doesn't exist" -e "should have 0 for days_visited"`

We should delete Redis key after each spec which is evaluating `update_last_seen!`
2020-05-15 09:24:07 +10:00
Sam Saffron 7f841dc21f
PERF: avoid race conditions when creating topic links
Previously the code was very race condition prone leading to
odd failures in production

It was re-written in raw SQL to avoid conditions where rows
conflict on inserts

There is no clean way in ActiveRecord to do:

Insert, on conflict do nothing and return existing id.

This also increases test coverage, we were previously not testing
the code responsible for crawling external sites directly
2020-05-13 16:05:57 +10:00
Martin Brennan 9981fa4466
FIX: Prevent column name conflicts in reviewable code (#9753)
We were getting errors like this in Reviewables in some cases:

```
ActiveRecord::StatementInvalid (PG::AmbiguousColumn: ERROR:  column reference "category_id" is ambiguous
LINE 4: ...TRUE) OR (reviewable_by_group_id IN (NULL))) AND (category_i...
```

The problem that was making everything go boom is that plugins can add their own custom filters for Reviewables. If one is doing an INNER JOIN on topics, which has its own category_id column, we would get the above AmbiguousColumn error. The solution here is to just make all references to the reviewable columns in the list_for and viewable_by code prefixed by the table name e.g. reviewables.category_id.
2020-05-13 09:05:56 +10:00
Sam Saffron a29ae17d3a
FIX: saving drafts unconditionally increases sequence
Previously we only changed sequence on ownership change, this
cause a race condition between tabs where user could type for a
long time without being warned of an out of date draft.

This change is a radical change and we should watch closely.

Code was already in place to track sequence on the client so no
changes are needed there.
2020-05-12 16:55:42 +10:00
Guo Xiang Tan d301af39bd
PERF: Avoid hitting DB when fetching draft sequence of bot user. 2020-05-12 13:56:58 +08:00
Martin Brennan 37be2958f7
DEV: Allow specifying if a notification is high_priority on create (#9660)
This allows for special cases where we may not want a certain notification type to ALWAYS be high priority (e.g. a topic timer).
2020-05-07 14:35:32 +10:00
Gerhard Schlager ec2f3169ff FIX: Using the `default_locale` in locale fallbacks caused problems
Locale files get precompiled after deployment and they contained translations from the `default_locale`. That's especially bad in multisites, because the initial `default_locale` is `en_US`. Sites where the `default_locale` isn't `en_US` could see missing translations. The same thing could happen when users are allowed to chose a different locale.

This change simplifies the logic by not using the `default_locale` in the locale chain. It always falls back to `en` in case of missing translations.
2020-05-06 22:59:07 +02:00
Robin Ward 612284cef3
DEV: Remove `Discourse.RAW_TEMPLATES` (#9630)
We were sharing `Discourse` both as an application object and a
namespace which complicated things for Ember CLI. This patch
moves raw templates into `__DISCOURSE_RAW_TEMPLATES` and adds
a couple helper methods to create/remove them.
2020-05-05 12:15:03 -04:00
David Taylor 03818e642a
FEATURE: Include optimized thumbnails for topics (#9215)
This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this:

- Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact

- A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake

- Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued

- Topic lists and topics now include a `thumbnails` key, which includes all the available images:
   ```
   "thumbnails": [
   {
     "max_width": null,
     "max_height": null,
     "url": "//example.com/original-image.png",
     "width": 1380,
     "height": 1840
   },
   {
     "max_width": 1024,
     "max_height": 1024,
     "url": "//example.com/optimized-image.png",
     "width": 768,
     "height": 1024
   }
   ]
  ```

- Themes can request additional thumbnail sizes by using a modifier in their `about.json` file:
   ```
    "modifiers": {
      "topic_thumbnail_sizes": [
        [200, 200],
        [800, 800]
      ],
      ...
  ```
  Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated

- Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 09:07:50 +01:00
Krzysztof Kotlarek 9bff0882c3
FEATURE: Nokogumbo (#9577)
* FEATURE: Nokogumbo

Use Nokogumbo HTML parser.
2020-05-05 13:46:57 +10:00
Krzysztof Kotlarek 37e93914fc
FIX: the muted message should be sent after edit (#9593)
Recently, we added feature that we are sending `/muted` to users who muted specific topic just before `/latest` so the client knows to ignore those messages - https://github.com/discourse/discourse/pull/9482

Same `/muted` message should be included when the post is edited
2020-05-01 08:33:57 +10:00
Neil Lalonde 9d781d4959
FIX: error customizing text for badges from plugins 2020-04-28 14:34:41 -04:00
mentalstring 67f3fe14aa
FEATURE: support SSO website and location overrides
Add location and website + the ability to override using SSO using the `sso_overrides_location` and `sso_overrides_website` site settings.
2020-04-28 16:06:35 +10:00
Sam Saffron cbceadf48b
FEATURE: when blocking emails prefer blocking canonical
Previously we relied entirely on levenshtein_distance_spammer_emails site
setting to handle "similar looking" emails.

This commit improves the situation by always preferring to block (and check)
canonical emails.

This means that if:

`samevil+test@domain.com` is blocked the system will block `samevil@domain.com`

This means that `samevil+2@domain.com` (ad infinitum) will be blocked
2020-04-24 14:09:51 +10:00
Martin Brennan cd1c7d7560
FIX: Copying image markdown for secure media loading full image (#9488)
* When copying the markdown for an image between posts, we were not adding the srcset and data-small-image attributes which are done by calling optimize_image! in cooked post processor
* Refactored the code which was confusing in its current state (the consider_for_reuse method was super confusing) and fixed the issue
2020-04-24 10:29:02 +10:00
Krzysztof Kotlarek 52c1d7337e
FEATURE: don't display new/unread notification for muted topics (#9482)
* FEATURE: don't display new/unread notification for muted topics

Currently, even if user mute topic, when a new reply to that topic arrives, the user will get "See 1 new or updated topic" message. After clicking on that link, nothing is visible (because the topic is muted)

To solve that problem, we will send background message to all users who recently muted that topic that update is coming and they can ignore the next message about that topic.
2020-04-23 14:57:35 +10:00
Vinoth Kannan df0c386f8a
UX: drop the `automatic_membership_retroactive` column from groups model. (#9430) 2020-04-22 22:07:39 +05:30
Dan Ungureanu e733701887
FEATURE: Make report filters reusable (#9444)
This commit also adds 'include subcategories' report filter
2020-04-22 11:52:50 +03:00
Jarek Radosz c08753dc34
DEV: Fix topic_timer_spec (#9483)
File it under "how did it even work".
2020-04-21 14:38:35 +10:00
Mark VanLandingham 06796bec8b
FIX: Set category description to first posts cooked value (#9478) 2020-04-20 14:51:49 -05:00
Robin Ward 25bed4f643 FIX: Concurrency issues with making topic embedded posts visible 2020-04-20 15:11:59 -04:00
Robin Ward 56a23c68f1 FIX: Embedded topics couldn't update their titles 2020-04-20 14:27:43 -04:00
David Taylor 0e4497b6be
DEV: Update tl3 spec to remove hard-coded primary keys
In some test runs, the hard-coded ids could correspond to real topics created earlier in the tests, and cause random test failures
2020-04-17 17:24:14 +01:00
Neil Lalonde 074509fd95
FIX: don't demote users to TL2 when default trust level is 3
Within 24 hours of signing up, new users were losing their
default trust level of 3. With this fix, demotions from
trust level 3 won't happen when the "default trust level"
setting is 3 or 4.
2020-04-16 12:28:16 -04:00
Robin Ward b6b92a562c
FEATURE: New site setting `embed_unlisted` (#9391)
If enabled, posts imported to discourse via embeddings will default to
unlisted until they receive a reply.
2020-04-13 15:17:02 -04:00
Kane York 138d4aebde FEATURE: Webhooks and Event for user being granted a badge
Adding a webhook for badge revocation is left for future work as it's relatively rare.
2020-04-08 14:38:30 -07:00
Robin Ward e1f8014acd
FEATURE: Support for publishing topics as pages (#9364)
If the feature is enabled, staff members can construct a URL and publish a
topic for others to browse without the regular Discourse chrome.

This is useful if you want to use Discourse like a CMS and publish
topics as articles, which can then be embedded into other systems.
2020-04-08 12:52:36 -04:00
Roman Rizzi 8e1bdc9458
FIX: Prevent low score flags from auto-closing a topic if the reviewable default visibility is higher than low (#9375) 2020-04-08 10:44:31 -03:00
Neil Lalonde c45574c9e7
FIX: exclude private messages from TL3 requirements
It wasn't intended that people should be able to earn trust level
3 without participating in public topics. When counting topic
views and likes given/received, don't count private topics.
2020-04-07 17:36:07 -04:00
Krzysztof Kotlarek a811976023
FIX: reset gravatar cache by adding random param to URL (#9370)
Users noticed that sometimes, avatar from Gravatar is not correctly updated - https://meta.discourse.org/t/updated-image-on-gravatar-not-seeing-it-update-on-site/54357

A potential reason for that is that even if you update your avatar in Gravatar, URL stays the same and if the cache is involved, service is still receiving the old photo.

For example. In my case, when I click the button to refresh avatar the
new Upload record is created with `origin` URL to new avatar, and `url` to
old one

I made some tests in the rails console and adding random param to Gravatar URL is deceiving cache and correct, the newest avatar is downloaded
2020-04-08 07:35:42 +10:00
Roman Rizzi e52e7f6e75
FIX: Include pending queued users regardless of their score (#9372) 2020-04-07 11:42:12 -03:00
David Taylor 3814ca06a8
DEV: Allow using .js extension for javascript modules in themes (#9358) 2020-04-06 17:24:59 +01:00
Dan Ungureanu b9d411a4eb
FIX: Topic.time_to_first_response should include sub-sub-categories (#9349) 2020-04-04 13:31:34 +03:00
David Taylor 3cfe086a94
FIX: Limit personal message participants when converting from topic (#9343)
Previously all topic posters would be added which could lead to major performance issues. Now if there are too many posters, only the acting user will be added as a participant.
2020-04-03 16:42:01 +01:00
Martin Brennan b79ea986ac
FEATURE: High priority bookmark reminder notifications (#9290)
Introduce the concept of "high priority notifications" which include PM and bookmark reminder notifications. Now bookmark reminder notifications act in the same way as PM notifications (float to top of recent list, show in the green bubble) and most instances of unread_private_messages in the UI have been replaced with unread_high_priority_notifications.

The user email digest is changed to just have a section about unread high priority notifications, the unread PM section has been removed.

A high_priority boolean column has been added to the Notification table and relevant indices added to account for it.

unread_private_messages has been kept on the User model purely for backwards compat, but now just returns unread_high_priority_notifications count so this may cause some inconsistencies in the UI.
2020-04-01 09:09:20 +10:00
Arpit Jalan b2a0d34bb7
FEATURE: add setting `auto_approve_email_domains` to auto approve users (#9323)
* FEATURE: add setting `auto_approve_email_domains` to auto approve users

This commit adds a new site setting `auto_approve_email_domains` to
auto approve users based on their email address domain.

Note that if a domain already exists in `email_domains_whitelist` then
`auto_approve_email_domains` needs to be duplicated there as well,
since users won’t be able to register with email address that is
not allowed in `email_domains_whitelist`.

* Update config/locales/server.en.yml

Co-Authored-By: Robin Ward <robin.ward@gmail.com>
2020-03-31 23:59:15 +05:30
Krzysztof Kotlarek a0f59a55cc
FIX: when a post is moved copy notifications level (#9311)
This is a revert of 2c011252f1

More information on meta: https://meta.discourse.org/t/when-a-reply-is-moved-to-a-new-topic-the-followers-of-the-previous-topic-are-automatically-follower-of-the-new-topic-as-well/130025
2020-03-31 16:19:47 +11:00
Jarek Radosz 104b647ca4 DEV: Fix another time-sensitive flaky spec 2020-03-27 20:39:25 +01:00
Jarek Radosz 7ff889574d
DEV: Add rubocop-rspec (#9288)
This adds rubocop-rspec, and enables some cops that were either already passing or are passing now, after fixing them in this commit.

Some new cops are disabled for now, with annotation: "TODO" or "To be decided". Those either need to be discussed first, or require manual changes, or the number of found and fixed offenses is too large to bundle them up in a single PR.

Includes:

* DEV: Update rubocop's `TargetRubyVersion` to 2.6
* DEV: Enable RSpec/VoidExpect
* DEV: Enable RSpec/SharedContext
* DEV: Enable RSpec/EmptyExampleGroup (Removed an obsolete empty spec file)
* DEV: Enable RSpec/ItBehavesLike
* DEV: Remove RSpec/ScatteredLet (It's too strict, as it doesn't recognize fab! as a let-like)
* DEV: Remove RSpec/MultipleExpectations
2020-03-27 17:35:40 +01:00
Sam Saffron 40b6e278a0
FIX: ensure wiki editor is assigned consistently
`DISTINCT ON` was not providing stability, this ensures the backfill will
be stable on a specific post avoiding risk around double grants.
2020-03-27 12:41:06 +11:00
Jarek Radosz d21d80198c
DEV: Update rubocop-discourse (#9270)
Includes:
* DEV: Use `eq_time` matcher
2020-03-26 16:32:41 +01:00
Martin Brennan 097851c135
FIX: Change secure media to encompass attachments as well (#9271)
If the “secure media” site setting is enabled then ALL files uploaded to Discourse (images, video, audio, pdf, txt, zip etc. etc.) will follow the secure media rules. The “prevent anons from downloading files” setting will no longer have any bearing on upload security. Basically, the feature will more appropriately be called “secure uploads” instead of “secure media”.

This is being done because there are communities out there that would like all attachments and media to be secure based on category rules but still allow anonymous users to download attachments in public places, which is not possible in the current arrangement.
2020-03-26 07:16:02 +10:00
Jeff Wong 7282435e17 FEATURE: Unassign the review queue topic when a flag is handled 2020-03-24 14:57:44 -07:00
David Taylor 5ff505cea6
SECURITY: Respect topic permissions when loading draft metadata
Co-authored-by: Sam Saffron <sam.saffron@gmail.com>
2020-03-23 11:30:40 +00:00
David Taylor 3215f2b6ee
FIX: Permalinks should redirect to category URL including the ID
This is a temporary fix. Urls for third-level categories should function without the id. Once that is fixed, this change can be reverted
2020-03-20 22:06:20 +00:00
romanrizzi cfec10a568 FIX: Moderators should be able to review flagged PMs since this has always been like this 2020-03-20 12:28:36 -03:00
Daniel Waterworth 1b24a7b993 FIX: Include entire slug path in permalinks
This is a temporary fix since these URLs should contain the id as well.
2020-03-20 10:43:13 +00:00
Roman Rizzi 080960a15e
FIX: Show the envelope icon when the flagged post is a PM. Flagged PM must be exclusively reviewed by admins (#9232) 2020-03-19 15:12:42 -03:00
Vinoth Kannan f6d6f1701f FIX: use the new duration attribute in `set_or_create_timer` method.
New `duration` attribute is introduced for the `set_or_create_timer` method in the commit aad12822b7 for "based on last post" and "auto delete replies" topic timers.
2020-03-19 21:45:05 +05:30
Justin DiRose 9101227266
FEATURE: Demote muted categories on category list (#9226) 2020-03-17 15:33:15 -05:00
Blake Erickson 919e405c48
FIX: Don't display webhooks for inactive plugins (#9206)
* FIX: Don't display webhooks for inactive plugins

This commit ensures that we don't show webhooks for plugins that are not
installed or that are disabled.

Bug report:

https://meta.discourse.org/t/webhookeventtype-and-the-solved-and-assign-plugins/144180

* rename to just 'active', it's cleaner
2020-03-17 10:39:24 -06:00
Jarek Radosz e950471c0f
DEV: Replace User.unstage and User#unstage API with User#unstage! (#8906)
* DEV: Replace User.unstage and User#unstage API with User#unstage!

Quoting @SamSaffron:

> User.unstage mixes concerns of both unstaging users and updating params which is fragile/surprising.
> u.unstage destroys notifications and raises a user_unstaged event prior to the user becoming unstaged and the user object being saved.

User#unstage! no longer updates user attributes and saves the object before triggering the `user_unstaged` event.

* Update one more spec

* Assign attributes after unstaging
2020-03-17 16:48:24 +01:00
Roman Rizzi 4663304775
FEATURE: Show rejected posts count in user summary (#9204) 2020-03-16 09:52:08 -03:00
Neil Lalonde 7c27f9bba9
FIX: remove parent tag from tag group
Having a tag be a member of a tag group and the group's parent tag at
the same time causes some unexpected behavior. When a tag is assigned
as the parent, remove it from the group.
2020-03-13 12:25:58 -04:00
David Taylor ec2d49d48a
DEV: Allow plugins to add theme modifiers via db migrations (#9192) 2020-03-12 16:35:28 +00:00
Stasiek Michalski 1b8793e7a4
FEATURE: Add support for custom gravatar-like services (#9137)
Adds 3 config values that allow to set a custom provider of Gravatar-like API accessible from gravatar_base_url. The gravatar_name is purely cosmetic, but helps with associating name with the service that actually provides the avatars. gravatar_login_url is a link relative to gravatar_base_url, which provides the user with the login to the Gravatar service
2020-03-12 11:23:55 -04:00
Martin Brennan 793f39139a
FEATURE: Send notifications for time-based and At Desktop bookmark reminders (#9071)
* This PR implements the scheduling and notification system for bookmark reminders. Every 5 minutes a schedule runs to check any reminders that need to be sent before now, limited to **300** reminders at a time. Any leftover reminders will be sent in the next run. This is to avoid having to deal with fickle sidekiq and reminders in the far-flung future, which would necessitate having a background job anyway to clean up any missing `enqueue_at` reminders.

* If a reminder is sent its `reminder_at` time is cleared and the `reminder_last_sent_at` time is filled in. Notifications are only user-level notifications for now.

* All JavaScript and frontend code related to displaying the bookmark reminder notification is contained here. The reminder functionality is now re-enabled in the bookmark modal as well.

* This PR also implements the "Remind me next time I am at my desktop" bookmark reminder functionality. When the user is on a mobile device they are able to select this option. When they choose this option we set a key in Redis saying they have a pending at desktop reminder. The next time they change devices we check if the new device is desktop, and if it is we send reminders using a DistributedMutex. There is also a job to ensure consistency of these reminders in Redis (in case Redis drops the ball) and the at desktop reminders expire after 20 days.

* Also in this PR is a fix to delete all Bookmarks for a user via `UserDestroyer`
2020-03-12 10:16:00 +10:00
David Taylor d1474e94a1
FEATURE: Allow themes to specify modifiers in their about.json file (#9097)
There are three modifiers:
- serialize_topic_excerpts (boolean)
- csp_extensions (array of strings)
- svg_icons (array of strings)

When multiple themes are active, the values will be combined. The combination method varies based on the setting. CSP/SVG arrays will be combined. serialize_topic_excerpts will use `Enumerable#any`.
2020-03-11 13:30:45 +00:00
Jarek Radosz 29b35aa64c
DEV: Improve flaky time-sensitive specs (#9141) 2020-03-10 22:13:17 +01:00
Penar Musaraj e69b6379ad
FEATURE: Broader support for post uploads in video markup (#9152)
Ensures URLs in the following HTML attributes are included in post uploads:
- video poster
- source src
- track src
2020-03-10 09:01:40 -04:00
Robin Ward 91682408e0 Remove very bad tests 2020-03-06 09:35:55 -05:00
Sam Saffron 494379201d
DEV: attempt to stabilize flaky spec
Spec was checking implementation when it did not need to, temp file was blank
so optimized image should be blank
2020-03-06 12:57:12 +11:00
David Taylor 65cc61be7a
PERF: Allow preloading 'recent time read' for a user (#9076)
This will be used when serializing multiple user cards
2020-03-03 13:57:46 +00:00
Dan Ungureanu 60908a94ec
FIX: Skip 'invited' small action if user is in an invited group (#9056)
Inviting a user that is already invited through a group used to generate
a small action and a notification. This commit skips that small action.
2020-02-27 14:45:20 +02:00
Vinoth Kannan acf337d583
FEATURE: auto archive group message if topic is closed. (#9046)
Co-Authored-By: Régis Hanol <regis@hanol.fr>
2020-02-27 11:09:37 +05:30
Joffrey JAFFEUX baba1cc02e
FIX: ensures destroying a user with security keys doesn't fail (#9042) 2020-02-25 14:07:57 -05:00
Sam Saffron 372f6f4f22
FEATURE: limit number of notifications per user to 10,000
Introduces a new site setting `max_notifications_per_user`.

Out-of-the-box this is set to 10,000. If a user exceeds this number of
notifications, we will delete the oldest notifications keeping only 10,000.

To disable this safeguard set the setting to 0.

Enforcement happens weekly.

This is in place to protect the system from pathological states where a
single user has enormous amounts of notifications causing various queries
to time out. In practice nobody looks back more than a few hundred notifications.
2020-02-24 11:42:50 +11:00
Martin Brennan 6a2bde4d48 Fix broken secure media specs 2020-02-21 10:01:32 +10:00
Martin Brennan 04df3bd46d
FIX: Only mark attachments as secure media if SiteSetting.secure_media? (#9009)
* Attachments (non media files) were being marked as secure if just
SiteSetting.prevent_anons_from_downloading_files was enabled. this
was not correct as nothing should be marked as actually "secure" in
the DB without that site setting enabled
* Also add a proper standalone spec file for the upload security class
2020-02-21 09:35:16 +10:00
Robin Ward 041168c9b6 FIX: Group mentions were not being cooked the same was as previewed
If a group mention could be notified on preview it was given an `<a>`
tag with the `.notify` class. When cooked it would display differently.
This patch makes the server side cooking match the client preview.
2020-02-18 15:45:02 -05:00
Martin Brennan 5122826bde Rubocop lint 2020-02-18 15:13:34 +10:00
Martin Brennan 500185dc11 Try fix upload_spec flakys and remove logging from tasks/uploads_spec 2020-02-18 15:08:58 +10:00
romanrizzi 17aed8d84b DEV: Add a test to make sure that flagged replies are ignored when calling the delete_and_agree_replies method 2020-02-17 13:31:01 -03:00
Martin Brennan e8efdd60d4
FIX: Tweak upload security emoji check (#8981)
Further on from my earlier PR #8973 also reject upload as secure if its origin URL contains images/emoji. We still check Emoji.all first to try and be canonical.

This may be a little heavy handed (e.g. if an external URL followed this same path it would be a false positive), but there are a lot of emoji aliases where the actual Emoji url is something, but you can have another image that should not be secure that that thing is an alias for. For example slight_smile.png does not show up in Emoji.all BUT slightly_smiling_face does, and it aliases slight_smile e.g. /images/emoji/twitter/slight_smile.png?v=9 and /images/emoji/twitter/slightly_smiling_face.png?v=9 are equivalent.
2020-02-17 15:11:15 +10:00
Martin Brennan dac923379a
FIX: Never mark uploads based on regular emoji secure (#8973)
Sometimes PullHotlinkedImages pulls down a site emoji and creates a new upload record for it. In the cases where these happen the upload is not created via the normal path that custom emoji follows, so we need to check in UploadSecurity whether the origin of the upload is based on a regular site emoji. If it is we never want to mark it as secure (we don't want emoji not accessible from other posts because of secure media).

This only became apparent because the uploads:ensure_correct_acl rake task uses UploadSecurity to check whether an upload should be secure, which would have marked a whole bunch of regular-old-emojis as secure.
2020-02-17 12:30:47 +10:00