Commit Graph

5172 Commits

Author SHA1 Message Date
fzngagan c363189858 Accounted for the change while reading the fields added specs to confirm working 2020-08-25 09:52:18 -04:00
fzngagan 352ad826c1 FIX: symbols and strings should be treated similarly while saving custom fields 2020-08-25 09:52:18 -04:00
Roman Rizzi dd13304b81
FEATURE: More API scopes (#10493) 2020-08-24 12:15:08 -03:00
Guo Xiang Tan daddad7fd6
DEV: Update annotations. 2020-08-21 11:36:53 +08:00
Guo Xiang Tan aae9e6e5fd
FIX: `Topic.similar_to` results in error if cooked raw is blank. 2020-08-21 10:54:08 +08:00
Kane York ab0b034404
FIX: Wizard could not send custom color schemes to the client correctly (#10484)
This was likely introduced with the refactor to make ColorSchemeColor a database object. Add a test so this doesn't happen again.

Also test other basics of the WizardSerializer.

For some reason, the .as_json left Ruby objects in; I solved this with a round trip through JSON during the test.
2020-08-20 17:10:33 -07:00
Guo Xiang Tan 337f062f0f
PERF: Defer indexing post for search when saving a post.
Indexing a post for search is slow and there is no reason for us to have
to block saving a post due to search indexing.
2020-08-21 07:52:43 +08:00
Penar Musaraj 3c06dd9b99
FIX: Detect dark scheme server-side for better dark logo support (#10490)
* FIX: Use dark logo when dark scheme is default

* Small refactor
2020-08-20 14:23:18 -04:00
Vinoth Kannan 8348a41124
FEATURE: add `regular_categories` field in site setting & user option. (#10477)
Like "default watching" and "default tracking" categories option now the "regular" categories support is added. It will be useful for sites that are muted by default. The user option will be displayed only if `mute_all_categories_by_default` site setting is enabled.
2020-08-20 00:35:04 +05:30
Neil Lalonde 298ed5d021
FIX: delete unused tags shouldn't delete tags belonging to tag groups 2020-08-19 12:17:49 -04:00
Joffrey JAFFEUX dde3d63b33
DEV: adds event_reminder (27) and event_invitation (28) (#10473)
Note that these events are also added to the list of events sent to push notifications.
2020-08-19 12:07:51 +02:00
Roman Rizzi 390615fbcd
UX: Help users understand the meaning of each scope. (#10468) 2020-08-18 15:12:04 -03:00
Penar Musaraj 882b0aac19
DEV: Let themes extend color definitions (#10429)
Themes can now declare custom colors that get compiled in core's color definitions stylesheet, thus allowing themes to better support dark/light color schemes. 

For example, if you need your theme to use tertiary for an element in a light color scheme and quaternary in a dark scheme, you can add the following SCSS to your theme's `color_definitions.scss` file: 

```
:root {
  --mytheme-tertiary-or-quaternary: #{dark-light-choose($tertiary, $quaternary)};
}
```

And then use the `--mytheme-tertiary-or-quaternary` variable as the color property of that element. You can also use this file to add color variables that use SCSS color transformation functions (lighten, darken, saturate, etc.) without compromising your theme's compatibility with different color schemes.
2020-08-18 13:02:13 -04:00
Vinoth Kannan 562180dd9a
FEATURE: add option to skip new user tips in first notification. (#10462) 2020-08-18 13:43:40 +05:30
Sam Saffron 38e7b1a049
FIX: when destroying uploads clear card and profile background
There is an fk to user_profile that can make destroying uploads fail
if they happen to be set as user profile.

This ensures we clear this information when destroying uploads.

There are more relationships, but this makes some more progress.
2020-08-18 10:55:16 +10:00
Penar Musaraj 3745f2bb86
FEATURE: Dark mode alternative logos (#10441) 2020-08-17 15:43:20 -04:00
Joffrey JAFFEUX 6d0eb7178d
FIX: prevents errors on /tags when a tag `constructor` exists (#10449)
This is due to js objects having a constructor property:

```
const obj = {};
obj['constructor'] // return [native code] and not undefined
```
2020-08-17 17:40:34 +02:00
Guo Xiang Tan 872ef82f4d
Revert "PERF: Prefer joins over subquery for `User#private_posts_for_user`."
This reverts commit 05b43e5ae4.
2020-08-17 14:58:47 +08:00
Guo Xiang Tan 05b43e5ae4
PERF: Prefer joins over subquery for `User#private_posts_for_user`.
The subquery here prevents the planner from optimizing the query. As
such, we prefer joining against the requried tables instead.
2020-08-17 14:47:48 +08:00
Vinoth Kannan 476d26159a
FEATURE: add new user option `skip_new_user_tips`. (#10437)
And add new site setting `default_other_skip_new_user_tips` in user preferences category.
2020-08-14 19:10:56 +05:30
Martin Brennan 4670b62969
DEV: IMAP log to database (#10435)
Convert all IMAP logging to write to a database table for easier inspection. These logs are cleaned up daily if they are > 5 days old.

Logs can easily be watched in dev by setting DISCOURSE_DEV_LOG_LEVEL=\"debug\" and running tail -f development.log | grep IMAP
2020-08-14 12:01:31 +10:00
Blake Erickson c2f3c0dc44 FIX: generate_topic_thumbnails job infinitely running for corrupted images
It's possible that the original topic image is broken in some form, so
we shouldn't try and generate a topic thumbnail for it. The fix will
prevent the generate_topic_thumbnails job being enqueued every time the
topic is viewed.
2020-08-13 17:08:32 -06:00
Neil Lalonde d65a839577
FEATURE: allow group membership to unmute categories and tags
For sites that are configured to mute some or all categories and tags
for users by default, groups can now be configured to set members'
notification level to normal from the group manage UI.
2020-08-13 17:20:53 -04:00
Vinoth Kannan 310952fd6a FIX: generate topic excerpt when moving posts to new topic.
Currently, it's not generating the excerpt by default. We have to trigger the "Rebuild HTML" action to do it.
2020-08-13 11:30:14 +05:30
Rafael dos Santos Silva 28669dfeb2
PERF: Faster TL3 promotion replies needed calculation (#10416)
Removing the LIMIT makes PostgreSQL use index_posts_on_user_id_and_created_at
which is much faster overall.

Before: 22 seconds
After: 100 ms
2020-08-12 11:28:34 -03:00
Penar Musaraj c05aced094
FIX: Invalidate cache when updating color scheme colors (#10417) 2020-08-11 16:28:59 -04:00
David Taylor fe7a7ecf6c
FIX: Include secure media URLs when linking post uploads (#10404)
Normally, secure media urls are linked like `/secure-media-uploads/...`. In this case, uploads were already being linked correctly.

But sometimes (e.g. when pulling hotlinked onebox images) secure media is referenced with a full domain name (`//example.com/secure-media-uploads`). This commit ensures that those uploads are also linked correctly.
2020-08-10 17:59:29 +01:00
jbrw 3593e582a3
FIX - limit number of embedded media items in a post (#10391)
* FIX - limit number of embedded media items in a post

* Add renamed settings to DeprecatedSettings
2020-08-07 12:08:59 -04:00
Penar Musaraj 9c9aa21726
Add site setting to pick dark mode color scheme (#10390)
Co-authored-by: Robin Ward <robin.ward@gmail.com>
2020-08-07 08:52:47 -04:00
Krzysztof Kotlarek fb9c2e5f89
FEATURE: notification for vote plugin (#10384)
This must exist in core as we want to see a correct notification after disable of the voting plugin.
2020-08-07 09:51:16 +10:00
Neil Lalonde 1ca81fbb95
FEATURE: set notification levels when added to a group (#10378)
* FEATURE: set notification levels when added to a group

This feature allows admins and group owners to define default
category and tag tracking levels that will be applied to user
preferences automatically at the time when users are added to the
group. Users are free to change those preferences afterwards.
When removed from a group, the user's notification preferences aren't
changed.
2020-08-06 12:27:27 -04:00
David Taylor ceb858c70a
PERF: Release post_upload records when downloaded image is removed (#10379)
Previously we would unconditionally keep all images downloaded via pull_hotlinked_images, even if they are later removed from the post. This commit removes that logic, and relies on the existing link_post_uploads process to pick up the downloaded images in `cooked`. Specs are added to ensure this is working correctly for regular hotlinked images, and for oneboxes.
2020-08-06 10:06:34 +10:00
David Taylor cb12a721c4
REFACTOR: Refactor pull_hotlinked_images job
This commit should cause no functional change
- Split into functions to avoid deep nesting
- Register custom field type, and remove manual json parse/serialize
- Recover from deleted upload records

Also adds a test to ensure pull_hotlinked_images redownloads secure images only once
2020-08-05 12:14:59 +01:00
Robin Ward a1ec711373 FIX: Previous fix comitted the wrong thing and was broken :( 2020-08-04 13:46:26 -04:00
Robin Ward 6da7a97eee FIX: Exclude shared drafts from digests 2020-08-04 13:35:48 -04:00
Mark VanLandingham b76731d722
FEATURE: Invite emails to groups from add member modal (#10308) 2020-08-04 10:02:01 -05:00
Martin Brennan 5a3494b1e1
FIX: IMAP archive fix and group list mailbox code unification (#10355)
* Fixed an issue I introduced in the last PR where I am just archiving everything regardless of whether it is actually archived in Discourse man_facepalming
* Refactor group list_mailboxes IMAP code to use providers, add specs, and add provider code to get the correct prodivder
2020-08-04 14:19:57 +10:00
Bianca Nenciu 2682da81ad
FIX: Get correct selectable avatar from URL (#10339)
The URL for selectable avatars was 'cooked' which means that the find_by
method was not enough.
2020-08-03 17:15:41 +03:00
Gerhard Schlager 957e851ffe Revert "FIX: Regularly reset unknown extension of uploads"
This reverts commit cc7b24b88b as it shouldn't be needed anymore for new uploads.
2020-08-03 13:37:32 +02:00
Martin Brennan 2920988b3a
FIX: IMAP sync email update uniqueness across groups and minor improvements (#10332)
Adds a imap_group_id column to IncomingEmail to deal with an issue where we were trying to update emails in the mailbox, calling IncomingEmail.where(imap_sync: true). However UID and UIDVALIDITY could be the same across accounts. So if group A used IMAP details for Gmail account A, and group B used IMAP details for Gmail account B, and both tried to sync changes to an email with UID of 3 (e.g. changing Labels), one account could affect the other. This even applied to Archiving!

Also in this PR:

* Fix error occurring if we do a uid_fetch and no emails are returned
* Allow for creating labels within the target mailbox (previously we would not do this, only use existing labels)
* Improve consistency for log messages
* Add specs for generic IMAP provider (Gmail specs still to come)
* Add custom archiving support for Gmail
* Only use Message-ID for uniqueness of IncomingEmail if it was generated by us
* Various refactors and improvements
2020-08-03 13:10:17 +10:00
Blake Erickson 8de635fe92
DEV: Show message when cannot invite user to PM (#10336)
* DEV: Show message when cannot invite user to PM

When inviting a user to a PM return a message that says, "Sorry, this
user can't be invited." if they have been muted or are not in a users
allowed pm users list.

* Minor refactor & improved some text
2020-07-31 09:52:19 -06:00
Simon Cossar 3d9eb3e085
Rename options passed to Readability::Document back to whitelist and blacklist (#10340) 2020-07-30 12:56:48 -07:00
Bianca Nenciu d2380a01b6
FIX: Allow 'c' as a tag (#10305)
The new tag routes allow the use of tag 'c'.
2020-07-29 14:47:12 +03:00
Martin Brennan 36c11cb62b
FIX: Rename delete_when_reminder_sent? bookmark method to avoid conflict with AR (#10333)
I added delete_when_reminder_sent to ignored_columns because it no longer exists and added a shortcut method delete_when_reminder_sent? to the Bookmark model. However I have been seeing some weird errors like:

> Job exception: unknown attribute 'delete_when_reminder_sent' for Bookmark.

So I am very suspicious. I am just renaming the method to auto_delete_when_reminder_sent? to avoid any potential conflicts.

Also found include_bookmark_delete_on_owner_reply? in PostSerializer which is used for nothing; I must have forgotten to delete it before.
2020-07-29 17:02:36 +10:00
Martin Brennan 9e5b213089
FIX: Ensure topic user bookmarked synced on bookmark auto-delete (#10323)
For the following conditions, the TopicUser.bookmarked column was not updated correctly:

* When a bookmark was auto-deleted because the reminder was sent
* When a bookmark was auto-deleted because the owner of the bookmark replied to the topic

This adds another migration to fix the out-of-sync column and also some refactors to BookmarkManager to allow for more of these delete cases. BookmarkManager is used instead of directly destroying the bookmark in PostCreator and BookmarkReminderNotificationHandler.
2020-07-29 09:43:32 +10:00
Guo Xiang Tan 4f8262e0d5
FIX: Cooked snippet of raw in `Topic.similar_to`.
If we don't cook the raw, we end up trying to match uncooked raw against
`TopicSearchData#search_data` which consists of cooked raw.
2020-07-28 15:20:18 +08:00
Guo Xiang Tan 4b21b5aac1
FIX: Handle case where `Post#raw` is blank in `Topic.similar_to`. 2020-07-28 13:23:53 +08:00
Guo Xiang Tan dcfe765dac
Make rubocop happy. 2020-07-28 12:06:38 +08:00
Guo Xiang Tan 597d542c33
FIX: Improve `Topic.similar_to` with better `Topic#title` matches.
This changes PG text search to only match the given title against
lexemes that are formed from the title. Likewise, the given raw will
only be matched against lexemes that are formed from the post's raw.
2020-07-28 12:00:27 +08:00
David Taylor 407bb96a22
FIX: Avoid validation error when deleting users with locked trust level 2020-07-27 17:40:10 +01:00
Krzysztof Kotlarek f2342c2d32
FIX: reserve id for reaction notifications (#10315)
Use id 25 for reaction notifications
2020-07-27 11:39:50 +10:00
Krzysztof Kotlarek e0d9232259
FIX: use allowlist and blocklist terminology (#10209)
This is a PR of the renaming whitelist to allowlist and blacklist to the blocklist.
2020-07-27 10:23:54 +10:00
Guo Xiang Tan b979579c1b
DEV: Refactor draft attributes for `CategoryList` and `TopicList`.
Avoid repeating the same logic in a bunch of places which will allow us
to make changes to the draft attributes easier in the future.
2020-07-24 10:11:30 +08:00
jbrw 2aec92d0b4
FEATURE - allow Group Moderators to edit category description (#10292)
Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2020-07-23 09:50:00 -04:00
Krzysztof Kotlarek 4b053462c0
FIX: display correct status on unsubscribe page (#10294)
There was a bug that even when `email_digest` was set to false but
`digest_after_minutes` was positive, we were not displaying correct
status.

In addition, the message is improved when the user is unsubscribed +
unsubscribe from all is hidden.
2020-07-23 16:20:10 +10:00
Martin Brennan e027acd367
FIX: Move consts and translations for bookmark auto delete prefs (#10295) 2020-07-23 13:12:28 +10:00
Arpit Jalan db4e310376 DEV: annotate models 2020-07-22 16:02:31 +05:30
Martin Brennan 8dc3543134
FIX: Ignore removed delete_when_reminder_sent bookmarks column 2020-07-22 10:17:21 +10:00
jbrw 0ed784b4fc
FEATURE: Create logs for Group Moderator changes (#10271) 2020-07-21 14:29:02 -04:00
Guo Xiang Tan cebd8da299
FIX: Don't raise error when adding to cache fails in readonly mode. 2020-07-21 12:20:23 +08:00
Martin Brennan 41b43a2a25
FEATURE: Add "delete on owner reply" bookmark functionality (#10231)
This adds an option to "delete on owner reply" to bookmarks. If you select this option in the modal, then reply to the topic the bookmark is in, the bookmark will be deleted on reply.

This PR also changes the checkboxes for these additional bookmark options to an Integer column in the DB with a combobox to select the option you want.

The use cases are:

* Sometimes I will bookmark the topics to read it later. In this case we definitely don’t need to keep the bookmark after I replied to it.
* Sometimes I will read the topic in mobile and I will prefer to reply in PC later. Or I may have to do some research before reply. So I will bookmark it for reply later.
2020-07-21 10:00:39 +10:00
Blake Erickson 690f17bcbe
FEATURE: Allow List for PMs (#10270)
* FEATURE: Allow List for PMs

This feature adds a new user setting that is disabled by default that
allows them to specify a list of users that are allowed to send them
private messages. This way they don't have to maintain a large list of
users they don't want to here from and instead just list the people they
know they do want. Staff will still always be able to send messages to
the user.

* Update PR based on feedback
2020-07-20 15:23:49 -06:00
David Taylor fab8b8649e
PERF: Combine avatar_lookup and primary_group_lookup into user_lookup (#10253)
These two classes were running very similar queries, which could be expensive on large topics
2020-07-17 10:48:08 +01:00
Guo Xiang Tan ff7678e210
FIX: Reindex posts when `Topic#title` or `Category#name` changes. 2020-07-17 11:12:31 +08:00
Blake Erickson cc8540701e FIX: Be sure to use same units when comparing thumbnail size 2020-07-16 14:30:29 -06:00
Roman Rizzi f13ec11c64
FEATURE: Add scopes to API keys (#9844)
* Added scopes UI

* Create scopes when creating a new API key

* Show scopes on the API key show route

* Apply scopes on API requests

* Extend scopes from plugins

* Add missing scopes. A mapping can be associated with multiple controller actions

* Only send scopes if the use global key option is disabled. Use the discourse plugin registry to add new scopes

* Add not null validations and index for api_key_id

* Annotate model

* DEV: Move default mappings to ApiKeyScope

* Remove unused attribute and improve UI for existing keys

* Support multiple parameters separated by a comma
2020-07-16 15:51:24 -03:00
Vinoth Kannan 3252cb847c FIX: : trigger `user_updated` event only if email changed after user creation.
Follow-up to 1460d7957c
2020-07-16 18:21:30 +05:30
Vinoth Kannan 1460d7957c FEATURE: trigger `user_updated` event if email changed.
It will trigger `user_updated` Discourse event if a user_email record is created or updated or destroyed.
2020-07-16 14:20:41 +05:30
Blake Erickson 1e9ce51151
FIX: Prevent thumbnail gen if image too large (#10247)
It's possible through an import or other means to have images larger
than the current max allowed image size in the db.

If this happens the thumbnail generation job will keep running
indefinitely trying to download a new copy of the original but
discarding it because it is larger than the max_file_size eventually
causing this error

`Job exception: undefined method `path' for nil:NilClass`

because the newly downloaded image is now nil.

This fix stops the enqueuing of the `GenerateTopicThumbnails` job for
all images that happen to be larger than the max image size.
2020-07-16 11:15:53 +10:00
jbrw 06073fe8c6
FEATURE: Allow group moderators to close/archive topics
* FEATURE: Allow group moderators to close/archive topics
2020-07-14 12:36:19 -04:00
David Taylor 3d65678a13
DEV: Add timestamp columns to optimized_images table (#10199)
This allows us to filter by created/updated date when comparing to an S3 inventory.
2020-07-14 11:50:33 +01:00
Rafael dos Santos Silva a4d5d1f51d PERF: Add user_id condition so we can use another index in the query 2020-07-13 15:26:56 -03:00
Arpit Jalan ba57dc57cc DEV: use group_ids instead of group_names in invite params 2020-07-13 17:37:04 +05:30
Bianca Nenciu 4492718864
FIX: Skip whisper posts when updating topic like count (#10157) 2020-07-13 16:30:00 +10:00
Dan Ungureanu c72bc27888
FEATURE: Implement support for IMAP and SMTP email protocols. (#8301)
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2020-07-10 12:05:55 +03:00
Guo Xiang Tan cbe1dd8ec7
Revert "FIX: Delete related search data when record has been deleted."
This reverts commit ecc799ab56.

This commit does not fix anything because we've always been deleting
records in `Searchable`.
2020-07-09 10:08:35 +08:00
Arpit Jalan 78beb4368a FIX: 'resend all invite' button was not working as expected 2020-07-08 15:52:44 +05:30
Martin Brennan 6be7a66ba7
FIX: Cap bookmark name at 100 chars and truncate existing names (#10189)
We have a couple of examples of enormous amounts of text being entered in the name column of bookmarks. This is not desirable...it is just meant to be a short note / reminder of why you bookmarked this.

This PR caps the column at 100 characters and truncates existing names in the database to 100 characters.
2020-07-08 17:19:01 +10:00
Martin Brennan 07ad243603
FIX: Stop updating bookmarked column from TopicUser.update_post_action_cache (#10188)
* This is causing issues where sometimes bookmarked is out of sync with what is in the Bookmark table. The BookmarkManager handles updating this column now.
* Add migration to fix bookmarked column that is incorrectly marked false when a Bookmark record exists.
2020-07-08 15:27:42 +10:00
Kris 66257ca8b6 FEATURE: Add "smallest" option to user text size preferences 2020-07-07 13:08:19 -04:00
Dan Ungureanu e08b860e88
FIX: Improve category hashtag lookup (#10133)
* FIX: Improve category hashtag lookup

This commit improves support for sub-sub-categories and does not include
the ID of the category in the slug, which fixes the composer preview.

* FIX: Sub-sub-categories can be mentioned using only two levels

* FIX: Remove support for three-level hashtags

* DEV: Simplify code
2020-07-07 10:19:01 +10:00
Blake Erickson 2e13ed061d FIX: Generate Topic Thumbnails nil to Array error
If any value, including nil, is passed in as an argument the default
won't be set, so we need to handle when a non-Array value is passed in
to the `generate_thumbnails!` method.
2020-07-06 15:35:33 -06:00
Jarek Radosz 64ce12a758
FIX: `OptimizedImage#filesize` (#10095)
`OptimizedImage#filesize` calls `Discourse.store.download` with an OptimizedImage as an argument. It would in turn attempt to call `#original_filename` and `#secure?` on that object. Both would fail as these methods do not exist on OptimizedImage, only on Upload. We didn't know about these issues because:
1. `#calculate_filesize` is not called often, because the filesize is saved on OptimizedImage creation, so it's used mostly for manual filesize recalculation
2. we were using `rescue nil` which swallows all errors
2020-07-06 17:01:29 +02:00
David Taylor cb1b472a0f
PERF: Do not include thumbnail information in default topic list payload (#10163)
Now it is only included when a theme/plugin has requested it.
2020-07-06 10:59:21 +01:00
Régis Hanol 48b4ed41f5 FIX: uploading an existing image as a site setting
The previous fix (f43c0a5d85) wasn't working for images that were already uploaded.
The "metadata" (eg. 'for_*' and 'secure' attributes) were not added to existing uploads.

Also used 'Upload.get_from_url' is the admin/site_setting controller to properly retrieve
an upload from its URL.

Fixed the Upload::URL_REGEX to use the \h (hexadecimal) for the SHA

Follow-up-to: f43c0a5d85
2020-07-03 19:16:54 +02:00
Roman Rizzi 2df388ffd7
DEV: Plugins can extend ReviewableScore types. (#10156) 2020-07-02 11:47:43 -03:00
Guo Xiang Tan ecc799ab56
FIX: Delete related search data when record has been deleted. 2020-07-02 12:24:59 +08:00
Régis Hanol 860deeb072 FIX: identify slug-less topic urls everywhere
In 91c89df6, I fixed the onebox to support local topics with a slug-less URL.
This commit fixes all the other spots (search, topic links and user badges) where we look up for a local topic.

Follow-up-to: 91c89df6
2020-06-29 12:31:20 +02:00
Régis Hanol 7109d94ee7 FIX: properly invalidate inline oneboxes when rebaking
When rebaking a post we were invalidating _regular_ oneboxes but not inline oneboxes.

DEV: also renamed 'InlineOneboxer.purge' to 'InlineOneboxer.invalidate' to keep
the API consistent with 'Oneboxer.invalidate'
2020-06-24 11:54:54 +02:00
Kane York 52278ce6fd
FIX: Use Discourse.system_user when we need a placeholder admin (#9781) 2020-06-24 15:51:30 +10:00
Jarek Radosz f28ea4751b FIX: A variable name typo
Not that this whole method is used much anymore.
2020-06-19 19:29:19 +02:00
Jarek Radosz 58a5293d9b
FIX: Delete PostUploads on Post deletion (#10090)
Related uploads will then be removed by Jobs::CleanUpUploads
2020-06-19 17:45:08 +02:00
Vinoth Kannan ae47bcb269
DEV: move user references deletion code to `before_destroy`. (#10085)
Moving the `delete_source_user_references` method code from user merger service to user model.
2020-06-18 17:42:39 -04:00
Cassius d13d5e2b99
FIX: Allow multiple hyphens in host (#10078)
* FIX: Allow multiple hyphens in host

* FIX: added spec for multiple hyphens
2020-06-18 13:58:47 -04:00
Bernhard Suttner e31471585a
DEV: allow to have duplicate topic titles if categegory is different (#10034)
Co-authored-by: Robin Ward <robin.ward@gmail.com>

Co-authored-by: Robin Ward <robin.ward@gmail.com>
2020-06-18 11:19:47 -04:00
Dan Ungureanu d21a08c284
DEV: Deprecate Category#url_with_id in favor of Category#url (#9972) 2020-06-18 11:32:14 +03:00
David Taylor 6caad5c083
FIX: Do not send staff welcome message if user already has role 2020-06-17 12:12:55 +01:00
Joffrey JAFFEUX 9da3a7f436
FEATURE: allows published pages to be public (#10053) 2020-06-17 12:42:20 +02:00
David Taylor a2f80670e1
FIX: Do not count youtube thumbnail when counting post images (#10049) 2020-06-15 20:25:30 +01:00
Arpit Jalan a94387c088 FIX: check for inviter group permissions at the time of redeeming invite 2020-06-15 14:44:35 +05:30
Guo Xiang Tan 402b80f306
DEV: Make rails_failover compatible with `SKIP_DB_AND_REDIS` env. 2020-06-15 16:23:24 +08:00
Guo Xiang Tan 58e52c0e4f
DEV: Use rails_failover gem for ActiveRecord and Redis failover handling 2020-06-15 15:47:07 +08:00
Guo Xiang Tan d8cd912769
DEV: Switch to db config to disable advisory locks. 2020-06-15 14:33:41 +08:00
Sam Saffron 7211394e4d
FEATURE: include tags in unread/new messages
We need this information to maintain counts on tracked/watched
tags.
2020-06-12 16:20:30 +10:00
Guo Xiang Tan 2fbd4eb11f
DEV: Remove double query in `Group#remove_review_group`.
Probably a mistake during commit.
2020-06-12 10:14:45 +08:00
Dan Ungureanu b7e70850e4
FIX: Allow users to add emails which were deleted before 2020-06-11 14:54:11 +03:00
Sam Saffron a26b490047
FIX: present correct new/unread counts when filtered by tag
Previously we would incorrectly ignore tags.

This ensures tracking state is properly shipped to client if
show_filter_by_tag is enabled.
2020-06-11 16:47:57 +10:00
Dan Ungureanu 5bfe1ee4f1
FEATURE: Improve UX support for multiple email addresses (#9691) 2020-06-10 19:11:49 +03:00
Dan Ungureanu 3a7ca97c36
FIX: Use include-subcategories filter in report export (#10007)
Some filters were renamed and the conversion of the filter names and arguments
was removed.
2020-06-10 18:57:39 +03:00
David Taylor ae6c4cd237
FIX: Allow merging users when email domains are restricted 2020-06-09 18:51:59 +01:00
Arpit Jalan 3094459cd9
FEATURE: multiple use invite links (#9813) 2020-06-09 20:49:32 +05:30
Guo Xiang Tan edc940ce3e
DEV: Fix incorrect messagebus redis config for rails_failover. 2020-06-09 17:05:14 +08:00
Gerhard Schlager 8c6a42c589 FIX: Redirects containing Unicode usernames didn't work 2020-06-08 10:26:29 +02:00
Vinoth Kannan 0f20a6f0aa FIX: use `short_path` of flair upload to get signed url for secure media.
If we use `upload.url` for secure urls then the images won't render.
2020-06-05 07:43:15 +05:30
Sam Saffron 57a3d4e0d2
FEATURE: whitelist theme repo mode (experimental)
In some restricted setups all JS payloads need tight control.

This setting bans admins from making changes to JS on the site and
requires all themes be whitelisted to be used.

There are edge cases we still need to work through in this mode
hence this is still not supported in production and experimental.

Use an example like this to enable:

`DISCOURSE_WHITELISTED_THEME_REPOS="https://repo.com/repo.git,https://repo.com/repo2.git"`

By default this feature is not enabled and no changes are made.

One exception is that default theme id was missing a security check
this was added for correctness.
2020-06-03 13:19:57 +10:00
Guo Xiang Tan 062db10c52
FIX: `EmailValidator` needs to validate format of email. 2020-06-03 10:34:37 +08:00
Guo Xiang Tan 439db7ca1e
DEV: Add `REDIS_RAILS_FAILOVER` env to test our new redis failover. 2020-06-02 17:24:14 +08:00
David Taylor 75b1298e99
DEV: Drop unused image_url column from posts and topics (#9953)
This has been superseded by image_upload_id. The image_url value in API responses is now generated dynamically from the upload record.
2020-06-02 16:21:38 +10:00
Guo Xiang Tan 7897010dfd
Fix lint. 2020-06-02 10:45:18 +08:00
Guo Xiang Tan c21e3bbd1b
DEV: Prefer `create!` over `create` when not checking return value.
We don't want stuff to silently fail.
2020-06-02 10:34:05 +08:00
Penar Musaraj fa6aa7f627
FIX: Digest frequency issue on user creation
If `default email digest frequency` was set to "Never", users would get
a `digest_after_minutes` set to `nil` which triggered this error
in the logs if/when the site eventually changed that setting and
enabled digests:

```
NoMethodError (undefined method `>=' for nil:NilClass)
/var/www/discourse/app/mailers/user_notifications.rb:227:in `digest'
```
2020-06-01 17:39:16 -04:00
Jarek Radosz 00aab49829
FIX: Restore the ability to rebake posts in deleted topics (#9951)
Lost in #9852
2020-06-01 07:04:16 +02:00
Mark VanLandingham 94cb5ab172
DEV: Remove unused variable topic_tracking_state (#9915) 2020-05-29 09:47:25 -05:00
David Taylor 28f46c171c
FIX: Pull hotlinked images even when edited by system users (#9890)
Previously the pull hotlinked images job was skipped after system edits. This ensured that we never had an infinite loop of system-edit/pull-hotlinked/system-edit/pull-hotlinked etc.

A side effect was that edits made by system for any other reason (e.g. API, removing full quotes) would prevent pulling hotlinked images. This commit removes the system edit check, and replaces it with another method to avoid an infinite job scheduling loop.
2020-05-29 13:07:47 +01:00
David Taylor d29d69e10d
FIX: Invalidate database theme cache when hostname changes (#9908)
Hostname can vary per-site on a multisite cluster, so this change requires converting the compiler_version from a constant into a class method which is evaluated at runtime. The value is stored in the theme DistributedCache, so performance impact should be negligible.
2020-05-29 13:04:51 +01:00
Sam Saffron 2acec4370b
FIX: muted tags removed topics with no tags from counts
We previously did not account for completely untagged topics when
looking at muted tags, this caused new/unread counts to be off if

1. You had muted tags
2. You had an unread/new topic
3. This topic had no tags
2020-05-29 12:59:47 +10:00
Sam Saffron 136835370c
FEATURE: optionally allow tags in topic tracking state
This feature allows certain plugins to output tag information
to topic tracking state, this allows per tag stats which can be
used by sidebars and other plugins that need per tag stats

Not enabled by default cause this would add cost to a critical
query
2020-05-29 12:59:47 +10:00
Roman Rizzi a41476800b
FIX: Don't raise an exception if a topic cannot be retrieved (#9906) 2020-05-28 11:59:20 -03:00
Guo Xiang Tan 8c86a109bb
DEV: Add ENV flag to test out `ActiveRecord::Failover`. 2020-05-28 16:24:22 +08:00
Joffrey JAFFEUX 77801aa9be
FIX: allows to have custom emoji translation without static file (#9893) 2020-05-27 20:11:52 +02:00
Roman Rizzi 461df7d050
UX: Rename Priority to score for sorting. (#9846) 2020-05-27 12:50:28 -03:00
Blake Erickson da839e6d26 SECURITY: Use FinalDestination for topic embeds 2020-05-27 09:26:09 -06:00
Régis Hanol 2a4db15544 FIX: don't send digests to users with no primary email
It might happen that some User records have no associated primary emails.
In which case we don't ever want to send them a digest.

Also added a new "user_email_no_email" skipped email log to ensure these cases
are properly handled and surfaced.
2020-05-27 17:09:40 +02:00
Arpit Jalan 2152e70e0d DEV: annotate models 2020-05-27 19:05:24 +05:30
Joffrey JAFFEUX 1d685c22af
REVERT: removes translate_emoji (#9889)
This API is actually used in some plugins.
2020-05-27 12:08:24 +02:00
Bianca Nenciu f47400475e
FEATURE: Send a private message when a group membership is accepted (#9822)
* FEATURE: Send a private message when a group membership is accepted

* DEV: Small code improvements

* FIX: Send PM as group owner

* Copy edits
2020-05-26 16:28:03 +03:00
Sam Saffron 76c4bc925d
DEV: followup to prev commit
337bd9a0f7 did not account for optimized image being nil
2020-05-26 16:19:05 +10:00
Sam Saffron 337bd9a0f7
FIX: concurrency bug when creating topic thumbnails
We were failing erratically when backfilling topic thumbnails.

This ensures that racing threads/processes will not conflict.
2020-05-26 16:10:22 +10:00
Osama Sayegh 2211581a85
FIX: Don't responde with error 500 if domain is invalid when adding automatic membership domain (#9655) 2020-05-26 15:40:09 +10:00
Arpit Jalan 5462fe9462
FIX: do not allow tag with name 'none' (#9867)
https://meta.discourse.org/t/none-tag-is-uneditable/152003
2020-05-26 08:15:45 +05:30
Guo Xiang Tan 878f06f1fe DEV: Remove custom connection reaper.
Rails 6 fixed the reaper to use one thread to reap all the connection pools.
2020-05-26 09:09:46 +08:00
Sam Saffron fc97f7e0e7
FIX: properly ban non human users from draft system
Previously we had a partial fix in place where non human users
were not allowed draft sequences, this left edges around where non
human users asked for drafts yet had none.

For example system could already have a few drafts in place.

This also removes and extensibility point we added that is not in use
2020-05-26 10:07:09 +10:00
Vinoth Kannan 5fb9271878
DEV: ignore `flair_url` column in group model. (#9873) 2020-05-26 00:43:50 +05:30
Gerhard Schlager 631024ae5d FEATURE: Permalinks for tags 2020-05-25 14:51:01 +02:00
Sam Saffron c011b817b2
FIX: do not fallback to flair_url column
flair_url is not loaded by our query anymore so avoid falling back
to the old value
2020-05-25 16:25:42 +10:00
Vinoth Kannan 8e56197728
UX: use "icon-picker" & "image-uploader" fields to set group flair. (#9779) 2020-05-25 11:08:47 +05:30
Gerhard Schlager 13d5ccedf5 FIX: Destroying a user failed when it had title
...because updating the user caused a validation of the primary email and the UserEmail records are already deleted at that time.
2020-05-24 22:14:50 +02:00
Michael Brown d9a02d1336
Revert "Revert "Merge branch 'master' of https://github.com/discourse/discourse""
This reverts commit 20780a1eee.

* SECURITY: re-adds accidentally reverted commit:
  03d26cd6: ensure embed_url contains valid http(s) uri
* when the merge commit e62a85cf was reverted, git chose the 2660c2e2 parent to land on
  instead of the 03d26cd6 parent (which contains security fixes)
2020-05-23 00:56:13 -04:00
Jeff Atwood 20780a1eee Revert "Merge branch 'master' of https://github.com/discourse/discourse"
This reverts commit e62a85cf6f, reversing
changes made to 2660c2e21d.
2020-05-22 20:25:56 -07:00
Blake Erickson 03d26cd6f0 SECURITY: ensure embed_url contains valid http(s) uri 2020-05-22 14:54:56 -06:00
Roman Rizzi 671f882fa3
FIX: We don't create a Post object if the queued post gets rejected. We need to count review items directly. (#9856) 2020-05-22 11:50:28 -03:00
Martin Brennan f9d55b4941
FEATURE: Update the topic excerpt when the OP is rebaked (#9852)
* We now have a site setting "topic_excerpt_maxlength" that is used when the OP is created or revised to generate a topic excerpt.
* However, posts created before this setting was introduced cannot benefit from this change unless they are revised, and if the topic excerpt length setting is changed that situation is also not covererd.
* This PR makes a change to rebake! to update the topic excerpt IF the post is the OP.
2020-05-22 13:04:15 +10:00
Martin Brennan c0779df99d
FIX: Remove access control post FK from uploads (#9853) 2020-05-22 11:20:25 +10:00
Martin Brennan df68d11c38
FEATURE: Add topic excerpt max length site setting (#9847)
Adds a new topic_excerpt_maxlength site setting.

* When topic excerpt is requested for a post, use the new topic_excerpt_maxlength site setting to limit the size of the excerpt
* Remove code for getting/setting Post.excerpt_size as it is not used anywhere
2020-05-21 13:19:48 +10:00
Robin Ward 096eca0ee8 FIX: Requests were not being logged correctly
`enable` was defaulting to `nil` which is not what we wanted.
2020-05-20 15:09:17 -04:00
Régis Hanol 8851b79472 DEV: less code for Category#url 2020-05-20 18:01:06 +02:00
Bianca Nenciu fb15da43da
Remove old web hooks in favor of 'reviewable' web hook (#9776)
* FIX: Emit web hooks for flags

* FEATURE: Remove 'flag' web hook in favor of 'reviewable' web hook

* FEATURE: Remove 'queued post' web hook in favor of 'reviewable' web hook

* FIX: Do not set a default value for web hooks with no events
2020-05-20 12:07:48 +03:00
Guo Xiang Tan f7f436e536
DEV: Install `rails_failover` gem to test our Redis changes. 2020-05-20 15:40:27 +08:00
Guo Xiang Tan 234cd5c3e7
FIX: Switch discobot to pull avatar from gravatar. 2020-05-20 10:20:08 +08:00
Sam Saffron 6b92c78033
FIX: purge all associated data on user delete
This commit reorganises the delete dependencies on users and make sure
all are covered. We forgot some on bookmarks, security keys, anon users
and so on.
2020-05-20 11:27:57 +10:00
Dan Ungureanu 05ca0c908f
FIX: GroupArchivedMessage belongs to Group, not User 2020-05-19 14:13:28 +03:00
David Taylor 725e38f9d7
DEV: Allow plugins to request topic thumbnail sizes (#9828)
In plugin.rb, you can register new sizes like

```
register_topic_thumbnail_size [512, 512]
```

For more information about thumbnails see 03818e642a
2020-05-19 10:38:58 +01:00
Gerhard Schlager 6d5e9db883 FIX: Restoring backup didn't clear cached translation overrides 2020-05-18 18:51:51 +02:00
Gerhard Schlager 27c611b1d1 FIX: Reverting multiple translation overrides didn't clear cache 2020-05-18 18:51:29 +02:00
David Taylor f968155288
FIX: Ensure no image downloads during topic thumbnail serialization
If image uploads are missing a width/height, we re-download them to update the width/height in the database. However, this takes significant resources so we don't want to run this logic during topic serialization.
2020-05-18 12:04:29 +01:00
Guo Xiang Tan d01c336899
DEV: Clean up some Redis leaks in test env. 2020-05-18 17:27:37 +08:00
David Taylor 461b4e5cc6
DEV: Add framework for filtered plugin registers (#9763)
* DEV: Add framework for filtered plugin registers

Plugins often need to add values to a list, and we need to filter those lists at runtime to ignore values from disabled plugins. This commit provides a re-usable way to do that, which should make it easier to add new registers in future, and also reduce repeated code.

Follow-up commits will migrate existing registers to use this new system

* DEV: Migrate user and group custom field APIs to plugin registry

This gives us a consistent system for checking plugin enabled state, so we are repeating less logic. API changes are backwards compatible
2020-05-15 14:04:38 +01:00
David Taylor 0495a748d0
FIX: Use CDN URLs for topic thumbnails 2020-05-15 13:35:20 +01:00
Sam Saffron 0cbaa8d813
FEATURE: extend duration allowed for download
Previously we would raise a warning in the logs if downloading
a file (from s3) takes longer than 60 seconds.

At scale this happens reasonably frequently.

1. Raised the duration to 3 minutes

2. Pulled the resizing mutex out of the downloading mutex
so we have less and clearer error logs
2020-05-15 12:45:47 +10:00
Guo Xiang Tan 165fa13c9d
FIX: Always return 0 for non-human user current DraftSequence. 2020-05-15 10:40:36 +08:00
Kane York 869f9b20a2
PERF: Dematerialize topic_reply_count (#9769)
* PERF: Dematerialize topic_reply_count

It's only ever used for trust level promotions that run daily, or compared to 0. We don't need to track it on every post creation.

* UX: Add symbol in TL3 report if topic reply count is capped

* DEV: Drop user_stats.topic_reply_count column
2020-05-14 15:42:00 -07:00
Mark VanLandingham bec6bace53
DEV: Allow plugins to hide user stats by new guardian method (#9772) 2020-05-14 11:57:35 -05:00
Kane York 1059557ce1
DEV: Standardize ignored_columns removal comments (#9771) 2020-05-13 13:08:15 -07:00
Sam Saffron 7f841dc21f
PERF: avoid race conditions when creating topic links
Previously the code was very race condition prone leading to
odd failures in production

It was re-written in raw SQL to avoid conditions where rows
conflict on inserts

There is no clean way in ActiveRecord to do:

Insert, on conflict do nothing and return existing id.

This also increases test coverage, we were previously not testing
the code responsible for crawling external sites directly
2020-05-13 16:05:57 +10:00
Guo Xiang Tan 34c39b65bb
PERF: Avoid executing the same query twice. 2020-05-13 11:10:09 +08:00
Martin Brennan 9981fa4466
FIX: Prevent column name conflicts in reviewable code (#9753)
We were getting errors like this in Reviewables in some cases:

```
ActiveRecord::StatementInvalid (PG::AmbiguousColumn: ERROR:  column reference "category_id" is ambiguous
LINE 4: ...TRUE) OR (reviewable_by_group_id IN (NULL))) AND (category_i...
```

The problem that was making everything go boom is that plugins can add their own custom filters for Reviewables. If one is doing an INNER JOIN on topics, which has its own category_id column, we would get the above AmbiguousColumn error. The solution here is to just make all references to the reviewable columns in the list_for and viewable_by code prefixed by the table name e.g. reviewables.category_id.
2020-05-13 09:05:56 +10:00
Daniel Waterworth 497dc6eaa7 Add a global setting for CDN origin
This is so that, on a multisite cluster, when we handle a CDN request,
the hostname that is requested corresponds to one of the sites -
specifically the default site.
2020-05-12 16:43:40 +01:00
Robin Ward 088c11a12c FIX: Don't raise an exception if a TopicLink cannot be created
This can happen under concurrency and it's fine. If a `TopicLink`
already exists we can continue on without raising an error.
2020-05-12 11:15:59 -04:00
Sam Saffron a29ae17d3a
FIX: saving drafts unconditionally increases sequence
Previously we only changed sequence on ownership change, this
cause a race condition between tabs where user could type for a
long time without being warned of an out of date draft.

This change is a radical change and we should watch closely.

Code was already in place to track sequence on the client so no
changes are needed there.
2020-05-12 16:55:42 +10:00
Guo Xiang Tan 8fb99f218d
FEATURE: Allow plugins to register a callback to ignore DraftSequence. 2020-05-12 14:25:20 +08:00
Guo Xiang Tan d301af39bd
PERF: Avoid hitting DB when fetching draft sequence of bot user. 2020-05-12 13:56:58 +08:00
Bianca Nenciu 8149bfbaf1
FEATURE: Filter settings by plugin (#9692) 2020-05-10 14:07:45 +03:00
Robin Ward b2e4ca968b DEV: Remove `translate` API from plugin.
It appears to be completely unused at this point.
2020-05-08 14:14:01 -04:00
Guo Xiang Tan efd0e4d02b
FIX: Don't trigger user_updated event for bot users. 2020-05-08 09:28:16 +08:00
Martin Brennan 37be2958f7
DEV: Allow specifying if a notification is high_priority on create (#9660)
This allows for special cases where we may not want a certain notification type to ALWAYS be high priority (e.g. a topic timer).
2020-05-07 14:35:32 +10:00
Martin Brennan 6fb0f36ce1
FEATURE: Optionally delete bookmark when reminder sent (#9637)
We now show an options gear icon next to the bookmark name.

When expanded we show the "delete bookmark when reminder sent" option. The value of this checkbox is saved in local storage for the user.

If this is ticked, when a reminder is sent for the bookmark the bookmark itself is deleted. This is so people can use the reminder functionality by itself.

Also remove the blue alert reminder section from the "Edit Bookmark" modal as it just added clutter, because the user can already see they had a reminder set:

Adds a default false boolean column `delete_when_reminder_sent` to bookmarks.
2020-05-07 13:37:39 +10:00
Robin Ward f5bdbd347e Revert "FIX: Discobot has not been created with our custom avatar."
This reverts commit 1062dbc3e9.

Looks like it's causing some errors on migration.

```
/var/www/discourse/lib/image_sizer.rb:6:in `resize'
/var/www/discourse/lib/upload_creator.rb:120:in `block in create_for'
/var/www/discourse/lib/distributed_mutex.rb:33:in `block in synchronize'
/var/www/discourse/lib/distributed_mutex.rb:29:in `synchronize'
/var/www/discourse/lib/distributed_mutex.rb:29:in `synchronize'
/var/www/discourse/lib/distributed_mutex.rb:14:in `synchronize'
/var/www/discourse/lib/upload_creator.rb:37:in `create_for'
```
2020-05-06 11:28:42 -04:00
Martin Brennan fa572d3a7a
DEV: Remove all code referencing at_desktop bookmark reminders (#9650)
We have found no need for these reminder types, so we are removing the code for them.
2020-05-06 15:22:43 +10:00
Guo Xiang Tan 1062dbc3e9 FIX: Discobot has not been created with our custom avatar.
Previously the image was imported from a Discourse hosted CDN but the
URL has since become invalid. However, it was not caught since all
errors are rescued. This commit fixes the issue by shipping the user
avatar with the plugin.
2020-05-06 09:25:01 +08:00
Robin Ward 612284cef3
DEV: Remove `Discourse.RAW_TEMPLATES` (#9630)
We were sharing `Discourse` both as an application object and a
namespace which complicated things for Ember CLI. This patch
moves raw templates into `__DISCOURSE_RAW_TEMPLATES` and adds
a couple helper methods to create/remove them.
2020-05-05 12:15:03 -04:00
David Taylor 03818e642a
FEATURE: Include optimized thumbnails for topics (#9215)
This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this:

- Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact

- A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake

- Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued

- Topic lists and topics now include a `thumbnails` key, which includes all the available images:
   ```
   "thumbnails": [
   {
     "max_width": null,
     "max_height": null,
     "url": "//example.com/original-image.png",
     "width": 1380,
     "height": 1840
   },
   {
     "max_width": 1024,
     "max_height": 1024,
     "url": "//example.com/optimized-image.png",
     "width": 768,
     "height": 1024
   }
   ]
  ```

- Themes can request additional thumbnail sizes by using a modifier in their `about.json` file:
   ```
    "modifiers": {
      "topic_thumbnail_sizes": [
        [200, 200],
        [800, 800]
      ],
      ...
  ```
  Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated

- Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 09:07:50 +01:00
Krzysztof Kotlarek 9bff0882c3
FEATURE: Nokogumbo (#9577)
* FEATURE: Nokogumbo

Use Nokogumbo HTML parser.
2020-05-05 13:46:57 +10:00
David Taylor a51b8d9c66
FIX: Do not use cached settings during theme compilation
We compile within a database transaction, so using a cached value from redis can cause unwanted side effects
2020-05-04 09:43:06 +01:00
David Taylor 4f885d7da2
FIX: Clear theme caches after database transaction has committed
This was causing unusual side effects on high-traffic sites, where the cache would be rebuilt before the transaction had been committed
2020-05-04 09:42:50 +01:00
Kane York 784bf2a173
FIX: Unassign user titles when a badge is deleted (#9573) 2020-05-02 18:02:28 -07:00
Krzysztof Kotlarek 37e93914fc
FIX: the muted message should be sent after edit (#9593)
Recently, we added feature that we are sending `/muted` to users who muted specific topic just before `/latest` so the client knows to ignore those messages - https://github.com/discourse/discourse/pull/9482

Same `/muted` message should be included when the post is edited
2020-05-01 08:33:57 +10:00
Jeff Wong 2cb9e85d14
FEATURE: add category banner for why a user cannot post (#9576)
* FEATURE: add category banner for why a user cannot post

Adds a category banner for why a user is unable to post in a category.

Also adds an extra alert for the user when a user is unable to create a topic in a
category and they still try and click on the disabled-looking new topic
button.
2020-04-30 10:39:11 -07:00
Sam Saffron d0d5a138c3
DEV: stop freezing frozen strings
We have the `# frozen_string_literal: true` comment on all our
files. This means all string literals are frozen. There is no need
to call #freeze on any literals.

For files with `# frozen_string_literal: true`

```
puts %w{a b}[0].frozen?
=> true

puts "hi".frozen?
=> true

puts "a #{1} b".frozen?
=> true

puts ("a " + "b").frozen?
=> false

puts (-("a " + "b")).frozen?
=> true
```

For more details see: https://samsaffron.com/archive/2018/02/16/reducing-string-duplication-in-ruby
2020-04-30 16:48:53 +10:00
Sam Saffron 310a7edee5
DEV: remove unused columns from posts and topics
avg_time on posts and topics have not been used in a year.

This uses a re-runnable ddl transaction diasabled migration to
drop the column, cause it touchs very high traffic table and may
deadlock
2020-04-30 11:22:33 +10:00
Neil Lalonde 9d781d4959
FIX: error customizing text for badges from plugins 2020-04-28 14:34:41 -04:00
Arpit Jalan 198856f78d DEV: annotate models 2020-04-28 15:59:39 +05:30
mentalstring 67f3fe14aa
FEATURE: support SSO website and location overrides
Add location and website + the ability to override using SSO using the `sso_overrides_location` and `sso_overrides_website` site settings.
2020-04-28 16:06:35 +10:00
Mark VanLandingham 051e6be7b8
DEV: Trigger 'user_added_to_group' when accepting invite with groups (#9561) 2020-04-27 15:13:34 -05:00
Mark VanLandingham d8c884a856
DEV: Add has_many category_tag_stats in tag.rb (#9558) 2020-04-27 12:42:13 -05:00
David Taylor 8a112b7464
DEV: Automatically extend CSP when themes link to external scripts (#9531) 2020-04-24 09:47:01 +01:00
Sam Saffron cbceadf48b
FEATURE: when blocking emails prefer blocking canonical
Previously we relied entirely on levenshtein_distance_spammer_emails site
setting to handle "similar looking" emails.

This commit improves the situation by always preferring to block (and check)
canonical emails.

This means that if:

`samevil+test@domain.com` is blocked the system will block `samevil@domain.com`

This means that `samevil+2@domain.com` (ad infinitum) will be blocked
2020-04-24 14:09:51 +10:00
Sam Saffron 6a18c9aa0b
Revert "FEATURE: enforce_canonical_emails site setting"
This reverts commit 6f9177e2ed.

We decided on a completely different approach to the problem.

Instead we will let blocked emails be treated as canonical.
2020-04-24 13:52:06 +10:00
Martin Brennan cd1c7d7560
FIX: Copying image markdown for secure media loading full image (#9488)
* When copying the markdown for an image between posts, we were not adding the srcset and data-small-image attributes which are done by calling optimize_image! in cooked post processor
* Refactored the code which was confusing in its current state (the consider_for_reuse method was super confusing) and fixed the issue
2020-04-24 10:29:02 +10:00
Krzysztof Kotlarek 52c1d7337e
FEATURE: don't display new/unread notification for muted topics (#9482)
* FEATURE: don't display new/unread notification for muted topics

Currently, even if user mute topic, when a new reply to that topic arrives, the user will get "See 1 new or updated topic" message. After clicking on that link, nothing is visible (because the topic is muted)

To solve that problem, we will send background message to all users who recently muted that topic that update is coming and they can ignore the next message about that topic.
2020-04-23 14:57:35 +10:00
Robin Ward e4a0e0bead FIX: Ignore removed column 2020-04-22 14:27:01 -04:00
Vinoth Kannan df0c386f8a
UX: drop the `automatic_membership_retroactive` column from groups model. (#9430) 2020-04-22 22:07:39 +05:30
Daniel Waterworth c8fed90e4e FIX: Handle sub-sub-category paths without an id 2020-04-22 12:19:54 +01:00
Dan Ungureanu 4e5f9d4cd1
DEV: Drop 'key' column from user_api_keys (#9388) 2020-04-22 12:13:19 +03:00
Dan Ungureanu e733701887
FEATURE: Make report filters reusable (#9444)
This commit also adds 'include subcategories' report filter
2020-04-22 11:52:50 +03:00
Martin Brennan 628ba9d1e2
FEATURE: Promote bookmarks with reminders to core functionality (#9369)
The main thrust of this PR is to take all the conditional checks based on the `enable_bookmarks_with_reminders` away and only keep the code from the `true` path, making bookmarks with reminders the core bookmarks feature. There is also a migration to create `Bookmark` records out of `PostAction` bookmarks for a site.

### Summary

* Remove logic based on whether enable_bookmarks_with_reminders is true. This site setting is now obsolete, the old bookmark functionality is being removed. Retain the setting and set the value to `true` in a migration.
* Use the code from the rake task to create a database migration that creates bookmarks from post actions.
* Change the bookmark report to read from the new table.
* Get rid of old endpoints for bookmarks
* Link to the new bookmarks list from the user summary page
2020-04-22 13:44:19 +10:00
Jarek Radosz a781ef7662
FIX: Reject invalid Category slugs (#9473)
Previously it would sanitize given slug and then save the resulting empty slug.
2020-04-21 03:50:50 +02:00
Jarek Radosz 17cf300b71
DEV: Use more specific error responses (#9472)
* DEV: Use `render_json_error` (Adds specs for Admin::GroupsController)
* DEV: Use a specific error on blank category slug (Fixes a `render_json_error` warning)
* DEV: Use a specific error on reviewable claim conflict (Fixes a `render_json_error` warning)
* DEV: Use specific errors in Admin::UsersController (Fixes `render_json_error` warnings)
* FIX: PublishedPages error responses
* FIX: TopicsController error responses (There was an issue of two separate `Topic` instances for the same record. This makes sure there's only one up-to-date instance.)
2020-04-21 03:50:20 +02:00
Mark VanLandingham 06796bec8b
FIX: Set category description to first posts cooked value (#9478) 2020-04-20 14:51:49 -05:00
Robin Ward 98729c8e6e FIX: Allow embed updates of just the title 2020-04-20 14:31:24 -04:00
Robin Ward 56a23c68f1 FIX: Embedded topics couldn't update their titles 2020-04-20 14:27:43 -04:00
Neil Lalonde 074509fd95
FIX: don't demote users to TL2 when default trust level is 3
Within 24 hours of signing up, new users were losing their
default trust level of 3. With this fix, demotions from
trust level 3 won't happen when the "default trust level"
setting is 3 or 4.
2020-04-16 12:28:16 -04:00
Martin Brennan 51672b9121
FIX: Minor bookmark with reminder issue cleanup (#9436)
* Count user summary bookmarks from new Bookmark table if bookmarks with reminders enabled
* Update topic user bookmarked column when new topic bookmark changed
* Make in:bookmarks search work with new bookmarks
* Fix batch inserts for bookmark rake task (and thus migration). We were only inserting one bookmark at a time, completely defeating the purpose of batching!
2020-04-16 11:32:21 +10:00
Vinoth Kannan 884eea7a83 FEATURE: add support for `upload` format in theme settings. 2020-04-15 18:34:02 +05:30
Sam Saffron 6f9177e2ed
FEATURE: enforce_canonical_emails site setting
The new `enforce_canonical_emails` site setting ensures that emails in the
canonical form are unique.

This mean that if `s.a.m+1@gmail.com` is registered `sam@gmail.com` will
not be allowed.

The commit contains a blanket "tag strip" (stripping everything after +)
it also contains special handling of a "dot strip" for googlemail and gmail.

The setting only impacts new registrations after `enforce_canonical_emails`

The setting is default false so it will not impact any existing installs.
2020-04-14 14:16:30 +10:00
Robin Ward b6b92a562c
FEATURE: New site setting `embed_unlisted` (#9391)
If enabled, posts imported to discourse via embeddings will default to
unlisted until they receive a reply.
2020-04-13 15:17:02 -04:00
Dan Ungureanu 799ddeea3c
FIX: Include subcategories in 'posts' report (#9410) 2020-04-13 18:57:52 +03:00
Vinoth Kannan b57d4586d7 FIX: use correct command line attribute for `gifsicle` while scale down the gif. 2020-04-10 18:16:47 +05:30
Dan Ungureanu 3d9c320aab
PERF: Cache Category.subcategory_ids (#9350)
Also reset category cache after backup restore.
2020-04-09 15:42:24 +03:00
Kane York 138d4aebde FEATURE: Webhooks and Event for user being granted a badge
Adding a webhook for badge revocation is left for future work as it's relatively rare.
2020-04-08 14:38:30 -07:00
Robin Ward e1f8014acd
FEATURE: Support for publishing topics as pages (#9364)
If the feature is enabled, staff members can construct a URL and publish a
topic for others to browse without the regular Discourse chrome.

This is useful if you want to use Discourse like a CMS and publish
topics as articles, which can then be embedded into other systems.
2020-04-08 12:52:36 -04:00
Dan Ungureanu b64b590cfb
FIX: Add index on user_api_keys.key_hash (#9387) 2020-04-08 19:49:18 +03:00
Roman Rizzi 8e1bdc9458
FIX: Prevent low score flags from auto-closing a topic if the reviewable default visibility is higher than low (#9375) 2020-04-08 10:44:31 -03:00
Sam Saffron b824898f61
FIX: respect automatic group membership when sso changes email
Previously we were only updating group membership when a user record was
first created in an SSO setting.

This corrects it so we also update it if SSO changes the email
2020-04-08 16:33:50 +10:00
Neil Lalonde c45574c9e7
FIX: exclude private messages from TL3 requirements
It wasn't intended that people should be able to earn trust level
3 without participating in public topics. When counting topic
views and likes given/received, don't count private topics.
2020-04-07 17:36:07 -04:00
Krzysztof Kotlarek a811976023
FIX: reset gravatar cache by adding random param to URL (#9370)
Users noticed that sometimes, avatar from Gravatar is not correctly updated - https://meta.discourse.org/t/updated-image-on-gravatar-not-seeing-it-update-on-site/54357

A potential reason for that is that even if you update your avatar in Gravatar, URL stays the same and if the cache is involved, service is still receiving the old photo.

For example. In my case, when I click the button to refresh avatar the
new Upload record is created with `origin` URL to new avatar, and `url` to
old one

I made some tests in the rails console and adding random param to Gravatar URL is deceiving cache and correct, the newest avatar is downloaded
2020-04-08 07:35:42 +10:00
Roman Rizzi e52e7f6e75
FIX: Include pending queued users regardless of their score (#9372) 2020-04-07 11:42:12 -03:00
Dan Ungureanu 0653750fbf
FEATURE: Hash user API keys in the database (#9344)
The 'key' column will be dropped in a future commit.
2020-04-07 16:42:52 +03:00
David Taylor 3814ca06a8
DEV: Allow using .js extension for javascript modules in themes (#9358) 2020-04-06 17:24:59 +01:00
Vinoth Kannan fd39c85c1a FIX: add category hashtags support for sub-sub categories.
Hashtags will include last two levels only (ex: "parent:child").
2020-04-06 20:43:38 +05:30
Dan Ungureanu f8ec5f309a
FIX: Ensure category_id is an integer 2020-04-04 14:20:01 +03:00
Dan Ungureanu b9d411a4eb
FIX: Topic.time_to_first_response should include sub-sub-categories (#9349) 2020-04-04 13:31:34 +03:00
David Taylor 3cfe086a94
FIX: Limit personal message participants when converting from topic (#9343)
Previously all topic posters would be added which could lead to major performance issues. Now if there are too many posters, only the acting user will be added as a participant.
2020-04-03 16:42:01 +01:00
Martin Brennan d261a809e2
FEATURE: Bookmark reminder type changes and bugfixes (#9329)
New Reminder Types
-------------------------------------

* Add a "later this week" reminder which is today + 2 days, will not show if we are on the days Thu-Sun
* Add a "start of next business week" reminder which is 8am Monday

Bugfixes and Tweaks
--------------------------------------

* Move dates out of translation for reminder types and yield HTML for tap-tile for more customizable content and styling
* Make sure double clicking the bookmark icon in quick access takes users to the new bookmarks-with-reminders page
* Sane default to 8am (start of day) for custom reminder with no time
2020-04-02 09:57:48 +10:00
Kane York cdaa60b56b FEATURE: Allow admins to disable self-service account deletion
https://meta.discourse.org/t/-/146276
2020-04-01 15:16:07 -07:00
Martin Brennan debed81022 Get rid of named param double reference in UserBookmarkList 2020-04-01 15:04:04 +10:00
Martin Brennan c07dd0d22a
FEATURE: Add lazy loading to user bookmarks list (#9317)
This is so users with huge amount of bookmarks do not have to wait a long time to see results.

* Add a bookmark list and list serializer to server-side to be able to handle paging and load more URL
* Use load-more component to load more bookmark items, 20 at a time in user activity
* Change the way current user is loaded for bookmark ember models because it was breaking/losing resolvedTimezone when loading more items
2020-04-01 14:09:07 +10:00
Martin Brennan b79ea986ac
FEATURE: High priority bookmark reminder notifications (#9290)
Introduce the concept of "high priority notifications" which include PM and bookmark reminder notifications. Now bookmark reminder notifications act in the same way as PM notifications (float to top of recent list, show in the green bubble) and most instances of unread_private_messages in the UI have been replaced with unread_high_priority_notifications.

The user email digest is changed to just have a section about unread high priority notifications, the unread PM section has been removed.

A high_priority boolean column has been added to the Notification table and relevant indices added to account for it.

unread_private_messages has been kept on the User model purely for backwards compat, but now just returns unread_high_priority_notifications count so this may cause some inconsistencies in the UI.
2020-04-01 09:09:20 +10:00
Arpit Jalan b2a0d34bb7
FEATURE: add setting `auto_approve_email_domains` to auto approve users (#9323)
* FEATURE: add setting `auto_approve_email_domains` to auto approve users

This commit adds a new site setting `auto_approve_email_domains` to
auto approve users based on their email address domain.

Note that if a domain already exists in `email_domains_whitelist` then
`auto_approve_email_domains` needs to be duplicated there as well,
since users won’t be able to register with email address that is
not allowed in `email_domains_whitelist`.

* Update config/locales/server.en.yml

Co-Authored-By: Robin Ward <robin.ward@gmail.com>
2020-03-31 23:59:15 +05:30
Krzysztof Kotlarek a0f59a55cc
FIX: when a post is moved copy notifications level (#9311)
This is a revert of 2c011252f1

More information on meta: https://meta.discourse.org/t/when-a-reply-is-moved-to-a-new-topic-the-followers-of-the-previous-topic-are-automatically-follower-of-the-new-topic-as-well/130025
2020-03-31 16:19:47 +11:00
Robin Ward db35baba26 FEATURE: Display "Last Updated At" on user directory 2020-03-30 14:34:48 -04:00
Joffrey JAFFEUX 0996c3b7b3
FEATURE: allows multiple custom emoji groups (#9308)
Note: DBHelper would fail with a sql syntax error on columns like "group".

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2020-03-30 20:16:10 +02:00
Arpit Jalan 435fe8ac6e FIX: allow invite email field to be blank for invite tokens 2020-03-30 10:35:49 +05:30
Vinoth Kannan 257f59f366 FEATURE: option to update child theme components via theme CLI.
423ce44112
2020-03-27 03:41:56 +05:30
Robin Ward a12c7fcb2f Revert "FIX: Allow JS transpilation"
This reverts commit 6d1263812f.
2020-03-25 16:13:01 -04:00
Robin Ward 6d1263812f FIX: Allow JS transpilation 2020-03-25 16:05:16 -04:00
romanrizzi 55b8620b43 FIX: TopicEmbed#absolutize_urls was trying to modify a frozen string 2020-03-25 12:57:54 -03:00
Jeff Wong 7282435e17 FEATURE: Unassign the review queue topic when a flag is handled 2020-03-24 14:57:44 -07:00
David Taylor c939001876
DEV: Bump theme compiler version for CSP change
We need to regenerate cached themes, so that the JS paths are updated to include the subfolder prefix

Followup to 8e98f39d9f
2020-03-24 15:52:14 +00:00
Sam Saffron 46a9622246
FIX: prevent scheduled publishing to deleted category
We missed a dependency which left timers firing on missing categories.

Co-authored-by: tshenry
2020-03-24 16:59:42 +11:00
David Taylor 5ff505cea6
SECURITY: Respect topic permissions when loading draft metadata
Co-authored-by: Sam Saffron <sam.saffron@gmail.com>
2020-03-23 11:30:40 +00:00
David Taylor 3215f2b6ee
FIX: Permalinks should redirect to category URL including the ID
This is a temporary fix. Urls for third-level categories should function without the id. Once that is fixed, this change can be reverted
2020-03-20 22:06:20 +00:00
romanrizzi cfec10a568 FIX: Moderators should be able to review flagged PMs since this has always been like this 2020-03-20 12:28:36 -03:00
Daniel Waterworth 1b24a7b993 FIX: Include entire slug path in permalinks
This is a temporary fix since these URLs should contain the id as well.
2020-03-20 10:43:13 +00:00
Roman Rizzi 080960a15e
FIX: Show the envelope icon when the flagged post is a PM. Flagged PM must be exclusively reviewed by admins (#9232) 2020-03-19 15:12:42 -03:00
Rafael dos Santos Silva 226d81fcc5 FIX: Fix a PostgreSQL error when a draft was concurrently created
Moves the new draft creation concurrency handling to PostgreSQL
so the database doesn't error out when the draft is being created
by multiple backends.

Also removes `retry_not_unique` parameter from Draft#set` which is
not called anywhere.

Also fixes a draft update not bumping the `updated_at` column.
2020-03-19 13:31:11 -03:00
Vinoth Kannan f6d6f1701f FIX: use the new duration attribute in `set_or_create_timer` method.
New `duration` attribute is introduced for the `set_or_create_timer` method in the commit aad12822b7 for "based on last post" and "auto delete replies" topic timers.
2020-03-19 21:45:05 +05:30
Vinoth Kannan aad12822b7
FEATURE: automatically delete replies on a topic after N days. (#9209) 2020-03-19 21:06:31 +05:30
Kane York ccc9b64a99 FIX: theme-javascripts using incorrect subfolder setting 2020-03-17 19:09:06 -07:00
Justin DiRose 9101227266
FEATURE: Demote muted categories on category list (#9226) 2020-03-17 15:33:15 -05:00
Blake Erickson 919e405c48
FIX: Don't display webhooks for inactive plugins (#9206)
* FIX: Don't display webhooks for inactive plugins

This commit ensures that we don't show webhooks for plugins that are not
installed or that are disabled.

Bug report:

https://meta.discourse.org/t/webhookeventtype-and-the-solved-and-assign-plugins/144180

* rename to just 'active', it's cleaner
2020-03-17 10:39:24 -06:00
Jarek Radosz e950471c0f
DEV: Replace User.unstage and User#unstage API with User#unstage! (#8906)
* DEV: Replace User.unstage and User#unstage API with User#unstage!

Quoting @SamSaffron:

> User.unstage mixes concerns of both unstaging users and updating params which is fragile/surprising.
> u.unstage destroys notifications and raises a user_unstaged event prior to the user becoming unstaged and the user object being saved.

User#unstage! no longer updates user attributes and saves the object before triggering the `user_unstaged` event.

* Update one more spec

* Assign attributes after unstaging
2020-03-17 16:48:24 +01:00
Roman Rizzi 4663304775
FEATURE: Show rejected posts count in user summary (#9204) 2020-03-16 09:52:08 -03:00
Sam Saffron a1d660d951
FEATURE: optional global invite_code for account registration
On some sites when bootstrapping communities it is helpful to bootstrap
with a "light weight" invite code.

Use the site setting `invite_code` to set a global invite code.

In this case the administrator can share the code with
a community which is very easy to remember and then anyone who has
that code can easily register accounts.

People without the invite code are not allowed account registration.

Global invite codes are less secure than indevidual codes, in that they
tend to leak in the community however in some cases when starting a brand
new community the security guarantees of invites are not needed.
2020-03-15 21:17:28 +11:00
Neil Lalonde 7c27f9bba9
FIX: remove parent tag from tag group
Having a tag be a member of a tag group and the group's parent tag at
the same time causes some unexpected behavior. When a tag is assigned
as the parent, remove it from the group.
2020-03-13 12:25:58 -04:00
Martin Brennan af92444948
FIX: Make sure bookmark serializer works with deleted topics + posts (#9195) 2020-03-13 10:44:39 +10:00
Martin Brennan 2237ba8c9d
FIX: Add topic deleted check to email/sender (#9166)
It already had a deleted post check and log reason, add a topic one too to avoid errors
2020-03-13 10:04:15 +10:00
David Taylor ec2d49d48a
DEV: Allow plugins to add theme modifiers via db migrations (#9192) 2020-03-12 16:35:28 +00:00
Stasiek Michalski 1b8793e7a4
FEATURE: Add support for custom gravatar-like services (#9137)
Adds 3 config values that allow to set a custom provider of Gravatar-like API accessible from gravatar_base_url. The gravatar_name is purely cosmetic, but helps with associating name with the service that actually provides the avatars. gravatar_login_url is a link relative to gravatar_base_url, which provides the user with the login to the Gravatar service
2020-03-12 11:23:55 -04:00
Martin Brennan 03c012bd97 Unscope post + topic relation for bookmark
* we want to be able to get these records for the bookmark
  even if they are trashed, for serialization for the bookmark
  list
2020-03-12 16:00:45 +10:00
Martin Brennan 793f39139a
FEATURE: Send notifications for time-based and At Desktop bookmark reminders (#9071)
* This PR implements the scheduling and notification system for bookmark reminders. Every 5 minutes a schedule runs to check any reminders that need to be sent before now, limited to **300** reminders at a time. Any leftover reminders will be sent in the next run. This is to avoid having to deal with fickle sidekiq and reminders in the far-flung future, which would necessitate having a background job anyway to clean up any missing `enqueue_at` reminders.

* If a reminder is sent its `reminder_at` time is cleared and the `reminder_last_sent_at` time is filled in. Notifications are only user-level notifications for now.

* All JavaScript and frontend code related to displaying the bookmark reminder notification is contained here. The reminder functionality is now re-enabled in the bookmark modal as well.

* This PR also implements the "Remind me next time I am at my desktop" bookmark reminder functionality. When the user is on a mobile device they are able to select this option. When they choose this option we set a key in Redis saying they have a pending at desktop reminder. The next time they change devices we check if the new device is desktop, and if it is we send reminders using a DistributedMutex. There is also a job to ensure consistency of these reminders in Redis (in case Redis drops the ball) and the at desktop reminders expire after 20 days.

* Also in this PR is a fix to delete all Bookmarks for a user via `UserDestroyer`
2020-03-12 10:16:00 +10:00
Rafael dos Santos Silva 3f9b310beb
FIX: Typo on draft save 2020-03-11 12:12:28 -03:00
David Taylor d1474e94a1
FEATURE: Allow themes to specify modifiers in their about.json file (#9097)
There are three modifiers:
- serialize_topic_excerpts (boolean)
- csp_extensions (array of strings)
- svg_icons (array of strings)

When multiple themes are active, the values will be combined. The combination method varies based on the setting. CSP/SVG arrays will be combined. serialize_topic_excerpts will use `Enumerable#any`.
2020-03-11 13:30:45 +00:00
Dan Ungureanu 0754c7c404
FIX: Various fixes to support posts with no user (#8877)
* Do not grant badges for posts with no user
* Ensure instructions are correct in Change Owner modal
* Hide user-dependent actions from posts with no user
* Make PostRevisor work with posts with no user
* Ensure posts with no user can be deleted
* discourse-narrative-bot should ignore posts with no user
* Skip TopicLink creation for posts with no user
2020-03-11 14:03:20 +02:00
Blake Erickson 6fb4c333b0
FIX: Throw error when removing a user from group fails (#9162)
This commit ensures that an error is thrown when a user fails to be
removed from a group instead of silently failing.

This means when using the api you will receive a 400 instead of a 200 if
there is a failure. The remove group endpoint allows the removal of
multiple users, this change means that if you try to delete 10 users,
but 1 of them fails you will receive a 400 instead of 200 even though
the other 9 were removed successfully. Rather than adding a bunch more
complexity I think this is more than adequate for most use cases.
2020-03-10 15:25:00 -06:00
Penar Musaraj e69b6379ad
FEATURE: Broader support for post uploads in video markup (#9152)
Ensures URLs in the following HTML attributes are included in post uploads:
- video poster
- source src
- track src
2020-03-10 09:01:40 -04:00
Jarek Radosz 85e03a7f68
DEV: Replace `Time.new` with `Time.now` (#9142)
(or `Time.zone.now`)
2020-03-09 17:37:49 +01:00
Blake Erickson 43b54c631d
DEV: Reserve webhook event types to be used in plugins (#9110)
* DEV: Reserve webhook event types to be used in plugins

Based on feedback on the following PR's:

https://github.com/discourse/discourse-solved/pull/85

https://github.com/discourse/discourse-assign/pull/61

This commit reserves ID's to be used for webhook event types to ensure
that some other webhook or plugin doesn't end up using the same ID.

* Fix broken test

I don't think this test has to test ALL event types to verify that this
feature is working. Now that we added some event types that plugins are
using this test was failing for missing fabricators that exist in the
respective plugins.

* remove loop and just test first record
2020-03-06 10:16:19 -07:00
Rafael dos Santos Silva 54f67661ac FEATURE: Option to connect to Redis using SSL 2020-03-05 20:49:05 -03:00
Vinoth Kannan d953c908d2 FEATURE: add child theme components in theme metadata.
Now theme creators can add an array of child theme components in about.json file for a top level theme.
2020-03-05 18:28:18 +05:30
David Taylor 65cc61be7a
PERF: Allow preloading 'recent time read' for a user (#9076)
This will be used when serializing multiple user cards
2020-03-03 13:57:46 +00:00
David Taylor d23f7af3cb
PERF: Allow user serializer to make use of preloaded custom fields (#9074) 2020-03-03 13:56:54 +00:00
Martin Brennan 0388653a4d
DEV: Upload and secure media retroactive rake task improvements (#9027)
* Add uploads:sync_s3_acls rake task to ensure the ACLs in S3 are the correct (public-read or private) setting based on upload security

* Improved uploads:disable_secure_media to be more efficient and provide better messages to the user.

* Rename uploads:ensure_correct_acl task to uploads:secure_upload_analyse_and_update as it does more than check the ACL

* Many improvements to uploads:secure_upload_analyse_and_update

* Make sure that upload.access_control_post is unscoped so deleted posts are still fetched, because they still affect the security of the upload.

* Add escape hatch for capture_stdout in the form of RAILS_ENABLE_TEST_STDOUT. If provided the capture_stdout code will be ignored, so you can see the output if you need.
2020-03-03 10:03:58 +11:00
Joffrey JAFFEUX 11425f8adc
FEATURE: alows to add a description link to a report (#9065)
This commit adds a description link to users_per_trust_level report linking to our blog  article on the subject https://blog.discourse.org/2018/06/understanding-discourse-trust-levels/
2020-03-02 14:30:51 -05:00
David Taylor f9cc3dc4b7
PERF: Allow passing an existing list of user field ids when loading (#8970)
* PERF: Allow passing an existing list of user field ids when loading

This avoids the need for running `UserField.pluck(:id)` for each user that is serialized

* Memoize user_fields to avoid rebuilding hash ever time
2020-03-02 14:22:49 -05:00
Martin Brennan 8123538c94
DEV: Minor review fixes and fix bookmark spec logging (#9045)
As per:

https://review.discourse.org/t/fix-never-allow-custom-emoji-to-be-marked-secure-8965/9072
https://review.discourse.org/t/feature-improving-bookmarks-part-2-topic-bookmarking-8954/9038
2020-03-02 15:40:29 +10:00
Dan Ungureanu 60184a290c
FIX: Sync preload key format for category topic lists
The server and client used two different formats for preload keys. The
server was using 'topic_list_c/SLUG/l/latest', but the client was using
'topic_list_c/SLUG/ID/l/latest'.

This commit is an addition to 374534f00e.
2020-02-28 11:10:03 +02:00
Dan Ungureanu 60908a94ec
FIX: Skip 'invited' small action if user is in an invited group (#9056)
Inviting a user that is already invited through a group used to generate
a small action and a notification. This commit skips that small action.
2020-02-27 14:45:20 +02:00
Vinoth Kannan acf337d583
FEATURE: auto archive group message if topic is closed. (#9046)
Co-Authored-By: Régis Hanol <regis@hanol.fr>
2020-02-27 11:09:37 +05:30
Sam Saffron 372f6f4f22
FEATURE: limit number of notifications per user to 10,000
Introduces a new site setting `max_notifications_per_user`.

Out-of-the-box this is set to 10,000. If a user exceeds this number of
notifications, we will delete the oldest notifications keeping only 10,000.

To disable this safeguard set the setting to 0.

Enforcement happens weekly.

This is in place to protect the system from pathological states where a
single user has enormous amounts of notifications causing various queries
to time out. In practice nobody looks back more than a few hundred notifications.
2020-02-24 11:42:50 +11:00
Robin Ward a47e0a3fda FIX: TOTP could not be used on sites with colons in their names
This is because the TOTP gem identifies as a colon as an addressable
protocol. The solution for now is to remove the colon in the issuer
name.

Changing the issuer changes the token values, but now it was completely
broken for colons so this should not be breaking anyone new.
2020-02-20 16:35:30 -05:00
Sam Saffron 28292d2759
PERF: avoid shelling to get hostname aggressively
Previously we had many places in the app that called `hostname` to get
hostname of a server. This commit replaces the pattern in 2 ways

1. We cache the result in `Discourse.os_hostname` so it is only ever called once

2. We prefer to use Socket.gethostname which avoids making a shell command

This improves performance as we are not spawning hostname processes throughout
the app lifetime
2020-02-18 15:13:19 +11:00
Martin Brennan 56b16bc68e
FIX: Never allow custom emoji to be marked secure (#8965)
* Because custom emoji count as post "uploads" we were
marking them as secure when updating the secure status for post uploads.
* We were also giving them an access control post id, which meant
broken image previews from 403 errors in the admin custom emoji list.
* We now check if an upload is used as a custom emoji and do not
assign the access control post + never mark as secure.
2020-02-14 11:17:09 +10:00
Martin Brennan e1e74abd4f
FEATURE: Improving bookmarks part 2 -- Topic Bookmarking (#8954)
### UI Changes

If `SiteSetting.enable_bookmarks_with_reminders` is enabled:

* Clicking "Bookmark" on a topic will create a new Bookmark record instead of a post + user action
* Clicking "Clear Bookmarks" on a topic will delete all the new Bookmark records on a topic
* The topic bookmark buttons control the post bookmark flags correctly and vice-versa
Disabled selecting the "reminder type" for bookmarks in the UI because the backend functionality is not done yet (of sending users notifications etc.)

### Other Changes

* Added delete bookmark route (but no UI yet)
* Added a rake task to sync the old PostAction bookmarks to the new Bookmark table, which can be run as many times as we want for a site (it will not create duplicates).
2020-02-13 16:26:02 +10:00
Robin Ward 726d97b29d FIX: Linking to a category via hashtag had a broken URL. 2020-02-12 14:23:09 -05:00
Mark VanLandingham 3e89774908
DEV: Use .hbr for raw template file extension (#8883) 2020-02-11 13:38:12 -06:00
Roman Rizzi 3413ec0a5c
FEATURE: Pending queued posts are included even if they don't pass the minimum priority threshold (#8925) 2020-02-11 15:29:22 -03:00
Dan Ungureanu ecaf2c2f4e
FIX: Make category slug validation less strict (#8915)
This was changed recently and caused issues saving old categories which
already had digits at the beginning of the slug (for example, '30-days').
2020-02-11 17:01:12 +02:00
Jeff Wong 1a1bb7a2c9
FEATURE: Add logging when claiming and unclaiming reviewable flagged posts (#8920) 2020-02-10 15:40:01 -08:00
Jarek Radosz 6cfd16656f
FIX: Ignore group mentions inside quotes (#8905)
Also includes:
* DEV: Reuse found elements
2020-02-10 18:31:42 +01:00
Joffrey JAFFEUX 4de7d5ff90 FIX: removes limit for trust level growth report (#8908) 2020-02-10 11:56:29 +01:00
David Taylor 5919618a87
DEV: Drop legacy OpenID 2.0 support (#8894)
This is not used in core or official plugins, and has been printing a deprecation notice since v2.3.0beta4. All OpenID 2.0 code and dependencies have been dropped. The user_open_ids table remains for now, in case anyone has missed the deprecation notice, and needs to migrate their data.

Context at https://meta.discourse.org/t/-/113249
2020-02-07 17:32:35 +00:00
Penar Musaraj 99fd65328c FIX: Skip absolutizing URLs when source URI is invalid 2020-02-07 10:54:24 -05:00
Joffrey JAFFEUX 20944e69e4
FEATURE: adds trust_level_growth report (#8878) 2020-02-06 19:44:30 +01:00
Blake Erickson 926d5f1c0a REFACTOR: Edit title respects min trust to edit post
Follow up to: 241d8f6452
2020-02-05 10:36:24 -07:00
Jarek Radosz 53529a3427
DEV: Upgrade Ember to version 3.12.2 (#8753)
* DEV: Use Ember 3.12.2
* Add Ember version to ThemeField's DEPENDENT_CONSTANTS
* DEV: Use `id` instead of `elementId` (See: https://github.com/emberjs/ember.js/issues/18147)
* FIX: Don't leak event listeners (bug introduced in 999e2ff)
2020-02-05 14:51:00 +01:00
Blake Erickson 241d8f6452 FIX: Edit title respects min trust to edit post
This fix ensures that the site setting `post_edit_time_limit` does not
bypass the limit of the site setting `min_trust_to_edit_post`. This
prevents a bug where users that did not meet the minimum trust level to
edit could edit the title of topics.
2020-02-04 16:31:16 -07:00
Gerhard Schlager 71849242fa PERF: Speed up moving posts on large databases
Old exection plan:
```
Delete on post_replies pr  (cost=6.59..20462.62 rows=2254 width=24) (actual time=2.580..2.580 rows=0 loops=1)
  ->  Nested Loop  (cost=6.59..20462.62 rows=2254 width=24) (actual time=0.086..2.557 rows=4 loops=1)
        Join Filter: (p.topic_id <> r.topic_id)
        Rows Removed by Join Filter: 328
        ->  Nested Loop  (cost=6.16..16845.77 rows=2254 width=26) (actual time=0.020..1.886 rows=332 loops=1)
              ->  Nested Loop  (cost=5.74..13257.09 rows=2254 width=20) (actual time=0.016..1.361 rows=332 loops=1)
                    ->  Seq Scan on moved_posts mp  (cost=0.00..19.70 rows=970 width=10) (actual time=0.002..0.028 rows=263 loops=1)
                    ->  Bitmap Heap Scan on post_replies pr  (cost=5.74..13.63 rows=2 width=14) (actual time=0.004..0.005 rows=1 loops=263)
                          Recheck Cond: ((reply_post_id = mp.old_post_id) OR (post_id = mp.old_post_id))
                          Heap Blocks: exact=278
                          ->  BitmapOr  (cost=5.74..5.74 rows=2 width=0) (actual time=0.004..0.004 rows=0 loops=263)
                                ->  Bitmap Index Scan on index_post_replies_on_reply_post_id  (cost=0.00..2.87 rows=1 width=0) (actual time=0.001..0.001 rows=1 loops=263)
                                      Index Cond: (reply_post_id = mp.old_post_id)
                                ->  Bitmap Index Scan on index_post_replies_on_post_id_and_reply_post_id  (cost=0.00..2.87 rows=1 width=0) (actual time=0.002..0.002 rows=1 loops=263)
                                      Index Cond: (post_id = mp.old_post_id)
              ->  Index Scan using posts_pkey on posts p  (cost=0.42..1.59 rows=1 width=14) (actual time=0.001..0.001 rows=1 loops=332)
                    Index Cond: (id = pr.post_id)
        ->  Index Scan using posts_pkey on posts r  (cost=0.42..1.59 rows=1 width=14) (actual time=0.001..0.002 rows=1 loops=332)
              Index Cond: (id = pr.reply_post_id)
Planning Time: 0.305 ms
Execution Time: 2.600 ms
```

New execution plan:
```
Delete on post_replies pr  (cost=15.34..6538275.37 rows=364157 width=12) (actual time=1.961..1.961 rows=0 loops=1)
  ->  Nested Loop  (cost=15.34..6538275.37 rows=364157 width=12) (actual time=0.048..1.827 rows=187 loops=1)
        ->  Seq Scan on moved_posts mp  (cost=0.00..19.70 rows=970 width=10) (actual time=0.004..0.029 rows=188 loops=1)
        ->  Bitmap Heap Scan on post_replies pr  (cost=15.34..6736.72 rows=375 width=14) (actual time=0.009..0.009 rows=1 loops=188)
              Recheck Cond: ((reply_post_id = mp.old_post_id) OR (post_id = mp.old_post_id))
              Filter: ((SubPlan 1) <> (SubPlan 2))
              Heap Blocks: exact=187
              ->  BitmapOr  (cost=15.34..15.34 rows=377 width=0) (actual time=0.003..0.003 rows=0 loops=188)
                    ->  Bitmap Index Scan on index_post_replies_on_reply_post_id  (cost=0.00..4.33 rows=1 width=0) (actual time=0.001..0.001 rows=1 loops=188)
                          Index Cond: (reply_post_id = mp.old_post_id)
                    ->  Bitmap Index Scan on index_post_replies_on_post_id_and_reply_post_id  (cost=0.00..10.82 rows=376 width=0) (actual time=0.001..0.001 rows=0 loops=188)
                          Index Cond: (post_id = mp.old_post_id)
              SubPlan 1
                ->  Index Scan using posts_pkey on posts p  (cost=0.43..8.45 rows=1 width=4) (actual time=0.002..0.002 rows=1 loops=187)
                      Index Cond: (id = pr.post_id)
              SubPlan 2
                ->  Index Scan using posts_pkey on posts r  (cost=0.43..8.45 rows=1 width=4) (actual time=0.002..0.003 rows=1 loops=187)
                      Index Cond: (id = pr.reply_post_id)
Planning Time: 0.136 ms
Execution Time: 1.990 ms
```
2020-02-04 12:30:43 +01:00
Robin Ward ee17138c0f FIX: Better error messages when name is too long
Previously you'd get a server side generic error due to a password check
failing. Now the input element has a maxlength attribute and the server
side will respond with a nicer error message if the value is too long.
2020-02-03 14:14:32 -05:00
Robin Ward 37888d9818 FIX: Never return the same reply more than once via reply_ids
If our reply tree somehow ends up with cycles or other odd
structures, we only want to consider a reply once, at the first
level in the tree that it appears.
2020-02-03 13:41:18 -05:00
David Taylor 7640914552
UX: Include muted categories on the category page by default (#8842)
* DEV: Add data-notification-level attribute to category UI

* Show muted categories on the category page by default

This reverts commit ed9c21e42c.

* Remove redundant spec - muted categories are now visible by default
2020-02-03 10:40:02 -08:00
Robin Ward f83362b05b FIX: Don't return post replies from other topics
It seems in some situations replies have been moved to other topics but
the `PostReply` table has not been updated. I will try and fix this in a
follow up PR, but for now this fix ensures that every time we ask a post
for its replies that we restrict it to the same topic.
2020-02-03 13:12:27 -05:00
Stephen Chung 98e9302c26
Log error when optimized image file fails to store. (#8840) 2020-02-03 12:28:45 -05:00
Roman Rizzi df43ac901d
FIX: We don't want to update the post read count and user stats if the post timing wasn't created due to a conflict. (#8824) 2020-01-31 10:23:24 -03:00
Martin Brennan 1150cd4621
FIX: Stop secure media URLs being censored too liberally in emails (#8817)
For example /t/ URLs were being replaced if they contained secure-media-uploads so if you made a topic called "Secure Media Uploads Are Cool" the View Topic link in the user notifications would be stripped out.

Refactored code so this secure URL detection happens in one place.
2020-01-30 16:19:14 +10:00
Roman Rizzi 2ee6a615b7
FEATURE: Send suspect users to the review queue (#8811) 2020-01-29 15:38:27 -03:00
Martin Brennan ab3bda6cd0
FIX: Mitigate issue where legacy pre-secure hotlinked media would not be redownloaded (#8802)
Basically, say you had already downloaded a certain image from a certain URL
using pull_hotlinked_images and the onebox. The upload would be stored
by its sha as an upload record. Whenever you linked to the same URL again
in a post (e.g. in our case an og:image on review.discourse) we would
would reuse the original upload record because of the sha1.

However when you turned on secure media this could cause problems as
the first post that uses that upload after secure media is enabled
will set the access control post for the upload to the new post.
Then if the post is deleted every single onebox/link to that same image
URL will fail forever with 403 as the secure-media-uploads URL fails
if the access control post has been deleted.

To fix this when cooking posts and pulling hotlinked images, we only
allow using an original upload by URL if its access control post
matches the current post, and if the original_sha1 is filled in,
meaning it was uploaded AFTER secure media was enabled. otherwise
we just redownload the media again to be safe, as the URL will always
be new then.
2020-01-29 10:11:38 +10:00
Gerhard Schlager ea11ad4d99 DEV: Drop unused columns 2020-01-27 15:28:56 +01:00
Martin Brennan 45b37a8bd1
FIX: Resolve pull hotlinked image and broken link issues for secure media URLs (#8777)
When pull_hotlinked_images tried to run on posts with secure media (which had already been downloaded from external sources) we were getting a 404 when trying to download the image because the secure endpoint doesn't allow anon downloads.

Also, we were getting into an infinite loop of pull_hotlinked_images because the job didn't consider the secure media URLs as "downloaded" already so it kept trying to download them over and over.

In this PR I have also refactored secure-media-upload URL checks and mutations into single source of truth in Upload, adding a SECURE_MEDIA_ROUTE constant to check URLs against too.
2020-01-24 11:59:30 +10:00
Martin Brennan 1b3b0708c0
FEATURE: Update upload security status on post move, topic conversion, category change (#8731)
Add TopicUploadSecurityManager to handle post moves. When a post moves around or a topic changes between categories and public/private message status the uploads connected to posts in the topic need to have their secure status updated, depending on the security context the topic now lives in.
2020-01-23 12:01:10 +10:00
Dan Ungureanu 89bd7ba45f
FIX: Use new tag routes (#8683)
Commit 1fb7a62 added unambiguous routes for tags. This commit ensures
that the new routes are used.
2020-01-21 19:23:08 +02:00
Sam Saffron bff9880d63 DEV: increase the length of backup codes
16 ^ 8 though not tiny but is a workable search space in the event of
breach, 16 ^ 16 is not.
2020-01-21 15:32:06 +11:00
Gerhard Schlager ab07b945c2
Merge pull request #8736 from gschlager/rename_reply_id_column
REFACTOR: Rename `post_replies.reply_id` column to `post_replies.reply_post_id`
2020-01-17 17:24:49 +01:00
Roman Rizzi 28d09227f5
FIX: Reload the ReviewableScore types when extending flags (#8740)
ReviewableScore#types extend the PostActionTypes with their own, storing the result inside a class variable. To avoid overwriting an existing flag, we need to calculate the next flag ID using these types instead of the PostAction ones. Since we first call the score types to calculate the id, this list gets memoized, leaving us with an outdated list.

To fix this, we now reload ReviewableScore#types after replacing flags.
2020-01-17 11:59:38 -03:00
Gerhard Schlager f216c6d60b FEATURE: Drop "backup" schema 7 days after restore
The "backup" schema is used to rollback a failed restore. It isn't useful after a longer period of time and turns into a waste of disk space.
2020-01-16 17:48:47 +01:00
Martin Brennan 7c32411881
FEATURE: Secure media allowing duplicated uploads with category-level privacy and post-based access rules (#8664)
### General Changes and Duplication

* We now consider a post `with_secure_media?` if it is in a read-restricted category.
* When uploading we now set an upload's secure status straight away.
* When uploading if `SiteSetting.secure_media` is enabled, we do not check to see if the upload already exists using the `sha1` digest of the upload. The `sha1` column of the upload is filled with a `SecureRandom.hex(20)` value which is the same length as `Upload::SHA1_LENGTH`. The `original_sha1` column is filled with the _real_ sha1 digest of the file. 
* Whether an upload `should_be_secure?` is now determined by whether the `access_control_post` is `with_secure_media?` (if there is no access control post then we leave the secure status as is).
* When serializing the upload, we now cook the URL if the upload is secure. This is so it shows up correctly in the composer preview, because we set secure status on upload.

### Viewing Secure Media

* The secure-media-upload URL will take the post that the upload is attached to into account via `Guardian.can_see?` for access permissions
* If there is no `access_control_post` then we just deliver the media. This should be a rare occurrance and shouldn't cause issues as the `access_control_post` is set when `link_post_uploads` is called via `CookedPostProcessor`

### Removed

We no longer do any of these because we do not reuse uploads by sha1 if secure media is enabled.

* We no longer have a way to prevent cross-posting of a secure upload from a private context to a public context.
* We no longer have to set `secure: false` for uploads when uploading for a theme component.
2020-01-16 13:50:27 +10:00
Martin Brennan 66f2db4ea4 SECURITY: 2FA with U2F / TOTP 2020-01-15 11:27:12 +01:00
Régis Hanol 5d75f90b27 FIX: group membership leak
FIX: raised a proper NotFound exception when filtering groups by username with invalid username.
FIX: properly filter the groups based on current user visibility when viewing another user's groups.
DEV: Guardian.can_see_group?(group) is now using Guardian.can_see_groups(groups) instead of duplicating the same code.
FIX: spec for groups_controller#index when group directory is disabled for logged in user.
FIX: groups_controller.sortable specs to actually test all sorting combinations.
DEV: s/response_body/body/g for slightly shorter spec code.
FIX: rewrote the "view another user's groups" specs to test all group_visibility and members_group_visibility combinations.
DEV: Various refactoring for cleaner and more consistent code.
2020-01-15 11:21:58 +01:00
David Taylor cff6e941de
PERF: Cache ranks for featured badges, to simplify user serialization (#8698) 2020-01-14 14:26:49 +00:00
Martin Brennan cb660ef952 SECURITY: Improve second factor auth logic 2020-01-10 10:45:56 +10:00
Robin Ward dcbe527a82 FIX: Don't log a claimed topic database error during tests
We now test the uniqueness validation, but also rescue a DB
exception in case the controller fails this check.
2020-01-09 12:32:05 -05:00
Robin Ward d043a4c6fe FIX: Stop logging errors in postgres on reviewable conflict
The previous concurrency-safe implementation relied on catching an
index conflict and following through appropriately. Unfortunately
those conflicts were logged to Postgres and there is no easy way
to turn them off.

This solution approaches the problem differently. It should still
be safe under concurrency and not log errors.
2020-01-09 12:04:17 -05:00
Jarek Radosz 531016f99b
DEV: Add missing indexes to user_profiles (#8691)
* DEV: Update model annotations
* DEV: Add missing indexes to user_profiles

The columns were changed in 24347ace10 (diff-baa5914c0c7cddf3c8b5cd9139e0d091)
2020-01-09 17:08:55 +01:00
Osama Sayegh d3a64e34e7
DEV: Remove unnecessary debugging line
I was playing with groups locally and saw this line. I suspect this method isn't needed at all because I don't see any reference to it anywhere in the code, and as far as I know ActiveRecord objects don't have an `id!` method so if this method is called dynamically somewhere it's most likely failing.
2020-01-07 15:04:43 +03:00
David Taylor d1779346e8 FIX: topic_tracking_state when mute_all_categories_by_default is enabled 2020-01-06 18:22:42 +00:00
romanrizzi ca07a571c7 FIX: Only agree with the first post when using the 'Delete post + replies and agree' option 2020-01-06 13:38:23 -03:00
David Taylor 5df815c2ee
FIX: New/unread count after dismissing new topics in a regular category (#8659)
6e1fe22 introduced the possiblity for category_users to have a NULL notification_level, so that we can store `last_seen_at` dates without locking the notification level. At the time, this did not affect the topic-tracking-state query. However, the query changes in f434de2 introduced a slight change in behavior.

Previously, a subquery would look for a category_user with notification_level=mute. f434de2 refactored this to remove the subquery, and inverted some of the logic to suit.

The new query checked for `notification_level <> :muted`. If `notification_level` is NULL, this comparison will return NULL. In this scenario, notification_level=NULL means that we should fall back to the default tracking level (regular), and so we want the expression to resolve as true, not false. There was already a check for the existence of the category_users row, but it did not check for the existence of a NOT NULL notification_level.

This commit amends the expression so that the notification_level will only be compared if it is non-null.
2020-01-06 16:15:24 +00:00
Roman Rizzi 98853b217f
FIX: Bots accuracy should be zero (#8654) 2020-01-02 13:24:24 -03:00
David Taylor 45c5f56ffc
PERF: Reduce DB queries when serializing ignore/mute information (#8629)
* PERF: Cache ignored and muted user ids in the current_user object
* PERF: Avoid DB queries when checking ignore/mute permission in guardian
2020-01-02 13:04:08 +00:00
Sam Saffron 34ede90927 FIX: under rare conditions saving a new draft could error temporarily
Under rare conditions due to bad HTTP timing and so on a draft could be
set at the exact same time from 2 unicorn workers.

When this happened and all the stars aligned, one of the sets would win
and the other would raise an error.

This transparently handles the situation without adding any cost to the
draft system.

The alternative is to add a distributed mutex, tricky DB transaction or handle
the error in the controller. However this seems like a reasonable way to
work around a pretty big edge case.
2020-01-02 11:38:14 +11:00
Roman Rizzi c751291769
FIX: The 'reviewed' status filter should include deleted elements (#8630) 2019-12-30 14:56:17 -03:00
David Taylor df8444e813
FIX: Update topic/post counter correctly when category has zero topics (#8600)
Previously, categories without any topics were being excluded from the UPDATE query. This means the counter gets stuck, and the category cannot be deleted. This change ensures that the counters get correctly set to zero.
2019-12-30 11:20:44 +00:00
David Taylor 9348d2cfdf
PERF: Cache user badge count in user_stats table (#8610)
This means that we no longer need to run a `count()` on the user_badges table when serializing a user
2019-12-30 11:19:59 +00:00
Martin Brennan 473e39f4a4 DEV: model annotations 2019-12-30 16:13:27 +10:00
Krzysztof Kotlarek 9e4fcb1a0a
FIX: Correct description for out of love badge (#8615)
The bug mentioned here
https://meta.discourse.org/t/badge-not-triggering/135896/8

Basically, descriptions for 3 badges: "Out of Love", "Higher Love" and
"Crazy in Love" are granted based on on "max_likes_per_day" and the
description should reflect that.
2019-12-24 08:30:34 +11:00
Jeff Wong eca1c643d5 FEATURE: allow TL3 promotions for overturned penalties
Restore tl3 reachability by calculating penalty counts vs unsuspend/unsilence.

Only count unsuspend or unsilences that have been made by a user other than
the Discourse system user.
2019-12-20 15:25:21 -08:00
Dan Ungureanu 069bc70bd4
UX: Improve sub-sub-categories styling in categories list (#8593)
- Using h4 instead of h3 for sub-categories.
- Show category description if it does not have subcategories.
- Implemented equivalent for mobile-view.
- Include description_excerpt in basic serializer. This is needed for
  displaying second-level categories in category list.

Follow-up to 9253cb79e3.
2019-12-19 17:09:45 +02:00
Penar Musaraj ea05a68df7
FIX: Use updated_at date to denote expired invites (#8521) 2019-12-17 10:13:49 -05:00
Martin Brennan beb91e7eff
FIX: require: false for rotp gem (#8540)
The ROTP gem is only used in a very small amount of places in the app, we don't need to globally require it.

Also set the Addressable gem to not have a specific version range, as it has not been a problem yet.

Some slight refactoring of UserSecondFactor here too to use SecondFactorManager to avoid code repetition
2019-12-17 10:33:51 +10:00
Robin Ward a04a631674 Changed `CONFLICT` to `SQL` for multiline strings
This is more consistent with the rest of our codebase.
2019-12-13 11:51:40 -05:00
Roman Rizzi b6a2875749
FIX: Granting staff status should auto-approve users waiting approval (#8533)
* FIX: Granting staff status should auto-approve users waiting approval

* Update app/models/concerns/roleable.rb

Co-Authored-By: Robin Ward <robin.ward@gmail.com>
2019-12-12 16:26:38 -03:00
Robin Ward 76f676879c FIX: Constraint error when inserting the same topic group twice
If that happens, update the value instead of raising an error.
2019-12-12 13:10:46 -05:00
David Taylor 4c9ca24ccf
FEATURE: Hash API keys in the database (#8438)
API keys are now only visible when first created. After that, only the first four characters are stored in the database for identification, along with an sha256 hash of the full key. This makes key usage easier to audit, and ensures attackers would not have access to the live site in the event of a database leak.

This makes the merge lower risk, because we have some time to revert if needed. Once the change is confirmed to be working, we will add a second commit to drop the `key` column.
2019-12-12 11:45:00 +00:00
Daniel Waterworth b2ed17cf86 FIX: Disallow c as a tag 2019-12-12 11:26:57 +00:00
Daniel Waterworth 6aff510809 FIX: Disallow none as a category slug 2019-12-12 11:26:21 +00:00
Bianca Nenciu 3ec2081059
UX: Include public groups in mentionable groups set (#8516) 2019-12-12 13:13:40 +02:00
Martin Brennan edbc356593
FIX: Replace deprecated URI.encode, URI.escape, URI.unescape and URI.unencode (#8528)
The following methods have long been deprecated in ruby due to flaws in their implementation per http://blade.nagaokaut.ac.jp/cgi-bin/vframe.rb/ruby/ruby-core/29293?29179-31097:

URI.escape
URI.unescape
URI.encode
URI.unencode
escape/encode are just aliases for one another. This PR uses the Addressable gem to replace these methods with its own encode, unencode, and encode_component methods where appropriate.

I have put all references to Addressable::URI here into the UrlHelper to keep them corralled in one place to make changes to this implementation easier.

Addressable is now also an explicit gem dependency.
2019-12-12 12:49:21 +10:00
Martin Brennan 6261339da9
Improving bookmarks part 1 (#8466)
Note: All of this functionality is hidden behind a hidden, default false, site setting called `enable_bookmarks_with_reminders`. Also, any feedback on Ember code would be greatly appreciated!

This is part 1 of the bookmark improvements. The next PR will address the backend logic to send reminder notifications for bookmarked posts to users. This PR adds the following functionality:

* We are adding a new `bookmarks` table and `Bookmark` model to make the bookmarks a first-class citizen and to allow attaching reminders to them.
* Posts now have a new button in their actions menu that has the icon of an actual book
* Clicking the button opens the new bookmark modal.
* Both name and the reminder type are optional.
* If you close the modal without doing anything, the bookmark is saved with no reminder.
* If you click the Cancel button, no bookmark is saved at all.
* All of the reminder type tiles are dynamic and the times they show will be based on your user timezone set in your profile (this should already be set for you).
* If for some reason a user does not have their timezone set they will not be able to set a reminder, but they will still be able to create a bookmark.
* A bookmark can be deleted by clicking on the book icon again which will be red if the post is bookmarked.

This PR does NOT do anything to migrate or change existing bookmarks in the form of `PostActions`, the two features live side-by-side here. Also this does nothing to the topic bookmarking.
2019-12-11 14:04:02 +10:00
Sam Saffron 0c52537f10 DEV: update rubocop to version 0.77
We like to stay as close as possible to latest with rubocop cause the cops
get better.

This update required some code changes, specifically the default is to avoid
explicit returns where implicit is done

Also this renames a few rules
2019-12-10 11:48:39 +11:00
Krzysztof Kotlarek 81c7d6a462
FIX: Muted tags are respected by TopicTrackingState (#8467)
When the tag is muted and topic contains that tag, we should not mark that message as NEW.

There are 3 possible settings which site admin can set.
remove_muted_tags_from_latest - always
It means that if the topic got at least one muted tag, we should not mark that topic as NEW

remove_muted_tags_from_latest - only muted
Similar to above, however, if at least one tag is not muted, the topic is marked as NEW

remove_muted_tags_from_latest - never
Basically, mute tag setting is ignored and all topics are set as NEW
2019-12-10 09:50:05 +11:00
Mark VanLandingham 14cb386f1e
FEATURE: Featured topic for user profile & card (#8461) 2019-12-09 11:15:47 -08:00
dependabot-preview[bot] 9b30922109 Build(deps): Bump rotp from 3.3.1 to 5.1.0 (#8477)
* Build(deps): Bump rotp from 3.3.1 to 5.1.0

Bumps [rotp](https://github.com/mdp/rotp) from 3.3.1 to 5.1.0.
- [Release notes](https://github.com/mdp/rotp/releases)
- [Changelog](https://github.com/mdp/rotp/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mdp/rotp/compare/v3.3.1...v5.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Fix breaking ROTP changes
2019-12-09 14:29:48 +10:00
jwjwyoung 5cef71e885 DEV: remove uneeded distinct from relation
No need to call `.distinct` when a unique index already exists.
2019-12-09 14:24:38 +11:00
Vinoth Kannan 11c4d5eda7 Fix the build - take 3.
754e1238b0
2019-12-05 20:35:39 +05:30
Vinoth Kannan 754e1238b0 Fix the build - take 2.
b0e3062fdf
2019-12-05 19:36:52 +05:30
Vinoth Kannan b0e3062fdf Fix the 'PostMover' spec. 2019-12-05 19:09:50 +05:30
Vinoth Kannan e6dfcda0bc DEV: handle all notification consolidations in new 'NotificationConsolidator' class.
481c8314f0
2019-12-05 14:36:06 +05:30
Neil Lalonde 875f0d8fd8
FEATURE: Tag synonyms
This feature adds the ability to define synonyms for tags, and the ability to merge one tag into another while keeping it as a synonym. For example, tags named "js" and "java-script" can be synonyms of "javascript". When searching and creating topics using synonyms, they will be mapped to the base tag.

Along with this change is a new UI found on each tag's page (for example, `/tags/javascript`) where more information about the tag can be shown. It will list the synonyms, which categories it's restricted to (if any), and which tag groups it belongs to (if tag group names are public on the `/tags` page by enabling the "tags listed by group" setting). Staff users will be able to manage tags in this UI, merge tags, and add/remove synonyms.
2019-12-04 13:33:51 -05:00
Daniel Waterworth 2f5adbe1f4 DEV: Allow the creation of sub-sub-categories
This commits adds a new site setting (max_category_nesting), that
determines whether sub-sub-categories are allowable.
2019-12-04 12:07:05 +00:00
Krzysztof Kotlarek 46fc45de99
FIX: inbound link when the only slug available (#8457)
Problem mentioned in [meta](https://meta.discourse.org/t/inbound-links-dont-show-up-when-topic-id-is-not-included/100551)

When there is an internal link without ID, only slug, we should still try to create reflection link.
2019-12-04 17:13:20 +11:00
Joffrey JAFFEUX 0d3d2c43a0
DEV: s/\$redis/Discourse\.redis (#8431)
This commit also adds a rubocop rule to prevent global variables.
2019-12-03 10:05:53 +01:00
David Taylor cdec5f154b DEV: Remove remaining reference to instagram_user_infos
Follow-up to ad6f33e5b1
2019-11-29 16:53:53 +00:00
David Taylor ad6f33e5b1 DEV: Remove unused instagram_user_info model
This has been unused since 703c724cf3
2019-11-29 16:43:38 +00:00
David Taylor dd8fe1a9e7 DEV: Update annotations 2019-11-29 15:49:08 +00:00
Krzysztof Kotlarek b120728999
FEATURE: Ability to add components to all themes (#8404)
* FEATURE: Ability to add components to all themes

This is the first and functional step from that topic https://dev.discourse.org/t/adding-a-theme-component-is-too-much-work/15398/16

The idea here is that when a new component is added, the user can easily assign it to all themes (parents).

To achieve that, I needed to change a site-setting component to accept `setDefaultValues` action and `setDefaultValuesLabel` translated label.
Also, I needed to add `allowAny` option to disable that for theme selector.

I also refactored backend to accept both parent and child ids with one method to avoid duplication (Renamed `add_child_theme!` to more general `add_relative_theme!`)

* FIX: Improvement after code review

* FIX: Improvement after code review2

* FIX: use mapBy and filterBy directly
2019-11-28 16:19:01 +11:00
Sam Saffron 7371b427cd DEV: correct a few Ruby 2.7 deprecations
Note:

```
def foo(bar: 1)
end

foo({bar: 2})
# raises a deprecation, instead use:

foo(**{bar: 2})
```

Additionally when matching regexes always use strings. It does not make
sense to match a non string to a regex.
2019-11-28 13:13:29 +11:00
Vinoth Kannan fd8ce7256c DEV: don't send staff welcome message to first admin. 2019-11-28 05:19:46 +05:30
Vinoth Kannan 0fed417439 DEV: always trigger 'notification_created' event.
And make prettier happy.
2019-11-28 04:31:55 +05:30
Vinoth Kannan 481c8314f0 FEATURE: consolidate group membership request notifications. 2019-11-28 04:02:35 +05:30
Martin Brennan e7c7a05097
FIX: Mark secure media upload insecure automatically if used for theme component (#8413)
When uploading a file to a theme component, and that file is existing and has already been marked as secure, we now automatically mark the file as secure: false, change the ACL, and log the action as the user (also rebake the posts for the upload)
2019-11-28 07:32:17 +10:00
Roman Rizzi 1c9d18f094
FIX: Correctly ignore/approve replies when acting on a flagged post (#8425) 2019-11-27 17:19:44 -03:00
David Taylor a227083c1c
FEATURE: Limit the number of active sessions for a user (#8411)
If a user has more than 60 active sessions, the oldest sessions will be terminated automatically. This protects performance when logging in and when loading the list of recently used devices.
2019-11-27 12:39:31 +00:00
Sam Saffron 88ecb650a9 DEV: Implement a faster Discourse.cache
This is a bottom up rewrite of Discourse cache to support faster performance
and a limited surface area.

ActiveSupport::Cache::Store accepts many options we do not use, this partial
implementation only picks the bits out that we do use and want to support.

Additionally params are named which avoids typos such as "expires_at" vs "expires_in"

This also moves a few spots in Discourse to use Discourse.cache over setex
Performance of setex and Discourse.cache.write is similar.
2019-11-27 16:11:49 +11:00
Sam Saffron 0fb497eb23 DEV: use Discourse.cache over Rails.cache
Discourse.cache is a more consistent method to use and offers clean fallback
if you are skipping redis

This is part of a larger change that both optimizes Discoruse.cache and omits
use of setex on $redis in favor of consistently using discourse cache

Bench does reveal that use of Rails.cache and Discourse.cache is 1.25x slower
than redis.setex / get so a re-implementation will follow prior to porting
2019-11-27 12:36:19 +11:00
Arpit Jalan 7e08dabeac FIX: do not validate topic when resetting `bumped_at`
Migrated topics were failing when resetting `bumped_at` via
`reset-bump-date` endpoint because there was an existing
(imported) topic with same title.
2019-11-26 23:12:47 +05:30
Leo McArdle 2714149fd2 FEATURE: hide posts from incoming email based on dmarc verdict (#8333) 2019-11-26 15:55:22 +01:00
Dan Ungureanu a992caf741
DEV: Replace magic values (#8398)
Follow-up to 35942f7c7c.
2019-11-25 14:32:19 +02:00
Martin Brennan afb5533581
FEATURE: Add timezone to core user_options (#8380)
* Add timezone to user_options table

* Also migrate existing timezone values from UserCustomField,
  which is where the discourse-calendar plugin is storing them

* Allow user to change their core timezone from Profile

* Auto guess & set timezone on login & invite accept & signup

* Serialize user_options.timezone for group members. this is so discourse-group-timezones can access the core user timezone, as it is being removed in discourse-calendar.

* Annotate user_option with timezone

* Validate timezone values
2019-11-25 10:49:27 +10:00
Krzysztof Kotlarek 43ddf60cdf
FIX: When dismissing category inform via MessageBus (#8371)
When category is dismissed, `dismiss_new` message is sent to fronted to clean state.

In addition, I noticed that when old dismiss new button is clicked, no message is sent so I decided to kill two birds with one stone.
2019-11-25 06:17:31 +11:00
Roman Rizzi c92f0b8775
FEATURE: New API to apply custom filters to the review queue (#8392) 2019-11-22 16:33:10 -03:00
Penar Musaraj 11d22293fb FIX: Allow private media uploads to be reused in login_required sites
In non-login-required sites, we prevent secure uploads already used in PMs from being used in public topics.

In login_required sites, secure uploads should be reusable in any topic, PM or not.
2019-11-21 09:14:06 -05:00