Commit Graph

618 Commits

Author SHA1 Message Date
David Taylor 5919618a87
DEV: Drop legacy OpenID 2.0 support (#8894)
This is not used in core or official plugins, and has been printing a deprecation notice since v2.3.0beta4. All OpenID 2.0 code and dependencies have been dropped. The user_open_ids table remains for now, in case anyone has missed the deprecation notice, and needs to migrate their data.

Context at https://meta.discourse.org/t/-/113249
2020-02-07 17:32:35 +00:00
Krzysztof Kotlarek 5b03f35614
FIX: suppress notification flood when post is edited (#8838) 2020-02-03 11:27:18 +11:00
Roman Rizzi 9eb622985a
FEATURE: Replace existing badge owners when using the bulk award feature (#8770)
* FEATURE: Replace existing badge owners when using the bulk award feature

* Use ActiveRecord to sanitize title update query, Change replace checkbox text

Co-Authored-By: Robin Ward <robin.ward@gmail.com>

Co-authored-by: Robin Ward <robin.ward@gmail.com>
2020-01-23 14:14:58 -03:00
mintsaxon 4b54791bcc FIX: Notify staged users about private categories (#8765)
group membership and `CategoryUser` notification level should be
respected to determine whether to notify staged users about activity in
private categories, instead of only ever generating notifications for staged
users' own topics (which has been the behaviour since
0c4ac2a7bc)
2020-01-22 16:33:25 -03:00
Bianca Nenciu 7b7e1717f2
FIX: Quoting a quote preserves the original post information (#8746)
Let's say post #2 quotes post number #1. If a user decides to quote the
quote in post #2, it should keep the information of post #1
("user_1, post: 1, topic: X"), instead of replacing with current post
info ("user_2, post: 2, topic: X").
2020-01-22 16:10:23 +02:00
Krzysztof Kotlarek aa04349cfd
FIX: Incorrect locale in badge granter (#8749)
We want to use default locale when:
a) Site settings are not allowing for user locale
OR
b) User locale are blank
2020-01-21 09:08:48 +11:00
Blake Erickson 75069ff179 FIX: Users should be able to remove their primary group
This fix allows a user to remove their currently assigned primary group
if the Site Setting `user selected primary groups` is enabled.

Before this fix, if a user selected "none" for their primary group it
would silently fail and never be updated.
2020-01-17 15:43:54 -07:00
Roman Rizzi 0c4ac2a7bc
Merge pull request from GHSA-569c-22ff-pj3x 2020-01-16 15:17:16 -03:00
David Taylor b60b57cddd
FIX: Allow any protocol in wildcard url checker (#8651)
This is required for people using apps with custom protocols. We still verify the entire URL (including protocol) against the site setting value.

Refactored wildcard_url_checker so that it always returns a boolean, rather than sometimes returning a regex match.
2020-01-02 16:03:13 +00:00
Krzysztof Kotlarek d8360b4c82 SECURITY: vulnerability in WildcardUrlChecker 2019-12-13 09:29:09 -05:00
Joffrey JAFFEUX 0d3d2c43a0
DEV: s/\$redis/Discourse\.redis (#8431)
This commit also adds a rubocop rule to prevent global variables.
2019-12-03 10:05:53 +01:00
David Taylor 06c2e28bbb DEV: Remove two more references to instagram_user_infos
Follow-up to ad6f33e5b1
2019-11-29 17:12:06 +00:00
David Taylor a6aada16bd DEV: Refactor API key specs to avoid hard-coding keys
By hard-coding keys, we are not testing the API key system end to end. This change also makes the specs more resilient to upcoming API key changes
2019-11-29 15:16:22 +00:00
Krzysztof Kotlarek b120728999
FEATURE: Ability to add components to all themes (#8404)
* FEATURE: Ability to add components to all themes

This is the first and functional step from that topic https://dev.discourse.org/t/adding-a-theme-component-is-too-much-work/15398/16

The idea here is that when a new component is added, the user can easily assign it to all themes (parents).

To achieve that, I needed to change a site-setting component to accept `setDefaultValues` action and `setDefaultValuesLabel` translated label.
Also, I needed to add `allowAny` option to disable that for theme selector.

I also refactored backend to accept both parent and child ids with one method to avoid duplication (Renamed `add_child_theme!` to more general `add_relative_theme!`)

* FIX: Improvement after code review

* FIX: Improvement after code review2

* FIX: use mapBy and filterBy directly
2019-11-28 16:19:01 +11:00
Sam Saffron e7cf4579a8 DEV: improve usability of subfolder specs
Previously people were not consistent about mocking which left internals in
a fragile state when running subfolder specs.

This introduces a simple helper `set_subfolder` which you can use to set
the subfolder for the spec. It takes care of proper configuration of subfolder
and teardown.

```
# usage
set_subfolder "/my_amazing_subfolder"
```

You should no longer stub base_uri or global_settings
2019-11-15 16:48:24 +11:00
Martin Brennan 56d3e29a69
FIX: Badge and user title interaction fixes (#8282)
* Fix user title logic when badge name customized
* Fix an issue where a user's title was not considered a badge granted title when the user used a badge for their title and the badge name was customized. this affected the effectiveness of revoke_ungranted_titles! which only operates on badge_granted_titles.
* When a user's title is set now it is considered a badge_granted_title if the badge name OR the badge custom name from TranslationOverride is the same as the title
* When a user's badge is revoked we now also revoke their title if the user's title matches the badge name OR the badge custom name from TranslationOverride
* Add a user history log when the title is revoked to remove confusion about why titles are revoked
* Add granted_title_badge_id to user_profile, now when we set badge_granted_title on a user profile when updating a user's title based on a badge, we also remember which badge matched the title
* When badge name (or custom text) changes update titles of users in a background job
* When the name of a badge changes, or in the case of system badges when their custom translation text changes, then we need to update the title of all corresponding users who have a badge_granted_title and matching granted_title_badge_id. In the case of system badges we need to first get the proper badge ID based on the translation key e.g. badges.regular.name
* Add migration to backfill all granted_title_badge_ids for both normal badge name titles and titles using custom badge text.
2019-11-08 15:34:24 +10:00
Krzysztof Kotlarek 17366d3bcc
FEATURE: notify tag watchers when tag was added to post (#8299)
Issue was mentioned in this [meta topic](https://meta.discourse.org/t/send-a-notification-to-watching-users-when-adding-tag/125314)

It is working well when category is changed because NotifyCategoryChange job already got that code:
```
if post&.topic&.visible?
  post_alerter = PostAlerter.new
  post_alerter.notify_post_users(post, User.where(id: args[:notified_user_ids]))
  post_alerter.notify_first_post_watchers(post, post_alerter.category_watchers(post.topic))
end
```

For NotifyTagChange job notify post users were missing so it worked only when your notification was set to `watching first post`
2019-11-07 08:20:15 +11:00
David Taylor 52c5cf33f8
FEATURE: Overhaul of admin API key system (#8284)
- Allow revoking keys without deleting them
- Auto-revoke keys after a period of no use (default 6 months)
- Allow multiple keys per user
- Allow attaching a description to each key, for easier auditing
- Log changes to keys in the staff action log
- Move all key management to one place, and improve the UI
2019-11-05 14:10:23 +00:00
Sam Saffron c5e67726fd FIX: under some conditions draft would say it was saving when not
This is a major change to draft internals. Previously there were quite a
few cases where the draft system would say "draft saved", when in fact
we just skipped saving.

This commit ensures the draft system deals with draft ownership handover in
a predictable way.

For example:

- Window 1 editing draft
- Window 2 editing same draft at the same time

Previously we would allow window 1 and 2 to just fight on the same draft
each window overwriting the same draft over an over.

This commit introduces an ownership concept where either window 1 or 2 win
and user is prompted on the loser window to reload screen to correct the issue

This also corrects edge cases where a user could have multiple browser windows
open and posts in 1 window, later to post in the second window. Previously
drafts would break in the second window, this corrects it.
2019-10-31 17:15:58 +11:00
Mark VanLandingham 4eb54f08b2
FEATURE: Site setting/UI to allow users to set their primary group (#8244)
* FEATURE: Site setting/ui to allow users to set their primary group

* prettier and remove logic from account template

* added 1 to 43 to make web_hook_user_serializer_spec pass
2019-10-28 12:46:27 -05:00
Neil Lalonde f061aee818 FEATURE: support SCSS in custom email style
In the CSS tab of Admin > Customize > Email Style, SCSS can now be used.
2019-10-23 15:42:37 -04:00
Daniel Waterworth 55a1394342 DEV: pluck_first
Doing .pluck(:column).first is a very common pattern in Discourse and in
most cases, a limit cause isn't being added. Instead of adding a limit
clause to all these callsites, this commit adds two new methods to
ActiveRecord::Relation:

pluck_first, equivalent to limit(1).pluck(*columns).first

and pluck_first! which, like other finder methods, raises an exception
when no record is found
2019-10-21 12:08:20 +01:00
Krzysztof Kotlarek 427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
Régis Hanol 0cfe47471f FIX: add support for version query parameter in InlineUploads 2019-09-13 18:21:19 +02:00
Régis Hanol aa511c5b59 FIX: support <img> in code blocks when inlining uploads
Simpler code is better :)

Also added moar specs to ensure <img> tag inside code blocks are properly ignored.
2019-09-12 21:25:14 +02:00
Vinoth Kannan 321f559c7c FIX: make markdown regexp patterns case insensitive. 2019-09-12 22:25:15 +05:30
David Taylor 73172f00d3
FIX: Clear authentication data from session after create account (#8040) 2019-09-12 12:11:12 +01:00
Sam Saffron 93b0ffc85c DEV: improve subfolder inline logic
This follows up on #503a11cc88f with a spec
2019-09-11 12:02:42 +10:00
Sam Saffron cd894b7d17 FIX: inline_uploads not working on subfolder setups 2019-09-05 15:32:33 +10:00
Roman Rizzi 7a70fce584
FIX: :reject_user_delete action can only be handled by ReviewableUser (#8068) 2019-09-04 14:12:48 -03:00
Daniel Waterworth 4ae8065f2c
DEV: Default to skipping creating a topic when fabricating categories (#7976)
This speeds up the test suite by 9%
2019-08-06 11:26:54 +01:00
David Taylor 39e0442de9 FIX: Various watched words improvements
- Client-side censoring fixed for non-chrome browsers. (Regular expression rewritten to avoid lookback)
- Regex generation is now done on the server, to reduce repeated logic, and make it easier to extend in plugins
- Censor tests are moved to ruby, to ensure everything works end-to-end
- If "watched words regular expressions" is enabled, warn the admin when the generated regex is invalid
2019-08-02 15:29:12 +01:00
Rishabh dcb47d902b
REFACTOR: Rename SiteSetting.disable_edit_notifications to disable_system_edit_notifications (#7958)
* REFACTOR: Rename SiteSetting.disable_edit_notifications to disable_system_edit_notifications

- The older name could cause some confusion because the setting does not disable all edit notifications, only system ones.

* FIX: Add frozen_string_literal: true in the migration

* DEV: Deprecate 'disable_edit_notifications'
2019-07-31 20:20:41 +05:30
Neil Lalonde 9656a21fdb
FEATURE: customization of html emails (#7934)
This feature adds the ability to customize the HTML part of all emails using a custom HTML template and optionally some CSS to style it. The CSS will be parsed and converted into inline styles because CSS is poorly supported by email clients. When writing the custom HTML and CSS, be aware of what email clients support. Keep customizations very simple.

Customizations can be added and edited in Admin > Customize > Email Style.

Since the summary email is already heavily styled, there is a setting to disable custom styles for summary emails called "apply custom styles to digest" found in Admin > Settings > Email.

As part of this work, RTL locales are now rendered correctly for all emails.
2019-07-30 15:05:08 -04:00
Guo Xiang Tan faea594436 DEV: Extract common regexps for multisite. 2019-07-29 19:01:36 +08:00
Osama Sayegh f14c6d81f4
FEATURE: Watched words improvements (#7899)
This commit contains 3 features:

- FEATURE: Allow downloading watched words
This introduces a button that allows admins to download watched words per action in a `.txt` file.

- FEATURE: Allow clearing watched words in bulk
This adds a "Clear All" button that clears all deleted words per action (e.g. block, flag etc.)

- FEATURE: List all blocked words contained in the post when it's blocked
When a post is rejected because it contains one or more blocked words, the error message now lists all the blocked words contained in the post.

-------

This also changes the format of the file for importing watched words from `.csv` to `.txt` so it becomes inconsistent with the extension of the file when watched words are exported.
2019-07-22 14:59:56 +03:00
Blake Erickson 092eeb5ca3 FEATURE: Create a rake task for destroying categories
Created a rake task for destroying multiple categories along with any
subcategories and topics the belong to those categories.

Also created a rake task for listing all of your categories.

Refactored existing destroy rake tasks to use new logging method, that
allows for puts output in the console but prevents it from showing in
the specs.
2019-07-17 12:44:14 -06:00
Vinoth Kannan 4bbf341ab1 SPEC: add additional test with 'cdn_url'.
dc6b13e4d2
2019-07-17 11:16:35 +05:30
Vinoth Kannan dc6b13e4d2 FIX: when 'raw' started with non-image upload url it's not converted to short-url.
dd0f0494c6
2019-07-17 11:13:50 +05:30
Vinoth Kannan dd0f0494c6 FIX: convert hotlinked non-image urls to short url.
3840ace978
2019-07-17 09:15:09 +05:30
Vinoth Kannan 7890f10693 SPEC: improve the code readability
3840ace978
2019-07-16 19:21:16 +05:30
Vinoth Kannan 3840ace978 FIX: skip markdown conversion for hotlinked non image urls 2019-07-16 18:05:17 +05:30
Penar Musaraj 7b0517895e FEATURE: Add "Group owners" to posting options for groups
Context: https://meta.discourse.org/t/121589

This new setting option lets group owners message/mention large groups
without granting that privilege to all members.
2019-07-08 17:14:11 -04:00
Régis Hanol 155cad8b85 FIX: only add image size when with & height are in pixels 2019-07-05 20:34:11 +02:00
Ralph Rooding 1318e0b288 FEATURE: Rake themes installer (#7848)
* Delete remote_theme when deleting the theme

* Install themes and theme components through rake

* Removed unnecessary test
2019-07-04 14:33:05 -04:00
Penar Musaraj 27387b0859 Do not collapse quote notifications
This prevents an issue where edits to a post with a quote would trigger push notifications indefinitely.
2019-06-27 18:20:43 -04:00
David Taylor afb5ec811d FIX: Don't use DistributedCache to store redis readonly state
This can cause unbound CPU usage in some cases, and excessive logging in other cases. This commit moves redis readonly information into the local process, but maintains the DistributedCache for postgres readonly state.
2019-06-25 11:20:34 +08:00
Penar Musaraj e51de4cc25
FEATURE: Add endpoint to individually update a theme setting (#7789)
- also adds a new staff action type called "change theme setting" for easier logging of staff changes to theme settings
2019-06-21 13:49:14 -04:00
Guo Xiang Tan 7c86f16aa3 FIX: Support carriage return in `InlineUploads`.
Follow up to 8deaef3872.
2019-06-21 14:07:06 +08:00
Guo Xiang Tan 8deaef3872 FIX: Don't replace img tags within anchor tags with markdown format.
Follow up to 9a25b0d614.
2019-06-21 12:32:02 +08:00
Guo Xiang Tan 9a25b0d614 FIX: Edge case with anchor tag in `InlineUploads`. 2019-06-21 09:55:56 +08:00
Guo Xiang Tan 53efb7bd24 FIX: BBcode edge case for `InlineUploads`. 2019-06-20 11:47:51 +08:00
Guo Xiang Tan 34f382b758 FIX: `InlineUploads` should replace attachment links with markdown text. 2019-06-19 11:15:02 +08:00
Guo Xiang Tan 5d16d10a9e DEV: Fix edge case for `InlineUploads`. 2019-06-14 13:48:03 +08:00
Guo Xiang Tan befb074c98 DEV: `InlineUploads` should process CDN upload URLs as well. 2019-06-14 13:14:37 +08:00
Guo Xiang Tan 41abebcbce DEV: Support both `http` and `https` for `InlineUploads`. 2019-06-14 12:48:31 +08:00
Guo Xiang Tan 7a0d031bc4 FIX: `InlineUploads` matching on external bbcode img url. 2019-06-13 13:47:36 +08:00
Guo Xiang Tan 782e583844 FIX: Edge cases with markdown references for `InlineUploads`. 2019-06-13 12:08:01 +08:00
Guo Xiang Tan 93c552afda FIX: `InlineUploads` does not correct urls with uppercase extension. 2019-06-13 11:19:33 +08:00
Guo Xiang Tan 641521896c FIX: Cover more edge cases in `InlineUploads`. 2019-06-12 17:06:58 +08:00
Sam Saffron 739696fdf0 DEV: improve spec to specify all code block formats
Previously we only covered a few, this covers a few more formats.
2019-06-12 18:34:30 +10:00
Guo Xiang Tan 73bf880f74 FIX: Correct more edge cases with `InlineUploads`. 2019-06-12 10:44:25 +08:00
Guo Xiang Tan ff48fbdfda FIX: `InlineUploads` raises an error when img tag is invalid. 2019-06-12 10:31:00 +08:00
Bianca Nenciu 934adb14d2
FIX: On tag change notify only users watching the tag. (#7707) 2019-06-11 18:06:54 +03:00
Guo Xiang Tan ee142c2173 DEV: More improvements to `InlineUploads`.
* Convert inline links to short path

```
<link> <link>
<link>
```

to

```
<short_path> <short_path>
<short_path>
```
2019-06-07 11:49:30 +08:00
Neil Lalonde 1f73a3ba6d FIX: round the calculated heat values
Views heats like 12135 will become 12000, like ratios like
1.666666666667 will become 1.67.
2019-06-06 15:44:55 -04:00
Guo Xiang Tan 95db609586 DEV: Support more formats of inline images. 2019-06-06 15:50:56 +08:00
Neil Lalonde ecc9c76692
FEATURE: dynamically update the topic heat settings monthly (#7670)
The site settings beginning with "topic views heat" and "topic post like
heat" are set to defaults when installing Discourse, but there has not
been a process or guidance for updating these values based on
community activity.

This feature will update them once a month. The low, medium, and
high settings will be based on the minimums of the 45th, 25th, and
10th percentile topics respectively, so that 45% of topics will have
some "heat".

Disable automatic changes with the automatic_topic_heat_values setting.
2019-06-04 10:34:07 -04:00
Guo Xiang Tan 5819e446dd Make lint happy. 2019-06-04 16:14:21 +08:00
Guo Xiang Tan 1991af2abb DEV: Switch `InlineUploads` to a regexp based implementation. 2019-06-04 15:54:25 +08:00
Guo Xiang Tan d93e5fb00d DEV: Class that converts MD with old attachment links to new MD. 2019-06-04 15:54:25 +08:00
Sam Saffron 6428aa5b1f FIX: search indexer had various cases where it could fail
Previous to this fix is a post had the test www.test.com/abc it would fail
to index.

This also simplifies the rules to avoid full url parsing which can be
expensive
2019-06-04 16:21:03 +10:00
Sam Saffron 5c524ea8a4 FEATURE: introduce dedicated storage and DB constraints for anon users
Previously we used custom fields to denote a user was anonymous, this was
risky in that custom fields are prone to race conditions and are not
properly dedicated, missing constraints and so on.

The new table `anonymous_users` is properly protected. There is only one
possible shadow account per user, which is enforced using a constraint.

Every anonymous user will have a unique row in the new table.
2019-05-29 14:26:24 +10:00
Ralph Rooding 0d9fdbf8fe FEATURE: Allow import of hidden site settings (#7601)
* Allow optional import of hidden site settings

* Always import hidden settings
2019-05-28 16:20:18 +10:00
Robin Ward 89b84651c3 Migrate score settings to use sensitivities
We hide scores so these settings no longer made sense.
2019-05-24 15:44:24 -04:00
Gerhard Schlager b788948985 FEATURE: English locale with international date formats
Makes en_US the new default locale
2019-05-20 13:47:20 +02:00
Penar Musaraj fc5bb39096 FIX: Do not send duplicate alerts for the same post (#7476) 2019-05-15 18:47:36 +02:00
Arpit Jalan a2b1789c3e FIX: do not log username change if the save process was unsuccessful 2019-05-13 22:20:27 +05:30
Daniel Waterworth bf7594b327 More prefabrication 2019-05-10 08:34:04 -04:00
Guo Xiang Tan f530048973 Fix broken spec in 405ba00 take 2. 2019-05-08 16:07:14 +08:00
Guo Xiang Tan 405ba00c08 FEATURE: Create notifications on wiki edits for watching users.
* Moves creation of notification into background job.
2019-05-08 15:31:10 +08:00
Daniel Waterworth e219588142 DEV: Prefabrication (test optimization) (#7414)
* Introduced fab!, a helper that creates database state for a group

It's almost identical to let_it_be, except:

 1. It creates a new object for each test by default,
 2. You can disable it using PREFABRICATION=0
2019-05-07 13:12:20 +10:00
Guo Xiang Tan 152238b4cf DEV: Prefer `public_send` over `send`. 2019-05-07 09:33:21 +08:00
Guo Xiang Tan d252dad4f0 FIX: Can't remove user profile uploads.
Follow up to 24347ace10.
2019-05-02 16:54:10 +08:00
Guo Xiang Tan 24347ace10 FIX: Properly associate user_profiles background urls via upload id.
`Upload#url` is more likely and can change from time to time. When it
does changes, we don't want to have to look through multiple tables to
ensure that the URLs are all up to date. Instead, we simply associate
uploads properly to `UserProfile` so that it does not have to replicate
the URLs in the table.
2019-05-02 14:58:24 +08:00
Sam Saffron 4ea21fa2d0 DEV: use #frozen_string_literal: true on all spec
This change both speeds up specs (less strings to allocate) and helps catch
cases where methods in Discourse are mutating inputs.

Overall we will be migrating everything to use #frozen_string_literal: true
it will take a while, but this is the first and safest move in this direction
2019-04-30 10:27:42 +10:00
Gerhard Schlager 876c4f20b3 FIX: Remove duplicate Emoji names from blurb
The blurb contained the value of the alt and title attribute of Emojis. Both values are always the same.
2019-04-29 17:26:39 +02:00
Gerhard Schlager 71d19f6e1f FIX: Reduce mentions in blurbs to @username or @groupname
The link to the user profile or group is useless and the URL encoded username or group name looks awful for Unicode names
2019-04-29 17:26:39 +02:00
Sam Saffron ac3dab93f5 DEV: lint the files 2019-04-29 17:40:53 +10:00
Sam Saffron 45285f1477 DEV: remove update_attributes which is deprecated in Rails 6
See: https://github.com/rails/rails/pull/31998

update_attributes is a relic of the past, it should no longer be used.
2019-04-29 17:32:25 +10:00
Daniel Waterworth ad44243a57 Removed unused let blocks (#7446)
The bodies of these blocks were never evaluated.
2019-04-29 15:08:56 +08:00
Gerhard Schlager a7bc1ecbae FEATURE: Add support for Unicode usernames and group names
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2019-04-23 13:00:27 +02:00
Daniel Waterworth bf313c1b68 DEV: User email test optimization (#7405) 2019-04-23 17:49:05 +08:00
Tarek Khalil 442fb2facb FEATURE: Remove ignore feature SiteSetting and enable ignore by default (#7349) 2019-04-10 12:54:59 +02:00
David Taylor 3987691b5e DEV: Correct specs which had exceptions raised in jobs 2019-04-08 16:00:33 +01:00
Maja Komel ca33d091b3 FIX: don't trigger notifications when changing category/tags of unlisted topics (#7323) 2019-04-05 15:06:38 +02:00
Robin Ward 1bce97a596 FIX: Allow queued posts from deleted users to be rejected 2019-04-03 11:40:46 -04:00
Guo Xiang Tan d8704c11ca PERF: Better use of index when queueing a topci for search reindex.
Also move `Search::INDEX_VERSION` to `SearchIndexer` which is where the
version is actually being used.
2019-04-02 09:53:37 +08:00
Guo Xiang Tan 2a69ab4a4c FIX: Keep `alt` and `title` in lightbox when indexing for search.
Follow up to cfd507822f
2019-04-01 16:20:33 +08:00
Guo Xiang Tan 16215f9d3b DEV: Correct spec added in cfd507822f.
Remove stub.
2019-04-01 10:32:25 +08:00
Guo Xiang Tan cfd507822f
PERF: Improve quality of `PostSearchData#raw_data`. (#7275)
This commit fixes the follow quality issue with `PostSearchData#raw_data`:

1. URLs are being tokenized and links with similar href and characters
are being duplicated in the raw data.

`Post#cooked`:

```
<p><a href=\"https://meta.discourse.org/some.png\" class=\"onebox\" target=\"_blank\" rel=\"nofollow noopener\">https://meta.discourse.org/some.png</a></p>
```

`PostSearchData#raw_data` Before:

```
This is a test topic 0 Uncategorized https://meta.discourse.org/some.png discourse org/some png https://meta.discourse.org/some.png discourse org/some png
```

`PostSearchData#raw_data` After:

```
This is a test topic 0 Uncategorized https://meta.discourse.org/some.png meta discourse org
```

2. Ligthbox being included in search pollutes the
`PostSearchData#raw_data` unncessarily.

From 28 March 2018 to 28 March 2019, searches for the term `image` on
`meta.discourse.org` had a click through rate of 2.1%. Non-lightboxed images are not included in indexing for search yet we were indexing content within a lightbox. Also, search for terms like `image` was affected we were using `Pasted image` as the filename for
uploads that were pasted.

`Post#cooked`

```
<p>Let me see how I can fix this image<br>\n<div class=\"lightbox-wrapper\"><a class=\"lightbox\" href=\"https://meta.discourse.org/some.png\" title=\"some.png\" rel=\"nofollow noopener\"><img src=\"https://meta.discourse.org/some.png\" width=\"275\" height=\"299\"><div class=\"meta\">\n<svg class=\"fa d-icon d-icon-far-image svg-icon\" aria-hidden=\"true\"><use xlink:href=\"#far-image\"></use></svg><span class=\"filename\">some.png</span><span class=\"informations\">1750×2000</span><svg class=\"fa d-icon d-icon-discourse-expand svg-icon\" aria-hidden=\"true\"><use xlink:href=\"#discourse-expand\"></use></svg>\n</div></a></div></p>
```

`PostSearchData#raw_data` Before:

```
This is a test topic 0 Uncategorized Let me see how I can fix this image some.png png https://meta.discourse.org/some.png discourse org/some png some.png png 1750×2000
```

`PostSearchData#raw_data` After:

```
This is a test topic 0 Uncategorized Let me see how I can fix this image
```

In terms of indexing performance, we now have to parse the given HTML
through nokogiri twice. However performance is not a huge worry here since a string length of 194170 takes only 30ms
to scrub plus the indexing takes place in a background job.
2019-04-01 10:14:29 +08:00
Guo Xiang Tan daeda80ada
FIX: Don't index posts with empty `Post#raw` for search. (#7263)
* DEV: Remove unnecessary join in `Jobs::ReindexSearch`.

* FIX: Don't index posts with empty `Post#raw` for search.
2019-04-01 10:06:27 +08:00
Robin Ward b58867b6e9 FEATURE: New 'Reviewable' model to make reviewable items generic
Includes support for flags, reviewable users and queued posts, with REST API
backwards compatibility.

Co-Authored-By: romanrizzi <romanalejandro@gmail.com>
Co-Authored-By: jjaffeux <j.jaffeux@gmail.com>
2019-03-28 12:45:10 -04:00
Penar Musaraj 51e08feb7e DEV: Refactor icons used in lightbox HTML
Uses <svg> elements instead of hacky CSS pseudoelements

Adds a migration to mark posts with lightboxes as needing a rebake
2019-03-22 11:52:06 -04:00
Tarek Khalil a31a35b334 FEATURE: Ignored user notification behaviour should be as a muted user (#7227) 2019-03-21 12:15:34 +01:00
Tarek Khalil 1dd0fa0c4e
REFACTOR: Move redundant ignored user check into guardian (#7219)
* REFACTOR: Move redundant ignored user check into guardian
2019-03-20 19:55:46 +00:00
Tarek Khalil 5852e86226 FEATURE: Only allow TL2 Users to ignore other users (#7212) 2019-03-20 15:02:33 +01:00
Tarek Khalil 3b59ff0d02 [FEATURE] Disallow ignoring self, admins or moderators users (#7202) 2019-03-20 11:18:46 +01:00
Guo Xiang Tan d808f36fc4 FIX: Reindex post for search when post is moved to a different topic.
* This is causing certain posts to appear in searches incorrectly as `PostSearchData#raw_data` contains the outdated title, category name and tag names.
2019-03-19 17:19:14 +08:00
Penar Musaraj 9334d2f4f7
FEATURE: add more granular user option levels for email notifications (#7143)
Migrates email user options to a new data structure, where `email_always`, `email_direct` and `email_private_messages` are replace by

* `email_messages_level`, with options: `always`, `only_when_away` and `never` (defaults to `always`)
* `email_level`, with options: `always`, `only_when_away` and `never` (defaults to `only_when_away`)
2019-03-15 10:55:11 -04:00
Robin Ward fa5a158683 REFACTOR: Move `queue_jobs` out of `SiteSetting`
It is not a setting, and only relevant in specs. The new API is:

```
Jobs.run_later!        # jobs will be thrown on the queue
Jobs.run_immediately!  # jobs will run right away, avoid the queue
```
2019-03-14 10:47:38 -04:00
Guo Xiang Tan b0c8fdd7da FIX: Properly support defaults for upload site settings. 2019-03-13 16:36:57 +08:00
Bianca Nenciu c6ed86220e FIX: Notify on tag change. (#7119) 2019-03-12 18:09:34 +01:00
Robin Ward d1d9a4f128 Add new `run_jobs_synchronously!` helper for tests
Previously if you wanted to have jobs execute in test mode, you'd have
to do `SiteSetting.queue_jobs = false`, because the opposite of queue
is to execute.

I found this very confusing, so I created a test helper called
`run_jobs_synchronously!` which is much more clear about what it does.
2019-03-11 16:58:35 -04:00
Tarek Khalil d6eb892b04
FEATURE: Account for `ignored_users` when merging two users (#7142)
* FEATURE: Account for `ignored_users` when merging two users

## Why?

This is part of the [Ability to ignore a user feature](https://meta.discourse.org/t/ability-to-ignore-a-user/110254/8).
When we merge two users, we need to account for merging their list of `ignored_users` too.
2019-03-11 13:53:59 +00:00
David Taylor fc7938f7e0
REFACTOR: Migrate GoogleOAuth2Authenticator to use ManagedAuthenticator (#7120)
https://meta.discourse.org/t/future-social-authentication-improvements/94691/3
2019-03-07 11:31:04 +00:00
Tarek Khalil 0a9a11094d
FEATURE: Save ignored usernames in user preferences (#7117)
* FEATURE: Save ignored usernames in user preferences
2019-03-06 11:21:58 +00:00
Joffrey JAFFEUX 703c724cf3
REFACTOR: Migrate InstagramAuthenticator to use ManagedAuthenticator (#7081) 2019-03-04 14:54:28 +01:00
Arpit Jalan a960cbd97f fix the build ❤️ 2019-02-18 10:00:17 +05:30
Arpit Jalan 7cb194f2db Add more specs for word watcher service. 2019-02-18 09:55:16 +05:30
Arpit Jalan 99c6db21e6
FEATURE: allow blocking emojis (#7011)
https://meta.discourse.org/t/blocking-emojis-wont-work/105853
2019-02-15 20:55:48 +05:30
Robin Ward bc3efab816 FIX: When disagreeing with a flag that silenced a user, unsilence them
Previously it would unhide their post but leave them silenced.

This fix also cleans up some of the helper classes to make it easier
to pass extra data to the silencing code (for example, a link to the
post that caused the user to be silenced.)

This patch also refactors the auto_silence specs to avoid using
stubs.
2019-02-08 08:50:50 -05:00
Robin Ward e5a81aeb6e REFACTOR: Remove stubbed methods in tests 2019-02-04 15:06:00 -05:00
Gerhard Schlager 1e98929501 FIX: Email domain whitelist prevented user anonymizing
FIX: Secondary email addresses were not deleted during anonymizing
2019-01-30 15:14:36 +01:00
Gerhard Schlager f8ded9cec9 FIX: Changing owner of small action post failed
Small action posts for invites (CC from email) have a blank raw and revising of posts with blank raw didn't work anymore.
2019-01-19 13:38:45 +01:00
Guo Xiang Tan b4c2d49b0c DEV: Consolidate and remove duplicated liked notifications tests. 2019-01-17 11:14:17 +08:00
Sam 1022d69101 DEV: lint spec file
We were defining a lot of unused vars, this made my linter upset so
I prefixed them with _
2019-01-17 12:41:28 +11:00
Guo Xiang Tan 78748f1501 DEV: Refactor tests for notification when liking a post.
Tests were covering the same code path so I'm consolidating it.
2019-01-14 11:43:09 +08:00
Régis Hanol 88e861e895 FIX: prevent error when badge has already been awarded 2019-01-04 15:17:54 +01:00
Sam 59c56bd20f DEV: anonymizing should not delete uploads
We have another job for upload deletion cause uploads may be shared
2018-12-13 16:43:48 +11:00
David Taylor 160d29b18a
REFACTOR: Migrate TwitterAuthenticator to use ManagedAuthenticator (#6739)
No changes to functionality. TwitterAuthenticator goes from 136 lines to 24, and all twitter-specific logic elsewhere has been deleted 🎉
2018-12-07 15:39:06 +00:00
Bianca Nenciu 41e184280d FEATURE: Remove full quotes of direct replies. (#6729) 2018-12-07 13:07:11 +01:00
Sam 82e45f5485 FIX: method extraction caused push notifications to include incorrect post
Previously the push notification code path was not tested for notification
collapsing. This happens if you get multiple replies to a topic you are
watching.
2018-12-05 16:40:10 +11:00
Sam aa97f6fdba FEATURE: disable notifications for small actions that are whispers
Previously we would notify on small actions if they were whispers
this inconsistently lead to all sorts of problems including

- collapsed "N replies" after assign
- empty push notifications

New behavior adds an api to explicitly send push notifications as well
if needed: create_notification_alert
2018-12-04 17:54:42 +11:00
David Taylor 208005f9c9 REFACTOR: Migrate FacebookAuthenticator to use ManagedAuthenticator
Changes to functionality
  - Removed syncing of user metadata including gender, location etc.
    These are no longer available to standard Facebook applications.
  - Removed the remote 'revoke' functionality. No other providers have
    it, and it does not appear to be standard practice in other apps.
  - The 'facebook_no_email' event is no longer logged. The system can
    cope fine with a missing email address.

Data is migrated to the new user_associated_accounts table.
facebook_user_infos can be dropped once we are confident the data has
been migrated successfully.
2018-11-30 11:18:11 +00:00
Arpit Jalan 059e36a6ff FIX: log name changes only when the name is actually updated 2018-11-30 15:30:46 +05:30
Gerhard Schlager 5640166b27 FIX: Notify only invited users about mentions in PMs 2018-11-26 22:42:56 +01:00
Guo Xiang Tan a57baeec2a Fix the build. 2018-11-22 16:37:24 +08:00
Guo Xiang Tan d298f00046 DEV: Improve specs to be more specific about what has changed. 2018-11-22 10:10:07 +08:00
Guo Xiang Tan e28af0429c DEV: Improve tests to be more specific. 2018-11-13 15:02:46 +08:00
Maja Komel 27e732a58d FEATURE: allow multiple secrets for Discourse SSO provider
This splits off the logic between SSO keys used incoming vs outgoing, it allows to far better restrict who is allowed to log in using a site.

This allows for better auditing of the SSO provider feature
2018-10-15 16:03:53 +11:00
Sam 550e108a8c FEATURE: only export settings that changed via rake task 2018-10-08 11:54:52 +11:00
Neil Lalonde dc1e7bb645 UX: when admin is deleted, make it clear in staff action logs when records belong to a deleted user and show their username in the details 2018-10-02 13:46:54 -04:00
Régis Hanol 4481836de2 FEATURE: new 'search_ignore_accents' site setting 2018-09-17 10:42:30 +02:00
Régis Hanol 30619c244c FIX: don't index urls to local files 2018-09-13 18:53:53 +02:00
David Taylor 84fc7abb73 FIX: Allow `rake destroy:topics` to delete topics in sub-categories 2018-09-10 12:52:14 +01:00
Gerhard Schlager 797cbf8653 FIX: Remove user fields when anonymizing user 2018-09-07 00:02:56 +02:00
Gerhard Schlager b8fc699164 FIX: Detect {{foo}} as interpolation key 2018-09-05 00:47:39 +02:00
Guo Xiang Tan 8dc1463ab3 Enable `Lint/ShadowingOuterLocalVariable` for Rubocop. 2018-09-04 10:16:42 +08:00
Sam 9b7cab589a FIX: revert diacritic stripping
See more details in test case and at: https://meta.discourse.org/t/discourse-should-ignore-if-a-character-is-accented-when-doing-a-search/90198/16?u=sam
2018-08-31 11:46:55 +10:00
James Kiesel a4001c1ea0 FEATURE: Pop revise modal on post edited notification (#6287)
* Add revision number to notification url

* Pop modal on route change

* Add semicolon

* Ensure modal pops even when navigating within a topic

* Ensure modal pops when visiting from other page

* Fix eslint errors

* Fix prettier errors

* Add callback for notification item click

* Remove stray revisionUrl function

* Rename to afterRouteComplete
2018-08-24 09:13:07 -04:00
Osama Sayegh e0cc29d658 FEATURE: themes and components split
* FEATURE: themes and components split

* two seperate methods to switch theme type

* use strict equality operator
2018-08-24 11:30:00 +10:00
Régis Hanol bc7b530b0a FIX: remove diacritics instead of transliterating 2018-08-24 00:38:44 +02:00
Régis Hanol 2fcf2b899e FIX: remove diacritics when tokenizing html for search 2018-08-23 17:13:52 +02:00
Bianca Nenciu 975a72ab7a FEATURE: Make links indexable. (#6285) 2018-08-20 10:39:19 +10:00
Osama Sayegh 865cb3feb9
FIX: allow selecting site's default theme from preference 2018-08-10 14:12:02 +03:00
Osama Sayegh 0b7ed8ffaf FEATURE: backend support for user-selectable components
* FEATURE: backend support for user-selectable components

* fix problems with previewing default theme

* rename preview_key => preview_theme_id

* omit default theme from child themes dropdown and try a different fix

* cache & freeze stylesheets arrays
2018-08-08 14:46:34 +10:00
Guo Xiang Tan 919e8db686 FIX: Check for group name availability should skip reserved usernames. 2018-08-01 11:09:33 +08:00
David Taylor eda1462b3b
FEATURE: List, revoke and reconnect associated accounts. Phase 1 (#6099)
Listing connections is supported for all built-in auth providers. Revoke and reconnect is currently only implemented for Facebook.
2018-07-23 16:51:57 +01:00
Vinoth Kannan 84ab825e41
FEATURE: Webhook for user destroyed event (#6124) 2018-07-23 13:19:49 +05:30
Maja Komel 368d4e8eae FIX: notify staff about whispers in watched categories (#6128) 2018-07-21 11:20:21 +02:00
OsamaSayegh decf1f27cf FEATURE: Groundwork for user-selectable theme components
* Phase 0 for user-selectable theme components

- Drops `key` column from the `themes` table
- Drops `theme_key` column from the `user_options` table
- Adds `theme_ids` (array of ints default []) column to the `user_options` table and migrates data from `theme_key` to the new column.
- Removes the `default_theme_key` site setting and adds `default_theme_id` instead.
- Replaces `theme_key` cookie with a new one called `theme_ids`
- no longer need Theme.settings_for_client
2018-07-12 14:18:21 +10:00
Maja Komel ec3e6a81a4 FEATURE: Second factor backup 2018-06-28 10:12:32 +02:00
Arpit Jalan f1d1207725 FIX: improve context when user deletes self 2018-06-18 11:36:22 +05:30
Gerhard Schlager 89bbc94331 Add spec for a26889a 2018-06-11 18:43:56 +02:00
Gerhard Schlager 150ae21489 FEATURE: Log user merge in staff logs 2018-06-11 18:43:56 +02:00
Gerhard Schlager 2564a8285d REFACTOR: Run some parts of user anonymizing in background job 2018-06-08 15:50:07 +02:00
Gerhard Schlager cf038cf72a FEATURE: Remove more PII during user anonymizing
Removes
  * invite if the user was invited
  * email tokens
  * incoming emails
  * email log entries
  * raw emails from posts
2018-06-08 15:50:07 +02:00
Sam 89ad2b5900 DEV: Rails 5.2 upgrade and global gem upgrade
This updates tests to use latest rails 5 practice
and updates ALL dependencies that could be updated

Performance testing shows that performance has not regressed
if anything it is marginally faster now.
2018-06-07 14:21:33 +10:00
Arpit Jalan 36f9af4fa4 minor optimizations for post rejected logs 2018-06-02 09:44:55 +05:30
Arpit Jalan 89eca87f16 FEATURE: add staff action log for post rejections 2018-06-01 21:48:27 +05:30
Gerhard Schlager b970b072f6 FIX: User merge should not fail when primary email address is missing
The merge process might move all email addresses of the source user to the target user. Destroying the source user failed in that case.
2018-06-01 16:23:21 +02:00
Guo Xiang Tan 7fc8a36529 DEV: Take 2 Queue jobs in tests by default.
On my machine this cuts the time taken to run our test suite
from ~11mins to ~9mins.
2018-05-31 16:23:23 +08:00
Guo Xiang Tan 56e9ff6853 Revert "DEV: Queue jobs in tests by default."
Too risky for now

This reverts commit be28154d3b.
2018-05-31 15:34:46 +08:00
Guo Xiang Tan 2b783997fa Refactor tests to complete assertions within a single `it` block. 2018-05-31 15:05:47 +08:00
Guo Xiang Tan be28154d3b DEV: Queue jobs in tests by default. 2018-05-31 14:45:47 +08:00
Gerhard Schlager ed2ae3cb0a FIX: Load post revisions in correct order 2018-05-28 11:25:24 +02:00
Gerhard Schlager 20b94bc714 FIX: Extraction of quoted posts failed in some cases
* It stored only oneboxed "quotes" when [quote] and links to topics or posts were mixed.
* Revising a post didn't add or remove records from the quoted_posts table.
2018-05-25 12:00:17 +02:00
Sam 720a12ce76 correct time dependent spec 2018-05-25 14:26:05 +10:00
Guo Xiang Tan 755b511b5c PERF: Destroy collapsed notifications in 1 query instead of 3. 2018-05-25 12:01:36 +08:00
Guo Xiang Tan ad9e0d6bea
Merge pull request #5848 from OsamaSayegh/fix-social-login-groups
FIX: apply automatic group rules when using social login providers
2018-05-23 08:17:42 +08:00
OsamaSayegh f6d412465b FIX: apply automatic group rules when using social login providers 2018-05-23 02:26:07 +03:00
Gerhard Schlager 95db5f0c8c FIX: Do not replace wrong avatars when renaming user 2018-05-22 13:41:51 +02:00
Sam 661d89f489 skip erratic spec 2018-05-22 09:42:50 +10:00
Guo Xiang Tan aae5575d33 Remove spec file that was accidentally added. 2018-05-21 17:35:55 +08:00
Guo Xiang Tan bf84037f79 FIX: Payload for webhooks should be current as of the time the event was triggered.
https://meta.discourse.org/t/group-category-tag-user-deleted-webhooks-not-firing/87752
2018-05-21 17:29:58 +08:00
Gerhard Schlager ae6236d090 FIX: Changing owner of deleted reply didn't work 2018-05-16 17:03:09 +02:00
Gerhard Schlager 74c4af279a Improvements to user renaming
* don't update search index if post belongs to deleted topic
* log errors instead of crashing when updating post or revision fails
* update mentions even when the href attribute is missing
* run the background job with low priority
* replace username in all notifications
* update `action_code_who` used by small action posts
2018-05-15 21:05:51 +02:00
Gerhard Schlager 2e1b5bc8d3 FIX: Transaction in UserAnonymizer prevented avatar from updating 2018-05-15 20:47:58 +02:00
Gerhard Schlager c67c2dc638 FIX: Username update should ignore revisions without raw 2018-05-14 15:22:42 +02:00
Gerhard Schlager 8232aba743 FIX: Update search index after renaming user within posts 2018-05-14 13:20:30 +02:00
Gerhard Schlager 02cb84847c FIX: Update avatar in oneboxed posts when renaming user 2018-05-14 13:20:29 +02:00
Gerhard Schlager 49392fb105 FIX: Anonymizing user didn't replace uploaded avatar in posts 2018-05-14 13:20:29 +02:00
Gerhard Schlager 2e67998319 Improvements for user renaming (#5810)
* FEATURE: Update avatars in posts and revisions when user gets renamed

* FIX: Replace username in deleted posts when user gets renamed

* FEATURE: Replace username in notifications when user gets renamed

FEATURE: Update mentions and quotes when user gets merged
2018-05-08 10:02:43 -04:00
Jeff Wong 91b31860a1
Feature: Push notifications for Android (#5792)
* Feature: Push notifications for Android

Notification config for desktop and mobile are merged.

Desktop notifications stay as they are for desktop views.

If mobile mode, push notifications are enabled.

Added push notification subscriptions in their own table, rather than through
custom fields.

Notification banner prompts appear for both mobile and desktop when enabled.
2018-05-04 15:31:48 -07:00
Robin Ward 05dddcb2fd Add an event that is triggered when a user is anonymized 2018-05-02 12:25:29 -04:00
Gerhard Schlager 3be3c50c7e FEATURE: Rename user in mentions and quotes
Co-authored-by: Robin Ward <robin.ward@gmail.com>
2018-05-01 13:49:14 -04:00
Robin Ward 079caf08d6 FIX: leftover debug statement 2018-04-30 14:41:11 -04:00
Robin Ward e21a4ce1dd FEATURE: Support anonymizing a user's IP addresses 2018-04-30 14:09:03 -04:00
Neil Lalonde f7c4c71409 FIX: title selector needs to flag whether title comes from badge or not 2018-04-26 16:51:11 -04:00