Commit Graph

6464 Commits

Author SHA1 Message Date
Robin Ward a5d9afe397 FEATURE: Include a user's pending posts in the topic view
Also includes a refactor to TopicView's serializer which was not
building our attributes using serializers properly.
2019-04-12 14:25:12 -04:00
Robin Ward 1760069419 FIX: Our rspec suite should not require a network connection to github 2019-04-12 14:25:12 -04:00
Penar Musaraj 55942224ed FIX: strip spoilers in notification excerpts 2019-04-12 12:00:49 -04:00
David Taylor 268d4d4c82
FEATURE: Multiple SCSS file support for themes (#7351)
Theme developers can include any number of scss files within the /scss/ directory of a theme. These can then be imported from the main common/desktop/mobile scss.
2019-04-12 11:36:08 +01:00
David Taylor 24ef4f7b2b Remove support for disable_jump_reply user setting (#7359) 2019-04-12 09:03:06 +10:00
Robin Ward 6f7b6a3333 FIX: Disagree and Restore should not revert edits 2019-04-11 16:21:18 -04:00
Robin Ward d21dd521d2 FIX: Don't try to create a reviewable user twice 2019-04-11 12:30:54 -04:00
Robin Ward 331a809738 FEATURE: Display the reason for many reviewable items
Queued Posts and Users will now display a reason why they are in the
review queue.
2019-04-11 12:04:45 -04:00
Arpit Jalan 7143572e0c
FIX: correctly retrieve 'login required' setting value on wizard (#7355)
* FIX: correctly retrieve 'login required' setting value on wizard

FEATURE: extract 'invite only' setting in a separate checkbox control

* Update invite_only checkbox locale on wizard.

Co-Authored-By: techAPJ <arpit@techapj.com>
2019-04-11 20:25:08 +05:30
Arpit Jalan 318aba1f53 fix flaky spec 2019-04-11 12:25:41 +05:30
Sam Saffron baa7a9836c FEATURE: remove "COMPRESS_BROTLI" optional behavior
The compress brotli functionality is no longer optional, this has worked
well for years. The name of the ENV var is also confusing cause it does
not have a `DISCOURSE_` prefix which caused issues with the web upgrader

Brotli support is now unconditionally on
2019-04-11 12:36:18 +10:00
Robin Ward 87de5d9e8f Record the reason that a post was put into the queue 2019-04-10 17:42:49 -04:00
Dan Ungureanu 47c50d9a95
DEV: Add test for post notices. (#7352) 2019-04-10 22:06:27 +03:00
Robin Ward abf4ca9774 FIX: Should be able to recover a user deleted post if raw didn't change 2019-04-10 13:10:40 -04:00
Robin Ward cab83596ec FIX: Sites with SSO that required approval were not creating reviewables 2019-04-10 12:53:50 -04:00
Régis Hanol f665862ee8 SPEC: ensure we don't send activation reminder to staged users 2019-04-10 17:40:11 +02:00
Régis Hanol 4fe562ece5 fix build 2019-04-10 17:37:38 +02:00
Robin Ward 5d99346740 FIX: Always allow us to reject users, even if they are deleted 2019-04-10 11:00:39 -04:00
Bianca Nenciu 3d545d66df FEATURE: Send user activation reminders. (#7280) 2019-04-10 16:53:52 +02:00
Vinoth Kannan af15ba86a0 Fix the build.
8d5c900142
2019-04-10 19:14:41 +05:30
Roman Rizzi 536b90e0ef
FEATURE: Support searching custom staff actions (#7346) 2019-04-10 09:53:17 -03:00
Tarek Khalil 442fb2facb FEATURE: Remove ignore feature SiteSetting and enable ignore by default (#7349) 2019-04-10 12:54:59 +02:00
Roman Rizzi 82e051077d Show link to the original topic when it was deleted (#7319)
* Moved i18n keys to core. We show a links that points to the original topic when it was deleted

* Use link-to since it's more idiomatic
2019-04-10 11:25:45 +02:00
Vinoth Kannan d0fe42e2ef FIX: should look through posts for image markdown
Downloaded onebox images only included in the cooked HTML content. So we have to check 'post.cooked' instead of 'raw'. bfdd0fe64c
2019-04-10 13:52:35 +05:30
Sam Saffron f6db30a5da FEATURE: mark last notification unread when removing timings
This calls a bit more attention to the deferred topic by amending the "read"
state of the last notification on the topic.
2019-04-09 12:42:21 +10:00
Vinoth Kannan f5ac01253a FIX: use new changed method name and remove whitespaces 2019-04-09 02:25:26 +05:30
Vinoth Kannan 914ada1c74 DEV: convert scheduled job EnsurePostUploadsExistence into a rake task 2019-04-09 02:07:35 +05:30
Roman Rizzi aec457e09a
DEV: Expose a way to extend a method that returns a list 2019-04-08 14:42:36 -03:00
Robin Ward df1ab9259b FIX: Take action should agree with all pending flags 2019-04-08 12:39:18 -04:00
David Taylor 3987691b5e DEV: Correct specs which had exceptions raised in jobs 2019-04-08 16:00:33 +01:00
David Taylor 20cda5d689 DEV: Raise exceptions when jobs fail in test mode
Previously jobs would fail silently in test mode. Now they will raise the exception and cause the relevant test to fail. This identified a few broken tests, which I will fix in a followup commit
2019-04-08 15:59:04 +01:00
Guo Xiang Tan fd6b594625 Fix the build. 2019-04-08 21:30:17 +08:00
David Taylor f524f8f811
Remove Yahoo login support from core and deprecate OpenID2.0 (#7310)
- Plugin developers using OpenID2.0 should migrate to OAuth2 or OIDC. OpenID2.0 APIs will be removed in v2.4.0

- For sites requiring Yahoo login, it can be implemented using the OpenID Connect plugin: https://meta.discourse.org/t/103632

For more information, see https://meta.discourse.org/t/113249
2019-04-08 10:38:25 +01:00
David Taylor 6a05f190c6
PERF: Do not create staged users for most rejected incoming emails (#7301)
Previously we would create users, then destroy them at the end of the job if the post was rejected. Now we do not create users unless required.
2019-04-08 10:36:39 +01:00
Guo Xiang Tan 108c231d1c FIX: Clean up `topic_search_data` of trashed topics.
This keeps the index and table smaller.
2019-04-08 16:53:39 +08:00
Guo Xiang Tan c4997ce85f FIX: Don't try and reindex posts that have been trashed. 2019-04-08 16:38:43 +08:00
Robin Ward 8181ad646b FIX: Queued Posts were not saving tags properly 2019-04-05 14:07:47 -04:00
Robin Ward cbc311e4ed UX: Update the reviewable count before the message bus
In certain edge cases, the message bus won't send the message to the
user about the updated review count and it can go out of sync.

This patch synchronizes the review count every time:

1. The user visits the "Needs Review" page

2. Every time the user performs an action
2019-04-05 10:35:38 -04:00
Tarek Khalil 131eba0366 FIX: Hide post replies content for ignored users (#7320) 2019-04-05 16:13:36 +02:00
Erick Guan 9dd325f805 FIX: skip some checks for CJK locale in TextSentinel (#7322) 2019-04-05 15:07:49 +02:00
Maja Komel ca33d091b3 FIX: don't trigger notifications when changing category/tags of unlisted topics (#7323) 2019-04-05 15:06:38 +02:00
Sam Saffron 5f896ae8f7 PERF: Keep track of when a users first unread is
This optimisation avoids large scans joining the topics table with the
topic_users table.

Previously when a user carried a lot of read state we would have to join
the entire read state with the topics table. This operation would slow down
home page and every topic page. The more read state you accumulated the
larger the impact.

The optimisation helps people who clean up unread, however if you carry
unread from years ago it will only have minimal impact.
2019-04-05 12:44:45 +11:00
Guo Xiang Tan d299197392 DEV: Add missing test cases for `Topic.for_digest` and `Topic.secured`. 2019-04-05 09:00:48 +08:00
Robin Ward c9ddc6ce30 FIX: Make sure the site setting works and fix build 2019-04-04 17:44:10 -04:00
Neil Lalonde a0f7953f0c remove extra lines 2019-04-04 17:03:57 -04:00
Neil Lalonde 47eefb9f37 remove wip comments 2019-04-04 16:47:02 -04:00
Neil Lalonde 83996fc8ea FEATURE: ability to restrict some tags to a category while allowing all others
A new checkbox has been added to the Tags tab of the category settings modal
which is used when some tags and/or tag groups are restricted to the category,
and all other unrestricted tags should also be allowed.
Default is the same as the previous behaviour: only allow the specified set of
tags and tag groups in the category.
2019-04-04 16:40:15 -04:00
Robin Ward df48c657fd FIX: Reviewable counts were not updating properly
Sometimes sidekiq is so fast that it starts jobs before transactions
have comitted. This patch moves the message bus stuff until after things
have comitted.
2019-04-04 16:07:34 -04:00
Robin Ward 2055804e95 FIX: The option to delete replies was missing from the new review queue 2019-04-04 15:51:36 -04:00
Vinoth Kannan 64ec4a6a1b fix the EnsurePostUploadsExistence job spec 2019-04-04 21:53:58 +05:30
Vinoth Kannan df6ef856e6 DEV: save missing s3 uploads count in redis 2019-04-04 19:05:57 +05:30
Vinoth Kannan b3fb0a7039 FEATURE: ensure consistency of post uploads in cooked content 2019-04-04 02:23:28 +05:30
Robin Ward 111a502231 FIX: Deleting Users should work nicely with Reviewable Users
"Rejecting" a user in the queue is equivalent to deleting them, which
would then making it impossible to review rejected users. Now we store
information about the user in the payload so if they are deleted things
still display in the Rejected view.

Secondly, if a user is destroyed outside of the review queue, it will
now automatically "Reject" that queue item.
2019-04-03 16:42:39 -04:00
Robin Ward c1ea63bdc1 FIX: Reviewables should not be created for users until they are active
Conversely, if a user is deactivated the reviewable should automatically
be rejected.

Before this fix, if a user was not active they'd still show in the
review queue but without an "Approve" button which was confusing.
2019-04-03 15:25:00 -04:00
Robin Ward 1bce97a596 FIX: Allow queued posts from deleted users to be rejected 2019-04-03 11:40:46 -04:00
Robin Ward 62956003c3 FEATURE: Allow users to customize bonuses for reviewable types
A new settings section in the review queue allows admins to specify that
certain types of flags should be weighted higher than others.
2019-04-03 11:18:34 -04:00
Guo Xiang Tan c10861da2f DEV: Fix randomly failing search spec. 2019-04-03 16:42:14 +08:00
Arpit Jalan 7b194743d7
FIX: clean up user export csv upload records in scheduled job (#7309) 2019-04-03 13:31:19 +05:30
Sam Saffron 1c57ae6657 FIX: grant first quote at the date post was created
Previously due to #b2dc65f9534ea date on the quoted_posts table could not
be trusted.

This changes it so we use the date on the actual post as the grant date.

Note: there is an edge case where you create a post and only add a quote
a week later. In this case the badge will not be awarded at the correct
time (it will display it was granted a week ago).
That said this is far more rare than the current situation.
2019-04-03 16:41:52 +11:00
Sam Saffron b2dc65f953 FIX: on rebakes should not recreate quoted_posts records
Previously every rebake would remove and recreate records in this table
This caused created_at and updated_at to keep changing

Yes, I know the SQL is somewhat complex, but this makes quote extraction
more efficient cause we do everything in 2 round trips.

This also removes some concurrency protection we should no longer need
2019-04-03 16:17:15 +11:00
Guo Xiang Tan d151425353
PERF: Delete search data of posts from trashed topics periodically. (#7302)
This keeps both the index and table smaller.
2019-04-03 10:10:41 +08:00
venarius c185109380 FIX: Restricted site text better error 2019-04-02 11:16:27 -04:00
Robin Ward 123c05cb07 FIX: category id wasn't stored when enqueueing a topic 2019-04-02 11:12:17 -04:00
Robin Ward 6ebadaed2c FIX: Do not allow `invite_only` and `enable_sso` at the same time
This functionality was never supported but before the new review queue
it didn't have any errors. Now the combination of settings is prevented
and existing sites with sso enabled will be migrated to remove invite
only.
2019-04-02 10:26:27 -04:00
Robin Ward af04318aff FIX: Allow us to serialize reviewables with deleted topics 2019-04-02 09:36:49 -04:00
Arpit Jalan d68d29f37a FIX: only staff can banner topics 2019-04-02 12:46:55 +05:30
Guo Xiang Tan 6a95d3fded FIX: `UploadRecovery` should look through posts for img src and bbcode. 2019-04-02 11:41:00 +08:00
Penar Musaraj 1c0f885a5b FIX: double render error with delegated authentication
Makes sure delegated authentication is checked before other login redirects

Updates specs to cover login_required = true cases
2019-04-01 22:13:53 -04:00
Guo Xiang Tan d8704c11ca PERF: Better use of index when queueing a topci for search reindex.
Also move `Search::INDEX_VERSION` to `SearchIndexer` which is where the
version is actually being used.
2019-04-02 09:53:37 +08:00
Guo Xiang Tan a2e6a8a51e DEV: Refactor test to assert for actual content.
The other assertion is already asserting for `Post#url` and I think it
makes the test clearer here.
2019-04-02 07:43:13 +08:00
Guo Xiang Tan aa2311a7b0 FIX: Don't reindex posts belonging to a deleted topic for search.
Posts belonging to a deleted topic can't be index for search so we need
to avoid loading those post ids.
2019-04-02 07:36:53 +08:00
Guo Xiang Tan 3fc5dbb045 FIX: Don't attempt to reindex posts that have an empty raw.
If the post ids keep loading, we might end up in a situations where
we're always loading the same post ids over and over again without
indexing anything new.

Follow up to daeda80ada.
2019-04-02 07:13:33 +08:00
Robin Ward d5a61ab167 FIX: Don't log a second pending action
If two people flagged the same post you'd get a "Pending" history event
logged which looked bad.
2019-04-01 17:07:47 -04:00
Robin Ward 76669bb5a6 FIX: Don't refer to pending review items as flags
They could be queued posts or users, and the notice should reflect that
properly.
2019-04-01 14:46:56 -04:00
Robin Ward 6470c85fda UX: Link directly to the post, not to the topic 2019-04-01 14:29:14 -04:00
Penar Musaraj fdf4145d4b
FEATURE: Delegated authentication via user api keys (#7272) 2019-04-01 13:18:53 -04:00
Joffrey JAFFEUX e986e96227
DEV: removes old dashboard (#7295) 2019-04-01 12:39:49 +02:00
Guo Xiang Tan 2a69ab4a4c FIX: Keep `alt` and `title` in lightbox when indexing for search.
Follow up to cfd507822f
2019-04-01 16:20:33 +08:00
Guo Xiang Tan e87ca59401 FIX: Relevance search will now consider document length in ranking.
The default ranking options ranks by the number of matches which is
highly problematic when posts are stuffed with a keyword. The ranking
will now be divided by the document length which is a much fairer way to
rank.
2019-04-01 14:37:45 +08:00
Maja Komel 88128f1ced UX: show which groups are missing permissions for parent category (#7252) 2019-04-01 15:34:52 +11:00
Guo Xiang Tan 16215f9d3b DEV: Correct spec added in cfd507822f.
Remove stub.
2019-04-01 10:32:25 +08:00
Guo Xiang Tan cfd507822f
PERF: Improve quality of `PostSearchData#raw_data`. (#7275)
This commit fixes the follow quality issue with `PostSearchData#raw_data`:

1. URLs are being tokenized and links with similar href and characters
are being duplicated in the raw data.

`Post#cooked`:

```
<p><a href=\"https://meta.discourse.org/some.png\" class=\"onebox\" target=\"_blank\" rel=\"nofollow noopener\">https://meta.discourse.org/some.png</a></p>
```

`PostSearchData#raw_data` Before:

```
This is a test topic 0 Uncategorized https://meta.discourse.org/some.png discourse org/some png https://meta.discourse.org/some.png discourse org/some png
```

`PostSearchData#raw_data` After:

```
This is a test topic 0 Uncategorized https://meta.discourse.org/some.png meta discourse org
```

2. Ligthbox being included in search pollutes the
`PostSearchData#raw_data` unncessarily.

From 28 March 2018 to 28 March 2019, searches for the term `image` on
`meta.discourse.org` had a click through rate of 2.1%. Non-lightboxed images are not included in indexing for search yet we were indexing content within a lightbox. Also, search for terms like `image` was affected we were using `Pasted image` as the filename for
uploads that were pasted.

`Post#cooked`

```
<p>Let me see how I can fix this image<br>\n<div class=\"lightbox-wrapper\"><a class=\"lightbox\" href=\"https://meta.discourse.org/some.png\" title=\"some.png\" rel=\"nofollow noopener\"><img src=\"https://meta.discourse.org/some.png\" width=\"275\" height=\"299\"><div class=\"meta\">\n<svg class=\"fa d-icon d-icon-far-image svg-icon\" aria-hidden=\"true\"><use xlink:href=\"#far-image\"></use></svg><span class=\"filename\">some.png</span><span class=\"informations\">1750×2000</span><svg class=\"fa d-icon d-icon-discourse-expand svg-icon\" aria-hidden=\"true\"><use xlink:href=\"#discourse-expand\"></use></svg>\n</div></a></div></p>
```

`PostSearchData#raw_data` Before:

```
This is a test topic 0 Uncategorized Let me see how I can fix this image some.png png https://meta.discourse.org/some.png discourse org/some png some.png png 1750×2000
```

`PostSearchData#raw_data` After:

```
This is a test topic 0 Uncategorized Let me see how I can fix this image
```

In terms of indexing performance, we now have to parse the given HTML
through nokogiri twice. However performance is not a huge worry here since a string length of 194170 takes only 30ms
to scrub plus the indexing takes place in a background job.
2019-04-01 10:14:29 +08:00
Guo Xiang Tan 7ac76fe935 DEV: Remove warning.
Library has already been loaded in application.rb.
2019-04-01 10:11:08 +08:00
Guo Xiang Tan daeda80ada
FIX: Don't index posts with empty `Post#raw` for search. (#7263)
* DEV: Remove unnecessary join in `Jobs::ReindexSearch`.

* FIX: Don't index posts with empty `Post#raw` for search.
2019-04-01 10:06:27 +08:00
Guo Xiang Tan 730ebdfcba DEV: Refactor `Jobs::EmitWebHookEvent` specs. 2019-04-01 09:46:39 +08:00
Vinoth Kannan 904ba266cf SPEC: Add test case in emit_web_hook_event_spec for commit 4c6bfb9 2019-03-31 16:28:40 +05:30
Bianca Nenciu 034b8a7ecc FIX: Let users delete topics.
Follow-up to 31053f30de.
2019-03-29 22:00:36 +02:00
Robin Ward 370355d754 FIX: Allow users with posts to be rejected 2019-03-29 13:53:46 -04:00
Bianca Nenciu 31053f30de FEATURE: Let users delete their own topics. (#7267) 2019-03-29 17:10:05 +01:00
Maja Komel 4a3daacb1b FIX: reset embedding settings when no embeddable host, log host changes (#7264) 2019-03-29 17:05:51 +01:00
Roman Rizzi 7740b1570b FIX: Avoid the deleted_at scope when recovering a topic from a recently recovered post 2019-03-29 09:40:15 -04:00
Tarek Khalil b1cb95fc23
FEATURE: Introduce ignore duration selection (#7266)
* FEATURE: Introducing new UI for tracking User's ignored or muted states
2019-03-29 10:14:53 +00:00
Guo Xiang Tan f458cba4cb FIX: Admin search logs should filter by date instead of timestamp.
The client side filters by date so it is confusion when the data changes as each second passes.
2019-03-29 11:50:25 +08:00
Guo Xiang Tan d8faf5f79e FIX: `SearcLog.term_details` generating incorrect data because of case.
Also match on equality rather than "LIKE ?" which is quite strange.
2019-03-29 11:31:01 +08:00
Guo Xiang Tan 8c2fa99f78 FIX: Remove `:term` from `admin/search_logs/term/:term` route.
Search log terms is a string that can contain characters like `/` which
messes with the route.
2019-03-29 09:48:20 +08:00
Robin Ward 118f98b6ed Linting error 2019-03-28 12:52:46 -04:00
Robin Ward b58867b6e9 FEATURE: New 'Reviewable' model to make reviewable items generic
Includes support for flags, reviewable users and queued posts, with REST API
backwards compatibility.

Co-Authored-By: romanrizzi <romanalejandro@gmail.com>
Co-Authored-By: jjaffeux <j.jaffeux@gmail.com>
2019-03-28 12:45:10 -04:00
Sam Saffron 9ebabc1de8 FEATURE: unconditionally update Topic updated_at when posts change in topic
Previously we would bypass touching `Topic.updated_at` for whispers and post
recovery / deletions.

This meant that certain types of caching can not be done where we rely on
this information for cache accuracy.

For example if we know we have zero unread topics as of yesterday and whisper
is made I need to bump this date so the cache remains accurate

This is only half of a larger change but provides the groundwork.

Confirmed none of our serializers leak out Topic.updated_at so this is safe
spot for this info

At the moment edits still do not change this but it is not relevant for the
unread cache.

This commit also cleans up some specs to use the new `eq_time` matcher for
millisecond fidelity comparison of times

Previously `freeze_time` would fudge this which is not that clean.
2019-03-28 17:28:01 +11:00
David Taylor 95d5819218 FIX: Re-download hotlinked optimized images (#7249)
* FIX: Download local images, even if download remote is disabled
2019-03-27 21:31:12 +01:00
Tim Lange 12181599db FIX: Staff action records now also accepts action_name as filter (#7256) 2019-03-27 21:29:15 +01:00