Commit Graph

38308 Commits

Author SHA1 Message Date
Guo Xiang Tan d8c796bc44
FIX: Ensure that aggregating search shows the post with the higest rank.
Previously, we would only take either the `MIN` or `MAX` for
`post_number` during aggregation meaning that the ranking is not
considered.

```
require 'benchmark/ips'

Benchmark.ips do |x|
  x.config(time: 10, warmup: 2)

  x.report("current aggregate search query") do
    DB.exec <<~SQL
    SELECT "posts"."id", "posts"."user_id", "posts"."topic_id", "posts"."post_number", "posts"."raw", "posts"."cooked", "posts"."created_at", "posts"."updated_at", "posts"."reply_to_post_number", "posts"."reply_count", "posts"."quote_count", "posts"."deleted_at", "posts"."off_topic_count", "posts"."like_count", "posts"."incoming_link_count", "posts"."bookmark_count", "posts"."score", "posts"."reads", "posts"."post_type", "posts"."sort_order", "posts"."last_editor_id", "posts"."hidden", "posts"."hidden_reason_id", "posts"."notify_moderators_count", "posts"."spam_count", "posts"."illegal_count", "posts"."inappropriate_count", "posts"."last_version_at", "posts"."user_deleted", "posts"."reply_to_user_id", "posts"."percent_rank", "posts"."notify_user_count", "posts"."like_score", "posts"."deleted_by_id", "posts"."edit_reason", "posts"."word_count", "posts"."version", "posts"."cook_method", "posts"."wiki", "posts"."baked_at", "posts"."baked_version", "posts"."hidden_at", "posts"."self_edits", "posts"."reply_quoted", "posts"."via_email", "posts"."raw_email", "posts"."public_version", "posts"."action_code", "posts"."locked_by_id", "posts"."image_upload_id" FROM "posts" JOIN (SELECT *, row_number() over() row_number FROM (SELECT topics.id, min(posts.post_number) post_number FROM "posts" INNER JOIN "post_search_data" ON "post_search_data"."post_id" = "posts"."id" INNER JOIN "topics" ON "topics"."id" = "posts"."topic_id" AND ("topics"."deleted_at" IS NULL) LEFT JOIN categories ON categories.id = topics.category_id WHERE ("posts"."deleted_at" IS NULL) AND "posts"."post_type" IN (1, 2, 3, 4) AND (topics.visible) AND (topics.archetype <> 'private_message') AND (post_search_data.search_data @@ TO_TSQUERY('english', '''postgres'':*ABCD')) AND (categories.id NOT IN (
      SELECT categories.id WHERE categories.search_priority = 1
    )
    ) AND ((categories.id IS NULL) OR (NOT categories.read_restricted)) GROUP BY topics.id ORDER BY MAX((
      TS_RANK_CD(
        post_search_data.search_data,
        TO_TSQUERY('english', '''postgres'':*ABCD'),
        1|32
      ) *
      (
        CASE categories.search_priority
        WHEN 2
        THEN 0.6
        WHEN 3
        THEN 0.8
        WHEN 4
        THEN 1.2
        WHEN 5
        THEN 1.4
        ELSE
          CASE WHEN topics.closed
          THEN 0.9
          ELSE 1
          END
        END
      )
    )
    ) DESC, topics.bumped_at DESC LIMIT 51 OFFSET 0) xxx) x ON x.id = posts.topic_id AND x.post_number = posts.post_number WHERE ("posts"."deleted_at" IS NULL) ORDER BY row_number;
    SQL
  end

  x.report("current aggregate search query with proper ranking") do
    DB.exec <<~SQL
    SELECT "posts"."id", "posts"."user_id", "posts"."topic_id", "posts"."post_number", "posts"."raw", "posts"."cooked", "posts"."created_at", "posts"."updated_at", "posts"."reply_to_post_number", "posts"."reply_count", "posts"."quote_count", "posts"."deleted_at", "posts"."off_topic_count", "posts"."like_count", "posts"."incoming_link_count", "posts"."bookmark_count", "posts"."score", "posts"."reads", "posts"."post_type", "posts"."sort_order", "posts"."last_editor_id", "posts"."hidden", "posts"."hidden_reason_id", "posts"."notify_moderators_count", "posts"."spam_count", "posts"."illegal_count", "posts"."inappropriate_count", "posts"."last_version_at", "posts"."user_deleted", "posts"."reply_to_user_id", "posts"."percent_rank", "posts"."notify_user_count", "posts"."like_score", "posts"."deleted_by_id", "posts"."edit_reason", "posts"."word_count", "posts"."version", "posts"."cook_method", "posts"."wiki", "posts"."baked_at", "posts"."baked_version", "posts"."hidden_at", "posts"."self_edits", "posts"."reply_quoted", "posts"."via_email", "posts"."raw_email", "posts"."public_version", "posts"."action_code", "posts"."locked_by_id", "posts"."image_upload_id" FROM "posts" JOIN (SELECT *, row_number() over() row_number FROM (SELECT subquery.topic_id id, (ARRAY_AGG(subquery.post_number))[1] post_number, MAX(subquery.rank) rank, MAX(subquery.bumped_at) bumped_at FROM (SELECT "posts"."id", "posts"."user_id", "posts"."topic_id", "posts"."post_number", "posts"."raw", "posts"."cooked", "posts"."created_at", "posts"."updated_at", "posts"."reply_to_post_number", "posts"."reply_count", "posts"."quote_count", "posts"."deleted_at", "posts"."off_topic_count", "posts"."like_count", "posts"."incoming_link_count", "posts"."bookmark_count", "posts"."score", "posts"."reads", "posts"."post_type", "posts"."sort_order", "posts"."last_editor_id", "posts"."hidden", "posts"."hidden_reason_id", "posts"."notify_moderators_count", "posts"."spam_count", "posts"."illegal_count", "posts"."inappropriate_count", "posts"."last_version_at", "posts"."user_deleted", "posts"."reply_to_user_id", "posts"."percent_rank", "posts"."notify_user_count", "posts"."like_score", "posts"."deleted_by_id", "posts"."edit_reason", "posts"."word_count", "posts"."version", "posts"."cook_method", "posts"."wiki", "posts"."baked_at", "posts"."baked_version", "posts"."hidden_at", "posts"."self_edits", "posts"."reply_quoted", "posts"."via_email", "posts"."raw_email", "posts"."public_version", "posts"."action_code", "posts"."locked_by_id", "posts"."image_upload_id", (
      TS_RANK_CD(
        post_search_data.search_data,
        TO_TSQUERY('english', '''postgres'':*ABCD'),
        1|32
      ) *
      (
        CASE categories.search_priority
        WHEN 2
        THEN 0.6
        WHEN 3
        THEN 0.8
        WHEN 4
        THEN 1.2
        WHEN 5
        THEN 1.4
        ELSE
          CASE WHEN topics.closed
          THEN 0.9
          ELSE 1
          END
        END
      )
    )
     rank, topics.bumped_at bumped_at FROM "posts" INNER JOIN "post_search_data" ON "post_search_data"."post_id" = "posts"."id" INNER JOIN "topics" ON "topics"."id" = "posts"."topic_id" AND ("topics"."deleted_at" IS NULL) LEFT JOIN categories ON categories.id = topics.category_id WHERE ("posts"."deleted_at" IS NULL) AND "posts"."post_type" IN (1, 2, 3, 4) AND (topics.visible) AND (topics.archetype <> 'private_message') AND (post_search_data.search_data @@ TO_TSQUERY('english', '''postgres'':*ABCD')) AND (categories.id NOT IN (
      SELECT categories.id WHERE categories.search_priority = 1
    )
    ) AND ((categories.id IS NULL) OR (NOT categories.read_restricted))) subquery GROUP BY subquery.topic_id ORDER BY rank DESC, bumped_at DESC LIMIT 51 OFFSET 0) xxx) x ON x.id = posts.topic_id AND x.post_number = posts.post_number WHERE ("posts"."deleted_at" IS NULL) ORDER BY row_number;
    SQL
  end

  x.compare!
end
```

```
Warming up --------------------------------------
current aggregate search query
                         1.000  i/100ms
current aggregate search query with proper ranking
                         1.000  i/100ms
Calculating -------------------------------------
current aggregate search query
                         17.726  (± 0.0%) i/s -    178.000  in  10.045107s
current aggregate search query with proper ranking
                         17.802  (± 0.0%) i/s -    178.000  in  10.002230s

Comparison:
current aggregate search query with proper ranking:       17.8 i/s
current aggregate search query:       17.7 i/s - 1.00x  (± 0.00) slower
```
2020-07-14 13:39:13 +08:00
Guo Xiang Tan 4009c9f711
DEV: Fix search specs to take note of order in assertions.
`contain_exactly` does not care about the order which isn't what we
want.
2020-07-14 13:37:44 +08:00
Martin Brennan bcc80e0ea8
FEATURE: Add search to user bookmark list (#10230)
User bookmarks can now be searched by name or post raw content. The q querystring param is hooked up from the Ember router as well.
2020-07-14 14:43:41 +10:00
Martin Brennan f4f3e8c401
FIX: Various improvements to bookmark modal UI (#10225)
* Do not autofocus name input on mobile
* Improve code for formatted reminder type times to not be computed, so the modal times update correctly
* Change wording of "Next Monday" to "Monday" for all days except when today is Monday
2020-07-14 14:02:13 +10:00
Guo Xiang Tan ce39733b1a
FIX: Incorrect search blurb when advanced search filters are used take2
Also remove include_blurbs attribute which isn't used.
2020-07-14 11:50:40 +08:00
Kane York 919c87a9ce
FIX: Get only the correct collapse title in emails (#10228)
Previously, this would get every details block title from the entire post, for every details block.
2020-07-13 18:49:36 -07:00
Kane York 8ddd45d524
PERF: topic_view participant post count: don't send back ID list (#10210)
On large topics, the cost of sending the entire post ID list back over to the database is signficant. Just have the DB recalculate the list of visible posts instead.
2020-07-13 18:42:09 -07:00
dependabot-preview[bot] ab77707c1e
DEV: Bump oj from 3.10.6 to 3.10.7 (#10227)
Bumps [oj](https://github.com/ohler55/oj) from 3.10.6 to 3.10.7.
- [Release notes](https://github.com/ohler55/oj/releases)
- [Changelog](https://github.com/ohler55/oj/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/ohler55/oj/compare/v3.10.6...v3.10.7)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-14 10:57:23 +10:00
Kris 277d09516e topic-admin-menu shouldn't be fixed width on mobile 2020-07-13 18:50:57 -04:00
Robin Ward 7045a2a87c FIX: Don't strip `noopener` from oneboxes 2020-07-13 16:54:42 -04:00
dependabot-preview[bot] c69983a30d
Build(deps): Bump multi_json from 1.14.1 to 1.15.0 (#10215)
Bumps [multi_json](https://github.com/intridea/multi_json) from 1.14.1 to 1.15.0.
- [Release notes](https://github.com/intridea/multi_json/releases)
- [Changelog](https://github.com/intridea/multi_json/blob/master/CHANGELOG.md)
- [Commits](https://github.com/intridea/multi_json/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-13 15:33:49 -04:00
Robin Ward a73da42691 FIX: Don't award new user of the month in the wrong month
see: https://meta.discourse.org/t/new-user-of-the-month-badge-awarded-before-registraton-date/157347/2?u=eviltrout
2020-07-13 15:05:30 -04:00
Penar Musaraj c02e358146
FIX: Remove social sharing icons from private contexts (#10213) 2020-07-13 14:35:39 -04:00
Rafael dos Santos Silva e0f97c707e PERF: Cache user summary data 2020-07-13 15:26:56 -03:00
Rafael dos Santos Silva a4d5d1f51d PERF: Add user_id condition so we can use another index in the query 2020-07-13 15:26:56 -03:00
Rafael dos Santos Silva 73a3e2ccbc DEV: Add spec for summary links 2020-07-13 15:26:56 -03:00
Blake Erickson a900c99993 DEV: Add external_id to notification payload
If sso is enabled the notification payload will now include the
external_id for the user.

This was requested on meta: https://meta.discourse.org/t/-/129052/10
2020-07-13 12:07:39 -06:00
dependabot-preview[bot] 33554e5cbc
Build(deps): Bump cose from 1.1.0 to 1.2.0 (#10223)
Bumps [cose](https://github.com/cedarcode/cose-ruby) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/cedarcode/cose-ruby/releases)
- [Changelog](https://github.com/cedarcode/cose-ruby/blob/master/CHANGELOG.md)
- [Commits](https://github.com/cedarcode/cose-ruby/compare/v1.1.0...v1.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-13 12:30:39 -04:00
Dan Ungureanu cf02c518b9
DEV: Merge category and tag hashtags code paths (#10216)
Category and tag hashtags used to be handled differently even though
most of the code was very similar. This design was the root cause of
multiple issues related to hashtags.

This commit reduces the number of requests (just one and debounced
better), removes the use of CSS classes which marked resolved hashtags,
simplifies a lot of the code as there is a single source of truth and
previous race condition fixes are now useless.

It also includes a very minor security fix which let unauthorized users
to guess hidden tags.
2020-07-13 19:13:17 +03:00
dependabot-preview[bot] c5da813ff5
Build(deps): Bump rubocop-rspec from 1.41.0 to 1.42.0 (#10207)
Bumps [rubocop-rspec](https://github.com/rubocop-hq/rubocop-rspec) from 1.41.0 to 1.42.0.
- [Release notes](https://github.com/rubocop-hq/rubocop-rspec/releases)
- [Changelog](https://github.com/rubocop-hq/rubocop-rspec/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rubocop-hq/rubocop-rspec/compare/v1.41.0...v1.42.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-13 12:01:43 -04:00
Bianca Nenciu 6c6c56613d FIX: Check if last poster exists before using it
There was a problem when admins viewed topics where the last poster was
deleted.
2020-07-13 18:55:14 +03:00
dependabot-preview[bot] c8856c7a43
Build(deps): Bump rake-compiler from 1.1.0 to 1.1.1 (#10206)
Bumps [rake-compiler](https://github.com/luislavena/rake-compiler) from 1.1.0 to 1.1.1.
- [Release notes](https://github.com/luislavena/rake-compiler/releases)
- [Changelog](https://github.com/rake-compiler/rake-compiler/blob/master/History.txt)
- [Commits](https://github.com/luislavena/rake-compiler/compare/v1.1.0...v1.1.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-13 11:52:56 -04:00
Jarek Radosz a96ff82e53
FIX: Set the lang/xml:lang html attrs in emails (#10218)
In some rare cases emails are incorrectly sent out with lang placeholders in body (`xml:lang="%{html_lang}"`)
2020-07-13 16:39:40 +02:00
Arpit Jalan 8e2f57a498 Fix the build. 2020-07-13 18:52:04 +05:30
Arpit Jalan 7532f24668 UX: better error message if moderator is not allowed to invite to group
UX: do not show invite to group option if mod is not owner of any group
2020-07-13 18:09:36 +05:30
Arpit Jalan ba57dc57cc DEV: use group_ids instead of group_names in invite params 2020-07-13 17:37:04 +05:30
Jarek Radosz eb73048b0f
DEV: Allow displaying both title and panels in modals (#10220) 2020-07-13 12:43:06 +02:00
Bianca Nenciu 4492718864
FIX: Skip whisper posts when updating topic like count (#10157) 2020-07-13 16:30:00 +10:00
Guo Xiang Tan 54d002f7db
DEV: Fix error introduced in d5c56a846. 2020-07-13 10:13:37 +08:00
Guo Xiang Tan 753477e684
FIX: Slightly reduce fake delay of discobot user.
Previously it would sleep for either 2 or 3 seconds instead of sleeping
between a range of 2 to 3 seconds. Also, 2 to 3 seconds seems to be
excessivly long when I tried out discobot again.
2020-07-13 09:08:31 +08:00
Krzysztof Kotlarek 180494185c
FIX: last visit bar regression (#10224)
Last visit regression bar is missing because of that change https://github.com/discourse/discourse/commit/9b7000dbf10

Order property was changed from "default" to null and it was picked by guard condition
2020-07-13 10:42:05 +10:00
Jarek Radosz 6c91f795d0
DEV: widget-dropdown CSS tweaks (#10219)
* DEV: Remove the margin from widget-dropdown

Generic components should not have a margin. Those should be styled in the place where they are used.

* DEV: Remove margin from the dropdown body

It triggered a warning in popper and was effectively a no-op as popper positions dropdowns on its own using `position: fixed` and `top/right/bottom/left` properties.
2020-07-13 01:27:31 +02:00
Jarek Radosz 942cc9b57a
DEV: Add rawTitle property support to modal-tab (#10221)
```js
const panels = [
  { id: "test1", rawTitle: "Test 1" },
  { id: "test2", rawTitle: "Test 2" }
];

showModal("a-modal", { panels }));
```
2020-07-13 01:26:45 +02:00
Blake Erickson 544a9865c6 DEV: Add more API Doc specs for topic endpoints
Added some more specs that will be used to auto generate the api docs.

Added timed topic and update timestamp endpoints.
2020-07-10 12:08:09 -06:00
David Taylor fbf29443bd
FIX: Allow signup when auth provider supplies no email 2020-07-10 17:56:33 +01:00
Bianca Nenciu 275b748016
FIX: Replace links to removed uploads from reviewables with a placeholder (#10180) 2020-07-10 17:57:06 +03:00
David Taylor 4b0d86d8d3
FIX: Do not display enabled inputs when username/name are locked 2020-07-10 10:49:32 +01:00
Dan Ungureanu c72bc27888
FEATURE: Implement support for IMAP and SMTP email protocols. (#8301)
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2020-07-10 12:05:55 +03:00
Dan Ungureanu e88b17c044
FIX: Sync client and server side behavior for category hashtag lookup
Category.query_from_hashtag_slug is the only source of truth.
2020-07-10 10:45:39 +03:00
Martin Brennan 56f42d89c5
FIX: Post menu bookmark icon and attributes not refreshing on notification click (#10214)
When creating a bookmark reminder that deletes the bookmark on reminder, if the user clicked on the notification and got taken to the post in the topic the bookmark icon still showed as blue with the reminder clock indicator. This was because the response JSON for reloading a topic post was not including the bookmark attributes, not even the bookmarked boolean.

We now return the correct attributes in the serializer, and if bookmarked is false we clear all the bookmark related attributes on the post for the notification to make sure nothing of the old bookmark remains in the UI.

This was only a problem if the user did not refresh the app completely inbetween setting the reminder and receiving the notification.
2020-07-10 14:35:53 +10:00
Martin Brennan 9a2955d471
FIX: Migrate topic_users.bookmarked to false when it is incorrectly true (#10211)
Follow up to https://github.com/discourse/discourse/pull/10188/files

There are still TopicUser records where bookmarked is true even though there are no Bookmark or PostAction records with the type of bookmark for the associated topic and user. This migration corrects this issue by setting bookmarked to false for these cases.
2020-07-10 12:22:15 +10:00
Penar Musaraj 37f7e41e60
DEV: Remove Google+ icon from sprite 2020-07-09 21:29:39 -04:00
Kris 706f1a6294
DEV: Remove text-ellipsis component and use CSS line-clamp instead (#10196) 2020-07-09 20:51:43 -04:00
Rafael dos Santos Silva 831f3cab56
DEV: Fix search rate limit tests 2020-07-09 20:44:17 -03:00
Blake Erickson e74817cbb9 DEV: Document set notification level endpoint
Another commit using rswag to document the api so that the api docs can
be automatically generated.
2020-07-09 17:41:11 -06:00
Rafael dos Santos Silva e866e3d609
FEATURE: Add global rate limit for anon searches (#10208) 2020-07-10 09:08:34 +10:00
Kane York 79b52b1e9a
DEV: Add SVG tests for 31e31ef44 (#10205) 2020-07-09 14:02:25 -07:00
Mark VanLandingham bfde665e76
DEV: Classes and plugin-outlet in admin user-list nav (#10204) 2020-07-09 15:10:25 -05:00
Robin Ward 5b276af921
Remove `Discourse.SiteSettings` from tests (#10193)
* Remove unused Discourse.SiteSettings

* Remove `Discourse.SiteSettings` from many tests

* REFACTOR: `lib:formatter` was using a lot of leaky state

* Remove more `Discourse.SiteSettings` from tests

* More SiteSettings removed from tests
2020-07-09 15:54:53 -04:00
Robin Ward b1c6ff9e1c FIX: Test output related to `Discourse::VERSION`
It's a little awkward to test constants by re-assigning them so
I've added a new parameter to `Discourse.find_compatible_resource`
which can be used by tests.
2020-07-09 14:57:27 -04:00