Commit Graph

7937 Commits

Author SHA1 Message Date
Sam Saffron 716629dc0a
DEV: correct flaky test relying on possibly valid user id
This test lasted about 7 years prior to it becoming flaky.

Today ... for whatever reason the test suite created 100 users
prior to running this spec. So the new user becomes user id 101

And... lo and behold:

```

  1) PostSerializer a hidden post with add_raw enabled a hidden post shows the raw post only if authorized to see it
     Failure/Error: expect(serialized_post_for_user(Fabricate(:user))[:raw]).to eq(nil)

       expected: nil
            got: "Raw contents of the post."

       (compared using ==)
     # ./spec/serializers/post_serializer_spec.rb:127:in `block (4 levels) in <main>'
```
2020-06-09 17:40:41 +10:00
Bianca Nenciu 052c91770f
FIX: Reply notifications should not appear as edited (#9965) 2020-06-08 15:23:33 +02:00
Gerhard Schlager 0bf753a739 DEV: Remove unused routes 2020-06-08 10:26:29 +02:00
Gerhard Schlager 8c6a42c589 FIX: Redirects containing Unicode usernames didn't work 2020-06-08 10:26:29 +02:00
Martin Brennan e84ffb4861
DEV: Add pry-byebug and optionally require pry behind a ENV var for better spec debugging (#9984)
We removed pry-nav a while back because it is not up to date with pry but it is super useful. Luckily pry-byebug is here to save us all from Satan's power.

To get this to work you need to add the following to your $HOME/.pryrc file.

```
if defined?(PryByebug)
  Pry.commands.alias_command 'c', 'continue'
  Pry.commands.alias_command 's', 'step'
  Pry.commands.alias_command 'n', 'next'
  Pry.commands.alias_command 'f', 'finish'
end

Pry::Commands.command /^$/, "repeat last command" do
  pry_instance.run_command Pry.history.to_a.last
end
```

The require-ing of pry, pry-rails, and pry-byebug in specs is controlled by the IMPROVED_SPEC_DEBUGGING flag (disabled by default).
2020-06-05 12:30:34 +10:00
Vinoth Kannan 0f20a6f0aa FIX: use `short_path` of flair upload to get signed url for secure media.
If we use `upload.url` for secure urls then the images won't render.
2020-06-05 07:43:15 +05:30
Jeff Wong de29b4a511
PERF: rate limit search, and add anon cache for search results (#9969)
Adds new hidden site settings for rate limits:
30 for logged in users, 15 for anon

Adds an anon cache for searching, caches results of searches for 1 minute
2020-06-04 09:26:08 -07:00
Penar Musaraj 2d880b42a3
UX: Add simple-list setting type (#9970) 2020-06-04 10:44:54 -04:00
Guo Xiang Tan 2188ccccd5 DEV: Remove `initiating_user` keyword arg from `EmailUpdater`.
The guardian contains the acting user.
2020-06-04 13:21:56 +08:00
Michael K Johnson 81e6bc7a0f
FEATURE: Add uploads:batch_migrate_from_s3 task to limit total posts migrated at once (#9933)
Allow limiting the number of migrations to do at once, both to do migrations that
have impact limited to multiple off-peak usage hours to reduce user impact from
a migration, and to allow tests that do only a very small number for test
purposes. ("Give me a ping, Vasili. One ping only, please.")
2020-06-04 09:48:11 +10:00
Blake Erickson a89574ccb9 FIX: Inline error when converting html to markdown
Looks like some html elements like `aside` and `section` will throw an error
when checking if they are inline or not. The commit simply handles

```
Job exception: undefined method `inline?' for nil:NilClass
```

and adds a test for it.
2020-06-03 15:59:19 -06:00
Gerhard Schlager f683c5d0e0 DEV: Check English locale for errors in CI
Moves the most important checks into a linter. It gets executed by Lefthook as well as the docker rake task and Github actions. Doing those checks in rspec takes too long and it produces errors when the discourse:test Docker image contains old, invalid locale files.
2020-06-03 21:54:58 +02:00
Vinoth Kannan 3e7f7fdde8
FEATURE: category setting for default list filter. (#9975) 2020-06-04 00:56:56 +05:30
Robin Ward 22789e0201 New `bootstrap.json` endpoint for starting up Discourse
Discourse needs a bunch of data preloaded before it can start up.
Normally we throw blobs of this into the HTML document that is requested
but in some cases that's awkward to retrieve.

For example with Ember CLI you have a separate javascript application
that needs to make its own HTML.

This API endpoint returns a JSON object with all the data Discourse needs to
bootstrap and start up.
2020-06-03 14:45:23 -04:00
Vinoth Kannan 7fe414d35d FIX: hide rss feed & json if a user profile is hidden. 2020-06-03 19:03:02 +05:30
Guo Xiang Tan 04a291ceea
DEV: Fix race conditions due to directory removal for uploads spec. 2020-06-03 12:28:39 +08:00
Sam Saffron 57a3d4e0d2
FEATURE: whitelist theme repo mode (experimental)
In some restricted setups all JS payloads need tight control.

This setting bans admins from making changes to JS on the site and
requires all themes be whitelisted to be used.

There are edge cases we still need to work through in this mode
hence this is still not supported in production and experimental.

Use an example like this to enable:

`DISCOURSE_WHITELISTED_THEME_REPOS="https://repo.com/repo.git,https://repo.com/repo2.git"`

By default this feature is not enabled and no changes are made.

One exception is that default theme id was missing a security check
this was added for correctness.
2020-06-03 13:19:57 +10:00
Guo Xiang Tan 062db10c52
FIX: `EmailValidator` needs to validate format of email. 2020-06-03 10:34:37 +08:00
Gerhard Schlager fc640f31df DEV: Skip interpolation key specs until build scripts can be fixed
Currently the build fails because it runs the specs for all plugins even though the current source of those plugins isn't pulled from git.
2020-06-02 19:51:47 +02:00
Gerhard Schlager 0cf297725f DEV: Use consistent interpolation key format in translations
From now on client strings can easily be reused on the server and you don’t have to think about choosing the right format anymore.
2020-06-02 19:05:10 +02:00
Robin Ward 4e3a84c687 FIX: If creating a flag for a watched word, include the reason 2020-06-02 11:49:02 -04:00
Penar Musaraj fa6aa7f627
FIX: Digest frequency issue on user creation
If `default email digest frequency` was set to "Never", users would get
a `digest_after_minutes` set to `nil` which triggered this error
in the logs if/when the site eventually changed that setting and
enabled digests:

```
NoMethodError (undefined method `>=' for nil:NilClass)
/var/www/discourse/app/mailers/user_notifications.rb:227:in `digest'
```
2020-06-01 17:39:16 -04:00
David Taylor 3106f85983
FIX: Support exporting reports which reference topics (#9957) 2020-06-01 18:23:58 +01:00
Guo Xiang Tan 389bdcf5ab
DEV: Fix implementation for `DiscourseRedis#exists`. 2020-06-01 13:11:32 +08:00
Jarek Radosz 00aab49829
FIX: Restore the ability to rebake posts in deleted topics (#9951)
Lost in #9852
2020-06-01 07:04:16 +02:00
Krzysztof Kotlarek 9a6ef80739
FEATURE: notify admins about old credentials (#9918)
* FEATURE: notify admins about old credentials

Security and API keys should be renewed periodically.
This additional notification should help admins keep their Discourse safe and secure.
2020-06-01 13:49:27 +10:00
Guo Xiang Tan df62407f35
DEV: Implement multiple keys support for `DiscourseRedis#exists`. 2020-06-01 11:20:26 +08:00
David Taylor ecfce93f28
FIX: Support IRIs (unicode URIs) when pulling hotlinked images (#9928) 2020-05-29 17:47:05 +01:00
David Taylor e159fb06df
FEATURE: Download remote images even for old posts (#9925)
When a post is rebaked, the admins expect it to work the same regardless of how old the post is.
2020-05-29 17:13:55 +01:00
David Taylor 28f46c171c
FIX: Pull hotlinked images even when edited by system users (#9890)
Previously the pull hotlinked images job was skipped after system edits. This ensured that we never had an infinite loop of system-edit/pull-hotlinked/system-edit/pull-hotlinked etc.

A side effect was that edits made by system for any other reason (e.g. API, removing full quotes) would prevent pulling hotlinked images. This commit removes the system edit check, and replaces it with another method to avoid an infinite job scheduling loop.
2020-05-29 13:07:47 +01:00
David Taylor d29d69e10d
FIX: Invalidate database theme cache when hostname changes (#9908)
Hostname can vary per-site on a multisite cluster, so this change requires converting the compiler_version from a constant into a class method which is evaluated at runtime. The value is stored in the theme DistributedCache, so performance impact should be negligible.
2020-05-29 13:04:51 +01:00
Sam Saffron 2acec4370b
FIX: muted tags removed topics with no tags from counts
We previously did not account for completely untagged topics when
looking at muted tags, this caused new/unread counts to be off if

1. You had muted tags
2. You had an unread/new topic
3. This topic had no tags
2020-05-29 12:59:47 +10:00
Sam Saffron 136835370c
FEATURE: optionally allow tags in topic tracking state
This feature allows certain plugins to output tag information
to topic tracking state, this allows per tag stats which can be
used by sidebars and other plugins that need per tag stats

Not enabled by default cause this would add cost to a critical
query
2020-05-29 12:59:47 +10:00
Vinoth Kannan ce1491e830
UX: remove `in:unpinned` filter from advanced search page. (#9911) 2020-05-29 00:47:28 +05:30
Andrew Schleifer 74d28a43d1
new S3 backup layout (#9830)
* DEV: new S3 backup layout

Currently, with $S3_BACKUP_BUCKET of "bucket/backups", multisite backups
end up in "bucket/backups/backups/dbname/" and single-site will be in
"bucket/backups/".

Both _should_ be in "bucket/backups/dbname/"

- remove MULTISITE_PREFIX,
- always include dbname,
- method to move to the new prefix
- job to call the method

* SPEC: add tests for `VacateLegacyPrefixBackups` onceoff job.

Co-authored-by: Vinoth Kannan <vinothkannan@vinkas.com>
2020-05-29 00:28:23 +05:30
Roman Rizzi a41476800b
FIX: Don't raise an exception if a topic cannot be retrieved (#9906) 2020-05-28 11:59:20 -03:00
David Taylor 8a3d9d7036
DEV: Run jobs sequentially in test mode (#9897)
When running jobs in tests, we use `Jobs.run_immediately!`. This means that jobs are run synchronously when they are enqueued. Jobs sometimes enqueue other jobs, which are also executed synchronously. This means that the outermost job will block until the inner jobs have finished executing. In some cases (e.g. process_post with hotlinked images) this can lead to a deadlock.

This commit changes the behavior slightly. Now we will never run jobs inside other jobs. Instead, we will queue them up and run them sequentially in the order they were enqueued. As a whole, they are still executed synchronously. Consider the example

```ruby
class Jobs::InnerJob < Jobs::Base
  def execute(args)
    puts "Running inner job"
  end
end

class Jobs::OuterJob < Jobs::Base
  def execute(args)
    puts "Starting outer job"
    Jobs.enqueue(:inner_job)
    puts "Finished outer job"
  end
end

Jobs.enqueue(:outer_job)
puts "All jobs complete"
```

The old behavior would result in:

```
Starting outer job
Running inner job
Finished outer job
All jobs complete
```

The new behavior will result in:
```
Starting outer job
Finished outer job
Running inner job
All jobs complete
```
2020-05-28 12:52:27 +01:00
Blake Erickson 6548cd1a96 FIX: sending messages to groups with non-lowercase names
Fixes a regression in

e8fb9d4066

which caused a bug where you couldn't send a message to a group that
contained an Uppercase letter. Added a test case for this.

Bug report: https://meta.discourse.org/t/-/152999
2020-05-27 14:52:08 -06:00
Jeff Wong a9d92f338e
SECURITY: make find topic by slug adhere to SiteSetting.detailed_404 (#9898) 2020-05-27 11:28:38 -07:00
Joffrey JAFFEUX 77801aa9be
FIX: allows to have custom emoji translation without static file (#9893) 2020-05-27 20:11:52 +02:00
Artem Vasiliev 12544c02c1
FIX: add X-Robots-Tag header for check_xhr-covered GET actions, too (#9868)
* FIX: add X-Robots-Tag header for check_xhr-covered GET actions, too

see https://meta.discourse.org/t/missing-x-robots-tag/152593/3 for context

* test: a spec making sure X-Robots-Tag header is present when needed

/groups path responds to anonymous requests and doesn't skip `check_xhr` method, so we can use it here.
2020-05-27 11:57:05 -04:00
Roman Rizzi 461df7d050
UX: Rename Priority to score for sorting. (#9846) 2020-05-27 12:50:28 -03:00
Blake Erickson da839e6d26 SECURITY: Use FinalDestination for topic embeds 2020-05-27 09:26:09 -06:00
Régis Hanol 2a4db15544 FIX: don't send digests to users with no primary email
It might happen that some User records have no associated primary emails.
In which case we don't ever want to send them a digest.

Also added a new "user_email_no_email" skipped email log to ensure these cases
are properly handled and surfaced.
2020-05-27 17:09:40 +02:00
Sam Saffron 5bfb6830c9
SECURITY: missing security check prior to redirect
In some rare cases, if a user knows the exact title of a topic
they could possibly determine that it really exists in the system
2020-05-27 10:58:22 +10:00
Krzysztof Kotlarek 34e5f0a9a3
Revert "FEATURE: notify admins about old credentials (#9854)" (#9886)
This reverts commit 349a67bee6.
2020-05-27 09:52:53 +10:00
Krzysztof Kotlarek 349a67bee6
FEATURE: notify admins about old credentials (#9854)
* FEATURE: notify admins about old credentials

Security and API keys should be renewed periodically.
This additional notification should help admins keep their Discourse safe and secure.
2020-05-27 08:13:47 +10:00
Penar Musaraj b1c726be0d
Remove support for FontAwesome 4.7 icon names (#9871) 2020-05-26 14:53:32 -04:00
Gerhard Schlager 69ee94b526 FIX: XML files could be detected as SVG files 2020-05-26 18:18:20 +02:00
Roman Rizzi b61a291cf3
FIX: returns false if the upload url is an invalid mailto link (#9877) 2020-05-26 10:32:48 -03:00
Bianca Nenciu f47400475e
FEATURE: Send a private message when a group membership is accepted (#9822)
* FEATURE: Send a private message when a group membership is accepted

* DEV: Small code improvements

* FIX: Send PM as group owner

* Copy edits
2020-05-26 16:28:03 +03:00
Osama Sayegh 2211581a85
FIX: Don't responde with error 500 if domain is invalid when adding automatic membership domain (#9655) 2020-05-26 15:40:09 +10:00
Arpit Jalan 5462fe9462
FIX: do not allow tag with name 'none' (#9867)
https://meta.discourse.org/t/none-tag-is-uneditable/152003
2020-05-26 08:15:45 +05:30
Sam Saffron fc97f7e0e7
FIX: properly ban non human users from draft system
Previously we had a partial fix in place where non human users
were not allowed draft sequences, this left edges around where non
human users asked for drafts yet had none.

For example system could already have a few drafts in place.

This also removes and extensibility point we added that is not in use
2020-05-26 10:07:09 +10:00
Vinoth Kannan 5fb9271878
DEV: ignore `flair_url` column in group model. (#9873) 2020-05-26 00:43:50 +05:30
Robin Ward fd2d7ca992 FIX: Email Styles were evaluated out of order
`yield` puts the content in the template right away unless explicitly
`capture`'d.
2020-05-25 12:47:23 -04:00
Arpit Jalan e8fb9d4066 FIX: when creating new PM username/groupname should be case-insensitive
(take 2)

https://meta.discourse.org/t/case-sensitivity-in-links-to-groupname/147596
https://meta.discourse.org/t/remove-case-sensitive-in-adding-users-to-a-message/151275
2020-05-25 21:34:05 +05:30
Arpit Jalan 302b37c805 Revert "FIX: when creating new PM username/groupname should be case-insensitive"
This reverts commit 2be79d94f5.

This is affecting multiple code path. Investigating.
2020-05-25 20:10:14 +05:30
Arpit Jalan 2be79d94f5 FIX: when creating new PM username/groupname should be case-insensitive
https://meta.discourse.org/t/case-sensitivity-in-links-to-groupname/147596
https://meta.discourse.org/t/remove-case-sensitive-in-adding-users-to-a-message/151275
2020-05-25 19:04:59 +05:30
Gerhard Schlager 631024ae5d FEATURE: Permalinks for tags 2020-05-25 14:51:01 +02:00
Vinoth Kannan 505122bb45 FIX: skip onceoff job for groups with invalid flair URL. 2020-05-25 13:11:00 +05:30
Sam Saffron c011b817b2
FIX: do not fallback to flair_url column
flair_url is not loaded by our query anymore so avoid falling back
to the old value
2020-05-25 16:25:42 +10:00
Vinoth Kannan 8e56197728
UX: use "icon-picker" & "image-uploader" fields to set group flair. (#9779) 2020-05-25 11:08:47 +05:30
Gerhard Schlager 13d5ccedf5 FIX: Destroying a user failed when it had title
...because updating the user caused a validation of the primary email and the UserEmail records are already deleted at that time.
2020-05-24 22:14:50 +02:00
Michael Brown d9a02d1336
Revert "Revert "Merge branch 'master' of https://github.com/discourse/discourse""
This reverts commit 20780a1eee.

* SECURITY: re-adds accidentally reverted commit:
  03d26cd6: ensure embed_url contains valid http(s) uri
* when the merge commit e62a85cf was reverted, git chose the 2660c2e2 parent to land on
  instead of the 03d26cd6 parent (which contains security fixes)
2020-05-23 00:56:13 -04:00
Jeff Atwood 20780a1eee Revert "Merge branch 'master' of https://github.com/discourse/discourse"
This reverts commit e62a85cf6f, reversing
changes made to 2660c2e21d.
2020-05-22 20:25:56 -07:00
Blake Erickson 03d26cd6f0 SECURITY: ensure embed_url contains valid http(s) uri 2020-05-22 14:54:56 -06:00
Roman Rizzi 671f882fa3
FIX: We don't create a Post object if the queued post gets rejected. We need to count review items directly. (#9856) 2020-05-22 11:50:28 -03:00
Arpit Jalan 61d8955fcb DEV: add spec for tag with unicode name 2020-05-22 14:42:22 +05:30
Martin Brennan f9d55b4941
FEATURE: Update the topic excerpt when the OP is rebaked (#9852)
* We now have a site setting "topic_excerpt_maxlength" that is used when the OP is created or revised to generate a topic excerpt.
* However, posts created before this setting was introduced cannot benefit from this change unless they are revised, and if the topic excerpt length setting is changed that situation is also not covererd.
* This PR makes a change to rebake! to update the topic excerpt IF the post is the OP.
2020-05-22 13:04:15 +10:00
Robin Ward d11c462104 SECURITY: ERB execution in custom Email Style 2020-05-21 14:48:30 -04:00
Guo Xiang Tan f4b82f1dc0
DEV: Fix randomly failing spec.
If a user is created with an id of 999, a `upload.user_id ==
user_avatar.user_id` will return true. This fix increases the id of the
upload to something that we will not hit in the foreseeable future.
2020-05-21 11:41:07 +08:00
Martin Brennan df68d11c38
FEATURE: Add topic excerpt max length site setting (#9847)
Adds a new topic_excerpt_maxlength site setting.

* When topic excerpt is requested for a post, use the new topic_excerpt_maxlength site setting to limit the size of the excerpt
* Remove code for getting/setting Post.excerpt_size as it is not used anywhere
2020-05-21 13:19:48 +10:00
Guo Xiang Tan 68db5deaec
FIX: Badge granter was disabled by default. 2020-05-21 06:55:22 +08:00
Robin Ward 096eca0ee8 FIX: Requests were not being logged correctly
`enable` was defaulting to `nil` which is not what we wanted.
2020-05-20 15:09:17 -04:00
Mark VanLandingham e1bd57007b
DEV: Specs for AboutController can_see_about_stats? impact (#9843) 2020-05-20 11:30:27 -05:00
Osama Sayegh 02f44def56
FIX: Don't blow up when trying to parse invalid or non-ASCII URLs (#9838)
* FIX: Don't blow up when trying to parseinvalid or non-ASCII URLs

Follow-up to 72f139191e
2020-05-20 12:46:27 +03:00
Bianca Nenciu fb15da43da
Remove old web hooks in favor of 'reviewable' web hook (#9776)
* FIX: Emit web hooks for flags

* FEATURE: Remove 'flag' web hook in favor of 'reviewable' web hook

* FEATURE: Remove 'queued post' web hook in favor of 'reviewable' web hook

* FIX: Do not set a default value for web hooks with no events
2020-05-20 12:07:48 +03:00
Martin Brennan 72f139191e
FIX: S3 store has_been_uploaded? was not taking into account s3 bucket path (#9810)
In some cases, between Discourse forums the hostname of a URL could match if they are hosting S3 files on the same bucket but the S3 bucket path might not. So e.g. https://testbucket.somesite.com/testpath/some/file/url.png vs https://testbucket.somesite.com/prodpath/some/file/url.png. So has_been_uploaded? was returning true for the second URL, even though it may have been uploaded on a different Discourse forum.

This is a very rare case but must be accounted for, because this impacts UrlHelper.is_local which mistakenly thinks the file has already been downloaded and thus allows the URL to be cooked, where we want to return the full URL to be downloaded using PullHotlinkedImages.
2020-05-20 10:40:38 +10:00
David Taylor e9f579bd3f
DEV: Clean up thumbnail tests correctly 2020-05-19 11:04:02 +01:00
David Taylor 725e38f9d7
DEV: Allow plugins to request topic thumbnail sizes (#9828)
In plugin.rb, you can register new sizes like

```
register_topic_thumbnail_size [512, 512]
```

For more information about thumbnails see 03818e642a
2020-05-19 10:38:58 +01:00
Guo Xiang Tan 96c02caba7
DEV: Change use of Redis `flushall` to `flushdb`.
FLUSHALL removes all keys from all databases. Instead we only want to
remove keys from the current Redis database.
2020-05-19 10:20:00 +08:00
Krzysztof Kotlarek f99f6ca111
FIX: randomize file name when created from fixtures (#9731)
* FIX: randomize file name when created from fixtures

When a temporary file is created from fixtures it should have a unique name.
It is to prevent a collision in parallel specs evaluation

* FIX: use /tmp/pid folder to keep fixture files
2020-05-19 09:09:36 +10:00
Gerhard Schlager 6d5e9db883 FIX: Restoring backup didn't clear cached translation overrides 2020-05-18 18:51:51 +02:00
Gerhard Schlager 27c611b1d1 FIX: Reverting multiple translation overrides didn't clear cache 2020-05-18 18:51:29 +02:00
David Taylor 96848b7649
UX: Allow secure media URLs to be cached for a short period of time
Signed S3 URLs are valid for 15 seconds, so we can safely allow the browser to cache them for 10 seconds. This should help with large numbers of requests when composing a post with many images.
2020-05-18 15:00:41 +01:00
Joffrey JAFFEUX 918bd75909
FIX: allows to remove a topic timer with delete_replies as type (#9800) 2020-05-18 11:47:08 +02:00
Guo Xiang Tan d01c336899
DEV: Clean up some Redis leaks in test env. 2020-05-18 17:27:37 +08:00
Sam Saffron 862773ec83
FIX: do not remove stop words when using English locale
PG already handles English stop words, the list in cppjieba is
bigger than the list PG uses, which in turn causes confusion cause
words such as "volume" are stripped using cppijieba stop word list

We will follow up with another commit here to apply the Chinese
word stopwords, but for now to eliminate the confusion we are
skipping applying the stopword list when the dictionary in PG is
in English.
2020-05-18 10:54:56 +10:00
Krzysztof Kotlarek ea63fa7de7
FIX: extract reset_last_seen_cache! in user_spec (#9808)
Helper method to avoid code duplication
2020-05-18 09:13:37 +10:00
David Taylor 461b4e5cc6
DEV: Add framework for filtered plugin registers (#9763)
* DEV: Add framework for filtered plugin registers

Plugins often need to add values to a list, and we need to filter those lists at runtime to ignore values from disabled plugins. This commit provides a re-usable way to do that, which should make it easier to add new registers in future, and also reduce repeated code.

Follow-up commits will migrate existing registers to use this new system

* DEV: Migrate user and group custom field APIs to plugin registry

This gives us a consistent system for checking plugin enabled state, so we are repeating less logic. API changes are backwards compatible
2020-05-15 14:04:38 +01:00
David Taylor 0495a748d0
FIX: Use CDN URLs for topic thumbnails 2020-05-15 13:35:20 +01:00
Guo Xiang Tan 858735b5da
DEV: Fix broken test due to 9ab5801a1b. 2020-05-15 15:00:04 +08:00
Guo Xiang Tan 9ab5801a1b
FIX: Allow post migrations using `#change` to carry out unsafe migration 2020-05-15 14:23:27 +08:00
Guo Xiang Tan 165fa13c9d
FIX: Always return 0 for non-human user current DraftSequence. 2020-05-15 10:40:36 +08:00
Blake Erickson 1a2b9435b0
DEV: Standardize table sorting verbiage (#9757)
* DEV: Standardize table sorting verbiage

This commit creates a common component that tables can use to make their
headers sortable. This commit also standardizes on using `desc` as the
default and passing in the `asc=true` flag to adjust the sorting
direction.

* Add deprecation warnings

Adds deprecation warnings if using previous params and maintains
backwards compatibility. Set the default sort value for group members to
be asc.

* switch group requests to use common table-header-toggle

* update fixture
2020-05-14 20:10:59 -06:00
Krzysztof Kotlarek 65a3fdbc57
FIX: randomly failing user_spec (#9754)
* FIX: randomly falling user_spec

When we evaluate `update_last_seen!` we relay on Redis to not run that code too often

https://github.com/discourse/discourse/blob/master/app/models/user.rb#L753

The problem is that not all specs which are running `update_last_seen!` are not cleaning after themselves

For examples specs in that block https://github.com/discourse/discourse/blob/master/spec/models/user_spec.rb#L901

So it can be replicated when you run a few times
`bundle exec rspec  ./spec/models/user_spec.rb -e "should not update the first seen value if it doesn't exist" -e "should have 0 for days_visited"`

We should delete Redis key after each spec which is evaluating `update_last_seen!`
2020-05-15 09:24:07 +10:00
Kane York 869f9b20a2
PERF: Dematerialize topic_reply_count (#9769)
* PERF: Dematerialize topic_reply_count

It's only ever used for trust level promotions that run daily, or compared to 0. We don't need to track it on every post creation.

* UX: Add symbol in TL3 report if topic reply count is capped

* DEV: Drop user_stats.topic_reply_count column
2020-05-14 15:42:00 -07:00
Penar Musaraj 5ff2a235f6 DEV: Allow 3-digit HEX color code in single icon route
Followup to aee8e62
2020-05-14 16:37:45 -04:00
Penar Musaraj aee8e62e21
FEATURE: Add endpoint for individual SVG icons (#9765) 2020-05-14 14:17:19 -04:00
Mark VanLandingham a047004c9a
FIX: Specific email error for replies to digest emails (#9770) 2020-05-14 09:04:58 -05:00
Dan Ungureanu 3ed6a0e904
FIX: Detect Wayback Machine using user agent (#9777) 2020-05-14 21:10:07 +10:00
David Taylor d4717f5d1e
DEV: Reduce repeated code in discourse_plugin_registry (#9705)
Use a helper method to simplify creating a new register. Previously this would require creating lots of different methods manually, and adding every register to the clear/reset functions
2020-05-13 12:25:34 +01:00
Sam Saffron 7f841dc21f
PERF: avoid race conditions when creating topic links
Previously the code was very race condition prone leading to
odd failures in production

It was re-written in raw SQL to avoid conditions where rows
conflict on inserts

There is no clean way in ActiveRecord to do:

Insert, on conflict do nothing and return existing id.

This also increases test coverage, we were previously not testing
the code responsible for crawling external sites directly
2020-05-13 16:05:57 +10:00
Martin Brennan a64cf265fd
FIX: Change /bookmarks URL back to topic list and add bookmark poster avatars (#9759)
* Changes the /bookmarks URL to show the original "topics filtered by bookmark" list instead of redirecting to user activity bookmarks (see https://meta.discourse.org/t/domain-com-bookmarks-is-showing-domain-com-u-user-activity-bookmarks-with-reminders/149252/12)
* Add the user avatar for the user who made the post that is bookmarked
2020-05-13 14:03:24 +10:00
Blake Erickson 4078b22887 FIX: Handle missing provider return sso url
This commit prevents a 500 error from occurring if someone is trying to
setup their discourse instance as a sso provider and they don't pass in
a `return_sso_url` in their payload.
2020-05-12 18:16:50 -06:00
Martin Brennan 9981fa4466
FIX: Prevent column name conflicts in reviewable code (#9753)
We were getting errors like this in Reviewables in some cases:

```
ActiveRecord::StatementInvalid (PG::AmbiguousColumn: ERROR:  column reference "category_id" is ambiguous
LINE 4: ...TRUE) OR (reviewable_by_group_id IN (NULL))) AND (category_i...
```

The problem that was making everything go boom is that plugins can add their own custom filters for Reviewables. If one is doing an INNER JOIN on topics, which has its own category_id column, we would get the above AmbiguousColumn error. The solution here is to just make all references to the reviewable columns in the list_for and viewable_by code prefixed by the table name e.g. reviewables.category_id.
2020-05-13 09:05:56 +10:00
Robin Ward 033644ea82 DEV: Adds an integrity spec for JS constants 2020-05-12 12:23:36 -04:00
Rafael dos Santos Silva 2298e14d38
FEATURE: Support for App Shortcuts Menu (#9749)
* FEATURE: Support for App Shortcuts Menu

This adds a list of shortcuts to a installed Discourse instance.

It can be accessed by right clicks or long press on the app icon.

See https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/master/Shortcuts/explainer.md

List of possible follow ups include:

- Making it admin customizable
- Making it user customizable
- Using SVG icons from the site icon sprite
- Picking an accent color for icons

* FIX: Add type to shortcut menu icons
2020-05-12 12:24:33 -03:00
Joshua Rosenfeld c94e6a9a66
FEATURE: add noindex header to tags pages (#9748) 2020-05-12 10:44:46 -04:00
David Taylor 6230f5c554
FEATURE: Allow parameter authentication for UserApiKeys (#9742)
This refactors default_current_user_provider in a few ways:
- Introduce a generic `api_parameter_allowed?` method which checks for whitelisted routes/formats
- Only read the api_key parameter on allowed routes. It is now completely ignored on other routes (previously it would raise a 403)
- Start reading user_api_key parameter on allowed routes
- Refactor tests as end-end integration tests

A plugin API for PARAMETER_API_PATTERNS will be added soon
2020-05-12 13:35:36 +01:00
Sam Saffron a29ae17d3a
FIX: saving drafts unconditionally increases sequence
Previously we only changed sequence on ownership change, this
cause a race condition between tabs where user could type for a
long time without being warned of an out of date draft.

This change is a radical change and we should watch closely.

Code was already in place to track sequence on the client so no
changes are needed there.
2020-05-12 16:55:42 +10:00
Guo Xiang Tan d301af39bd
PERF: Avoid hitting DB when fetching draft sequence of bot user. 2020-05-12 13:56:58 +08:00
Guo Xiang Tan 3c423faafd
DEV: Avoid repeating draft key logic in multiple places. 2020-05-12 11:14:03 +08:00
Krzysztof Kotlarek 4c8bece104
FEATURE: default canonical URL (#9738)
For pages that do not specify canonical URL we will default to `https://SITENAME/PATH`. 

This ensures that if a URL is crawled on the CDN the search ranking will transfer to the main site.

Additionally we whitelist the `?page` param
2020-05-12 09:13:20 +10:00
Blake Erickson ab919332dc
DEV: api documentation updates (#9612)
* DEV: api documentation updates

- Created a script to convert json responses to rswag
- Documented several api endpoints
- Switched rswag to use header based auth

* Update script, fix some schema missmatches
2020-05-11 13:06:49 -06:00
David Taylor 5fc51ed49c
DEV: Remove unused DiscoursePlugin class (#9715) 2020-05-11 15:46:54 +01:00
Joshua Rosenfeld 07b728c5e5
FEATURE: add noindex header to badges, groups, and /my pages (#9736) 2020-05-11 15:05:42 +10:00
Sam Saffron bb4e8899c4
FEATURE: let Google index pages so it can remove them
Google insists on indexing pages so it can figure out if they
can be removed from the index.

see: https://support.google.com/webmasters/answer/6332384?hl=en

This change ensures the we have special behavior for Googlebot
where we allow indexing, but block the actual indexing via
X-Robots-Tag
2020-05-11 12:15:18 +10:00
Sam Saffron 676be3a853
FEATURE: add no_index header if robots is disabled
Expand SiteSetting.allow_index_in_robots_txt so it also adds a
noindex header if set to false.

This makes sure that nothing is indexed even if it somehow reaches
Google.
2020-05-11 11:07:21 +10:00
Bianca Nenciu 8149bfbaf1
FEATURE: Filter settings by plugin (#9692) 2020-05-10 14:07:45 +03:00
Sam Saffron d8d54a92f1
FEATURE: tighten rate limiting rules for forgot password
1. Total 6 attempts per day per user
2. Total of 5 per unique email/login that is not found per hour
3. If an admin blocks an IP that IP can not request a reset
2020-05-08 13:30:51 +10:00
Martin Brennan 22dffe6f69 FIX: Do not allow null options for bookmark manager 2020-05-08 15:24:59 +00:00
Robin Ward f9608c0af5 DEV: Remove INLINE_ONEBOX_* constants
There were two constants here, `INLINE_ONEBOX_LOADING_CSS_CLASS` and
`INLINE_ONEBOX_CSS_CLASS` that were both longer than the strings they
were DRYing up: `inline-onebox-loading` and `inline-onebox`

I normally appreciate constants, but in this case it meant that we had
a lot of JS imports resulting in many more lines of code (and CPU cycles
spent figuring them out.)

It also meant we had an `.erb` file and had to invoke Ruby to create the
JS file, which meant the app was harder to port to Ember CLI.

I removed the constants. It's less DRY but faster and simpler, and
arguably the loss of DRYness is not significant as you can still search
for the `inline-onebox-loading` and `inline-onebox` strings easily if
you are refactoring.
2020-05-07 16:14:38 -04:00
Vinoth Kannan 744bbf6904 FEATURE: exclude muted categories from the "top" topics list. 2020-05-08 00:34:53 +05:30
Jarek Radosz 781e3f5e10
DEV: Use `response.parsed_body` in specs (#9615)
Most of it was autofixed with rubocop-discourse 2.1.1.
2020-05-07 17:04:12 +02:00
Martin Brennan 37be2958f7
DEV: Allow specifying if a notification is high_priority on create (#9660)
This allows for special cases where we may not want a certain notification type to ALWAYS be high priority (e.g. a topic timer).
2020-05-07 14:35:32 +10:00
Martin Brennan 6fb0f36ce1
FEATURE: Optionally delete bookmark when reminder sent (#9637)
We now show an options gear icon next to the bookmark name.

When expanded we show the "delete bookmark when reminder sent" option. The value of this checkbox is saved in local storage for the user.

If this is ticked, when a reminder is sent for the bookmark the bookmark itself is deleted. This is so people can use the reminder functionality by itself.

Also remove the blue alert reminder section from the "Edit Bookmark" modal as it just added clutter, because the user can already see they had a reminder set:

Adds a default false boolean column `delete_when_reminder_sent` to bookmarks.
2020-05-07 13:37:39 +10:00
tshenry 0d394e6b76
FEATURE: Include category position when exporting categories (#9658) 2020-05-07 12:17:15 +10:00
Krzysztof Kotlarek 05799538e6
FIX: correct edit notification username for PMs (#9649)
When the tag is added to PM by admin, notification has an incorrect username (post author username instead of admin username)
2020-05-07 07:52:21 +10:00
Gerhard Schlager ec2f3169ff FIX: Using the `default_locale` in locale fallbacks caused problems
Locale files get precompiled after deployment and they contained translations from the `default_locale`. That's especially bad in multisites, because the initial `default_locale` is `en_US`. Sites where the `default_locale` isn't `en_US` could see missing translations. The same thing could happen when users are allowed to chose a different locale.

This change simplifies the logic by not using the `default_locale` in the locale chain. It always falls back to `en` in case of missing translations.
2020-05-06 22:59:07 +02:00
Roman Rizzi 70bf1669be
DEV: Use a copy of the fixture file instead of the original one (#9645) 2020-05-06 11:54:08 -03:00
Sam Saffron f8c18ac888
DEV: remove fragile spec
The failover spec is very fragile and tests specific implementation
vs actual behavior

We rely on a different script during the build process to test
failover operates correctly
2020-05-06 16:21:48 +10:00
Martin Brennan fa572d3a7a
DEV: Remove all code referencing at_desktop bookmark reminders (#9650)
We have found no need for these reminder types, so we are removing the code for them.
2020-05-06 15:22:43 +10:00
Robin Ward 612284cef3
DEV: Remove `Discourse.RAW_TEMPLATES` (#9630)
We were sharing `Discourse` both as an application object and a
namespace which complicated things for Ember CLI. This patch
moves raw templates into `__DISCOURSE_RAW_TEMPLATES` and adds
a couple helper methods to create/remove them.
2020-05-05 12:15:03 -04:00
David Taylor 03818e642a
FEATURE: Include optimized thumbnails for topics (#9215)
This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this:

- Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact

- A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake

- Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued

- Topic lists and topics now include a `thumbnails` key, which includes all the available images:
   ```
   "thumbnails": [
   {
     "max_width": null,
     "max_height": null,
     "url": "//example.com/original-image.png",
     "width": 1380,
     "height": 1840
   },
   {
     "max_width": 1024,
     "max_height": 1024,
     "url": "//example.com/optimized-image.png",
     "width": 768,
     "height": 1024
   }
   ]
  ```

- Themes can request additional thumbnail sizes by using a modifier in their `about.json` file:
   ```
    "modifiers": {
      "topic_thumbnail_sizes": [
        [200, 200],
        [800, 800]
      ],
      ...
  ```
  Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated

- Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 09:07:50 +01:00
Krzysztof Kotlarek 9bff0882c3
FEATURE: Nokogumbo (#9577)
* FEATURE: Nokogumbo

Use Nokogumbo HTML parser.
2020-05-05 13:46:57 +10:00
David Taylor 5901717531
DEV: Allow DB.after_commit to be used outside of a transaction
In this case, it will execute the given block immediately
2020-05-04 09:42:41 +01:00
Sam Saffron 3877ef2cfa
FEATURE: Unconditionally notify on edits at least once a day
This ensures that at a minimum you are notified once a day of
repeat edits by the same user.

Long term we may consider winding this down to say 1 hour or
making it configurable.
2020-05-04 17:55:00 +10:00
Sam Saffron 3d0ccf8642
FIX: missing edit notifications in some rare cases
Due to a refactor in e90f9e5cc4 we stopped notifying on edits if
a user liked a post and then edited.

The like could have happened a long time ago so this gets extra
confusing.

This change makes the suppression more deliberate. We only want
to suppress quite/link/mention if the user already got a reply
notification.

We can expand this suppression if it is not enough.
2020-05-04 17:40:56 +10:00
Kane York 784bf2a173
FIX: Unassign user titles when a badge is deleted (#9573) 2020-05-02 18:02:28 -07:00
Martin Brennan bcc9ad6f57
FIX: Bookmark UI tweaks (#9604)
* When hovering over the bookmark icon for a post, show the name of the bookmark at the end of the tooltip _if_ it has been set.
* Order bookmarks by `updated_at DESC` in the user list and show that instead of created at.
2020-05-01 16:14:20 +10:00
Martin Brennan 867bc3b48e
FIX: Change base importer to create new Bookmark records (#9603)
Also add a spec and fixture with a mock importer that we can use to test the create_X methods of the base importer
2020-05-01 11:34:55 +10:00
Krzysztof Kotlarek 37e93914fc
FIX: the muted message should be sent after edit (#9593)
Recently, we added feature that we are sending `/muted` to users who muted specific topic just before `/latest` so the client knows to ignore those messages - https://github.com/discourse/discourse/pull/9482

Same `/muted` message should be included when the post is edited
2020-05-01 08:33:57 +10:00
Vinoth Kannan 71241a50f7 DEV: improve code readability & add tests for user guardian.
a511bea4cc
2020-04-30 20:59:33 +05:30
Régis Hanol 501b19b6e0
FIX: server-side HtmlToMarkdown improvements (#9586)
TLDR; this commit vastly improves how whitespaces are handled when converting from HTML to Markdown.
It also adds support for converting HTML <tables> to markdown tables.

The previous 'remove_whitespaces!' method was traversing the whole HTML tree and used a heuristic to remove
leading and trailing whitespaces whenever it was appropriate (ie. mostly before and after HTML block elements)

It was a good idea, but it was very limited and leaded to bad conversion when the html had leading whitespaces on several lines for example.
One such example can be found [here](https://meta.discourse.org/t/86782).

For various reasons, most of the whitespaces in a HTML file is ignored when the page is being displayed in a browser.
The rules that the browsers follow are the [CSS' White Space Processing Rules](https://www.w3.org/TR/css-text-3/#white-space-rules).
They can be quite complicated when you take into account RTL languages and other various tidbits but they boils down to the following:

- Collapse whitespaces down to one space (0x20) inside an inline context (ie. nodes/tags that are being displaying on the same line)
- Remove any leading/trailing whitespaces inside an inline context

One quick & dirty way of getting this 90% solved would be to do 'HTML.gsub!(/[[:space:]]+/, " ")'.
We would also need to hoist <pre> elements in order to not mess with their whitespaces.
Unfortunately, this solution let some whitespaces creep around HTML tags which leads to more '.strip!' calls than I can bear.

I decided to "emulate" the browser's handling of whitespaces and came up with a solution in 4 parts

1. remove_not_allowed!

The HtmlToMarkdown library is recursively "visiting" all the nodes in the HTML in order to convert them to Markdown.
All the nodes that aren't handled by the library (eg. <script>, <style> or any non-textual HTML tags) are "swallowed".
In order to reduce the number of nodes visited, the method 'remove_not_allowed!' will automatically delete all the nodes
that have no "visitor" (eg. a 'visit_<tag>' method) defined.

2. remove_hidden!

Similar purpose as the previous method (eg. reducing number of nodes visited), there's no point trying to convert something that is hidden.
The 'remove_hidden!' method removes any nodes that was hidden using the "hidden" HTML attribute, some CSS or with a width or height equal to 0.

3. hoist_line_breaks!

The 'hoist_line_breaks!' method is there to handle <br> tags. I know those tiny <br> don't do much but they can be quite annoying.
The <br> tags are inline elements but they visually work like a block element (ie. they create a new line).
If you have the following HTML "<i>Foo<br>Bar</i>", it ends up visually similar to "<i>Foo</i><br><i>Bar</i>".
The latter being much more easy to process than the former, so that's what this method is doing.
The "hoist_line_breaks" will hoist <br> tags out of inline tags until their parent is a block element.

4. remove_whitespaces!

The "remove_whitespaces!" is where all the whitespace removal is happening. It's broken down into 4 methods as well

- remove_whitespaces!
- is_inline?
- collapse_spaces!
- remove_trailing_space!

The 'remove_whitespace!' method is recursively walking the HTML tree (skipping <pre> tags).
If a node has any children, they will be chunked into groups of inline elements vs block elements.
For each chunks of inline elements, it will call the "collapse_space!" and "remove_trailing_space!" methods.
For each chunks of block elements, it will call "remote_whitespace!" to keep walking the HTML tree recursively.

The "is_inline?" method determines whether a node is part of a inline context.
A node is inline iif it's a text node or it's an inline tag, but not <br>, and all its children are also inline.

The "collapse_spaces!" method will collapse any kind of (white) space into a single space (" ") character, even accros tags.
For example, if we have "  Foo \n<i> Bar </i>\t42", it will return "Foo <i>Bar </i>42".

Finally, the "remove_trailing_space!" method is there to remove any trailing space that might creep in at the end of the inline chunk.

This solution is not 100% bullet-proof.
It does not support RTL languages at all and has some caveats that I felt were not worth the work to get properly fixed.

FIX: better detection of hidden elements when converting HTML to Markdown
FIX: take into account the 'allowed_href_schemes' site setting when converting HTML <a> to Markdown
FIX: added support for 'mailto:' scheme when converting <a> from HTML to Markdown
FIX: added support for <img> dimensions when converting from HTML to Markdown
FIX: added support for <dl>, <dd> and <dt> when converting from HTML to Markdown
FIX: added support for multilines emphases, strongs and strikes when converting from HTML to Markdown
FIX: added support for <acronym> when converting from HTML to Markdown
DEV: remove unused 'sanitize' gem

Wow, did you just read all that?! Congratz, here's a cookie: 🍪.
2020-04-30 12:21:25 +02:00
Sam Saffron d0d5a138c3
DEV: stop freezing frozen strings
We have the `# frozen_string_literal: true` comment on all our
files. This means all string literals are frozen. There is no need
to call #freeze on any literals.

For files with `# frozen_string_literal: true`

```
puts %w{a b}[0].frozen?
=> true

puts "hi".frozen?
=> true

puts "a #{1} b".frozen?
=> true

puts ("a " + "b").frozen?
=> false

puts (-("a " + "b")).frozen?
=> true
```

For more details see: https://samsaffron.com/archive/2018/02/16/reducing-string-duplication-in-ruby
2020-04-30 16:48:53 +10:00
David Taylor 367cbf5d2b
FEATURE: Allow user creation with admin api when local logins disabled (#9587) 2020-04-30 11:39:24 +10:00
Martin Brennan ca539fdccf
FIX: Rename all instances of bookmarkWithReminder to just bookmark (#9579)
* Rename all instances of bookmarkWithReminder and bookmark_with_reminder to just bookmark
* Delete old bookmark code at the same time
* Add migration to remove the bookmarkWithReminder post menu item if people have it set in site settings
2020-04-30 10:09:22 +10:00
Dan Ungureanu 0a85a7aef9
FEATURE: Add user_profile to user_archive CSV export (#9571) 2020-04-29 12:09:50 +03:00
Neil Lalonde 9d781d4959
FIX: error customizing text for badges from plugins 2020-04-28 14:34:41 -04:00
Joffrey JAFFEUX a6f986b50f
FEATURE: allows to to style published page with themes/plugins (#9570) 2020-04-28 18:24:24 +02:00
Penar Musaraj b19dcac272
FIX: Include default notification level in category serializer (#9572)
Fixes an issue where the notification level state goes missing when user edits a category in the UI.
2020-04-28 12:05:53 -04:00
David Taylor ba616ffb50
DEV: Use a tmp directory for storing uploads in tests (#9554)
This avoids development-mode upload files from polluting the test environment
2020-04-28 14:03:04 +01:00
Martin Brennan 5108cf8ddf
FIX: Topic user bookmarked column logic was not correct (#9563)
Make sure the topic_user.bookmarked column is set correctly when user bookmarks/unbookmarks any post in a topic. For example, you bookmarked a post in the topic that was not the OP, the bookmark icon in the topic list would not be shown. Same if deleting a bookmark for the last bookmarked post in a topic, the bookmark icon in the topic list would not be removed.

Previously this was only setting it to true if bookmarking the OP/topic, which was not correct -- we want to show the icon on the topic list if any post is bookmarked.
Also set to false if unbookmarking the last bookmarked post in the topic.
Also in this PR is a migration to correct any out of sync topic_user.bookmarked columns, based on the new logic.
2020-04-28 16:19:25 +10:00
mentalstring 67f3fe14aa
FEATURE: support SSO website and location overrides
Add location and website + the ability to override using SSO using the `sso_overrides_location` and `sso_overrides_website` site settings.
2020-04-28 16:06:35 +10:00
Blake Erickson a93ef2926d
DEV: Add rswag to aid in api documention (#9546)
Adding in rswag will allow us to write spec files to document and test
our api.
2020-04-27 16:40:07 -06:00
Joffrey JAFFEUX 01f2819dde
DEV: enables and fixes multisite-spec (#9557) 2020-04-27 20:55:36 +02:00
Roman Rizzi 394babcae3
FIX: Only show the review page to users that can see it. Do not publish the reviewable count update message to everyone. (#9556) 2020-04-27 14:51:25 -03:00
David Taylor f95609ae23
FIX: Add additional checks for automatic theme script CSP
- Skip absolute URLs
- Skip CDN URLs
- Skip invalid URLs
- Properly format protocol-less URLs
2020-04-27 15:56:29 +01:00
Benno 6e01acb3cb
FIX: Apply category priority for empty query (#9516) 2020-04-27 10:35:27 -04:00
Sam Saffron 8c1e008c59
DEV: Skip erratic spec for now
Spec fails intermittently due to CDN state.
2020-04-25 13:20:04 +10:00
David Taylor 8a112b7464
DEV: Automatically extend CSP when themes link to external scripts (#9531) 2020-04-24 09:47:01 +01:00
Sam Saffron cbceadf48b
FEATURE: when blocking emails prefer blocking canonical
Previously we relied entirely on levenshtein_distance_spammer_emails site
setting to handle "similar looking" emails.

This commit improves the situation by always preferring to block (and check)
canonical emails.

This means that if:

`samevil+test@domain.com` is blocked the system will block `samevil@domain.com`

This means that `samevil+2@domain.com` (ad infinitum) will be blocked
2020-04-24 14:09:51 +10:00
Sam Saffron 6a18c9aa0b
Revert "FEATURE: enforce_canonical_emails site setting"
This reverts commit 6f9177e2ed.

We decided on a completely different approach to the problem.

Instead we will let blocked emails be treated as canonical.
2020-04-24 13:52:06 +10:00
Martin Brennan cd1c7d7560
FIX: Copying image markdown for secure media loading full image (#9488)
* When copying the markdown for an image between posts, we were not adding the srcset and data-small-image attributes which are done by calling optimize_image! in cooked post processor
* Refactored the code which was confusing in its current state (the consider_for_reuse method was super confusing) and fixed the issue
2020-04-24 10:29:02 +10:00
Penar Musaraj 585e7bcfe8 DEV: update specs followup to 67e96f6 2020-04-23 16:11:17 -04:00
Arpit Jalan 39be639c37 FIX: update GitHub screen_name on login via GitHub 2020-04-23 20:54:26 +05:30
Krzysztof Kotlarek 52c1d7337e
FEATURE: don't display new/unread notification for muted topics (#9482)
* FEATURE: don't display new/unread notification for muted topics

Currently, even if user mute topic, when a new reply to that topic arrives, the user will get "See 1 new or updated topic" message. After clicking on that link, nothing is visible (because the topic is muted)

To solve that problem, we will send background message to all users who recently muted that topic that update is coming and they can ignore the next message about that topic.
2020-04-23 14:57:35 +10:00
Blake Erickson 9cbbaf4237 FIX: Don't throw 500 for invalid website url input
It's possible to cause a 500 error by putting in weird characters in the
input field for updating a users website on their profile.

Normal invalid input like not including the domain extension is already
handled by the user_profile model validation. This fix ensures a server
error doesn't occur for weird input characters.
2020-04-22 13:57:56 -06:00
Jarek Radosz 8adccaf98c
DEV: Use dynamic/static fabricator attrs correctly (#9519) 2020-04-22 20:49:53 +02:00
Vinoth Kannan df0c386f8a
UX: drop the `automatic_membership_retroactive` column from groups model. (#9430) 2020-04-22 22:07:39 +05:30
Krzysztof Kotlarek 81c216e04c
FIX: composer problem on the categories page (#9508) 2020-04-22 09:24:54 -07:00
Robin Ward 13f2723dcb FIX: Don't log an error to logster if a topic could not be updated.
If for some reason an update did not go through (for example,
concurrently updating the same topic twice), we were logging something
like:

```
create_errors_json called with unrecognized type: #<Topic
```

This happened because we knew an error occurred but the active record
object had no errors attached.

This patch fixes the issue by attaching a proper error message in the
event that this happens.
2020-04-22 11:56:04 -04:00
Dan Ungureanu 4e5f9d4cd1
DEV: Drop 'key' column from user_api_keys (#9388) 2020-04-22 12:13:19 +03:00
Dan Ungureanu e733701887
FEATURE: Make report filters reusable (#9444)
This commit also adds 'include subcategories' report filter
2020-04-22 11:52:50 +03:00
Vinoth Kannan a511bea4cc
FEATURE: admin UI to merge two users. (#9509) 2020-04-22 14:07:51 +05:30
Martin Brennan 628ba9d1e2
FEATURE: Promote bookmarks with reminders to core functionality (#9369)
The main thrust of this PR is to take all the conditional checks based on the `enable_bookmarks_with_reminders` away and only keep the code from the `true` path, making bookmarks with reminders the core bookmarks feature. There is also a migration to create `Bookmark` records out of `PostAction` bookmarks for a site.

### Summary

* Remove logic based on whether enable_bookmarks_with_reminders is true. This site setting is now obsolete, the old bookmark functionality is being removed. Retain the setting and set the value to `true` in a migration.
* Use the code from the rake task to create a database migration that creates bookmarks from post actions.
* Change the bookmark report to read from the new table.
* Get rid of old endpoints for bookmarks
* Link to the new bookmarks list from the user summary page
2020-04-22 13:44:19 +10:00
Martin Brennan e18aeb799e
FIX: Make sure reminder_type is parsed on bookmark update (#9503)
Otherwise we are trying to update the reminder type with a string which often evaluates to 0 (At Desktop) which causes reminders to come through early.
2020-04-22 10:44:04 +10:00
Jarek Radosz c08753dc34
DEV: Fix topic_timer_spec (#9483)
File it under "how did it even work".
2020-04-21 14:38:35 +10:00
Jarek Radosz a781ef7662
FIX: Reject invalid Category slugs (#9473)
Previously it would sanitize given slug and then save the resulting empty slug.
2020-04-21 03:50:50 +02:00
Jarek Radosz 17cf300b71
DEV: Use more specific error responses (#9472)
* DEV: Use `render_json_error` (Adds specs for Admin::GroupsController)
* DEV: Use a specific error on blank category slug (Fixes a `render_json_error` warning)
* DEV: Use a specific error on reviewable claim conflict (Fixes a `render_json_error` warning)
* DEV: Use specific errors in Admin::UsersController (Fixes `render_json_error` warnings)
* FIX: PublishedPages error responses
* FIX: TopicsController error responses (There was an issue of two separate `Topic` instances for the same record. This makes sure there's only one up-to-date instance.)
2020-04-21 03:50:20 +02:00
Jarek Radosz ab52bed014
DEV: Remove the return value of disable_if_low_on_disk_space (#9469)
It was used only in specs.
2020-04-21 03:48:33 +02:00
Jarek Radosz 5a81e3999c
DEV: Remove `bypass_bump` from CookedPostProcessor (#9468)
It was only passing it along to `PullHotlinkedImages` and that class have not used that arg since April 2014 (c52ee665b4)
2020-04-21 03:48:19 +02:00
Jarek Radosz 9a6e4b1fa1
FIX: Make InlineUploads handle more URL formats (#9467)
It previously failed to match URLs with characters other than `[a-zA-z0-9\.\/:-]`. This meant that `PullHotlinkedImages` would sometimes download an external image and then never use it in any posts.
2020-04-21 03:47:48 +02:00
Mark VanLandingham 06796bec8b
FIX: Set category description to first posts cooked value (#9478) 2020-04-20 14:51:49 -05:00
Robin Ward 25bed4f643 FIX: Concurrency issues with making topic embedded posts visible 2020-04-20 15:11:59 -04:00
Robin Ward 56a23c68f1 FIX: Embedded topics couldn't update their titles 2020-04-20 14:27:43 -04:00
Robin Ward 8f5314bf98 FIX: An `opts` hash was not, in fact, optional :) 2020-04-20 14:17:13 -04:00
Daniel Waterworth 7876ee2d67 DEV: upgrade Rails
Latest version of Rails contains compatibility fixes for Ruby 2.7 and some
minor security fixes we would like to have

It also broke some of the multisite tests.

Rails tries to use the same connection for reading from a replica as writing
to the leader during tests, because, with everything happening in a
transaction, changes to the DB wouldn't otherwise be reflected in the
replica connection.

The difference now is that Rails tries to do this for connections opened
after the test has started which affected rails multisite connections.

The upshot of this is that, as things stand, you are likely to
experience problems if you try to connect to a different multisite DB in
a test when the `current_db` is not 'default'.
2020-04-20 12:55:53 +01:00
Sam Saffron ee36382640
FEATURE: improve rendering of RSS feeds
- Eliminate superfluous "author wrote" block
- Eliminate block-quote for all posts
- Move participant count and reply count to 1 line
- Prioritize name over username if forum requests
- Use fabrication in list controller spec to speed up spec
2020-04-20 16:08:24 +10:00
Martin Brennan 344ef5226c
FEATURE: Edit bookmark reminders from post and explicit delete button (#9455)
There is now an explicit "Delete Bookmark" button in the edit modal. A confirmation is shown before deleting.

Along with this, when the bookmarked post icon is clicked the modal is now shown instead of just deleting the bookmark. Also, the "Delete Bookmark" button from the user bookmark list now confirms the action.

Add a `d d` shortcut in the modal to delete the bookmark.
2020-04-20 13:30:04 +10:00
tshenry e41f1e6ec2
FEATURE: Rake task to export groups (#9450) 2020-04-17 14:59:54 -07:00
Mark VanLandingham 0e74dd7d7c
FIX: Set user timezone on password reset login (#9461) 2020-04-17 13:39:04 -05:00
Roman Rizzi dce46086f4
FIX: Do not raise an error if the post action type is nil (#9458) 2020-04-17 14:23:33 -03:00
David Taylor 0e4497b6be
DEV: Update tl3 spec to remove hard-coded primary keys
In some test runs, the hard-coded ids could correspond to real topics created earlier in the tests, and cause random test failures
2020-04-17 17:24:14 +01:00
Martin Brennan 8f0544137a
FEATURE: Allow editing bookmark reminders (#9437)
Users can now edit the bookmark name and reminder time from their list of bookmarks.

We use "Custom" for the date and time in the modal because if the user set a reminder for "tomorrow" then edit the reminder "tomorrow", the definition of what "tomorrow" is has changed.
2020-04-17 11:08:07 +10:00
Krzysztof Kotlarek e8e764c064
FIX: flaky groups_controller_spec (#9439)
Sometimes spec which is testing order groups by user count is failing.

My theory is that cause is the randomness of Postgres when the order value is the same for 2 rows.

In spec, we got three groups
`moderator_group` - 0 users
`group` - 1 user
`other_group` - 1 user

And we are expecting that controller will return them in ascending order [moderator, group, other_group]

Because `group` and `other_group` contain the same amount of users, we are dealing with luck

Therefore, I believe that adding one more user to other_group should make that query reliable.

It was not crashing on my local machine, so I am not 100% sure.
2020-04-17 09:04:10 +10:00
David Taylor b6c19cba20
FIX: Abort emit_web_hook_event job cleanly if web hook was deleted (#9445)
Raising an error causes the job to be retried, and causes a lot of noise in the logs
2020-04-16 21:24:09 +01:00
Neil Lalonde 074509fd95
FIX: don't demote users to TL2 when default trust level is 3
Within 24 hours of signing up, new users were losing their
default trust level of 3. With this fix, demotions from
trust level 3 won't happen when the "default trust level"
setting is 3 or 4.
2020-04-16 12:28:16 -04:00