Commit Graph

6561 Commits

Author SHA1 Message Date
David Taylor 5755d3886f FIX: Do not raise exception if the authenticator email is missing
Followup to 296cdc53ee
2019-08-14 12:08:59 +01:00
Blake Erickson 296cdc53ee FIX: Downcase email coming back from auth-provider 2019-08-13 10:53:50 -06:00
Vinoth Kannan 9919ee1900 FIX: remove the tmp inventory files after the s3 uploads check. 2019-08-13 11:52:57 +05:30
Sam Saffron 7632fe0b58 PERF: rake posts:rebake_uncooked_posts runs inline
Running this inline makes more sense otherwise there is extreme risk in
saturating sidekiq queue.

This also reworks ordering and selection so we double check if a post needs
rebaking prior to rebaking, this unlocks the ability to run this rake task
from multiple consoles.
2019-08-13 10:28:42 +10:00
David Taylor 923c46f5df DEV: Provide method for auth plugins to generate a CSRF token 2019-08-13 01:13:08 +01:00
Gerhard Schlager d686318133 FIX: Prevent failed remaps during restores
Additional changes:
* Verbose logging of remaps during restores
* Exclude the backup_metadata table from restores
2019-08-12 17:15:01 +02:00
Gerhard Schlager c05739c3c8 FIX: Truncate topic_links.url to 500 chars during remap
This column often breaks remaps because of some weird, long URLs. The data isn't that important, so truncating a couple of URLs doesn't hurt that much.
2019-08-12 17:15:00 +02:00
Gerhard Schlager 8b6341669a REFACTOR: Less duplicate code in DbHelper.remap 2019-08-12 17:15:00 +02:00
Dan Ungureanu 2a98becfde
FIX: Do not create a double like notification. (#7999)
When a user liked, unliked and liked again the same post, the poster
would receive a notification such as "X and X liked ...". This happened
because PostActionNotifier.post_action_created was called twice.
2019-08-12 16:22:46 +03:00
David Taylor 1a8fee11a0 DEV: If only one auth provider is enabled allow GET request
In this case, the auth provider is acting as a SSO provider, and can be trusted to maintain its own CSRF protections.
2019-08-12 11:03:05 +01:00
David Taylor 750802bf56
UX: Improve error handling for common OmniAuth exceptions (#7991)
This displays more useful messages for the most common issues we see:
- CSRF (when the user switches browser)
- Invalid IAT (when the server clock is wrong)
- OAuth::Unauthorized for OAuth1 providers, when the credentials are incorrect

This commit also stops earlier for disabled authenticators. Now we stop at the request phase, rather than the callback phase.
2019-08-12 10:55:02 +01:00
Blake Erickson 87a0a6664e FEATURE: External auth when redeeming invites
This feature (when enabled) will allow for invite_only sites to require
external authentication before they can redeem an invite.

- Created hidden site setting to toggle this
- Enables sending invites with local logins disabled
- OAuth button added to invite form
- Requires OAuth email address to match invite email address
- Prevents redeeming invite if OAuth authentication fails
2019-08-11 12:20:02 -06:00
Gerhard Schlager e4f14ca3d7 FIX: Disallow user self-delete when user posted in PMs
All posts created by the user are counted unless they are deleted,
belong to a PM sent between a non-human user and the user or belong
to a PM created by the user which doesn't have any other recipients.

It also makes the guardian prevent self-deletes when SSO is enabled.
2019-08-10 12:30:16 +02:00
Gerhard Schlager 74d78e3636 FIX: Drop readonly function when dropping table 2019-08-09 11:39:46 +02:00
Gerhard Schlager 7cb51d0e40 FIX: Create readonly functions during backup
Temporarily recreate already dropped functions in the discourse_functions schema in order to allow restoring of backups which still reference dropped functions.
2019-08-09 11:39:46 +02:00
Dan Ungureanu 8aa5df69f0
FIX: Detect DiscourseHub user agent.
It used to show up as an unknown device and browser in user preferences
(Recently Used Devices).
2019-08-09 11:58:15 +03:00
Sam Saffron 1f47ed1ea3 PERF: message_bus will be deferred by server when flooded
The message_bus performs a fair amount of work prior to hijacking requests
this change ensures that if there is a situation where the server is flooded
message_bus will inform client to back off for 30 seconds + random(120 secs)

This back-off is ultra cheap and happens very early in the middleware.

It corrects a situation where a flood to message bus could cause the app
to become unresponsive

MessageBus update is here to ensure message_bus gem properly respects
Retry-After header and status 429.

Under normal state this code should never trigger, to disable raise the
value of DISCOURSE_REJECT_MESSAGE_BUS_QUEUE_SECONDS, default is to tell
message bus to go away if we are queueing for 100ms or longer
2019-08-09 17:48:01 +10:00
Bianca Nenciu 7c83d2eeb2 FIX: Award 'First Onebox' badge just for Oneboxed URLs. (#7974) 2019-08-08 18:45:18 +02:00
David Taylor 3b8c468832 SECURITY: Require POST with CSRF token for OmniAuth request phase 2019-08-08 11:58:00 +01:00
David Taylor 7bd54eaceb
UX: Add login button icon to no_ember page (#7982) 2019-08-08 10:44:04 +01:00
Neil Lalonde 5f87089b67 FIX: remove dependency on present? in distributed_mutex lib 2019-08-07 15:39:51 -04:00
Gerhard Schlager 24877a7b8c FIX: Correctly encode non-ASCII filenames in HTTP header
Backport of fix from Rails 6: 890485cfce
2019-08-07 19:10:50 +02:00
Arpit Jalan e90aac11cb fix the build 2019-08-07 16:39:58 +05:30
Arpit Jalan b0e781e2d4 FIX: do not follow redirect on same host with path /login or /session 2019-08-07 16:26:55 +05:30
Leo McArdle 6296ae3d31 FEATURE: add setting to show content of forwarded emails in topics (#7935) 2019-08-07 12:32:19 +02:00
Joffrey JAFFEUX 5d72561f4f
FIX: allows replacement of digits and symbols emojis (#7978) 2019-08-07 11:38:58 +02:00
Rafael dos Santos Silva 606c0ed14d
FIX: S3 uploads were missing a cache-control header (#7902)
Admins still need to run the rake task to fix the files who where uploaded previously.
2019-08-06 14:55:17 -03:00
Bianca Nenciu b60b2a342f
FIX: Show membership requests link just for group owners. (#7543) 2019-08-06 13:28:22 +03:00
Joffrey JAFFEUX dbb410697e
FIX: generated controller should be in an underscored folder (#7975) 2019-08-06 10:40:38 +02:00
Sam Saffron 67f5ad5ac0 FEATURE: allow post process mutex to be held longer
Previously we would only hold the post process mutex for 1 minute, that is
not enough when processing a post with lots of images. This raises the bar
to 10 minutes.

It also cleans up error reporting around distributed mutexes expiring. We
used to double report.
2019-08-05 11:57:35 +10:00
David Taylor 39e0442de9 FIX: Various watched words improvements
- Client-side censoring fixed for non-chrome browsers. (Regular expression rewritten to avoid lookback)
- Regex generation is now done on the server, to reduce repeated logic, and make it easier to extend in plugins
- Censor tests are moved to ruby, to ensure everything works end-to-end
- If "watched words regular expressions" is enabled, warn the admin when the generated regex is invalid
2019-08-02 15:29:12 +01:00
Sam Saffron 4c6a0313f2 FEATURE: improve stats provided by rake db:stats
New stats include index size and sort by index + table size

Eg:

```
table_name                  | row_estimate | table_size | index_size | total_size
--------------------------------------------------------------------------
topic_views                 | 18260776     | 796 MB     | 2015 MB    | 2811 MB
post_timings                | 15813837     | 668 MB     | 1153 MB    | 1821 MB
posts                       | 386890       | 1532 MB    | 215 MB     | 1747 MB
post_search_data            | 317575       | 580 MB     | 162 MB     | 742 MB
incoming_emails             | 34412        | 734 MB     | 7160 kB    | 741 MB
incoming_links              | 5169302      | 306 MB     | 377 MB     | 683 MB
email_logs                  | 5            | 32 kB      | 418 MB     | 418 MB
topic_users                 | 2410166      | 217 MB     | 155 MB     | 372 MB
user_actions                | 1361253      | 101 MB     | 266 MB     | 366 MB
...

```
2019-08-02 17:25:42 +10:00
Vinoth Kannan 3e456d5c0b FIX: don't include multisite upload path to source URL if already exist. 2019-08-02 07:57:27 +05:30
Vinoth Kannan a1fc35fdf7 FIX: make uploads recovery compatible with multisite. 2019-08-02 07:08:21 +05:30
Vinoth Kannan e44d56e4d2 DEV: raise error only when 'STOP_ON_ERROR' env variable is available. 2019-08-01 23:54:06 +05:30
Guo Xiang Tan 7bd93eba3e FIX: Gravatar uploads being dependent on authorized_extensions. 2019-08-01 16:24:09 +08:00
Daniel Waterworth 20bc4a38a5
FIX: DistributedMutex (#7953) 2019-08-01 09:12:05 +01:00
Guo Xiang Tan a1a61fcd2d DEV: Use `File#file?` in `posts:missing_uploads` rake task. 2019-08-01 10:59:34 +08:00
Régis Hanol 19dda59932 FIX: add back verbose option to DbHelper.remap 2019-07-31 17:30:08 +02:00
Rishabh dcb47d902b
REFACTOR: Rename SiteSetting.disable_edit_notifications to disable_system_edit_notifications (#7958)
* REFACTOR: Rename SiteSetting.disable_edit_notifications to disable_system_edit_notifications

- The older name could cause some confusion because the setting does not disable all edit notifications, only system ones.

* FIX: Add frozen_string_literal: true in the migration

* DEV: Deprecate 'disable_edit_notifications'
2019-07-31 20:20:41 +05:30
Neil Lalonde 43365a2bf1 Fix some broken styles 2019-07-30 16:46:20 -04:00
Neil Lalonde 9656a21fdb
FEATURE: customization of html emails (#7934)
This feature adds the ability to customize the HTML part of all emails using a custom HTML template and optionally some CSS to style it. The CSS will be parsed and converted into inline styles because CSS is poorly supported by email clients. When writing the custom HTML and CSS, be aware of what email clients support. Keep customizations very simple.

Customizations can be added and edited in Admin > Customize > Email Style.

Since the summary email is already heavily styled, there is a setting to disable custom styles for summary emails called "apply custom styles to digest" found in Admin > Settings > Email.

As part of this work, RTL locales are now rendered correctly for all emails.
2019-07-30 15:05:08 -04:00
Penar Musaraj a1fb718340 DEV: Tweak plugin rake task console message
Show "Allowing write to all repos!" message only once.
2019-07-30 09:53:56 -04:00
Penar Musaraj b5705348b3 FIX: Correctly identify Chromium-based Edge
Chromium-based Edge browsers has only "Edg" in user agent string.
2019-07-29 16:51:08 -04:00
Régis Hanol 89fce2ce71 DEV: remove duplicate Remap class and use DbHelper.remap instead
Follow-up to 9cd3f96dee
2019-07-29 18:43:40 +02:00
David Taylor 933d279811
FIX: Support multi-file stylesheets in theme components (#7950) 2019-07-29 10:28:24 +01:00
Guo Xiang Tan 8a64b0c8e8 Revert "DEV: Remove unused kwarg and properly check for local missing uploads."
This reverts commit 97769f3d02.

The code is confusing but this change is quite risky. Defer for now
until we can look at it properly.
2019-07-29 14:35:34 +08:00
Guo Xiang Tan 97769f3d02 DEV: Remove unused kwarg and properly check for local missing uploads. 2019-07-29 14:21:06 +08:00
Guo Xiang Tan e9dc48ccae DEV: Fix multisite support for `posts:missing_uploads`. 2019-07-29 10:56:11 +08:00
Joffrey JAFFEUX d83f99fc2e
DEV: improves rails plugin generator (#7949)
Fixes bugs, simplifies code, more default files. General idea, more is more here as it's easier to just delete things than reading and passing all the options.
2019-07-27 18:21:44 +02:00
Osama Sayegh 525920a979
FIX: Better error when SSO fails due to blank secret (#7946)
* FIX: Better error when SSO fails due to blank secret

* Update spec/requests/session_controller_spec.rb

Co-Authored-By: Robin Ward <robin.ward@gmail.com>
2019-07-26 17:37:23 +03:00
Robin Ward a727968112 FIX: Provide an error message if no valid tags were selected 2019-07-25 12:46:16 -04:00
Gerhard Schlager fd12c414e7 DEV: Refactor helper methods for upload markdown
Follow-up to a61ff167
2019-07-25 16:36:35 +02:00
Gerhard Schlager 7e0eeed292 FEATURE: Add attachments to outgoing emails
This feature is off by default and can can be configured with the `email_total_attachment_size_limit_kb` site setting.

Co-authored-by: Maja Komel <maja.komel@gmail.com>
2019-07-25 15:57:45 +02:00
Gerhard Schlager a61ff16740 DEV: Make attachment markdown reusable 2019-07-25 14:04:18 +02:00
Vinoth Kannan 47deb8b3da FIX: use same id for both original & optimized inventories in multisite setup. 2019-07-25 14:16:47 +05:30
Joffrey JAFFEUX cc46de8f46
s/discourse-staff-notes/discourse-user-notes (#7936) 2019-07-24 20:04:27 +02:00
Régis Hanol ae05245b00 DEV: plugin API to register User custom field types 2019-07-24 18:38:44 +02:00
David Taylor b3e5f7a8c6 SECURITY: Sanitize email id for use as mutex key 2019-07-24 13:45:02 +01:00
David Taylor b084d6c8df DEV: Add missing parenthesis for 47ad2a4d 2019-07-24 11:29:18 +01:00
David Taylor 0a6cae654b SECURITY: Add confirmation screen when connecting associated accounts 2019-07-24 10:28:15 +01:00
David Taylor da4c1c5afc DEV: Remove trailing whitespace from 47ad2a4d 2019-07-24 10:27:44 +01:00
Angus McLeod 47ad2a4d7a DEV: Handle both name formats in managed authenticator (#7925) 2019-07-24 10:22:26 +01:00
Michael Brown 31f583855a DEV: pull static check out of loop
* followup to 08b28680
* as per https://review.discourse.org/t/4713/2
2019-07-23 17:18:16 -04:00
Gerhard Schlager 68b082e1a4 FIX: Ensure that jobs don't run immediately after migrate_to_s3 2019-07-23 17:42:12 +02:00
romanrizzi c4be8541f6 FIX: use uniq instead of uniq! when checking for uncompressed root path. Use rails naming convention for ZipUtils 2019-07-23 07:57:50 -03:00
OsamaSayegh f364317625 PERF: Improve query speed when looking up direct PMs
Follow up to 5fc5a7f5ae
2019-07-23 03:52:52 +00:00
Osama Sayegh 5fc5a7f5ae FEATURE: Add search operator to see all direct messages from a user (#7913)
* FEATURE: Add search operator to see all direct messages from a user

* Only show message if related messages >= 5

* Make "all messages" the hyperlink

* Review
2019-07-22 10:55:49 -04:00
Osama Sayegh f14c6d81f4
FEATURE: Watched words improvements (#7899)
This commit contains 3 features:

- FEATURE: Allow downloading watched words
This introduces a button that allows admins to download watched words per action in a `.txt` file.

- FEATURE: Allow clearing watched words in bulk
This adds a "Clear All" button that clears all deleted words per action (e.g. block, flag etc.)

- FEATURE: List all blocked words contained in the post when it's blocked
When a post is rejected because it contains one or more blocked words, the error message now lists all the blocked words contained in the post.

-------

This also changes the format of the file for importing watched words from `.csv` to `.txt` so it becomes inconsistent with the extension of the file when watched words are exported.
2019-07-22 14:59:56 +03:00
Roman Rizzi eb26bee046
DEV: group_list site settings should store IDs instead of group names (#7860)
* DEV: group_list site settings should store IDs instead of group names

* Ship site setting to know when we should migrate group_list settings

* Migrate existing group_list site settings

* Bump migration timestamp and don't set null when migrating is not possible.
2019-07-19 15:17:58 -03:00
Robin Ward e47e0af123
FEATURE: Allow viewing of raw emails for reviewable queued posts (#7910)
If a post arrives via email but must be reviewed, we now show an
icon that can be clicked to view the raw contents of the email.

This is useful if Discourse's email parser is acting odd and the user
reviewing the post wants to know what the original contents were before
approving/rejecting the post.
2019-07-19 11:56:14 -04:00
Bianca Nenciu 9ba2c7cd8b
FIX: Set a minimum reading time per post. (#7842)
Topics containing only images could generate a reading time of zero minutes.
2019-07-19 18:15:38 +03:00
Gerhard Schlager b73bd7fc1b FIX: Always backup local uploads in addition to files stored on S3 2019-07-19 15:13:05 +02:00
Blake Erickson d26aa6e71e REFACTOR: Cleanup rake tasks based on feedback
Follow up to: [FEATURE: Create a rake task for destroying categories][1]

- `Discourse.system_user` is my friend
- Remove puts statements from rake tasks that don't return anything
- `for_each` is also my friend
- Use `human_users` to also exclude discobot
- Sort/format categories:list

[1]: 092eeb5ca3
2019-07-18 19:15:01 -06:00
Blake Erickson b0c92bb0b9 REFACTOR: Clean up parameterized title
Follow up to [FIX: Empty backup names with unicode site titles][1]

- Use .presence - "It's cleaner"
- Update spec to use System.system_user so it is more readable

[1]: c8661674d4
2019-07-18 15:49:16 -06:00
Vinoth Kannan ad04ce9f43 FIX: remove post upload record creation inside 'find_missing_uploads' method. 2019-07-19 01:44:08 +05:30
Roman Rizzi f5c707c97a
FEATURE: Gz to zip for exports (#7889)
* Revert "Revert "FEATURE: admin/user exports are compressed using the zip format (#7784)""

This reverts commit f89bd55576.

* Replace .tar.zip with .zip
2019-07-18 09:34:48 -03:00
Gerhard Schlager 7e69c5cc36 Revert "FEATURE: Use configured quotation marks in fancy topic title"
This reverts most of commit ce8e099639.

The rake task to update fancy topic titles is still there, because that's useful even without this feature.
2019-07-18 11:55:49 +02:00
Blake Erickson c8661674d4 FIX: Empty backup names with unicode site titles
If a site title contains unicode it may end up with an empty backup
filename because of the rails `parameterize` method we are calling.

This fix ensures that the backup filenames default to "discourse" if the
parameterized site title is empty.

Bug reported [here][1].

[1]: https://meta.discourse.org/t/backup-checksum-and-backup-name-missing-when-unicode-site-name/123192?u=blake
2019-07-17 17:07:10 -06:00
Blake Erickson 092eeb5ca3 FEATURE: Create a rake task for destroying categories
Created a rake task for destroying multiple categories along with any
subcategories and topics the belong to those categories.

Also created a rake task for listing all of your categories.

Refactored existing destroy rake tasks to use new logging method, that
allows for puts output in the console but prevents it from showing in
the specs.
2019-07-17 12:44:14 -06:00
Joffrey JAFFEUX b3eb67976d
DEV: Upgrades to Ember 3.10 (#7871)
Co-Authored-By: majakomel <maja.komel@gmail.com>
2019-07-16 12:45:15 +02:00
Michael Brown 08b286808a
FIX: backups taken by pg_dump >= 11 are nonportable (#7893) 2019-07-15 18:07:44 -04:00
Rafael dos Santos Silva 1221d34284
FEATURE: Make Discourse work offline with WorkboxJS (#7870) 2019-07-15 13:05:55 -03:00
Vinoth Kannan 839916aa49
DEV: Debundle plugin javascript assets and don't load if disabled (#7566)
And don't load javascript assets if plugin is disabled.

* precompile auto generated plugin js assets

* SPEC: remove spec test functions

* remove plugin js from test_helper

Co-Authored-By: Régis Hanol <regis@hanol.fr>

* DEV: using equality is slightly easier to read than inequality

Co-Authored-By: Régis Hanol <regis@hanol.fr>

* DEV: use `select` method instead of `find_all` for readability

Co-Authored-By: Régis Hanol <regis@hanol.fr>
2019-07-15 20:22:54 +05:30
Neil Lalonde 6a2f30c277 Version bump to v2.4.0.beta2 2019-07-15 10:12:45 -04:00
Gerhard Schlager ce8e099639 FEATURE: Use configured quotation marks in fancy topic title 2019-07-12 21:10:10 +02:00
Gerhard Schlager 9a11a8b33b FEATURE: Site setting for typographic quotation marks
Adds locale defaults for German and French
2019-07-11 23:19:28 +02:00
Robin Ward 1d38040579 SECURITY: SQL injection with default categories
This is a low severity security fix because it requires a logged in
admin user to update a site setting via the API directly to an invalid
value.

The fix adds validation for the affected site settings, as well as a
secondary fix to prevent injection in the event of bad data somehow
already exists.
2019-07-11 13:41:51 -04:00
Joffrey JAFFEUX 550e811652
DEV: allows lodash to be updated with `rake javascript:update` (#7881) 2019-07-11 16:57:03 +02:00
Blake Erickson c76732722a FIX: Turn off search logging when read-only (#7877)
If `SiteSetting.log_search_queries` is enabled 500 errors will occur
when searching if the master db is down. This fix allows searching to
still work under these conditions.
2019-07-10 17:05:31 -07:00
romanrizzi f89bd55576 Revert "FEATURE: admin/user exports are compressed using the zip format (#7784)"
This reverts commit 8b2580e20f.
2019-07-10 11:38:51 -03:00
Roman Rizzi 8b2580e20f
FEATURE: admin/user exports are compressed using the zip format (#7784)
* FEATURE: admin/user exports are compressed using the zip format

* Update translations. Theme exporter now exports .zip file. Theme importer supports .zip and .gz files

* Fix controller test, updated locale and skip saving the csv export to disk
2019-07-10 11:13:03 -03:00
Gerhard Schlager 5f0d38341e FIX: Remapping during restore was wrong for CDN URLs 2019-07-09 17:34:41 +02:00
Gerhard Schlager 4c1b8c7559 FIX: Remap differently when backup comes from multisite 2019-07-09 16:11:32 +02:00
Gerhard Schlager a65a9a85d5 FEATURE: Remap uploads during restore when S3 or CDN changes
In order for this to work the Backuper stores a couple of site settings
in the new backup_metadata table, because the old setting values might
not be available on restore anymore.
2019-07-09 14:04:16 +02:00
Gerhard Schlager 5ffb722999 DEV: Less verbose remapping
It's hard to see which columns have been remapped when remapping prints
lots of "0 rows affected" lines. This changes it to output the row count
only for affected columns.
2019-07-09 14:04:16 +02:00
Gerhard Schlager f2dc59d61f FEATURE: Add hidden setting to include S3 uploads in backups 2019-07-09 14:04:16 +02:00
Dan Ungureanu 9f5cfa192e
FEATURE: Allow Markdown in post notices. (#7864) 2019-07-09 14:42:02 +03:00
Daniel Waterworth c3db5925a8 FIX: Turbo tests exit codes 2019-07-09 08:51:23 +01:00
Neil Lalonde 9cd3f96dee FIX: Remap shouldn't try to change read-only columns
Read-only columns are obsolete and not used in the code anymore.
Previously, remap would fail when trying to update a read-only column.
2019-07-08 16:52:52 -04:00