Commit Graph

7132 Commits

Author SHA1 Message Date
Robin Ward eaa324ecbd Revert "Move the widget-hbs compiler to js from es6"
This reverts commit 5d66a2c16e.
2020-03-25 16:13:26 -04:00
Robin Ward 5d66a2c16e Move the widget-hbs compiler to js from es6 2020-03-25 15:03:21 -04:00
Robin Ward 2abe85b834 Rename all test files from JS -> ES6 2020-03-25 15:03:21 -04:00
Mark VanLandingham c14f6d4ced
FEATURE: Allow plugins to exclude wizard steps (#9275) 2020-03-25 11:36:42 -05:00
Robin Ward f2f8ede22c Last ES6 files in our app/assets/javascripts 2020-03-25 12:30:20 -04:00
Dan Ungureanu 49395ec577
FIX: Fix image optimization pipeline (#9257)
* FIX: Do not use original filename to extract the original filename

Prefer extracting filename from the destination path, which is build
using extracted image information.

* UX: Show better error images
2020-03-25 12:59:16 +02:00
Robin Ward a11938d58d Rename more es6 files to js 2020-03-24 16:32:56 -04:00
Robin Ward 4d190c93b6 Migrate ember-addons from ES6 -> JS 2020-03-24 16:11:56 -04:00
Bianca Nenciu d8640fd042
DEV: Move requested_group_id custom field from post to topic (#9127)
Follow-up-to accbbded15
2020-03-24 11:12:52 +02:00
Robin Ward 27641f21e4 Migrate `discourse-common` from es6 -> js 2020-03-23 15:05:58 -04:00
Robin Ward 1ac0242201 Convert wizard es6 files to js 2020-03-23 14:15:16 -04:00
David Taylor 5db41cd578
SECURITY: Respect topic permissions when loading bookmark metadata
Co-authored-by: Martin Brennan <martin@discourse.org>
Co-authored-by: Sam Saffron <sam.saffron@gmail.com>
2020-03-23 11:30:48 +00:00
Roman Rizzi c4bc734b11
FIX: Backfill topic timer duration (#9249) 2020-03-23 09:38:18 +05:30
Robin Ward 07813c4a91
Convert select-kit from es6 to js (#9246)
* Convert select-kit from es6 to js

* Hide more git blames
2020-03-20 12:40:32 -04:00
Robin Ward c150566506
Migrate pretty-text to `.js` extensions (#9243) 2020-03-20 09:55:42 -04:00
Sam Saffron 145c90419d
DEV: increase timeout for initial context eval
Due to JS refactors our initial payload transpile is taking a bit longer
instead of failing the test suite attempt to give this a bit more time
2020-03-20 15:48:25 +11:00
Sam Saffron 10b37e1e36
FIX: add support for sub-sub category slugs in search
Previous to this change slugs for leaves in 3 level nestings would not work

Our UX picks only the last two levels

This also makes the results consistent for slugs as it enforces order.
2020-03-20 15:36:50 +11:00
David Taylor 19814c5e81
FIX: Allow CSP to work correctly for non-default hostnames/schemes (#9180)
- Define the CSP based on the requested domain / scheme (respecting force_https)
- Update EnforceHostname middleware to allow secondary domains, add specs
- Add URL scheme to anon cache key so that CSP headers are cached correctly
2020-03-19 19:54:42 +00:00
Joffrey JAFFEUX 86eb03af85
DEV: allows to run up/down migrations of a plugin (#9241)
This is mostly useful while developing a plugin, to avoid manual actions of deleting tables and schema_migrations rows.

Usage:
bundle exec rake plugin:migrate:down[discourse-calendar]
2020-03-19 19:30:08 +01:00
Vinoth Kannan f6d6f1701f FIX: use the new duration attribute in `set_or_create_timer` method.
New `duration` attribute is introduced for the `set_or_create_timer` method in the commit aad12822b7 for "based on last post" and "auto delete replies" topic timers.
2020-03-19 21:45:05 +05:30
Martin Brennan 0cd502a558
FIX: N1 issues for bookmark list (#9236)
* Preload custom fields for BookmarkQuery and add preload callback. Copy TopicQuery preload methodology to allow plugins to preload data for the BookmarkQuery. This fixes assigned plugin custom fields N1
* Include topic tags in initial query to avoid tags N1

Related: discourse/discourse-assign#63
2020-03-19 15:48:23 +10:00
Dan Ungureanu 1393950dbc
FIX: Improve HTML to Markdown conversion (#9231)
This commit ensures that whitespaces are preserved in <pre>, but removed
inside text paragraphs.
2020-03-18 19:31:10 +02:00
Bianca Nenciu 778454e26b
FIX: Condense line codes in emails (#9225) 2020-03-18 16:21:24 +02:00
Kane York ccc9b64a99 FIX: theme-javascripts using incorrect subfolder setting 2020-03-17 19:09:06 -07:00
Jarek Radosz e950471c0f
DEV: Replace User.unstage and User#unstage API with User#unstage! (#8906)
* DEV: Replace User.unstage and User#unstage API with User#unstage!

Quoting @SamSaffron:

> User.unstage mixes concerns of both unstaging users and updating params which is fragile/surprising.
> u.unstage destroys notifications and raises a user_unstaged event prior to the user becoming unstaged and the user object being saved.

User#unstage! no longer updates user attributes and saves the object before triggering the `user_unstaged` event.

* Update one more spec

* Assign attributes after unstaging
2020-03-17 16:48:24 +01:00
Martin Brennan 16799da580
FIX: Improve bookmark modal on mobile and bookmark sync rake task (#9221)
* Improve the bookmark mobile on modal so it doesn't go all the way to the edge and the custom datetime input is easier to use
* Improve the rake task for syncing so it does not error for topics that no longer exist and batches 2000 inserts at a time, clearing the array each time
2020-03-17 15:15:22 +10:00
Rafael dos Santos Silva e50abe1317
FIX: sync-alt is used on composer draft indicator 2020-03-16 15:32:38 -03:00
Blake Erickson 269ec70ba8 DEV: Only show deprecation message for non-whitelisted requests
As another step toward fully dreprecating query parameter authentication
in API requests this change prevents an admin dashboard message from
showing up if using a whitelisted route like rss feeds or the
mail-receiver route.
2020-03-16 12:05:24 -06:00
Daniel Waterworth 15ae3145ec FIX: Plugins may have relative symlinks 2020-03-15 11:26:25 +00:00
Sam Saffron a14313e9d0
FIX: RANDOM_PASSWORD not working rake admin:create
We regressed behavior of this env var which enable admin account
creation with a random password
2020-03-15 17:24:11 +11:00
Gerhard Schlager 8022e51179 FIX: Failed to restore backups from versions without translation overrides
Rails calls I18n.translate during initialization and by default translation overrides are used. Database migrations would fail if the system tried to migrate from an old version that didn't have the `translation_overrides` table with all its columns yet.

This makes restoring really old backups work again. Running `DISABLE_TRANSLATION_OVERRIDES=1 rake db:migrate` will allow you to upgrade such an old database as well.
2020-03-14 00:00:22 +01:00
Robin Ward e40e06d78c Rename `.js.es6` to `.js` in the admin application 2020-03-13 17:03:08 -04:00
David Taylor 3d71b68195
DEV: Introduce plugin api for conditionally rendering assets (#9200) 2020-03-13 15:30:31 +00:00
Martin Brennan af92444948
FIX: Make sure bookmark serializer works with deleted topics + posts (#9195) 2020-03-13 10:44:39 +10:00
Martin Brennan 2237ba8c9d
FIX: Add topic deleted check to email/sender (#9166)
It already had a deleted post check and log reason, add a topic one too to avoid errors
2020-03-13 10:04:15 +10:00
David Taylor ec2d49d48a
DEV: Allow plugins to add theme modifiers via db migrations (#9192) 2020-03-12 16:35:28 +00:00
Daniel Waterworth 59578dfc5b FIX: Notification emails with attachments are incorrectly structured
Two behaviors in the mail gem collide:

 1. Attachments are added as extra parts at the top level,
 2. When there are both text and html parts, the content type is set to
    'multipart/alternative'.

Since attachments aren't alternative renderings, for emails that contain
attachments and both html and text parts, some coercing is necessary.
2020-03-12 15:42:24 +00:00
Martin Brennan 89a1b3a791 Make sure reminder not sent for deleted post bookmark 2020-03-12 16:10:56 +10:00
Martin Brennan e1eb5fb9b3
FEATURE: MVP Bookmarks with reminders user list changes (#8999)
* This PR changes the user activity bookmarks stream to show a new list of bookmarks based on the Bookmark record.
* If a bookmark has a name or reminder it will be shown as metadata above the topic title in the list
* The categories, tags, topic status, and assigned show for each bookmarked post based on the post topic
* Bookmarks can be deleted from the [...] menu in the list
* As well as this, the list of bookmarks from the quick access panel is now drawn from the Bookmarks table for a user:
* All of this new functionality is gated behind the enable_bookmarks_with_reminders site setting
The /bookmarks/ route now redirects directly to /user/:username/activity/bookmarks-with-reminders
* The structure of the Ember for the list of bookmarks is not ideal, this is an MVP PR so we can start testing this functionality internally. There is a little repeated code from topic.js.es6. There is an ongoing effort to start standardizing these lists that will be addressed in future PRs.
* This PR also fixes issues with feature detection for at_desktop bookmark reminders
2020-03-12 15:20:56 +10:00
Martin Brennan 849631188f
FEATURE: Allow custom date + time for bookmark reminders (#9185)
A custom date and time can now be selected for a bookmark reminder

The reminder will not happen at the exact time but rather at the next 5 minute interval of the bookmark reminder schedule.

This PR also fixes issues with bulk deleting topic bookmarks.
2020-03-12 10:52:15 +10:00
Martin Brennan 793f39139a
FEATURE: Send notifications for time-based and At Desktop bookmark reminders (#9071)
* This PR implements the scheduling and notification system for bookmark reminders. Every 5 minutes a schedule runs to check any reminders that need to be sent before now, limited to **300** reminders at a time. Any leftover reminders will be sent in the next run. This is to avoid having to deal with fickle sidekiq and reminders in the far-flung future, which would necessitate having a background job anyway to clean up any missing `enqueue_at` reminders.

* If a reminder is sent its `reminder_at` time is cleared and the `reminder_last_sent_at` time is filled in. Notifications are only user-level notifications for now.

* All JavaScript and frontend code related to displaying the bookmark reminder notification is contained here. The reminder functionality is now re-enabled in the bookmark modal as well.

* This PR also implements the "Remind me next time I am at my desktop" bookmark reminder functionality. When the user is on a mobile device they are able to select this option. When they choose this option we set a key in Redis saying they have a pending at desktop reminder. The next time they change devices we check if the new device is desktop, and if it is we send reminders using a DistributedMutex. There is also a job to ensure consistency of these reminders in Redis (in case Redis drops the ball) and the at desktop reminders expire after 20 days.

* Also in this PR is a fix to delete all Bookmarks for a user via `UserDestroyer`
2020-03-12 10:16:00 +10:00
Robin Ward c9d1577087 Let's not log the username/password
This could easily be seen by someone who shouldn't.
2020-03-11 12:48:10 -04:00
Robin Ward a3f0543f99
Support for transpiling `.js` files (#9160)
* Remove some `.es6` from comments where it does not matter

* Use a post processor for transpilation

This will allow us to eventually use the directory structure to
transpile rather than the extension.

* FIX: Some errors and clean up in confirm-new-email

It would throw an error if the webauthn element wasn't present.
Also I changed things so that no-module is not explicitly
referenced.

* Remove `no-module`

Instead we allow a magic comment: `// discourse-skip-module` to prevent
the asset pipeline from creating a module.

* DEV: Enable babel transpilation based on directory

If it's in `app/assets/javascripts/dicourse` it will be transpiled
even without the `.es6` extension.

* REFACTOR: Remove Tilt/ES6ModuleTranspiler
2020-03-11 09:43:55 -04:00
Joffrey JAFFEUX fd4ce6ab8f
DEV: hbs extensions are misleading in this case (#9170)
This would also prevent any linting tool to attempt to lint this incorrectly.
2020-03-11 14:42:14 +01:00
David Taylor d1474e94a1
FEATURE: Allow themes to specify modifiers in their about.json file (#9097)
There are three modifiers:
- serialize_topic_excerpts (boolean)
- csp_extensions (array of strings)
- svg_icons (array of strings)

When multiple themes are active, the values will be combined. The combination method varies based on the setting. CSP/SVG arrays will be combined. serialize_topic_excerpts will use `Enumerable#any`.
2020-03-11 13:30:45 +00:00
Dan Ungureanu 0754c7c404
FIX: Various fixes to support posts with no user (#8877)
* Do not grant badges for posts with no user
* Ensure instructions are correct in Change Owner modal
* Hide user-dependent actions from posts with no user
* Make PostRevisor work with posts with no user
* Ensure posts with no user can be deleted
* discourse-narrative-bot should ignore posts with no user
* Skip TopicLink creation for posts with no user
2020-03-11 14:03:20 +02:00
Sam Saffron 1f34f653bf
FIX: last ip address could point at wrong ip
Due to unicorn env object recycling request.ip could point at the wrong
ip address by the time defer block is called. This usually would happen
under load.

This also avoids keeping the entire request object as referenced by the
closure.
2020-03-11 17:42:56 +11:00
romanrizzi f795c1b8e8 Revert "DEV: enforces ember-template-lint: no-triple-curlies (#9150)"
This reverts commit d436b600fb.

Triple curlies are still necessary for some raw templates.
2020-03-10 15:00:12 -03:00
Michael Brown 74acbad505
FIX: don't break the private key when writing it out during theme import
* when importing a private theme using the themes:install rake task the SSH key is written out to a file for use by the git-clone command
* if the private key is written out without a newline at end-of-file (i.e. after it's been stripped) it's not recognized as a valid key by SSH
* so: don't strip it when writing it out, we should be fine
2020-03-10 13:20:11 -04:00
Joffrey JAFFEUX d436b600fb
DEV: enforces ember-template-lint: no-triple-curlies (#9150)
This pr replaces `{{{ }}}` usage by a {{html-safe}} helper. While it doesn't solve the underlying issue, it gives us a path forward without risking breaking too much existing behavior.

Also introduces an htmlSafe computed macro:

```
import { htmlSafe } from "discourse/lib/computed";

htmlDescription: htmlSafe("description")
```

Overtime {{html-safe}} usage should be removed and moved to components properties or specialized components/helpers.
2020-03-10 16:46:57 +01:00
Jarek Radosz aec26ad2f0
FIX: Preserve TopicCreator's timestamp resolution (#9158)
Continuation of #9140 (e35bc8b). It's the last piece required for #9141.
2020-03-10 15:35:40 +01:00
Roman Rizzi 826b4793c0
FEATURE: Approve suspect users is now true by default. The suspect users list was removed (#9151) 2020-03-10 08:56:42 -03:00
Mark VanLandingham 3ad5cb0cbc
FIX: Error message for 403 when featuring topic on profile (#9149) 2020-03-09 11:41:07 -05:00
Jarek Radosz e35bc8bebd
FIX: Preserve PostCreator's created_at resolution (#9140)
PostMover passes to PostCreator a `created_at` that is a `ActiveSupport::WithTimeZone` instance (and also `is_a? Time`). Previously it was always being passed through `Time.zone.parse` so it would lose sub-second information. Now, it takes `Time` input as-is, while still parsing other types.
2020-03-09 17:38:13 +01:00
Jarek Radosz 85e03a7f68
DEV: Replace `Time.new` with `Time.now` (#9142)
(or `Time.zone.now`)
2020-03-09 17:37:49 +01:00
Mark VanLandingham 174764be25
FEATURE: Add embed_set_canonical_url setting (#9134) 2020-03-09 09:31:24 -05:00
Joffrey JAFFEUX 60b47d622e
UX: adds support for a color setting type (#9016) 2020-03-09 10:07:03 +01:00
Roman Rizzi 87687c0819
Drop unnecessary readonly_during_backup setting (#9112) 2020-03-06 14:29:00 -03:00
Mark VanLandingham 475f545557
DEV: Test theme js compiler append_raw_template (#8950)
* DEV: Test for append_raw_template

* remove empty line
2020-03-06 11:35:52 -05:00
Robin Ward a5f61729e0 Revert "Revert "FIX: Don't allow people to clear the upload bucket while it's enabled""
This reverts commit d4fc76b335.
2020-03-06 09:35:55 -05:00
Robin Ward d4fc76b335 Revert "FIX: Don't allow people to clear the upload bucket while it's enabled"
This reverts commit 4bb8db024c.
2020-03-05 16:29:59 -05:00
Neil Lalonde 3e21d40bc4
Version bump to v2.5.0.beta2 2020-03-05 16:10:28 -05:00
Robin Ward 4bb8db024c FIX: Don't allow people to clear the upload bucket while it's enabled 2020-03-05 16:04:52 -05:00
Kane York 10ddb8a9c4 FIX: Use destroy_all instead of delete_all for shared drafts
Rails has an odd behavior for calling .delete_all on a has_many relation - the
default behavior is to nullify the foreign key fields instead of actually
'DELETE'ing the records.

Additionally, publishing a shared draft topic creates a PostRevision that the
NotifyPostRevision job picks up which is then promptly deleted.

Use destroy_all when cleaning up the revisions and have the NotifyPostRevision
job tolerate deleted PostRevision records.

This takes a small performance hit (several SQL DELETEs instead of just one)
but shouldn't be too much of an issue (high cardinalities range from 30-100).
2020-03-05 11:13:43 -08:00
David Taylor 5b3630dba3
FIX: Do not raise an error when in:all search is performed by anon (#9113)
Also improve in:all specs to catch to catch similar failures
2020-03-05 17:50:29 +00:00
Robin Ward e01d5e2adc SECURITY: Add more restrictions on invite emails
They could be filtered and returned in some circumstances where they
shouldn't have been.
2020-03-05 09:23:21 -05:00
Daniel Waterworth 83e649d08e FIX: Clobbering plugin files on startup is problematic
On startup, (including when starting a rails console) we manipule a
collection of plugin files. Writing these files is done in multiple
observable steps, which presents opportunities for race conditions and
causes temporary corruption.

This commit uses the write, fsync and rename trick to atomically
overwrite these files instead, but reads them first to avoid unnecessary
writes.

c457d3bf was a previous attempt to fix the same problem.
2020-03-05 11:51:56 +00:00
tshenry 1c60c731ba
DEV: Mark `discourse-categories-suppressed` as official 2020-03-04 09:35:04 -08:00
Gerhard Schlager 8fa8bab9ff FIX: Don't optimize icons during db:migrate when restoring backup
Uploads are extracted after the DB migration, so this could lead to a failure during the restore. Site icons get optimized after extracting uploads.
2020-03-04 16:59:49 +01:00
Gerhard Schlager 93b8b04b06 FIX: Migrating uploads to S3 could miss files
The rake task aborted the migration with "Already migrated" when all upload URLs linked to the correct S3 bucket even though the files didn't exist on S3. By removing the first check we force the rake task to check for the existance of uploads on S3.
2020-03-04 12:50:48 +01:00
Sam Saffron 9cbed7ee46
Revert "FIX: under concurrent usage booting rails could cause plugin corruption"
This reverts commit c457d3bf28.

Something is not right about this fix, reverting.
2020-03-04 17:10:36 +11:00
Sam Saffron 0054ccc522
Revert "DEV: correct typo in param name"
This reverts commit dae5b3d673.

Something is not right about my fix reverting
2020-03-04 17:10:14 +11:00
Sam Saffron dae5b3d673
DEV: correct typo in param name
Followup to c457d3bf

It did cause a test suite failure as expected
2020-03-04 16:55:53 +11:00
Sam Saffron c457d3bf28
FIX: under concurrent usage booting rails could cause plugin corruption
Previously on boot we were always removing and adding the same pre-generated
files and symlinks.

This change attempts to avoid writing any automatically generated content if
it is exactly what it should be on disk.

This corrects issues where running a rails console can temporarily corrupt
internal state in production.
2020-03-04 16:48:06 +11:00
Jeff Wong 97545ee4a0
REFACTOR: separate post_can_act logic in post action creator (#9103)
Package logic of the post can act a little bit better so that overriding
methods can easily reuse the logic.
2020-03-03 14:56:37 -10:00
Martin Brennan 3e54e0191e
FIX: Use full URL for secure attachments when secure media enabled (#9037)
When secure media is enabled and an attachment is marked as secure we want to use the full url instead of the short-url so we get the same access control post protections as secure media uploads.
2020-03-04 10:11:08 +11:00
Martin Brennan 0388653a4d
DEV: Upload and secure media retroactive rake task improvements (#9027)
* Add uploads:sync_s3_acls rake task to ensure the ACLs in S3 are the correct (public-read or private) setting based on upload security

* Improved uploads:disable_secure_media to be more efficient and provide better messages to the user.

* Rename uploads:ensure_correct_acl task to uploads:secure_upload_analyse_and_update as it does more than check the ACL

* Many improvements to uploads:secure_upload_analyse_and_update

* Make sure that upload.access_control_post is unscoped so deleted posts are still fetched, because they still affect the security of the upload.

* Add escape hatch for capture_stdout in the form of RAILS_ENABLE_TEST_STDOUT. If provided the capture_stdout code will be ignored, so you can see the output if you need.
2020-03-03 10:03:58 +11:00
Gerhard Schlager 5889309d3b FIX: Restoring with `disable_emails: false` didn't work anymore 2020-03-02 17:44:01 +01:00
Sam Saffron b4999acadd
PERF: improve performance of category topic list
In some cases CTE caused pathologically bad query plans.
This optimises it so query runs by itself and caches for lifetime
of the topic query object.

This lightweight caching is done cause topic query will often
execute two queries (one for pinned and one for non pinned)
2020-02-29 15:40:54 +11:00
Sam Saffron 18209e1daf
DEV: remove dead code
This code is not called anywhere, remove it
2020-02-29 15:05:09 +11:00
Neil Lalonde 9f528f0ec2 Version bump to v2.5.0.beta1 2020-02-26 16:56:11 -05:00
spirobel 41ee7cad0e
FIX plugin generator: mobile, desktop stylesheets registering (#9039)
After adding styles in the mobile scss they will also be applied to desktop without this change.

https://meta.discourse.org/t/registering-assets-for-mobile-and-desktop-making-site-responsive/65041/2
2020-02-25 11:43:17 +01:00
Joffrey JAFFEUX 0ea11a9d49
FIX: ensures we don't attempt to create a new PM on an existing topic (#9029)
This fix attempts to both fix it at UI level and server side. A previous attempt related to this behavior has been made in commit: 49c750ca78
2020-02-24 08:55:12 -06:00
Martin Brennan 3af2670bd5
FIX: Consider webp a supported image format for upload (#9015)
* Also fixes an issue where if webp was a downloaded hotlinked
  image and then secure + sent in an email, it was not being
  redacted because webp was not a supported media format in
  FileHelper
* Webp originally removed as an image format in
  https://github.com/discourse/discourse/pull/6377
  and there was a spec to make sure a .bin webp
  file did not get renamed from its type to webp.

  However we want to support webp images now to make
  sure they are properly redacted if secure media is
  on, so change the example in the spec to use tiff,
  another banned format, instead
2020-02-21 13:08:01 +10:00
Martin Brennan 04df3bd46d
FIX: Only mark attachments as secure media if SiteSetting.secure_media? (#9009)
* Attachments (non media files) were being marked as secure if just
SiteSetting.prevent_anons_from_downloading_files was enabled. this
was not correct as nothing should be marked as actually "secure" in
the DB without that site setting enabled
* Also add a proper standalone spec file for the upload security class
2020-02-21 09:35:16 +10:00
Roman Rizzi c7787464cd
FEATURE: Admins can configure the reflag cooldown window and if posts flagged as spam by TL3+ users get automatically hidden (#9010) 2020-02-20 14:43:33 -03:00
Arpit Jalan 7b92280b97 UX: use same styling for username and user-name 2020-02-20 17:51:37 +05:30
Arpit Jalan f36719c1f5 FIX: respect prioritize_username_in_ux setting in email
UX: only the first attribute should be hyperlinked
UX: add margin based on attribute position
2020-02-20 17:47:16 +05:30
Sam Saffron a3d576534a
FIX: correct upload statistics report for external storage
Follows up #64b35120

This also corrects it so bytes used for internal storage counts all the space
used, previously it was only counting uploads not optimized images.

Additionally we now correctly count storage for optimized images.
2020-02-20 15:15:53 +11:00
Martin Brennan 254b57c812
FIX: When admin changes staff email still enforce old email confirm (#9007)
A follow-up correction to this change https://github.com/discourse/discourse/pull/9001.

When admin changes staff email still enforce old email confirm. Only allow auto-confirm of a new email by admin IF the target user is not also an admin. If an admin gets locked out of their email the site admin can use the rails console to solve the issue in a pinch.
2020-02-20 13:42:57 +10:00
Martin Brennan 5dc6100acc Work around deleted upload.access_control_post scoping issue
* TODO to come back and deal with this in a better way, it
  just needs to be done for the rake task to set secure media
  to work correctly
2020-02-20 10:57:40 +10:00
Martin Brennan 97d8f19387
FIX: When admin changes another user's email auto-confirm the change (#9001)
When admin changes a user's email from the preferences page of that user:

* The user will not be sent an email to confirm that their
  email is changing. They will be sent a reset password email
  so they can set the password for their account at the new
  email address.
* The user will still be sent an email to their old email to inform
  them that it was changed.
* Admin and staff users still need to follow the same old + new
  confirm process, as do users changing their own email.
2020-02-20 09:52:21 +10:00
David Taylor 836ab73d59
FIX: Patch ActiveRecord SchemaCache for safe concurrency support
A single SchemaCache instance is maintained by the connection pool, and made available via a schema_cache method on each connection. When the SchemaCache instance is fetched from the pool, its internal connection reference is updated to equal the requesting connection. However, since there is only one instance of SchemaCache, this internal connection reference is updated everywhere, and can ultimately result in multiple threads accessing the same database connection. In Discourse, this could result in Sidekiq jobs getting 'stuck' in database connections.

This patch modifies SchemaCache so that it caches the internal connection on a per-thread basis

Co-authored-by: Sam Saffron <sam.saffron@gmail.com>
Co-authored-by: Matt Palmer <mpalmer@hezmatt.org>
2020-02-19 16:26:02 +00:00
Robin Ward 041168c9b6 FIX: Group mentions were not being cooked the same was as previewed
If a group mention could be notified on preview it was given an `<a>`
tag with the `.notify` class. When cooked it would display differently.
This patch makes the server side cooking match the client preview.
2020-02-18 15:45:02 -05:00
Dan Ungureanu 66893c020f
FIX: Use 'hostname' when Discourse.os_hostname is not available
This may be the case when DiscourseLogstashLogger is initialized before
the application (see unicorn.conf.rb)

This commit is a follow-up to 28292d2759.

Co-authored-by: David Taylor <david@taylorhq.com>
Co-authored-by: Sam Saffron <sam.saffron@gmail.com>
2020-02-18 13:37:39 +02:00
Martin Brennan 500185dc11 Try fix upload_spec flakys and remove logging from tasks/uploads_spec 2020-02-18 15:08:58 +10:00
Sam Saffron 64b3512084
DEV: use DiskSpace module for all disk space calculations
This normalizes it so we only carry one place for grabbing disk space size

It also normalizes the command made so it uses Discourse.execute_command
which splits off params in a far cleaner way.
2020-02-18 15:13:19 +11:00
Sam Saffron 28292d2759
PERF: avoid shelling to get hostname aggressively
Previously we had many places in the app that called `hostname` to get
hostname of a server. This commit replaces the pattern in 2 ways

1. We cache the result in `Discourse.os_hostname` so it is only ever called once

2. We prefer to use Socket.gethostname which avoids making a shell command

This improves performance as we are not spawning hostname processes throughout
the app lifetime
2020-02-18 15:13:19 +11:00
Martin Brennan cfd56e9159 Include access control post when loading uploads in rake task
* to avoid N+1 query
2020-02-18 10:35:15 +10:00
Penar Musaraj 708190802c FIX: Strip video oneboxes from excerpts
Follows up on 49843f327e and removes onebox markup for videos too in the excerpt, this was previously being counted as part of the excerpt length.
2020-02-17 13:52:23 -05:00
Martin Brennan e8efdd60d4
FIX: Tweak upload security emoji check (#8981)
Further on from my earlier PR #8973 also reject upload as secure if its origin URL contains images/emoji. We still check Emoji.all first to try and be canonical.

This may be a little heavy handed (e.g. if an external URL followed this same path it would be a false positive), but there are a lot of emoji aliases where the actual Emoji url is something, but you can have another image that should not be secure that that thing is an alias for. For example slight_smile.png does not show up in Emoji.all BUT slightly_smiling_face does, and it aliases slight_smile e.g. /images/emoji/twitter/slight_smile.png?v=9 and /images/emoji/twitter/slightly_smiling_face.png?v=9 are equivalent.
2020-02-17 15:11:15 +10:00
Martin Brennan 9dcc454a07
FIX: Improvements and fixes for update_upload_acl rake task (#8980)
The rake task was broken, because the addition of the
UploadSecurity check returned true/false instead of the
upload ID to determine which uploads to set secure.
Also it was rebaking the posts in the wrong place and
pretty inefficiently at that. Also it was rebaking before
the upload was being changed to secure in the DB.
This also updates the task to set the access_control_post_id
for all uploads. the first post the upload is linked to is used
for the access control. if the upload doesn't get changed to
secure this doesn't affect anything.
Added a spec for the rake task to cover common cases.
2020-02-17 14:21:43 +10:00
Martin Brennan dac923379a
FIX: Never mark uploads based on regular emoji secure (#8973)
Sometimes PullHotlinkedImages pulls down a site emoji and creates a new upload record for it. In the cases where these happen the upload is not created via the normal path that custom emoji follows, so we need to check in UploadSecurity whether the origin of the upload is based on a regular site emoji. If it is we never want to mark it as secure (we don't want emoji not accessible from other posts because of secure media).

This only became apparent because the uploads:ensure_correct_acl rake task uses UploadSecurity to check whether an upload should be secure, which would have marked a whole bunch of regular-old-emojis as secure.
2020-02-17 12:30:47 +10:00
Sam Saffron 3b062f79fc
FIX: invalid syntax in elsif in Theme importer
elsif had a check that assigned to vars, this is not valid syntax and
could explode at runtime
2020-02-16 20:37:54 +11:00
Neil Lalonde c31039d51f FIX: tag topic counts wrong after adding synonyms
After adding a tag as a synonym of another tag,
both tags will have the wrong topic counts. It's
corrected within 12 hours by the EnsureDbConsistency
job. This fix ensures the topic counts are updated
much sooner.
2020-02-14 12:15:29 -05:00
Roman Rizzi fadb2b7157
FEATURE: Users cannot reflag recently handled items using the same reason unless the post was edited, or it was reviewed more than 24 hours ago. (#8969) 2020-02-14 13:43:48 -03:00
David Taylor b37d2f09ba
PERF: Memoize allowed user fields more efficiently (#8968)
Previously we were caching by user_id, but the there are only two possible outcomes. Therefore we only need to cache two values.

This removes another N+1 query when serializing multiple user cards.
2020-02-14 14:47:16 +00:00
Dan Ungureanu 8eae7d2799
DEV: Improve code readability (#8967)
Follow-up to b0f72ca1 and 8e27fa1b.
2020-02-14 13:05:52 +02:00
Martin Brennan 56b16bc68e
FIX: Never allow custom emoji to be marked secure (#8965)
* Because custom emoji count as post "uploads" we were
marking them as secure when updating the secure status for post uploads.
* We were also giving them an access control post id, which meant
broken image previews from 403 errors in the admin custom emoji list.
* We now check if an upload is used as a custom emoji and do not
assign the access control post + never mark as secure.
2020-02-14 11:17:09 +10:00
Neil Lalonde 2136d4b5d5 Version bump to v2.4.0.beta11 2020-02-13 16:56:46 -05:00
Robin Ward c2e58b6b85 FIX: Don't remove the topic image if posts don't have them 2020-02-13 14:00:30 -05:00
Martin Brennan e1e74abd4f
FEATURE: Improving bookmarks part 2 -- Topic Bookmarking (#8954)
### UI Changes

If `SiteSetting.enable_bookmarks_with_reminders` is enabled:

* Clicking "Bookmark" on a topic will create a new Bookmark record instead of a post + user action
* Clicking "Clear Bookmarks" on a topic will delete all the new Bookmark records on a topic
* The topic bookmark buttons control the post bookmark flags correctly and vice-versa
Disabled selecting the "reminder type" for bookmarks in the UI because the backend functionality is not done yet (of sending users notifications etc.)

### Other Changes

* Added delete bookmark route (but no UI yet)
* Added a rake task to sync the old PostAction bookmarks to the new Bookmark table, which can be run as many times as we want for a site (it will not create duplicates).
2020-02-13 16:26:02 +10:00
Mark VanLandingham c0ccfdb45e
FIX: Correctly compile theme template overrides (#8946) 2020-02-12 10:45:53 -08:00
Dan Ungureanu 82c84c5141
PERF: Memoize PostRevisor#diff_size (#8939)
Although this method is usually called twice, its execution can take
about 1s.
2020-02-12 19:47:48 +02:00
Gerhard Schlager 453bec9394 FEATURE: Add small action post to indicate forwarded email
This happens only when the sender of the email didn't write anything in their email.
2020-02-12 16:23:17 +01:00
Gerhard Schlager 0adab26e45 FIX: Don't count ignored, missing uploads in migration to S3 2020-02-12 16:18:52 +01:00
Dan Ungureanu 8e27fa1bd8
PERF: Optimize post editing
Reordering the conditions reduced the execution time by a factor of 5.
2020-02-12 15:53:28 +02:00
Dan Ungureanu ec40242b5c
FIX: Make inline oneboxes work with secured topics in secured contexts (#8895) 2020-02-12 12:11:28 +02:00
Mark VanLandingham 3e89774908
DEV: Use .hbr for raw template file extension (#8883) 2020-02-11 13:38:12 -06:00
Vinoth Kannan b0f72ca1d6 DEV: improve the code readability.
a0e0b1ef4b
2020-02-11 23:11:59 +05:30
Jarek Radosz 6cfd16656f
FIX: Ignore group mentions inside quotes (#8905)
Also includes:
* DEV: Reuse found elements
2020-02-10 18:31:42 +01:00
David Taylor 5919618a87
DEV: Drop legacy OpenID 2.0 support (#8894)
This is not used in core or official plugins, and has been printing a deprecation notice since v2.3.0beta4. All OpenID 2.0 code and dependencies have been dropped. The user_open_ids table remains for now, in case anyone has missed the deprecation notice, and needs to migrate their data.

Context at https://meta.discourse.org/t/-/113249
2020-02-07 17:32:35 +00:00
Vinoth Kannan a0e0b1ef4b FIX: reset `edit_reason` in posts when creating a new version. 2020-02-07 09:40:16 +05:30
Penar Musaraj 49843f327e
FIX: Strip audio/video content from excerpt (#8881) 2020-02-06 15:08:13 -05:00
Penar Musaraj 4b6a47be48 DEV: do not persist force_custom_user_agent_hosts setting
Followup to f029e2
2020-02-06 11:56:54 -05:00
Penar Musaraj 0fd39cc511 FIX: Remove post/topic image_url on post edits
- resets image_url when image is removed from first post on edit
- excludes onebox icons from being featured as topic/post images
2020-02-06 11:23:08 -05:00
Penar Musaraj f029e2eaf6 FEATURE: Add site setting for specific hosts using custom user agent when oneboxing
Followup to #00c406
2020-02-06 10:32:42 -05:00
Blake Erickson 9207c370d9 FIX: Spoiler logic should live inside of spoiler plugin
This commit removes logic about spoilers because it should live inside
of the discourse-spoiler-alert plugin.

This PR:

https://github.com/discourse/discourse-spoiler-alert/pull/38

also completely removes spoilers from excerpts in order to keep them
from leaking in topic previews and notifications.
2020-02-06 07:46:46 -07:00
Dan Ungureanu c5e3faac00
FIX: Recovered posts with no user will be taken over by system user (#8834) 2020-02-06 10:19:04 +02:00
David Taylor 88779d849f
UX: Ignore name parameter from IDP when it is equal to email (#8869)
Some auth providers (e.g. Auth0 with default configuration) send the email address in the name field. In Discourse, the name field is made public, so this commit adds a safeguard to prevent emails being made public.
2020-02-05 16:03:18 +00:00
Régis Hanol 13d4b05963 FIX: allow both meta_data and custom_fields when creating a topic
For some reasons, we have two ways of associating "custom fields" to a new topic:
using 'meta_data' and 'custom_fields'.

However, if we were to provide both arguments, the 'meta_data' would be overwritten
by any 'custom_fields' provided.

This commit ensures we can use both and merges the 'custom_fields' with the 'meta_data'.
2020-02-05 10:40:22 +01:00
Joffrey JAFFEUX f0fe2ba9ac
UX: introduces icon-picker component for badges (#8844) 2020-02-05 00:41:10 +01:00
Joffrey JAFFEUX 87e92da085
DEV: makes popper.js part of javascript rake task (#8847) 2020-02-04 15:34:46 +01:00
Gerhard Schlager e84d88ddea FIX: Email attachments with a size of 0 bytes caused error 2020-02-04 12:30:43 +01:00
Matt Marjanović ad2aa7b52c
FEATURE: Add logout functionality to SSO Provider protocol (#8816)
This commit adds support for an optional "logout" parameter in the
payload of the /session/sso_provider endpoint.  If an SSO Consumer
adds a "logout=true" parameter to the encoded/signed "sso" payload,
then Discourse will treat the request as a logout request instead
of an authentication request.  The logout flow works something like
this:

 * User requests logout at SSO-Consumer site (e.g., clicks "Log me out!"
   on web browser).
 * SSO-Consumer site does whatever it does to destroy User's session on
   the SSO-Consumer site.
 * SSO-Consumer then redirects browser to the Discourse sso_provider
   endpoint, with a signed request bearing "logout=true" in addition
   to the usual nonce and the "return_sso_url".
 * Discourse destroys User's discourse session and redirects browser back
   to the "return_sso_url".
 * SSO-Consumer site does whatever it does --- notably, it cannot request
   SSO credentials from Discourse without the User being prompted to login
   again.
2020-02-03 12:53:14 -05:00
Daniel Waterworth 7c7098c700 FIX: Off-by-one error setting the distributed mutex key to expire
Accounting for fractional seconds, a distributed mutex can be held for
almost a full second longer than its validity.

For example: if we grab the lock at 10.5 seconds passed the epoch with a
validity of 5 seconds, the lock would be released at 16 seconds passed
the epoch. However, in this case assuming that all other processing
takes a negligible amount of time, the key would be expired at 15.5
seconds passed the epoch.

Using expireat, the key is now expired exactly when the lock is released.
2020-02-03 14:54:50 +00:00
Martin Brennan 5f1749a1c0 Minor change to case-insensitive regex for s3_presigned_url? 2020-02-03 14:22:35 +10:00
Jarek Radosz 73b04976e5
FIX: Use updated_at in the S3 inventory job (#8823)
When we change upload's sha1 (e.g. when resizing images) it won't match the data in the most recent S3 inventory index. With this change the uploads that have been updated since the inventory has been generated are ignored.
2020-01-31 11:02:44 +01:00
Martin Brennan 8d77e99827
FIX: Stop encoding presigned URLs with UrlHelper (#8818)
When FinalDestination is given a URL it encodes it before doing anything else. however S3 presigned URLs should not be messed with in any way otherwise we can end up with 400 errors when downloading the URL e.g.

<Error><Code>InvalidToken</Code><Message>The provided token is malformed or otherwise invalid.</Message>

The signature of presigned URLs is very important and is automatically generated and should be preserved.
2020-01-31 09:09:34 +10:00
Dan Ungureanu 62b9a432bd
FIX: Import sub-sub-categories (#8810)
This should make the importer more resilient to incomplete or damaged
backups. It will disable some validations and attempt to automatically
repair category permissions before importing.
2020-01-30 18:46:33 +02:00
Mark VanLandingham 089d38e9d3
FIX: Disallow featuring hidden topics (#8814) 2020-01-30 10:00:49 -06:00
Dan Ungureanu bf50087d72
Revert "FIX: Create post notices only for public posts (#8708)"
This reverts commit c2d051315d.
2020-01-30 14:26:08 +02:00
Arpit Jalan 62c21ba649 Remove `bounce_score_threshold_deactivate` setting.
Removed `bounce_score_threshold_deactivate` setting as the deactivate
threshold is not getting reached.
2020-01-30 16:17:31 +05:30
Martin Brennan 1150cd4621
FIX: Stop secure media URLs being censored too liberally in emails (#8817)
For example /t/ URLs were being replaced if they contained secure-media-uploads so if you made a topic called "Secure Media Uploads Are Cool" the View Topic link in the user notifications would be stripped out.

Refactored code so this secure URL detection happens in one place.
2020-01-30 16:19:14 +10:00
Martin Brennan c84652eb8b Get rid of no longer needed target_usernames warning log 2020-01-30 12:30:11 +10:00
Neil Lalonde 34f564acd6 FIX: customized email subjects was ignored for some notifications
Email templates for notifications about new posts and topics weren't
using customized values.
2020-01-29 14:49:08 -05:00
Dan Ungureanu 09e8be3209
UX: Introduce automatic 'categories topics' setting (#8804)
When 'categories topics' setting is set to 0, the system will
automatically try to find a value to keep the two columns (categories
and topics) symmetrical.

The value is computed as 1.5x the number of top level categories and at
least 5 topics will always be returned.
2020-01-29 20:30:48 +02:00
Mark VanLandingham e29fef9e99
FEATURE: Users can feature any public topic on his/her profile (#8809) 2020-01-29 10:10:23 -06:00
Sam Saffron 7f3a30d79f FIX: blank cooked markdown could raise an exception in logs
Previously if somehow a user created a blank markdown document using tag
tricks (eg `<p></p><p></p><p></p><p></p><p></p><p></p>`) and so on, we would
completely strip the document down to blank on post process due to onebox
hack.

Needs a followup cause I am still unclear about the reason for empty p stripping
and it can cause some unclear cases when we re-cook posts.
2020-01-29 11:37:25 +11:00
Martin Brennan ab3bda6cd0
FIX: Mitigate issue where legacy pre-secure hotlinked media would not be redownloaded (#8802)
Basically, say you had already downloaded a certain image from a certain URL
using pull_hotlinked_images and the onebox. The upload would be stored
by its sha as an upload record. Whenever you linked to the same URL again
in a post (e.g. in our case an og:image on review.discourse) we would
would reuse the original upload record because of the sha1.

However when you turned on secure media this could cause problems as
the first post that uses that upload after secure media is enabled
will set the access control post for the upload to the new post.
Then if the post is deleted every single onebox/link to that same image
URL will fail forever with 403 as the secure-media-uploads URL fails
if the access control post has been deleted.

To fix this when cooking posts and pulling hotlinked images, we only
allow using an original upload by URL if its access control post
matches the current post, and if the original_sha1 is filled in,
meaning it was uploaded AFTER secure media was enabled. otherwise
we just redownload the media again to be safe, as the URL will always
be new then.
2020-01-29 10:11:38 +10:00