Commit Graph

1312 Commits

Author SHA1 Message Date
Sam Saffron 70c6831125 PERF: add a filtered index for banners
This ensures we can very quickly figure out which topics are banners if
a banner is set.

Previously you would have to scan an entire table to find banners
2019-11-01 11:22:19 +11:00
Neil Lalonde d777844ed6 FEATURE: categories can require topics have a tag from a tag group
In a category's settings, the Tags tab has two new fields to
specify the number of tags that must be added to a topic
from a tag group. When creating a new topic, an error will be
shown to the user if the requirement isn't met.
2019-10-31 16:10:19 -04:00
Sam Saffron c5e67726fd FIX: under some conditions draft would say it was saving when not
This is a major change to draft internals. Previously there were quite a
few cases where the draft system would say "draft saved", when in fact
we just skipped saving.

This commit ensures the draft system deals with draft ownership handover in
a predictable way.

For example:

- Window 1 editing draft
- Window 2 editing same draft at the same time

Previously we would allow window 1 and 2 to just fight on the same draft
each window overwriting the same draft over an over.

This commit introduces an ownership concept where either window 1 or 2 win
and user is prompted on the loser window to reload screen to correct the issue

This also corrects edge cases where a user could have multiple browser windows
open and posts in 1 window, later to post in the second window. Previously
drafts would break in the second window, this corrects it.
2019-10-31 17:15:58 +11:00
Gerhard Schlager 6a0e232dea PERF: Add index to notifications to speed up moving of posts 2019-10-30 12:35:12 +01:00
Sam Saffron 5ae35f9906 FIX: allow storage of non unique rows in oauth2_user_infos
Certain DBs have duplicates already, if we want to ensure uniqueness here
we need to decide first how to clean up existing data and confirm all the
plugins expect this.
2019-10-25 11:57:34 +11:00
Dan Ungureanu a7301c8671 PERF: Add index on group to category_groups (#8231) 2019-10-23 10:30:43 +01:00
Dan Ungureanu 09a569aaba PERF: Add unique index oauth2_user_infos(user_id, provider) (#8230) 2019-10-23 10:27:56 +01:00
Sam Saffron f5d1aff8dd FEATURE: experimental hidden setting for draft backups
Under exceptional situations the automatic draft feature can fail.

This new **hidden, default off** site setting
`backup_drafts_to_pm_length` will automatically backup any draft that is
saved by the system to a dedicated PM (originating from self)

The body of that PM will contain the text of the reply.

We can enable this feature strategically on sites exhibiting issues to
diagnose issues with the draft system and offer a recourse to users who
appear to lose drafts. We automatically checkpoint these drafts every 5
minutes forcing a new revision each 5 minutes so you can revert to old
content.

Longer term we are considering automatically enabling this kind of feature
for extremely long drafts where the risk is really high one could lose
days of writing.
2019-10-17 16:58:21 +11:00
Daniel Waterworth 7ba914f1e1 FIX: Add migrations to fix index on category slugs
Slugs can be the empty string, but the added index didn't account for
that. This commit changes the migration, stopping it from being unique
so that it can be applied everywhere and adds another migration that
recreates the index properly.
2019-10-16 14:02:11 +00:00
Daniel Waterworth c71da3f335 FIX: Add unique index to prevent duplicate slugs for categories 2019-10-15 19:44:50 +00:00
Roman Rizzi 01bc465db8
DEV: Split max decompressed setting for themes and backups (#8179) 2019-10-11 14:38:10 -03:00
Mark VanLandingham 057f698e37 FIX: Store user_accuracy_bonus to clarify explanations (#8165)
* FIX: Store user_accuracy_bonus to clarify explanations

* Fixed specs from rounding change

* migration cleanup

* user_accuracy_bonus column not nullable
2019-10-08 09:49:07 -04:00
Martin Brennan 68d35b14f4 FEATURE: Webauthn authenticator management with 2FA login (Security Keys) (#8099)
Adds 2 factor authentication method via second factor security keys over [web authn](https://developer.mozilla.org/en-US/docs/Web/API/Web_Authentication_API).

Allows a user to authenticate a second factor on login, login-via-email, admin-login, and change password routes. Adds registration area within existing user second factor preferences to register multiple security keys. Supports both external (yubikey) and built-in (macOS/android fingerprint readers).
2019-10-01 19:08:41 -07:00
Krzysztof Kotlarek 32b8a2ccff DEV: Upgrade Discourse to Rails 6 (#8083)
* Adjustments to pass specs on Rails 6.0.0
* Use classic autoloader instead of Zeitwerk
* Update Rails 6.0.0 deprecated methods
* Rails 6.0.0 not allowing column with integer name
* Drop freedom_patches/rails6.rb
* Default value for trigger_transactional_callbacks? is true
* Bump rspec-rails version to 4.0.0.beta2
2019-09-12 10:41:50 +10:00
Rimian Perkins d17425cbb8 FIX: Migrate post_edit_time_limit to tl2_post_edit_time_limit (#8082)
If a user amended edit_time_limit keep the behavior as is, instead of introducing a potentially tighter time for tl2 edit time than what they had set in the past.
2019-09-09 11:27:41 +10:00
Sam Saffron dc9110cc43 FEATURE: track date api key was last used
Start tracking the date an api key was last used. This has already been
the case for user_api_keys.

This information can provide us with the ability to automatically expire
unused api keys after N days.
2019-09-03 18:11:30 +10:00
Roman Rizzi 7c741fa0d6
FEATURE: Publish read state on group messages. (Originally introduced in #7989) (#8025)
* Revert "Revert "FEATURE: Publish read state on group messages. (#7989) [Undo revert] (#8024)""

This reverts commit 36425eb9f0.

* Fix: Show who read only if the attribute is enabled

* PERF: Precalculate the last post  readed by a group member

* Use book-reader icon instear of far-eye

* FIX: update topic groups correctly

* DEV: Tidy up read indicator update on write
2019-08-27 09:09:00 -03:00
romanrizzi 36425eb9f0 Revert "FEATURE: Publish read state on group messages. (#7989) [Undo revert] (#8024)"
This reverts commit 5dda5c2f7c.
2019-08-20 13:29:22 -03:00
Roman Rizzi 5dda5c2f7c
FEATURE: Publish read state on group messages. (#7989) [Undo revert] (#8024)
* Reenable: "FEATURE: Publish read state on group messages. (#7989)"

This reverts commit 67f5cc1ce8.

* FIX: Read indicator only appears when the group setting is enabled
2019-08-20 11:57:25 -03:00
romanrizzi 67f5cc1ce8 Revert "FEATURE: Publish read state on group messages. (#7989)"
This reverts commit 1630dae2db.
2019-08-20 10:24:34 -03:00
Roman Rizzi 1630dae2db
FEATURE: Publish read state on group messages. (#7989)
* Enable or disable read state based on group attribute

* When read state needs to be published, the minimum unread count is calculated in the topic query. This way, we can know if someone reads the last post

* The option can be enabled/disabled from the UI

* The read indicator will live-updated using message bus

* Show read indicator on every post

* The read indicator now shows read count and can be expanded to see user avatars

* Read count gets updated everytime someone reads a message

* Simplify topic-list read indicator logic

* Unsubscribe from message bus on willDestroyElement, removed unnecesarry values from post-menu, and added a comment to explain where does minimum_unread_count comes from
2019-08-20 09:46:57 -03:00
Robin Ward b4878cde6f FEATURE: Add a webhook for user notifications
If enabled, this will fire a webhook whenever a user's notification has
been created. This could potentially be a lot of data depending on your
forum, and should be used carefully since it includes everything all users
will see in their feeds.
2019-08-15 14:47:25 -04:00
Vinoth Kannan 88359b0f16
FEATURE: add support for group members visibility level (#8004)
There are 5 visibility levels (similar to group visibility)

public (default)
logged-in users
members only
staff
owners

Admins & group owners always have visibility to group members.
2019-08-14 19:00:04 +05:30
Gerhard Schlager 7cb51d0e40 FIX: Create readonly functions during backup
Temporarily recreate already dropped functions in the discourse_functions schema in order to allow restoring of backups which still reference dropped functions.
2019-08-09 11:39:46 +02:00
Leo McArdle 6296ae3d31 FEATURE: add setting to show content of forwarded emails in topics (#7935) 2019-08-07 12:32:19 +02:00
Rishabh dcb47d902b
REFACTOR: Rename SiteSetting.disable_edit_notifications to disable_system_edit_notifications (#7958)
* REFACTOR: Rename SiteSetting.disable_edit_notifications to disable_system_edit_notifications

- The older name could cause some confusion because the setting does not disable all edit notifications, only system ones.

* FIX: Add frozen_string_literal: true in the migration

* DEV: Deprecate 'disable_edit_notifications'
2019-07-31 20:20:41 +05:30
Régis Hanol afe2be4f62 Follow-up for 89efcfcf to properly fix nested arrays 2019-07-23 10:29:18 +02:00
Gerhard Schlager 89efcfcf28 Follow-up for a8cdd685 to fix nested array 2019-07-22 22:30:57 +02:00
Gerhard Schlager a8cdd68518 FIX: Migrations tried to change frozen string
Also, makes the migration work with locales which use more than just the "one" and "other" keys.
2019-07-22 22:16:43 +02:00
Roman Rizzi eb26bee046
DEV: group_list site settings should store IDs instead of group names (#7860)
* DEV: group_list site settings should store IDs instead of group names

* Ship site setting to know when we should migrate group_list settings

* Migrate existing group_list site settings

* Bump migration timestamp and don't set null when migrating is not possible.
2019-07-19 15:17:58 -03:00
Arpit Jalan eb9155f3fe
FEATURE: send max 200 emails every minute for bulk invites (#7875)
DEV: deprecate `invite.via_email` in favor of `invite.emailed_status`

This commit adds a new column `emailed_status` in `invites` table for
 tracking email sending status.
 0 - not required
 1 - pending
 2 - bulk pending
 3 - sending
 4 - sent

For normal email invites, invite record is created with emailed_status
 set to 'pending'.

When bulk invites are sent invite record is created with emailed_status
 set to 'bulk pending'.

For invites that generates link, invite record is created with
 emailed_status set to 'not required'.

When invite email is in queue emailed_status is updated to 'sending'

Once the email is sent via `InviteEmail` job the invite emailed_status
 is updated to 'sent'.
2019-07-19 11:29:12 +05:30
Penar Musaraj 98866ca043 Adds down SQL statements to badge icons migration
follows up on a571efba
2019-07-17 09:15:29 -04:00
Penar Musaraj a571efba35
FIX: Rename deprecated icons, allow custom icons in badges
- adds a migration renaming FA4 icon names in badges
- allows all icons to be used in badges (previously was limited to icons prefixed with fa-)
- renames remaining FA 4.7 icons equivalents
2019-07-16 11:13:44 -04:00
Gerhard Schlager a65a9a85d5 FEATURE: Remap uploads during restore when S3 or CDN changes
In order for this to work the Backuper stores a couple of site settings
in the new backup_metadata table, because the old setting values might
not be available on restore anymore.
2019-07-09 14:04:16 +02:00
Penar Musaraj b690fc3d98
FEATURE: Add new group visibility option for "logged on users" (#7814)
Groups can now be marked as visible to "logged on users". All automatic groups (except `everyone`) are now visible to "logged on users", previously they were marked as public but suppressed in the group page for non-staff.
2019-07-08 15:09:50 -04:00
Osama Sayegh 3d64532273 FEATURE: allow disabling theme components (#7812)
This allows you to temporarily disable components without having to remove them from a theme. 

This feature is very handy when doing quick fix engineering.
2019-07-03 18:18:11 +10:00
Jeff Wong 88ef5e55fe
FEATURE: add ability to have multiple totp factors (#7626)
Adds a second factor landing page that centralizes a user's second factor configuration.

This contains both TOTP and Backup, and also allows multiple TOTP tokens to be registered and organized by a name. Access to this page is authenticated via password, and cached for 30 minutes via a secure session.
2019-06-26 16:58:06 -07:00
Daniel Waterworth bc03c509ab FIX: CategoryUser#batch_set (#7787)
* Remove unused method

* Prefabricate user in category_user_spec.rb

* FIX: Remove notification_level from category_users unique indexes

* FIX: CategoryUser#batch_set wasn't updating pre-existing records

* Improve tests for CategoryUser#batch_set

* FIX: changed was being reported incorrectly

* DEV: Rewrote query to do a bulk insert

* DEV: remove unnecessary parentheses
2019-06-25 12:13:27 +10:00
David Taylor 6afd7f0eae FIX: Clean broken relations before adding javascript_caches foreign key 2019-06-04 10:01:20 +01:00
David Taylor 81dcadf788
DEV: Stop compiling themes during DB migration. Recompile on cdn change. (#7676)
This is an improved implementation for bc8b7b13
2019-06-03 16:38:02 +01:00
David Taylor 7500eed4c0
FEATURE: Multi-file javascript support for themes (#7526)
You can now add javascript files under `/javascripts/*` in a theme, and they will be loaded as if they were included in core, or a plugin. If you give something the same name as a core/plugin file, it will be overridden. Support file extensions are `.js.es6`, `.hbs` and `.raw.hbs`.
2019-06-03 10:41:00 +01:00
Dan Ungureanu c1e7a1b292 UX: Merge settings related to muted tags. (#7656) 2019-06-03 12:23:23 +10:00
Sam Saffron 3b8819f0ab FEATURE: add support for defer topic functionality
This feature allows end users to "defer" topics by marking them unread

The functionality is default disabled. This also introduces the new site
setting default_other_enable_defer: to enable this by default on new user
accounts.
2019-05-31 15:44:35 +10:00
Sam Saffron 5c524ea8a4 FEATURE: introduce dedicated storage and DB constraints for anon users
Previously we used custom fields to denote a user was anonymous, this was
risky in that custom fields are prone to race conditions and are not
properly dedicated, missing constraints and so on.

The new table `anonymous_users` is properly protected. There is only one
possible shadow account per user, which is enforced using a constraint.

Every anonymous user will have a unique row in the new table.
2019-05-29 14:26:24 +10:00
Arpit Jalan 2e3b74c353 fix the build 2019-05-28 16:47:01 +05:30
Arpit Jalan 028121b95b
FIX: delete system generated message when user_export record is deleted (#7595)
FIX: system generated message for user export should be closed by default
2019-05-28 16:38:41 +05:30
Guo Xiang Tan fd0eff6ce3 Revert "FIX: Add missing unique index on user_id for user_profiles."
Oops `user_id` is the primary key.

This reverts commit 5478aa63ab.
2019-05-24 12:58:36 +08:00
Guo Xiang Tan 5478aa63ab FIX: Add missing unique index on user_id for user_profiles. 2019-05-24 12:11:49 +08:00
Robin Ward e74cd54fc6 REFACTOR: Replace score bonuses with low/med/high priorities
We removed score from the UX so it makes more sense to have sites set
priorities instead of score bonuses.
2019-05-23 11:54:45 -04:00
Gerhard Schlager 98a17edc90 Remove unused translations
The migration for the ToS signup field happend in 2014. Everyone who hasn't updated yet needs to live with the English text "Terms of Service". There's no need to keep these unused translations forever.
2019-05-22 16:15:22 +02:00
Gerhard Schlager b788948985 FEATURE: English locale with international date formats
Makes en_US the new default locale
2019-05-20 13:47:20 +02:00
Sam Saffron 624184560e PERF: improve performance of publish_notifications_state
User.publish_notifications_state is called every time a notification is
created, this can become a very critical code path.

On some heavy notification related sites this can be a major CPU user on PG

This index makes it much cheaper to publish notification state, cause a
simple index lookup does the trick.
2019-05-14 16:02:55 +10:00
Guo Xiang Tan 8165ceb320 Make rubocop happy. 2019-05-13 09:55:44 +08:00
Sam Saffron 30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Robin Ward b380ed5282 FEATURE: Claim Reviewables by Topic
This is a feature that used to be present in discourse-assign but is
much easier to implement in core. It also allows a topic to be assigned
without it claiming for review and vice versa and allows it to work with
category group reviewers.
2019-05-09 13:40:36 -04:00
Guo Xiang Tan 414a36184b DEV: Prefer `create!` over `create` when not checking return value. 2019-05-09 15:10:24 +08:00
Vinoth Kannan 87cd4701b8 FEATURE: option to skip posts with ignored missing uploads 2019-05-09 05:11:15 +05:30
Robin Ward b409dab77f Rename `flags_default_topics` to `reviewable_default_topics`
This functionality regressed with the reviewable update. I took the
opporunity to rename it when fixing it for clarity.
2019-05-08 10:07:10 -04:00
Guo Xiang Tan 152238b4cf DEV: Prefer `public_send` over `send`. 2019-05-07 09:33:21 +08:00
Robin Ward 31e100530f FEATURE: Flag count in post menu
This change shows a notification number besides the flag icon in the
post menu if there is reviewable content associated with the post.
Additionally, if there is pending stuff to review, the icon has a red
background.

We have also removed the list of links below a post with the flag
status. A reviewer is meant to click the number beside the flag icon to
view the flags. As a consequence of losing those links, we've removed
the ability to undo or ignore flags below a post.
2019-05-06 16:13:31 -04:00
Sam Saffron 413a54e7be DEV: follow up on bb8cdf9e, assume no dupe bounce_keys
We may have tons of emails in the email logs, assume none are duplicate
hunting for duplication can be very expensive.
2019-05-03 10:52:15 +10:00
Sam Saffron bb8cdf9e5f PERF: add missing bounce_key index to email_logs
We perform lookups based off bounce_key when emails bounce, we need the
index.
2019-05-03 08:48:01 +10:00
Guo Xiang Tan 24347ace10 FIX: Properly associate user_profiles background urls via upload id.
`Upload#url` is more likely and can change from time to time. When it
does changes, we don't want to have to look through multiple tables to
ensure that the URLs are all up to date. Instead, we simply associate
uploads properly to `UserProfile` so that it does not have to replicate
the URLs in the table.
2019-05-02 14:58:24 +08:00
Guo Xiang Tan c9f6beba05 FIX: Incorrect DB query for `AddUploadsToCategories` migration.
This is a regression as a result of 7896c74c2b. Most instances would have ran the migrations and some might have run this migration with the incorrect query. Impact of this is small for now but I'm fixing this for correctness purposes.
2019-05-02 14:58:24 +08:00
David Taylor 0e303c7f5d
FEATURE: Automatically generate optimized site metadata icons (#7372)
This change automatically resizes icons for various purposes. Admins can now upload `logo` and `logo_small`, and everything else will be auto-generated. Specific icons can still be uploaded separately if required.

## Core

- Adds an SiteIconManager module which manages automatic resizing and fallback

- Icons are looked up in the OptimizedImage table at runtime, and then cached in Redis. If the resized version is missing for some reason, then most icons will fall back to the original files. Some icons (e.g. PWA Manifest) will return `nil` (because an incorrectly sized icon is worse than a missing icon). 

- `SiteSetting.site_large_icon_url` will return the optimized version, including any fallback. `SiteSetting.large_icon` continues to return the upload object. This means that (almost) no changes are required in core/plugins to support this new system.

- Icons are resized whenever a relevant site setting is changed, and during post-deploy migrations

## Wizard

- Allows `requiresRefresh` wizard steps to reload data via AJAX instead of a full page reload

- Add placeholders to the **icons** step of the wizard, which automatically update from the "Square Logo"

- Various copy updates to support the changes

- Remove the "upload-time" resizing for `large_icon`. This is no longer required.

## Site Settings UX

- Move logo/icon settings under a new "Branding" tab

- Various copy changes to support the changes

- Adds placeholder support to the `image-uploader` component

- Automatically reloads site settings after saving. This allows setting placeholders to change based on changes to other settings

- Upload site settings will be assigned a placeholder if SiteIconManager `responds_to?` an icon of the same name

## Dashboard Warnings

- Remove PWA icon and PWA title warnings. Both are now handled automatically.

## Bonus

- Updated the sketch logos to use @awesomerobot's new high-res designs
2019-05-01 14:44:45 +01:00
Robin Ward 404b35bd04 FEATURE: Category Reviewable by Group
Allow a group to review content in a particular category.
2019-04-30 15:23:06 -04:00
Sam Saffron b8c65cc6cf PERF: add index on user auth tokens (for user lookups)
This speeds up user destroy cause we cascade looking up all the tokens
for a user
2019-04-26 22:38:54 +10:00
Sam Saffron 378fe1c1cf PERF: allow looking up incoming email by user_id
This speeds up the ability to remove users from the system
2019-04-26 22:33:01 +10:00
Sam Saffron e82b90d6ac correct typo 2019-04-26 18:14:28 +10:00
Sam Saffron c009a6ae36 PERF: speed up user deletion logic
Previously any user deletion would scan a very large number of tables

This avoids scans on 6 tables/indexes on delete
2019-04-26 18:11:39 +10:00
Sam Saffron 8d697bbbd3 FIX: ensure image tracking custom fields have no dupes
All these columns must never include duplicates for a single post
otherwise code breaks. All are defined in post.rb in the top in constants
but we usually prefer not to ref constants in case they change and migration
becomes inconsistent.
2019-04-24 17:21:42 +10:00
David Taylor a2ddb6cf2d DEV: Increase max length of theme_field name column
Now that we have custom SCSS files with arbitrary names, it's easy to exceed 30 characters
2019-04-23 12:34:32 +01:00
Bianca Nenciu 9050b1bf5a FIX: Add unique index on group_requests(group_id, user_id). (#7399) 2019-04-23 12:51:30 +10:00
Dan Ungureanu 57d1dea8a2
FEATURE: Let staff add custom post notices. (#7377) 2019-04-19 17:53:58 +03:00
Penar Musaraj 7cd621778d FEATURE: Native app banner improvements
This commit adds some improvements to native app banners for iOS and Android

- iOS and Android now have separate settings for native app banners

- app banners will now only show for users on TL1 and up

- app ids are now in a hidden site setting to allow sites to switch to their own app, if desired

- iOS only: the site URL is passed to the app arguments
2019-04-17 12:25:13 -04:00
Robin Ward a5d9afe397 FEATURE: Include a user's pending posts in the topic view
Also includes a refactor to TopicView's serializer which was not
building our attributes using serializers properly.
2019-04-12 14:25:12 -04:00
David Taylor dc703adad7 FEATURE: Add user preference for title counter mode (#7364) 2019-04-12 09:02:18 +10:00
Robin Ward 331a809738 FEATURE: Display the reason for many reviewable items
Queued Posts and Users will now display a reason why they are in the
review queue.
2019-04-11 12:04:45 -04:00
Vinoth Kannan 8d5c900142 DEV: add unique missing uploads index in post custom fields
https://review.discourse.org/t/feature-send-missing-post-uploads-stat-to-prometheus/2609/6?u=vinothkannans
2019-04-10 18:09:35 +05:30
Tarek Khalil 442fb2facb FEATURE: Remove ignore feature SiteSetting and enable ignore by default (#7349) 2019-04-10 12:54:59 +02:00
Guo Xiang Tan bac1bcc79f PERF: Add `index_reply_id_on_post_replies`.
Speeds up the reference posts query in `Email::Sender#send`.
2019-04-10 13:58:29 +08:00
Guo Xiang Tan f40b28d4c7 DEV: Safeguard for migration that disables DDL transactions.
Follow up to c82a929025.
2019-04-09 14:14:30 +08:00
Guo Xiang Tan c82a929025 PERF: Add `index_for_rebake_old` to `posts`.
The index becomes smaller over time and is much faster.

Follow up to 4791d992dc.
2019-04-09 13:59:46 +08:00
Guo Xiang Tan b7d8bbc095 PERF: Add index for `post_search_data` and `topic_search_data`. 2019-04-08 16:37:54 +08:00
Guo Xiang Tan 4791d992dc PERF: Add index `id DESC, baked_version` ON `posts`.
A scheduled job runs `Post.rebake_old` with a limit of 80 which does a
look up for the latest posts that have not been baked to the latest
version. Before this commit, the query would run using the primary key's
index and does a reverse scan. However the query performance quicky
becomes bad as more and more of the latest posts have been baked to the
latest version.
2019-04-08 15:33:16 +08:00
Robin Ward 13c6bf54d0 FIX: Sometimes queued post would have a string for a category
In this case, don't migrate the old queued post category
2019-04-06 20:38:38 -04:00
Robin Ward 9c2e6b9a54 FIX: Migrated flags should be reviewable by moderators 2019-04-05 10:04:23 -04:00
Sam Saffron 8f4527c1f1 FIX: recover from migration failure
If the migration somehow failed halfway then just run through, when ddl
transactions are disabled stuff can get somewhat odd.
2019-04-05 21:53:07 +11:00
Guo Xiang Tan ca4c9abec7 PERF: Add index for (action_type,created_at)` on `user_actions` table.
Speeds up counting of likes in `About#stats`.
2019-04-05 12:49:45 +08:00
Sam Saffron 5f896ae8f7 PERF: Keep track of when a users first unread is
This optimisation avoids large scans joining the topics table with the
topic_users table.

Previously when a user carried a lot of read state we would have to join
the entire read state with the topics table. This operation would slow down
home page and every topic page. The more read state you accumulated the
larger the impact.

The optimisation helps people who clean up unread, however if you carry
unread from years ago it will only have minimal impact.
2019-04-05 12:44:45 +11:00
Neil Lalonde 83996fc8ea FEATURE: ability to restrict some tags to a category while allowing all others
A new checkbox has been added to the Tags tab of the category settings modal
which is used when some tags and/or tag groups are restricted to the category,
and all other unrestricted tags should also be allowed.
Default is the same as the previous behaviour: only allow the specified set of
tags and tag groups in the category.
2019-04-04 16:40:15 -04:00
Robin Ward 111a502231 FIX: Deleting Users should work nicely with Reviewable Users
"Rejecting" a user in the queue is equivalent to deleting them, which
would then making it impossible to review rejected users. Now we store
information about the user in the payload so if they are deleted things
still display in the Rejected view.

Secondly, if a user is destroyed outside of the review queue, it will
now automatically "Reject" that queue item.
2019-04-03 16:42:39 -04:00
Robin Ward c1ea63bdc1 FIX: Reviewables should not be created for users until they are active
Conversely, if a user is deactivated the reviewable should automatically
be rejected.

Before this fix, if a user was not active they'd still show in the
review queue but without an "Approve" button which was confusing.
2019-04-03 15:25:00 -04:00
Robin Ward 6ebadaed2c FIX: Do not allow `invite_only` and `enable_sso` at the same time
This functionality was never supported but before the new review queue
it didn't have any errors. Now the combination of settings is prevented
and existing sites with sso enabled will be migrated to remove invite
only.
2019-04-02 10:26:27 -04:00
Joffrey JAFFEUX c66ae19037 FIX: ensures score is 0 and not null if no associated reviewable_scores 2019-03-29 09:13:31 -04:00
Tarek Khalil b1cb95fc23
FEATURE: Introduce ignore duration selection (#7266)
* FEATURE: Introducing new UI for tracking User's ignored or muted states
2019-03-29 10:14:53 +00:00
Robin Ward f3ed002c30 FIX: Approved is status 1 2019-03-28 15:31:14 -04:00
Robin Ward a282ade9c6 FIX: Missing events for ReviewableUser migration 2019-03-28 15:25:26 -04:00
Robin Ward fdaf17ec7c FIX: Approved should not be assumed in the migration 2019-03-28 15:22:58 -04:00
Robin Ward d873b9b2ad FIX: ReviewableUser is not created yet 2019-03-28 14:57:44 -04:00
Robin Ward b58867b6e9 FEATURE: New 'Reviewable' model to make reviewable items generic
Includes support for flags, reviewable users and queued posts, with REST API
backwards compatibility.

Co-Authored-By: romanrizzi <romanalejandro@gmail.com>
Co-Authored-By: jjaffeux <j.jaffeux@gmail.com>
2019-03-28 12:45:10 -04:00
Bianca Nenciu a9798f0c47
FEATURE: Add page for all group membership requests. (#6909) 2019-03-27 13:30:59 +02:00
Guo Xiang Tan cf9d8c1ad3 Add index for `SearchLog#created_at`.
Admin search logs filters by `created_at` so this will help.
2019-03-26 20:39:22 +08:00
Gerhard Schlager cb70a328f5 FIX: Remove duplicate site settings and add unique index 2019-03-25 17:30:53 +01:00
Gerhard Schlager 467c17da53 FIX: Make migration resilient against duplicate site settings 2019-03-25 16:57:23 +01:00
Penar Musaraj 51e08feb7e DEV: Refactor icons used in lightbox HTML
Uses <svg> elements instead of hacky CSS pseudoelements

Adds a migration to mark posts with lightboxes as needing a rebake
2019-03-22 11:52:06 -04:00
Guo Xiang Tan 6450969c7b PERF: Add index to speed up scopes for `UserSecondFactor`. 2019-03-21 15:23:36 +08:00
Rishabh ad6ad3f679 DEV: Remove SiteSetting.s3_force_path_style (#7210)
- s3_force_path_style was added as a Minio specific url scheme but it has never been well supported in our code base.
- Our new migrate_to_s3 rake task does not work reliably with path style urls too
- Minio has also added support for virtual style requests i.e the same scheme as AWS S3/DO Spaces so we can rely on that instead of using path style requests.
- Add migration to drop s3_force_path_style from the site_settings table
2019-03-20 14:58:20 +01:00
Guo Xiang Tan f0f1e878fe PERF: Add index to speed up `User#number_of_flagged_posts`.
On meta, we were seeing the query take almost 100ms.
2019-03-20 17:16:30 +08:00
Gerhard Schlager 8a99d59422 FIX: seed data classes couldn't always be found 2019-03-18 23:13:47 +01:00
Gerhard Schlager 3fd04df781
FEATURE: Locale support for seeded categories and topics (#7110) 2019-03-18 21:09:13 +01:00
Guo Xiang Tan 5e410dc5e0
FEATURE: Ability to exclude category from search results. (#7194)
This commit also adds `Category#search_priority` which sets the ground
work to enable prioritizing of posts for certain categories when searching.
2019-03-18 15:25:45 +08:00
Penar Musaraj 9334d2f4f7
FEATURE: add more granular user option levels for email notifications (#7143)
Migrates email user options to a new data structure, where `email_always`, `email_direct` and `email_private_messages` are replace by

* `email_messages_level`, with options: `always`, `only_when_away` and `never` (defaults to `always`)
* `email_level`, with options: `always`, `only_when_away` and `never` (defaults to `only_when_away`)
2019-03-15 10:55:11 -04:00
Joffrey JAFFEUX 93490fbfaf FIX: prevents duplicate error on instragam migration (#7178)
Previous instagram records didn’t have unique index, as a result it's seems like in some cases we have duplicate ids, which is causing issues with the associated_accounts which does have unique index.

It's sorted by `updated_at DESC` on the thought that the most recent one is more likely to be the one we want to keep.
2019-03-15 15:46:07 +01:00
Guo Xiang Tan cb1f909fa4 FIX: Migrate old null override for upload site settings.
For some reason, setting a text site setting to empty in the past can
result in either an empty string or null as the value. This migration
is a follow up to 1981add261.
2019-03-15 13:59:29 +08:00
Guo Xiang Tan 1981add261 FIX: Migrate old blank override for upload site settings.
If the depercated SiteSetting.logo_url was set to "" previously, it will
not have been migrated by the onceoff job which we added. As such, we
need a migration to migrate the blank override so that the default for
certain upload site settings will not be used.
2019-03-15 11:03:58 +08:00
Tarek Khalil bd6d31c9ec
FEATURE: Add `IgnoredUsersSummary` daily job (#7144)
* FEATURE: Add `IgnoredUsersSummary` daily job

## Why?

This is part of the [Ability to ignore a user feature](https://meta.discourse.org/t/ability-to-ignore-a-user/110254/8).

We want to:

1. Send an automatic group PM that goes out to moderators
2. When {x} users have Ignored the same user, threshold defined by a site setting, default of 5
3. Only send this message every X days which is defined by another site setting
2019-03-14 22:51:43 +00:00
Gerhard Schlager c34a6ba674 REFACTOR: Rename site settings to make them less confusing 2019-03-14 13:40:14 +01:00
Guo Xiang Tan b0c8fdd7da FIX: Properly support defaults for upload site settings. 2019-03-13 16:36:57 +08:00
David Taylor fc7938f7e0
REFACTOR: Migrate GoogleOAuth2Authenticator to use ManagedAuthenticator (#7120)
https://meta.discourse.org/t/future-social-authentication-improvements/94691/3
2019-03-07 11:31:04 +00:00
Gerhard Schlager ccb48bef75 UX: Make Uncategorized category less confusing
* Adds warnings to the "Edit Category" dialog
* Doesn't hide the "Security" tab on the "Edit Category" dialog anymore. Instead, it shows an explanation why permissions can't be changed.
* Makes the category name translatable
* Hides the category name from the edit dialog (it can be customized by overriding the translation)
* Creates a translation override if the category has been renamed in the past
2019-03-05 14:42:41 +01:00
Joffrey JAFFEUX 703c724cf3
REFACTOR: Migrate InstagramAuthenticator to use ManagedAuthenticator (#7081) 2019-03-04 14:54:28 +01:00
Tarek Khalil 986cc8a0fb FEATURE: Introduce Ignore user (#7072) 2019-02-27 14:49:07 +01:00
Gerhard Schlager d96d561180 Fix defect db migration
Follow-up to fc999c04b3
2019-02-08 16:02:33 +01:00
Sam 19386ec2ea FIX: old migration was loading up invalid model schema
Generally we should never be touching AR objects in migrations, this is
super risky as we may end up with invalid schema cache.

This code from 2013 did it unconditionally. This change amends it so:

1. We only load up schema if we have no choice
2. We flush the cache before and after

This makes this migration far less risky.
2019-02-07 15:16:28 +11:00
David Taylor fc999c04b3
Drop facebook_user_infos and twitter_user_infos (#6970)
Data was migrated to user_associated_accounts in 208005f and 160d29b
2019-02-05 15:12:39 +00:00
Guo Xiang Tan ac004297fa PERF: Add index on `user_id` to `single_sign_on_records`. 2019-01-30 09:34:19 +08:00
Penar Musaraj fb7f21fbe9 Clearer syntax in migration
Following up on 1c1fd205
2019-01-29 10:24:32 -05:00
David Taylor aca0b32fda
FEATURE: Allow overriding text size from a different device (#6955)
This brings the feature in line with the theme selection system
2019-01-28 11:19:50 +00:00
Arpit Jalan 244628ed98
Copy `welcome_message` translation override to `welcome_message_invite_only` (#6949)
This commit copies `login_required.welcome_message` translation override to `login_required.welcome_message_invite_only` if the `login_required.welcome_message_invite_only` override is blank. This is done so as to ensure that login page customization remains same after [this commit](93eb0a0690).
2019-01-28 13:59:21 +05:30
David Taylor a48731e359
FEATURE: Support additional metadata in theme about.json (#6944)
New `about.json` fields (all optional):
 - `authors`: An arbitrary string describing the theme authors
 - `theme_version`: An arbitrary string describing the theme version
 - `minimum_discourse_version`: Theme will be auto-disabled for lower versions. Must be a valid version descriptor.
 - `maximum_discourse_version`: Theme will be auto-disabled for lower versions. Must be a valid version descriptor.

A localized description for a theme can be provided in the language files under the `theme_metadata.description` key

The admin UI has been re-arranged to display this new information, and give more prominence to the remote theme options.
2019-01-25 14:19:01 +00:00
Gerhard Schlager d82628987a FIX: Badges appeared in wrong badge group
Some badges always appeared in the "Other" group (the default group) and some badges were always moved back into the original group during seeding.

Now badges are either in the correct, seeded group or stay in a custom group if the admin moved the badge into a custom group.
2019-01-21 21:09:33 +01:00
Penar Musaraj 77fda4dc00 Code styling fix
See 1c1fd2051f (r31953997)
2019-01-17 12:40:05 -05:00
David Taylor 880311dd4d
FEATURE: Support for localized themes (#6848)
- Themes can supply translation files in a format like `/locales/{locale}.yml`. These files should be valid YAML, with a single top level key equal to the locale being defined. For now these can only be defined using the `discourse_theme` CLI, importing a `.tar.gz`, or from a GIT repository.

- Fallback is handled on a global level (if the locale is not defined in the theme), as well as on individual keys (if some keys are missing from the selected interface language).

- Administrators can override individual keys on a per-theme basis in the /admin/customize/themes user interface.

- Theme developers should access defined translations using the new theme prefix variables:
  JavaScript: `I18n.t(themePrefix("my_translation_key"))`
  Handlebars: `{{theme-i18n "my_translation_key"}}` or `{{i18n (theme-prefix "my_translation_key")}}`

- To design for backwards compatibility, theme developers can check for the presence of the `themePrefix` variable in JavaScript

- As part of this, the old `{{themeSetting.setting_name}}` syntax is deprecated in favour of `{{theme-setting "setting_name"}}`
2019-01-17 11:46:11 +00:00
Penar Musaraj 1c1fd2051f
FEATURE: enable CSP by default on new sites (#6873)
- adds migration to enable CSP for new sites
- removes "EXPERIMENTAL" labels from setting names
- sets CSP violation report to default off
- adds CSP-related note to GTM setting
2019-01-15 08:58:46 -05:00
David Taylor 1ebd3dbbd0
FEATURE: Allow the base font size to be changed on a per-user basis (#6859) 2019-01-14 13:21:46 +00:00
Gerhard Schlager f43345b75a FIX: Remove google+ from share_links site setting
Follow-up to 2748822576
2019-01-10 21:31:20 +01:00
Joffrey JAFFEUX 1fdbf0fc9b
FIX: uses compiler version to force rebaking of themes (#6870)
Co-Authored-By: David Taylor <david@taylorhq.com>
2019-01-10 12:36:13 +01:00
Joffrey JAFFEUX f9648de897
DEV: upgrades from Ember 2.13 to Ember 3.5.1 (#6808)
Co-Authored-By: Bianca Nenciu <nbianca@users.noreply.github.com>
Co-Authored-By: David Taylor <david@taylorhq.com>
2019-01-10 11:06:01 +01:00
Guo Xiang Tan 7896c74c2b DEV: Remove use of ActiveRecord in migration.
It makes the migration brittle to changes. If I could I would go back in
time and tell the 2016 version of me that.
2019-01-10 11:10:21 +08:00
Guo Xiang Tan cb0f6d653b DEV: Minor fixes to b63b399799. 2019-01-10 09:38:22 +08:00
Saurabh Patel b63b399799 DEV: remove uploaded_meta_id column from category (#6725)
* DEV: remove uploaded_meta_id column from category

* remove uploaded_meta part
2019-01-10 09:37:21 +08:00
Sam 37088c4221 PERF: index on topic_id for user_histories table
We query this table when getting composer messages with the queries
such as:

```
SELECT  1 AS one FROM "user_histories"
WHERE "user_histories"."target_user_id" = 1 AND
 "user_histories"."action" = 9 AND
 "user_histories"."topic_id" = 105794
LIMIT 1
```

This index ensures this query remains very quick, regardless of user
history size.
2019-01-06 15:14:41 +11:00
Vinoth Kannan 75dbb98cca FEATURE: Add S3 etag value to uploads table (#6795) 2019-01-04 14:16:22 +08:00
Sam 6961a4f43e Revert "Revert "FEATURE: force rebake of all posts with images""
This reverts commit cb317430a1.
No longer needed per: 70269c7c97
2019-01-04 09:25:04 +11:00
David Taylor cb317430a1 Revert "FEATURE: force rebake of all posts with images"
This reverts commit 30a1d29a7e due to performance issues on large clusters
2019-01-03 17:52:31 +00:00
Sam 30a1d29a7e FEATURE: force rebake of all posts with images
This was done to pick up 3 changes

1. New pngquant which will result in much smaller images
2. Placeholder images which are missing from old posts
3. Retina images missing from old posts

Also picks up on Image Magick upgrade which slightly alters resize algorithm.

Rebake trickles per: `rebake_old_posts_count` site setting. (100 per 15 minutes)
2019-01-03 17:19:38 +11:00
Sam 570877da3c FEATURE: store thumbnail algorithm version in optimized image table
Previously we had no idea what algorithm generated thumbnails, this starts tracking the version.

We also bumped up the version to force all optimized images to be generated. This is important cause we recently introduced pngquant which results in much smaller images.
2019-01-03 17:07:30 +11:00
Gerhard Schlager 7977b09025 FEATURE: Activate users invited via email when invite is redeemed
Do not send an activation email to users invited via email. They
already confirmed their email address by clicking the invite link.
Users invited via link will need to confirm their email address before
they can login.
2018-12-11 00:09:53 +01:00
David Taylor 9db829134c
FIX: Use database to persist metadata during social registration (#6750)
Previously was using the cookie_store, which is limited to 4kb. This caused issues for providers sending large volumes of metadata about a user.
2018-12-10 15:10:06 +00:00
David Taylor 160d29b18a
REFACTOR: Migrate TwitterAuthenticator to use ManagedAuthenticator (#6739)
No changes to functionality. TwitterAuthenticator goes from 136 lines to 24, and all twitter-specific logic elsewhere has been deleted 🎉
2018-12-07 15:39:06 +00:00
Saurabh Patel 9e3143445b DEV:add uploaded_meta option in category for category meta image (#6724) 2018-12-07 16:24:07 +01:00
Vinoth Kannan d33d031742
FEATURE: Filter topic and post web hook events by tags (#6726)
* FEATURE: Filter topic and post web hook events by tags

* Add a spec test with unmatched tags
2018-12-05 14:44:06 +05:30
David Taylor 9248ad1905 DEV: Enable `Style/SingleLineMethods` and `Style/Semicolon` in Rubocop (#6717) 2018-12-04 11:48:13 +08:00
David Taylor 208005f9c9 REFACTOR: Migrate FacebookAuthenticator to use ManagedAuthenticator
Changes to functionality
  - Removed syncing of user metadata including gender, location etc.
    These are no longer available to standard Facebook applications.
  - Removed the remote 'revoke' functionality. No other providers have
    it, and it does not appear to be standard practice in other apps.
  - The 'facebook_no_email' event is no longer logged. The system can
    cope fine with a missing email address.

Data is migrated to the new user_associated_accounts table.
facebook_user_infos can be dropped once we are confident the data has
been migrated successfully.
2018-11-30 11:18:11 +00:00
David Taylor 534e1b1b18 DEV: Introduce Auth::ManagedAuthenticator
A generic implementation of Auth::Authenticator which stores data in the
new UserAssociatedAccount model. This should help significantly reduce the duplicated
logic across different auth providers.
2018-11-30 11:18:11 +00:00
Maja Komel 4a8f21d387 FIX: prevent minimum_required_tags on category being set to null (#6703)
* FIX: prevent minimum_required_tags on category being set to null

* add migration for NOT_NULL constraint for minimum_required_tags

* add specs
2018-11-29 18:10:14 +01:00
Kyle E. Mitchell 15e793fd3b FEATURE: Terms of Service v1.0.0
Co-authored-by: Gerhard Schlager <mail@gerhard-schlager.at>
2018-11-21 00:45:16 +01:00
Maja Komel 592f8c1637 FIX: escape sso_secret string when migrating to sso_provider_secret (#6634) 2018-11-20 15:28:37 +01:00
Régis Hanol 4459665dee
REFACTOR: use tables instead of custom fields for polls (#6359)
Co-authored-by: Guo Xiang Tan <tgx_world@hotmail.com>
2018-11-19 14:50:00 +01:00
Guo Xiang Tan 44d7249a17
Stop seeding assets for site design topic. (#6609) 2018-11-16 12:57:04 +08:00
Guo Xiang Tan 222a7c2270 Fix restores failing due to missing migration file. 2018-11-16 11:49:24 +08:00
Guo Xiang Tan df111259fe More URL site settings into a onceoff job.
* Doing it in a post migration was a bad idea
  because the migration will fail if the site
  is down while trying to download uploads
  which points to the instance. This mainly
  affects self-hosters using `discourse_docker`
  where `./launcher rebuild` will take the
  existing container down.
2018-11-14 20:29:20 +08:00
Guo Xiang Tan 777364df01 Rescue from read and connect timeouts when migration url site settings. 2018-11-14 18:46:34 +08:00
Guo Xiang Tan 52a756b98c Generate URL without CDN when migrating upload site settings. 2018-11-14 16:16:55 +08:00
Guo Xiang Tan 44391ee8ab
FEATURE: Upload Site Settings. (#6573) 2018-11-14 15:03:02 +08:00
Gerhard Schlager 24e5be3f0c FIX: Relative links in translations should work with subfolder 2018-11-08 23:31:05 +00:00
Guo Xiang Tan ee42a53436 Missing require. 2018-11-07 10:50:47 +08:00
Bianca Nenciu 2070edf889 FIX: Clarify User.group_locked_trust_level.
* Rename User.group_locked_trust_level to User.group_granted_trust_level.

* Remove the column from users table.
2018-11-07 10:27:44 +08:00
Robin Ward ec91450aae FEATURE: Track how many user flags are agreed/disagreed/ignored
Display the percentage when reviewing flags.
2018-11-01 09:59:50 -04:00
Robin Ward c2c99c7c39 FIX: Don't seed flags if ids don't exist
This can happen if you use the `replace_flags` plugin API to remove
a flag.
2018-10-25 15:36:24 -04:00
Bianca Nenciu b69652278f FEATURE: Add Wiki Editor badge. (#6511) 2018-10-19 15:30:27 +02:00
Maja Komel 27e732a58d FEATURE: allow multiple secrets for Discourse SSO provider
This splits off the logic between SSO keys used incoming vs outgoing, it allows to far better restrict who is allowed to log in using a site.

This allows for better auditing of the SSO provider feature
2018-10-15 16:03:53 +11:00
Kyle Zhao 6acdea37c4 DEV: extract inline js when baking theme fields (#6447)
* extract inline js when baking theme fields
* destroy javascript cache when destroying theme fields

This work is needed to support CSP work
2018-10-15 15:55:23 +11:00
Guo Xiang Tan 84d4c81a26 FEATURE: Support backup uploads/downloads directly to/from S3.
This reverts commit 3c59106bac.
2018-10-15 09:43:31 +08:00
Guo Xiang Tan 3c59106bac Revert "FEATURE: Support backup uploads/downloads directly to/from S3."
This reverts commit c29a4dddc1.

We're doing a beta bump soon so un-revert this after that is done.
2018-10-11 11:08:23 +08:00
Gerhard Schlager c29a4dddc1 FEATURE: Support backup uploads/downloads directly to/from S3. 2018-10-11 10:38:43 +08:00
Robin Ward a566ed42ae FEATURE: Option to disable user presence and profile
This allows users who are privacy conscious to disable the presence
features of the forum as well as their public profile.
2018-10-10 17:34:33 -04:00
Erin Kosewic 51aba32651 FEATURE: add branch option to remote theme import
* FEATURE: add branch option to remote theme import

* FIX: Add missing variable in params

* FIX: Add missing param for import_theme method

* SPEC: Add test methods for branch support in git import

* FIX: Add missing space to scss style

* Do not assume default branch as master

* Change branch field placeholder

* FIX: add missing div start tag
2018-10-09 17:01:08 +11:00
Guo Xiang Tan 40fa96777d
FEATURE: Post deployment migrations. (#6406)
This moves us away from the delayed drops pattern which
was problematic on two counts. First, it uses a hardcoded "delay for"
duration which may be too short for certain deployment strategies.
Second, delayed drop doesn't ensure that it only runs after
the latest application code has been deployed. If the migration runs
and the application code fails to deploy, running the migration after
"delay for" has been met will cause the application to blow up.

The new strategy allows post deployment migrations to be skipped if the
env `SKIP_POST_DEPLOYMENT_MIGRATIONS` is provided.

```
SKIP_POST_DEPLOYMENT_MIGRATIONS=1 rake db:migrate
-> deploy app servers
SKIP_POST_DEPLOYMENT_MIGRATIONS=0 rake db:migrate
```

To aid with the generation of a post deployment migration, a generator
has been added. Simply run `rails generate post_migration`.
2018-10-08 15:47:38 +08:00
David Taylor 9bf522f227
FEATURE: Mixed case tagging (#6454)
- By default, behaviour is not changed: tags are made lowercase upon creation and edit.

- If force_lowercase_tags is disabled, then mixed case tags are allowed.

- Tags must remain case-insensitively unique. This is enforced by ActiveRecord and Postgres.

- A migration is added to provide a `UNIQUE` index on `lower(name)`. Migration includes a safety to correct any current tags that do not meet the criteria.

- A `where_name` scope is added to `models/tag.rb`, to allow easy case-insensitive lookups. This is used instead of `Tag.where(name: "blah")`.

- URLs remain lowercase. Mixed case URLs are functional, but have the lowercase equivalent as the canonical.
2018-10-05 10:23:52 +01:00
Arpit Jalan 3e17ef0507
Merge pull request #6414 from techAPJ/user-field-unique-key
FEATURE: add external details to user fields
2018-09-20 22:29:39 +05:30
Sam df45e82377 SECURITY: only allow picking of avatars created by self (#6417)
* SECURITY: only allow picking of avatars created by self

Also adds origin tracking to all uploads including de-duplicated uploads
2018-09-19 22:33:10 -07:00
Arpit Jalan 72be638728 FEATURE: add external details to user fields 2018-09-20 08:10:51 +05:30
Guo Xiang Tan 6dbe1d832d Rename `Jobs::RecoverPostUploads` to rerun the job.
Running a migration to destroy the onceoff job doesn't work
because an instance running the old code may execute the job.
2018-09-19 21:59:44 +08:00
Guo Xiang Tan 95293a6997 Re-run `Jobs::RecoverPostUploads`.
Previously, `UploadRecovery` did not account for uploads which
may be present in links. This meant that attachments were not
recovered.
2018-09-19 12:17:01 +08:00
OsamaSayegh c7d81e2682 FIX/FEATURE: don't blow up when can't reach theme's repo, show problem themes on dashboard 2018-09-17 09:49:53 +10:00
Régis Hanol 0bfb1f83fc make migration irreversible 2018-09-14 11:09:36 +02:00
Bianca Nenciu aca195e4a7 Remove unused site setting. (#6398) 2018-09-14 07:49:32 +00:00
Guo Xiang Tan 45f092a49d Extract `badge_posts` view management logic into a service object.
* Avoid defininig a global constant and method.

Fixes https://github.com/discourse/discourse/pull/6318
2018-09-03 09:51:35 +08:00
Sam 740308675b FEATURE: erode bounce score every time an email is sent
Introduces a hidden setting (default is 0.1) that erodes bounce score
every time we send an email. This means that erratic failures are less
painful cause system auto corrects
2018-08-28 17:02:12 +10:00
Sam 9ab1fb7dfc FEATURE: correctly store width and height on uploads
Previously we used width and height for thumbnails, new code ensures

1. We auto correct width and height
2. We added extra columns for thumbnail_width and height, this is determined
 by actual upload and no longer passed in as a side effect
3. Optimized Image now stores filesize which can be used for analysis, decisions

Also

- fixes Android image manifest as a side effect
- fixes issue where a thumbnail generated that is smaller than the upload is no longer used
2018-08-28 12:59:22 +10:00
Kris faf09bb8c8 Replacing default brown category color 2018-08-24 14:18:14 -04:00
Osama Sayegh e0cc29d658 FEATURE: themes and components split
* FEATURE: themes and components split

* two seperate methods to switch theme type

* use strict equality operator
2018-08-24 11:30:00 +10:00
Bianca Nenciu 860c1c3dcd FEATURE: Automatically expire keys if not used for a configurable amount of time. (#6264) 2018-08-20 17:36:14 +02:00
Osama Sayegh 0b7ed8ffaf FEATURE: backend support for user-selectable components
* FEATURE: backend support for user-selectable components

* fix problems with previewing default theme

* rename preview_key => preview_theme_id

* omit default theme from child themes dropdown and try a different fix

* cache & freeze stylesheets arrays
2018-08-08 14:46:34 +10:00
Arpit Jalan ffc8c52bf5 FIX: store welcome topic id in custom field 2018-08-06 23:46:03 +05:30
Guo Xiang Tan 6ede7c4623 PERF: Add missing index on `EmailLog#bounced`.
* Also remove index on `user_id_and_created_at` which
  isn't being used alot.
2018-08-03 17:07:18 +08:00
Guo Xiang Tan 87537b679c Drop `reply_key`, `skipped` and `skipped_reason` from `email_logs`. 2018-07-30 11:39:28 +08:00
Arpit Jalan fc3b904e1f remove "track external right clicks" feature 2018-07-29 15:01:33 +05:30
Vinoth Kannan af5b88f8e2 Rename approval web hook event type to queued post 2018-07-26 10:29:38 +05:30
Vinoth Kannan 5059dad8f0 FEATURE: Webhook for post approval events 2018-07-25 23:43:19 +05:30
Joffrey JAFFEUX 7a3c541077
UX: Preview multiple color schemes in wizard (#6151)
It was a dropdown to provide choices of color schemes,
and only one scheme could be shown.
With this commit, multiple color scheme previews can be displayed on
one page at the same time, making admins choose color schemes more
easily.

Theme preview windows are shrinked.

Imported default color schemes.

Co-Authored-By: Misaka 0x4e21 <misaka4e21@gmail.com>
2018-07-24 09:00:20 -04:00
Sam 178cb480fa correct migration 2018-07-24 18:36:18 +10:00
Guo Xiang Tan 01a63f8b4b Drop `EmailLogs#topic_id`. 2018-07-24 15:18:57 +08:00
Guo Xiang Tan fad9c2b971 PERF: Move `EmailLog#reply_key` into new `post_reply_keys` table. 2018-07-24 13:51:53 +08:00
Guo Xiang Tan ae8b0a517f PERF: Split skipped email logs into a seperate table. 2018-07-24 13:14:37 +08:00
Guo Xiang Tan 09a00016ad PERF: Tune indexes to speed up lookup for bounced email. 2018-07-19 19:03:00 +08:00
Guo Xiang Tan 3553375dd2 PERF: Store `EmailLog#reply_key` as `uuid` data type. 2018-07-17 17:05:42 +08:00
Guo Xiang Tan a6e9f4f44b FIX: DROP NOT NULL instead of changing all the rows in the table.
```
change_column :topic_views, :ip_address, :inet, null: true
```

translates to

```
"ALTER TABLE \"topic_views\" ALTER COLUMN \"ip_address\" TYPE inet"
```

which locks the whole table while the migration is taking place.
2018-07-17 11:03:41 +08:00
David Taylor f4c902b434 FIX: set default on theme.key, if it has not yet been dropped (#6095) 2018-07-17 08:40:35 +10:00
Guo Xiang Tan 59e72b466a Add uniq_ip_or_user_id_topic_views index concurrently. 2018-07-16 23:45:38 +08:00
Guo Xiang Tan c0c263405a
PERF: Store `EmailLog#bounce_key` as `uuid` data type. (#6093)
PERF: Store `EmailLog#bounce_key` as `uuid` data type.
2018-07-16 20:05:54 +08:00
Sam ac0053f491 FEATURE: navigate to first post and auto bump category settings
### navigate_to_first_post_after_read setting for categories

When enabled on categories logged on users will return to OP after
reading the entire category. (useful for documentation categories)

### num_auto_bump_daily

Set a number of topics that will automatically bump daily on a category.

- Every 15 minutes we will check if any category has this setting
- Categories with the setting are shuffled
- We exclude pinned, closed, category description and archived topics
- Maximum of 1 topic for the list of categories is bumped till limit reached per category
- We always try to bump oldest first
- Limit is elastic using a RateLimiter that ensures that we only bump N per day

Also some minor organisation on category settings

Froze strings on category.rb
2018-07-16 18:10:35 +10:00
Guo Xiang Tan 6761f8ecbf PERF: Add missing index on `user_id` for `user_second_factors` table. 2018-07-16 14:21:07 +08:00
OsamaSayegh decf1f27cf FEATURE: Groundwork for user-selectable theme components
* Phase 0 for user-selectable theme components

- Drops `key` column from the `themes` table
- Drops `theme_key` column from the `user_options` table
- Adds `theme_ids` (array of ints default []) column to the `user_options` table and migrates data from `theme_key` to the new column.
- Removes the `default_theme_key` site setting and adds `default_theme_id` instead.
- Replaces `theme_key` cookie with a new one called `theme_ids`
- no longer need Theme.settings_for_client
2018-07-12 14:18:21 +10:00
Vinoth Kannan b49c662aa4 Remove the clean up migration for vote post action 2018-07-10 16:57:12 +05:30
Guo Xiang Tan 21f333654c REFACTOR: Reduce dependency on the post stream in `TopicView`.
This will allow us to drop the post stream from the payload for
mega-topics. On smaller topics, the extra query is fast because
of an existing index.
2018-07-10 15:53:00 +08:00
Guo Xiang Tan 96aca6d7e6
Remove legacy vote post action code. (#6009) 2018-07-09 16:54:18 +08:00
Leo McArdle c3129444ea FIX: allow multiple secondary emails 2018-07-03 18:21:35 +08:00
Guo Xiang Tan c1972f8438 PERF: Add index_topic_id_percent_rank_on_posts.
Speeds up the `Posts#summary` mode query for large topics.
2018-06-21 09:49:15 +08:00
Sam cb824a6b33 DEV: remove all calls to SqlBuilder use DB.build instead
This is part of the migration to mini_sql, SqlBuilder.new is being
deprecated and replaced with DB.build
2018-06-20 17:53:49 +10:00
riking 1de0ef137e FIX: SearchLog: Do not store IP of logged in users 2018-06-20 10:44:11 +10:00
Sam 5f64fd0a21 DEV: remove exec_sql and replace with mini_sql
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
Sam Saffron 003df147a5 workaround fabricator not working correctly 2018-06-12 14:44:59 +10:00
riking 4b2e2ea8ef Change TopicViewItem to allow null IP address 2018-06-12 12:43:56 +10:00
Arpit Jalan f9ab3848ed FEATURE: support disabling emails for non-staff users 2018-06-07 18:31:08 +05:30
Sam 799081e846 attempt all column dropping upfront prior to loading cache 2018-06-07 20:30:19 +10:00
Sam e031b06afa attempt to reset column info on user table explicitly 2018-06-07 20:20:15 +10:00
Sam 8b88b71309 Attempt very aggressively to clear cache 2018-06-07 20:10:52 +10:00
Sam 855b0df085 attempt to reset cache 2018-06-07 16:18:45 +10:00
Sam 24c7d2913b DEV: organise fixture so all drops happen at the end 2018-06-07 15:53:54 +10:00
Guo Xiang Tan 78c705f5de Remove migration code that is no longer valid. 2018-06-07 08:57:01 +08:00
riking 501bc0e9af FIX: UserProfileView: Do not log IP of logged-in users 2018-05-25 15:54:14 +10:00
riking 9b0efe9c84 FIX: TopicLinkClick: do not log IP of logged in users 2018-05-22 16:05:38 +10:00
Arpit Jalan 9f422c93f6 FIX: restrict updates on `confirm_old_email` email templates 2018-05-19 12:19:59 +05:30
Régis Hanol e9abdaebbe UX: show an enveloppe icon when a badge is used in messages
- the badge count now includes messages
- only show the message badges to admins
2018-05-14 19:02:00 +02:00
Arpit Jalan 27efa45800 fix deprecated setting migration 2018-05-09 08:14:45 +05:30
Arpit Jalan 83245aa508 FIX: better handling of invite links after they are redeemed
FIX: deprecate invite_passthrough_hours setting
2018-05-08 20:17:57 +05:30
Misaka 0x4e21 ff6be3c2e3 FEATURE: add profile_background fields into SSO (#5701)
Add profile_background and card_background fields into Discourse SSO.
2018-05-07 10:03:26 +02:00
Jeff Wong 91b31860a1
Feature: Push notifications for Android (#5792)
* Feature: Push notifications for Android

Notification config for desktop and mobile are merged.

Desktop notifications stay as they are for desktop views.

If mobile mode, push notifications are enabled.

Added push notification subscriptions in their own table, rather than through
custom fields.

Notification banner prompts appear for both mobile and desktop when enabled.
2018-05-04 15:31:48 -07:00
Sam 3a06cb461e FEATURE: remove support for legacy auth tokens 2018-05-04 10:12:10 +10:00
Neil Lalonde bd77795d7a REFACTOR: move support for user card badge images to a plugin discourse-user-card-badges 2018-04-26 13:25:24 -04:00
Neil Lalonde 70f2c5d3fd FEATURE: move staff tags setting to tag group settings 2018-04-20 15:34:23 -04:00
Arpit Jalan 91bf10bd12 FIX: create upload record for exported csv files 2018-04-20 00:27:49 +05:30
Arpit Jalan a16b616861 FEATURE: webhook for flag events 2018-04-13 07:47:58 +05:30
Arpit Jalan 9ca6ebe8fe FEATURE: enforce tagging on categories 2018-04-11 07:15:24 +05:30
Vinoth Kannan 434cbc649f FEATURE: Webhook for tag events 2018-04-04 17:49:20 +05:30
Vinoth Kannan efb19dbdaf
Merge pull request #5705 from discourse/new_webhooks
FEATURE: Webhook for group and category events
2018-04-02 10:53:21 +05:30
Robin Ward 6aef8f9cd9 Add an index on category_id to shared drafts 2018-03-28 14:05:09 -04:00
Neil Lalonde 7311023a52
Merge pull request #5700 from discourse/crawl-block
FEATURE: control web crawlers access with white/blacklist
2018-03-27 15:06:03 -04:00
Sam 6c70925c6f PERF: add missing index for akismet
Note, current practice if for plugins to submit PRs to core for any
migrations required for plugins, so we can better control schema.

Especially if core tables are being touched.

In this case index has close to zero cost unless akismet is installed

This reduces the akismet admin query from 20ms on every new page load to 0.5ms
2018-03-27 17:36:13 +11:00
Vinoth Kannan e7407d0adc FEATURE: Webhook for group and category events 2018-03-27 11:53:35 +05:30
Neil Lalonde f2c060bdf2 FEATURE: option for tags in a tag group to be visible only to staff 2018-03-26 17:05:09 -04:00
Robin Ward 58ec4d04c0 FIX: Rails has bugs when a `has_one` has no `id` column :( 2018-03-23 12:25:10 -04:00
Neil Lalonde ced7e9a691 FEATURE: control which web crawlers can access using a whitelist or blacklist 2018-03-22 15:41:02 -04:00
Gerhard Schlager 43f63c435d FIX: Drop unused tables with a delay 2018-03-21 16:45:44 +01:00
Gerhard Schlager eebe1d8c56 Allow delayed dropping and renaming of tables 2018-03-21 12:05:12 +01:00
Sam 6a3c8fe69c FEATURE: protect against accidental column or table drops
Often we need to amend our schema, it is tempting to use
drop_table, rename_column and drop_column to amned schema
trouble though is that existing code that is running in production
can depend on the existance of previous schema leading to application
breaking until new code base is deployed.

The commit enforces new rules to ensure we can never drop tables or
columns in migrations and instead use Migration::ColumnDropper and
Migration::TableDropper to defer drop the db objects
2018-03-21 15:43:32 +11:00
Robin Ward b9abd7dc9e FEATURE: Shared Drafts
This feature can be enabled by choosing a destination for the
`shared drafts category` site setting.

* Staff members can create shared drafts, choosing a destination
category for the topic when it is published.

* Shared Drafts can be viewed in their category, or above the
topic list for the destination category where it will end up.

* When the shared draft is ready, it can be published to the
appropriate category by clicking a button on the topic view.

* When published, Drafts change their timestamps to the current
time, and any edits to the original post are removed.
2018-03-20 17:15:26 -04:00
Sam 39e679d3cb FEATURE: allow themes to live in private git repos
This feature allows themes sourced from git to live on private
servers, it automatically generates key pairs.
2018-03-09 16:14:38 +11:00
Guo Xiang Tan 10d6101eae Fix incorrect function name. 2018-03-08 15:52:07 +08:00
Guo Xiang Tan 3ef5661608 Clean up unused function left in the database. 2018-03-08 15:25:10 +08:00
Gerhard Schlager 8e48b339fa Drop unused tables (#5630) 2018-03-05 17:27:30 -05:00
OsamaSayegh 282f53f0cd FEATURE: Theme settings (2) (#5611)
Allows theme authors to specify custom theme settings for the theme. 

Centralizes the theme/site settings into a single construct
2018-03-04 19:04:23 -05:00
Neil Lalonde 3313072957 Remove censored_pattern site setting, which is replaced by watched words 2018-02-26 16:29:27 -05:00
Guo Xiang Tan 4ac5fc8cd3 Fix incorrect data type for `SiteSetting.google_oauth2_prompt`. 2018-02-23 12:20:21 +08:00
Sam 720e1965e3 FEATURE: add category suppress from latest
In the past we used suppress_from_homepage, it had mixed semantics
it would remove from category list if category list was on home and
unconditionally remove from latest.

New setting explicitly only removes from latest list but leaves the
category list alond
2018-02-22 09:56:35 +11:00
Guo Xiang Tan 14f3594f9f Review Changes for f4f8a293e7. 2018-02-21 14:55:49 +08:00
Jeff Wong f4f8a293e7 FEATURE: Implement 2factor login TOTP
implemented review items.

Blocking previous codes - valid 2-factor auth tokens can only be authenticated once/30 seconds.
I played with updating the “last used” any time the token was attempted but that seemed to be overkill, and frustrating as to why a token would fail.
Translatable texts.
Move second factor logic to a helper class.
Move second factor specific controller endpoints to its own controller.
Move serialization logic for 2-factor details in admin user views.
Add a login ember component for de-duplication
Fix up code formatting
Change verbiage of google authenticator

add controller tests:
second factor controller tests
change email tests
change password tests
admin login tests

add qunit tests - password reset, preferences

fix: check for 2factor on change email controller
fix: email controller - only show second factor errors on attempt
fix: check against 'true' to enable second factor.

Add modal for explaining what 2fa with links to Google Authenticator/FreeOTP

add two factor to email signin link

rate limit if second factor token present

add rate limiter test for second factor attempts
2018-02-21 09:04:07 +08:00
Neil Lalonde 76c309fe6b PERF: a faster way to count tags used per category 2018-02-12 15:16:47 -05:00
Sam Saffron fb91d24deb PERF: add index on user email logs
This index makes it much faster to figure out if a user reached
email limit on sites where lots of emails are skipped
2018-02-07 11:33:28 -05:00
Arpit Jalan f88b8a8945 rename 'default_email_private_messages' to 'default_email_personal_messages' 2018-02-01 13:25:29 +05:30
Arpit Jalan 6be536ca50 rename 'max_private_messages_per_day' to 'max_personal_messages_per_day' 2018-02-01 13:25:29 +05:30
Arpit Jalan 7cda3a37af rename 'private_email_time_window_seconds' to 'personal_email_time_window_seconds' 2018-02-01 13:25:29 +05:30
Arpit Jalan 7e48c47d37 rename 'enable_private_email_messages' to 'enable_personal_email_messages' 2018-02-01 13:25:29 +05:30
Arpit Jalan ff0376a80b rename 'enable_private_messages' to 'enable_personal_messages' 2018-02-01 13:25:29 +05:30
Arpit Jalan 25ec077eca rename 'min_private_message_{post/title}_length' to 'min_personal_message_{post/title}_length' 2018-02-01 13:25:29 +05:30
Guo Xiang Tan 55f47491a4 Remove stale GA site setting records from the DB. 2018-01-27 09:15:46 +08:00
Robin Ward 6b04967e2f FEATURE: Staff members can lock posts
Locking a post prevents it from being edited. This is useful if the user
has posted something which has been edited out, and the staff members don't
want them to be able to edit it back in again.
2018-01-26 14:01:30 -05:00
Guo Xiang Tan 9c66473c4c Add index for `EmailLog#topic_id` and `EmailLog#post_id`. 2018-01-11 17:28:14 +08:00
Arpit Jalan 84026ad1fd FIX: check for existing translation keys before replacing blocked with silenced 2017-12-28 18:58:18 +05:30
Arpit Jalan 5ca558c7c5 FIX: add migration to replace 'blocked_by_staff' with 'silenced_by_staff' 2017-12-28 18:11:39 +05:30
Sam 67aecff59c FEATURE: store twitter supplied email for auditing 2017-12-14 15:54:32 +11:00
Arpit Jalan 895cc0666b FIX: replace invitee_name to inviter_name in translation overrides 2017-12-13 18:17:52 +05:30
typeoneerror d568402102 Updates api_test_seeds for multiple auth_token support (#5414) 2017-12-09 08:56:11 +11:00
Arpit Jalan e3925278e2 FEATURE: support search click through tracking for user, category and tags
https://meta.discourse.org/t/search-logs-page/73281/11?u=techapj

This commit adds following features:

- support for tracking click through to user, tag and category
- new filter for search type (header, full page)

This commit also removes "most viewed topic" field from search logs page because we are now tracking multiple click through entities, so topic is not a special entity anymore. This also improves query perf. The query now takes `20.5ms` to runs, as opposed to `655.9ms` previously.
2017-12-01 12:04:55 +05:30
Robin Ward 77f90876d3 REFACTOR: Track manual locked user levels separately from groups 2017-11-27 11:23:44 -05:00
Gerhard Schlager 8f6d35aa59 FEATURE: category setting for mailinglist mirror 2017-11-17 15:29:14 +01:00
Neil Lalonde b37e40eea9 FEATURE: show read time in last 60 days 2017-11-16 15:46:51 -05:00
Robin Ward 971e302ff2 FEATURE: Support an end date for user silencing 2017-11-14 13:20:19 -05:00
Robin Ward 1f14350220 Rename "Blocked" to "Silenced" 2017-11-10 14:10:27 -05:00
Michael Howell 38b8d68c68 FEATURE: Allow the user to select a custom home page (#5268)
* Add user_home configuration option

* Use the new user_home preference to actually show the right home page

* Fix trailing whitespace

* Update user_option_serializer.rb

* Fix JavaScript default homepage tests

* Use an object instead of a giant switch

* Remove trailing whitespace

* Make the default `user_home` set to `null` instead of `0`

* Rename user_home to homepage_id
2017-11-10 06:45:19 +11:00
Guo Xiang Tan b3237d37f0 Drop unused email column from users table. 2017-11-07 10:12:33 +08:00
Guo Xiang Tan 79de10b212 FEATURE: Allow users to disable new PMs.
https://meta.discourse.org/t/is-it-possible-to-disable-private-messaging-for-a-specific-user/46391
2017-10-19 12:32:55 +08:00
Guo Xiang Tan eb85922083 Revert "Remove unnecessary `SiteSetting.refresh!`."
This reverts commit 4556e95837.
2017-10-12 17:43:58 +08:00
Guo Xiang Tan 4556e95837 Remove unnecessary `SiteSetting.refresh!`.
* It is already `refresh!`ed in our initializer.
2017-10-12 17:15:26 +08:00
Neil Lalonde 1faae3c765 rename forgot_password_strict to hide_email_address_taken 2017-10-03 15:28:31 -04:00
Guo Xiang Tan 77d4c4d8dc Fix all the errors to get our tests green on Rails 5.1. 2017-09-25 13:48:58 +08:00
Régis Hanol 8a935a4b5f FEATURE: new badges when visiting the forum for 10, 100 and 365 consecutive days 2017-09-06 22:35:08 +02:00