Commit Graph

68 Commits

Author SHA1 Message Date
David Taylor 8fd46c04ea
Drop flash video onebox (#12261)
Flash was discontinued by Adobe at the end of 2020. There is no need to continue OneBox support for it
2021-03-02 17:11:14 +00:00
David Taylor b22ea7911c
DEV: Drop old SSO site setting rows from the database (#12148)
These were copied to their new names in 821bb1e8cb
2021-02-19 19:05:49 +00:00
Sam 58de9e85be
FIX: ensure corrected migration runs (#12137)
Some instances may have ran earier version of the migration. Ensure newer
one runs instead.
2021-02-19 11:48:32 +11:00
Sam 4ef642b300
FIX: optimise MoveNewSinceToTable (#12136)
* FIX: optimise MoveNewSinceToTable

Avoids shuffling all ids around to the app (only use min / max)
Ensure the query for boundaries is ordered by user_id
2021-02-19 11:35:52 +11:00
Dan Ungureanu bddf94c0ab
FIX: Delete topic timers far in the future (#12125)
The migration used to fail because the same duration in minutes was out
of the integer range. The '20 years' limit was introduced in e0f0fe5.
2021-02-18 14:18:43 +02:00
Krzysztof Kotlarek 7829558c6d
FIX: dismiss new when topic_user exists without last read (#12103)
The bug was mentioned on meta: https://meta.discourse.org/t/pressing-dismiss-new-doesnt-clear-new-topics/179858

Problem is that sometimes the user has TopicUser records with `last_read_post_number` set as NULL. In that case, the topic is still "new" to them and should be dismissed when they click dismiss button.

In addition, I added that condition to post_migration and bumped the number to fix existing records. Migration is written to be idempotent so it will make no harm to already deployed instances.
2021-02-18 10:39:05 +11:00
Martin Brennan 9f0f801ae3
FIX: Do not mark group_flair images as secure on upload (#12081)
See https://meta.discourse.org/t/secure-media-uploads-breaks-group-flair-image/173671/4

Group flair image uploads definitely do not need to be secure.
2021-02-16 12:34:03 +10:00
Krzysztof Kotlarek ad3ec5809f
FIX: Dismiss new with better migration (#12062)
Original PR was reverted because of broken migration https://github.com/discourse/discourse/pull/12058

I fixed it by adding this line
```
          AND topics.id IN(SELECT id FROM topics ORDER BY created_at DESC LIMIT :max_new_topics)
```

This time it is left joining a limited amount of topics. I tested it on few databases and it worked quite smooth
2021-02-15 08:50:33 +11:00
Martin Brennan 18da1d5b07
FIX: Topic timer duration_minutes was not backfilled correctly (#12004)
Because of a where clause of duration_minutes != duration, where
duration_minutes was NULL, the previous migration to fill the new
duration_minutes column failed. This corrects the failed migration
by just running the update where duration_minutes is NULL and duration
IS NOT NULL.

Previous commit is 4af77f1
2021-02-08 09:33:08 +10:00
Mark VanLandingham 809274fe0d
DEV: Replace 'processed' column on notifications with new table (#11864) 2021-01-27 10:29:24 -06:00
Régis Hanol cd3d24ed8c
FIX: move post_search_data migration into onceoff job (#11851)
And reduce the size of the batches to 100k.

That should hopefully make the migrations run smoother...
2021-01-26 16:29:00 +01:00
Régis Hanol c56ba6c9bd
PERF: batch expensive post-migration (#11845)
Run the 'MigrateSearchDataAfterDefaultLocaleRename' post migration in batches of 500k records.

This will hopefully prevent any potential deadlocks on large tables.
2021-01-25 22:58:58 +01:00
Gerhard Schlager e65c5b0aad
PERF: Migrate search data after locale rename (#11831)
The default locale `en_US` has been renamed into `en`. This tries to migrate existing search data to avoid resource intensive reindexing.
2021-01-25 14:30:17 +01:00
David Taylor e8452a55a6
DEV: Drop github_user_infos table (#11181)
Follow-up to cf21de0e7a
2020-11-10 11:33:27 +00:00
Jarek Radosz 2f4a1ff61b
DEV: Update rubocop-discourse from 2.3.2 to 2.4.0 (#11079)
Also fixes whitespace related issues raised by rubocop.
2020-10-30 15:04:29 +01:00
Guo Xiang Tan 9b75d95fc6 PERF: Keep track of first unread PM and first unread group PM for user.
This optimization helps to filter away topics so that the joins on
related tables when querying for unread messages is not expensive.
2020-09-09 14:05:41 +08:00
Guo Xiang Tan 87de8948c0
DEV: Drop search index on non-pm posts.
The problem with this index is that on sites with a high non-pm to pm
posts ratio, the index is esstentially duplicating the existing index on
`PostSearchData#search_data`. If the site is huge, the index ends up
taking up more diskspace.
2020-08-21 07:21:34 +08:00
Sam Saffron 54cf3c6766
PERF: Drop index idx_regular_post_search_data concurrently
This can slightly help with the drop command.

That said if a giant vacuum is running we may still time out.
2020-08-20 13:39:46 +10:00
Sam Saffron 628319aad3
PERF: drop idx_regular_post_search_data during migration
Rebuilding this index while amending the boolean is very expensive.

Avoid this work
2020-08-20 12:48:48 +10:00
Sam Saffron d2c504ea86
PERF: Improve performance of post_search_data migration
Very large batches can take an enormous amount of time due to churn

Limiting to 200k changes at a time gives us a far larger chance of finishing
the job without timing out or deadlocking.
2020-08-20 08:45:04 +10:00
Sam Saffron fcfaa8b063
PERF: Ensure transaction is of minimal size
A giant transaction in a post migration can be very risky.

This splits the large amount of work this migration needs to do into 2 parts:

1. A re-runnable cleanup job prior to transaction
2. A minimally sized transaction to add the database constraint

This avoids large amounts of churn on the table
2020-08-19 17:15:14 +10:00
Guo Xiang Tan 7e414da0d9
DEV: Fix lint. 2020-08-18 16:59:57 +08:00
Guo Xiang Tan 2161abfabd
DEV: Move data migration of `PostSearchData#private_message` into post_migration.
Follow-up to 92b7fe4c62
2020-08-18 16:46:14 +08:00
Krzysztof Kotlarek 14003abc37
FIX: Improve allowed_path column migration (#10321)
Because previous migration was already deployed and some databases were already migrated, I needed to add some conditions to the migration.

Previous migration - https://github.com/discourse/discourse/blob/master/db/post_migrate/20200629232159_rename_path_whitelist_to_allowed_paths.rb

What will happen in a scenario when previous migration was not run.
1. column allowed_paths will be created
2. allowed_path will be populated with data from path_whitelist
3. path_whitelist column will be dropped

What will happen in a scenario when previous migration was already run.
1. column allowed_paths will not be created because already exists - `unless column_exists?(:embeddable_hosts, :allowed_paths)`
2. Data will not be copied because path_whitelist is missing - `if column_exists?(:embeddable_hosts, :path_whitelist) && column_exists?(:embeddable_hosts, :allowed_paths)`
3. path_whitelist column deletion will be skipped - `if column_exists?(:embeddable_hosts, :path_whitelist)`
2020-07-28 13:31:51 +10:00
Krzysztof Kotlarek e0d9232259
FIX: use allowlist and blocklist terminology (#10209)
This is a PR of the renaming whitelist to allowlist and blacklist to the blocklist.
2020-07-27 10:23:54 +10:00
Martin Brennan 41b43a2a25
FEATURE: Add "delete on owner reply" bookmark functionality (#10231)
This adds an option to "delete on owner reply" to bookmarks. If you select this option in the modal, then reply to the topic the bookmark is in, the bookmark will be deleted on reply.

This PR also changes the checkboxes for these additional bookmark options to an Integer column in the DB with a combobox to select the option you want.

The use cases are:

* Sometimes I will bookmark the topics to read it later. In this case we definitely don’t need to keep the bookmark after I replied to it.
* Sometimes I will read the topic in mobile and I will prefer to reply in PC later. Or I may have to do some research before reply. So I will bookmark it for reply later.
2020-07-21 10:00:39 +10:00
David Taylor c74f2b949f
DEV: Correct historical posts table schema discrepancies (#10014)
This is a no-op for modern installations. It only affects older sites.

Follow-up to f7d676dce1

This needs to be done in a transaction, because we need to drop and recreate the badge_posts view.
2020-06-11 11:45:46 +01:00
Dan Ungureanu 5bfe1ee4f1
FEATURE: Improve UX support for multiple email addresses (#9691) 2020-06-10 19:11:49 +03:00
Arpit Jalan 274db95fb2
FIX: MigrateInviteRedeemedDataToInvitedUsers should be normal migration (#10013) 2020-06-10 16:09:36 +05:30
Arpit Jalan 3094459cd9
FEATURE: multiple use invite links (#9813) 2020-06-09 20:49:32 +05:30
David Taylor 75b1298e99
DEV: Drop unused image_url column from posts and topics (#9953)
This has been superseded by image_upload_id. The image_url value in API responses is now generated dynamically from the upload record.
2020-06-02 16:21:38 +10:00
Michael Brown d9a02d1336
Revert "Revert "Merge branch 'master' of https://github.com/discourse/discourse""
This reverts commit 20780a1eee.

* SECURITY: re-adds accidentally reverted commit:
  03d26cd6: ensure embed_url contains valid http(s) uri
* when the merge commit e62a85cf was reverted, git chose the 2660c2e2 parent to land on
  instead of the 03d26cd6 parent (which contains security fixes)
2020-05-23 00:56:13 -04:00
Jeff Atwood 20780a1eee Revert "Merge branch 'master' of https://github.com/discourse/discourse"
This reverts commit e62a85cf6f, reversing
changes made to 2660c2e21d.
2020-05-22 20:25:56 -07:00
Martin Brennan c0779df99d
FIX: Remove access control post FK from uploads (#9853) 2020-05-22 11:20:25 +10:00
Martin Brennan e334133294
PERF: Remove foreign keys from bookmarks (#9837)
As a general rule we do not add FKs in Discourse and instead rely on consistency checks at weekly/daily intervals
2020-05-20 10:45:49 +10:00
Kane York 869f9b20a2
PERF: Dematerialize topic_reply_count (#9769)
* PERF: Dematerialize topic_reply_count

It's only ever used for trust level promotions that run daily, or compared to 0. We don't need to track it on every post creation.

* UX: Add symbol in TL3 report if topic reply count is capped

* DEV: Drop user_stats.topic_reply_count column
2020-05-14 15:42:00 -07:00
Sam Saffron 53e0d8fbab
DEV: correct drop logic for columns in post table
We need to recreate the badge_posts view each time we change
column on the posts table.

The p.* is auto expanded on view create, cascade should not have
been used
2020-04-30 12:58:09 +10:00
Sam Saffron 310a7edee5
DEV: remove unused columns from posts and topics
avg_time on posts and topics have not been used in a year.

This uses a re-runnable ddl transaction diasabled migration to
drop the column, cause it touchs very high traffic table and may
deadlock
2020-04-30 11:22:33 +10:00
Sam Saffron 6a18c9aa0b
Revert "FEATURE: enforce_canonical_emails site setting"
This reverts commit 6f9177e2ed.

We decided on a completely different approach to the problem.

Instead we will let blocked emails be treated as canonical.
2020-04-24 13:52:06 +10:00
Vinoth Kannan df0c386f8a
UX: drop the `automatic_membership_retroactive` column from groups model. (#9430) 2020-04-22 22:07:39 +05:30
Dan Ungureanu 4e5f9d4cd1
DEV: Drop 'key' column from user_api_keys (#9388) 2020-04-22 12:13:19 +03:00
Martin Brennan b79ea986ac
FEATURE: High priority bookmark reminder notifications (#9290)
Introduce the concept of "high priority notifications" which include PM and bookmark reminder notifications. Now bookmark reminder notifications act in the same way as PM notifications (float to top of recent list, show in the green bubble) and most instances of unread_private_messages in the UI have been replaced with unread_high_priority_notifications.

The user email digest is changed to just have a section about unread high priority notifications, the unread PM section has been removed.

A high_priority boolean column has been added to the Notification table and relevant indices added to account for it.

unread_private_messages has been kept on the User model purely for backwards compat, but now just returns unread_high_priority_notifications count so this may cause some inconsistencies in the UI.
2020-04-01 09:09:20 +10:00
Gerhard Schlager ea11ad4d99 DEV: Drop unused columns 2020-01-27 15:28:56 +01:00
Gerhard Schlager cb54bf4f45 FIX: Don't cause exceptions due to rename of `reply_id` column
This syncs the value of the `reply_id` column into the `reply_post_id` column until all servers have been deployed and the post migrations ran.

Follow-up to ab07b945c2
2020-01-17 19:43:00 +01:00
David Taylor e8beccdaa4 DEV: Drop unused api_keys.key column
This completes the migration to hashed API keys which was started in 4c9ca24ccf
2019-12-19 11:21:49 +00:00
David Taylor 88dce41be1 DEV: Drop unused google and instagram auth_info tables
Information was migrated to user_associated_accounts in fc7938f7e0 and 703c724cf3
2019-11-29 16:28:39 +00:00
Sam Saffron f49ef851df FIX: do not insert conflict rows into category user
This could fail under certain race conditions
2019-11-19 12:23:11 +11:00
Vinoth Kannan 3bb7ad4be1
FEATURE: remove support for 'suppress_from_latest' category setting. (#8308) 2019-11-18 12:28:35 +05:30
Krzysztof Kotlarek 6e1fe22a9d
FEATURE: Dismiss new per category (#8330)
Ability to dismiss new topics per category.
2019-11-14 11:16:13 +11:00
Gerhard Schlager 7cb51d0e40 FIX: Create readonly functions during backup
Temporarily recreate already dropped functions in the discourse_functions schema in order to allow restoring of backups which still reference dropped functions.
2019-08-09 11:39:46 +02:00