Commit Graph

1411 Commits

Author SHA1 Message Date
Andrei Prigorshnev 033f72c65f
FEATURE: user status emoji (#17025) 2022-06-22 18:15:33 +04:00
Vinoth Kannan deee3c6f02
DEV: drop unused column `flair_url` from groups table. (#17179)
It's already included in the `ignored_columns` list in the group model. 03ffb0bf27/app/models/group.rb (L9)

Also, removed the `MigrateGroupFlairImages` onceoff job and spec.
2022-06-22 00:15:05 +05:30
Osama Sayegh 5176c689e9
UX: Change wording for 'regular' categories to 'normal' (#17134)
At some point in the past we decided to rename the 'regular' notification state of topics/categories to 'normal'. However, some UI copy was missed when the initial renaming was done so this commit changes the spots that were missed to the new name.
2022-06-20 06:49:33 +03:00
David Taylor 410ab5d587
FIX: Skip invalid values in site setting upload references migration (#17138) 2022-06-19 19:02:55 +01:00
Martin Brennan 4d3c1ceb44
FEATURE: Log the SMTP response in EmailLog (#17056)
When sending emails with delivery_method_options -> return_response
set to true, the SMTP sending code inside Mail will return the SMTP
response when calling deliver! for mail within the app. This commit
ensures that Email::Sender captures this response if it is returned
and stores it against the EmailLog created for the sent email.

A follow up PR will make this visible within the admin email UI.
2022-06-15 10:28:30 +10:00
David Taylor 002d62b847
FIX: Cleanup invalid historic site setting data (#17049)
`selectable_avatars_urls` contains invalid data (it's a backup from 20200810194943_change_selectable_avatars_site_setting.rb)

This migration is deliberately backdated so that it runs before `20220330160747_copy_site_settings_uploads_to_upload_references`
2022-06-09 09:51:29 +01:00
Martin Brennan 20f118c4c9
FIX: Handle empty string in theme_settings for upload_references (#17047)
Follow up to 9db8f00b3d,
the theme_settings.value field is not an integer and so
can be '', we need to account for this in the migration
otherwise we get this error:

> PG::InvalidTextRepresentation: ERROR:  invalid input syntax for type integer: ""
2022-06-09 10:19:44 +02:00
Bianca Nenciu 9db8f00b3d
FEATURE: Create upload_references table (#16146)
This table holds associations between uploads and other models. This can be used to prevent removing uploads that are still in use.

* DEV: Create upload_references
* DEV: Use UploadReference instead of PostUpload
* DEV: Use UploadReference for SiteSetting
* DEV: Use UploadReference for Badge
* DEV: Use UploadReference for Category
* DEV: Use UploadReference for CustomEmoji
* DEV: Use UploadReference for Group
* DEV: Use UploadReference for ThemeField
* DEV: Use UploadReference for ThemeSetting
* DEV: Use UploadReference for User
* DEV: Use UploadReference for UserAvatar
* DEV: Use UploadReference for UserExport
* DEV: Use UploadReference for UserProfile
* DEV: Add method to extract uploads from raw text
* DEV: Use UploadReference for Draft
* DEV: Use UploadReference for ReviewableQueuedPost
* DEV: Use UploadReference for UserProfile's bio_raw
* DEV: Do not copy user uploads to upload references
* DEV: Copy post uploads again after deploy
* DEV: Use created_at and updated_at from uploads table
* FIX: Check if upload site setting is empty
* DEV: Copy user uploads to upload references
* DEV: Make upload extraction less strict
2022-06-09 09:24:30 +10:00
Andrei Prigorshnev 5c596273a0
FEATURE: user status (#16875) 2022-05-27 13:15:14 +04:00
Martin Brennan 5ebad3be0a
FIX: Further refine duplicate bookmark delete query (#16931)
In f00e282067 we added this
DELETE query to delete duplicate for_topic bookmarks, we
just need this further refinement to the WHERE clause to
avoid deleting post bookmarks.
2022-05-27 08:22:23 +10:00
Daniel Waterworth f00e282067
FIX: Delete extraneous topic bookmarks (#16926)
Otherwise the migration can fail

Co-authored-by: David Taylor <david@taylorhq.com>
2022-05-26 11:58:15 -05:00
Martin Brennan fcc2e7ebbf
FEATURE: Promote polymorphic bookmarks to default and migrate (#16729)
This commit migrates all bookmarks to be polymorphic (using the
bookmarkable_id and bookmarkable_type) columns. It also deletes
all the old code guarded behind the use_polymorphic_bookmarks setting
and changes that setting to true for all sites and by default for
the sake of plugins.

No data is deleted in the migrations, the old post_id and for_topic
columns for bookmarks will be dropped later on.
2022-05-23 10:07:15 +10:00
David Taylor 991b62b6f1
DEV: Drop old hotlinked image data from post_custom_fields (#16594)
`20220428094026_create_post_hotlinked_media` moved this data into a dedicated table
2022-05-12 15:34:35 +01:00
David Taylor d90065e0ef
Revert "DEV: Update add_column migration to remove transaction (#16715)" (#16716)
This reverts commit e599b5b08f.

This didn't solve the problem we were trying to fix - reverting back to the standard migration pattern
2022-05-11 12:36:11 +01:00
David Taylor e599b5b08f
DEV: Update add_column migration to remove transaction (#16715)
This migration is failing to acquire a lock under some production conditions. We're only performing one action, so removing the transaction is safe and may help to resolve the issue.
2022-05-11 11:28:34 +01:00
Isaac Janzen 1a12e4cfc8
FEATURE: Introduce site setting to allow for non staff pm tagging (#16671)
Currently the only way to allow tagging on pms is to use the `allow_staff_to_tag_pms` site setting.  We are removing that site setting and replacing it with `pm_tags_allowed_for_groups` which will allow for non staff tagging. It will be group based permissions instead of requiring the user to be staff.

If the existing value of `allow_staff_to_tag_pms` is `true` then we include the `staff` groups as a default for `pm_tags_allowed_for_groups`.
2022-05-10 10:02:28 -05:00
Martin Brennan fbcc35b417
DEV: Remove PostAction/UserAction bookmark refs (#16681)
We have not used anything related to bookmarks for PostAction
or UserAction records since 2020, bookmarks are their own thing
now. Deleting all this is just cleaning up old cruft.
2022-05-10 10:42:18 +10:00
Roman Rizzi 6bdcd7afb2
FEATURE: Promote the "delete group" staff action log. (#16656)
We used to log group deletion as custom, which means we can't later search for them. Include group ID in the details.
2022-05-09 12:12:52 -03:00
Osama Sayegh 7e026e8ddf
DEV: Add last_seen_reviewable_id to the users table (#16666)
We're adding this column now in preparation for a future commit(s) that will
redesign the avatar/notifications menu. The reason the column is added in a
separate commit is because the redesign changes are going to be complex with a
high risk of getting (temporarily) reverted and if they included a database
migration, they wouldn't revert cleanly/easily.

Internal ticket: t65045.
2022-05-06 10:11:16 +03:00
David Taylor db9ae32e41
FIX: Skip invalid custom_field json in hotlinked_media migration (#16630)
Truly testing for JSON validity would require defining a new postgres function. Checking just the first character should take care of all the cases of invalid historic data that we've seen.
2022-05-04 15:05:18 +01:00
David Taylor 1c9d8b4999
FIX: Handle invalid JSON from downloaded_images custom fields (#16613)
custom_field data on some sites has duplicate values for a given url key in the JSON value. This is invalid, so we should drop the extra data.
2022-05-03 18:51:25 +01:00
David Taylor bebb4fd111
FIX: Handle duplicates in hotlinked_media migration (#16611)
In the old custom_field-based system, it was possible for a url to be both 'downloaded' and 'broken'. The new table enforces uniqueness, so we need to drop invalid data.
2022-05-03 17:17:27 +01:00
David Taylor 19d2d55011
FIX: Ensure post_hotlinked_media index does not exceed size limit (#16609)
On some installations, this would fail with 'index row size exceeds btree version 4 maximum'. This commit replaces the (post_id, url)` index with a `(post_id, md5(url))` index, which is much more space efficient.
2022-05-03 15:47:58 +01:00
David Taylor c1db968740
DEV: Move hotlinked image information into a dedicated table (#16585)
This will make future changes to the 'pull hotlinked images' system easier. This commit should not introduce any functional change.

For now, the old post_custom_field data is kept in the database. This will be dropped in a future commit.
2022-05-03 13:53:32 +01:00
Loïc Guitaut 008b700a3f DEV: Upgrade to Rails 7
This patch upgrades Rails to version 7.0.2.4.
2022-04-28 11:51:03 +02:00
Alan Guo Xiang Tan 98c49acad5
DEV: Setup experimental sidebar skeleton (#16575)
* hidden siteSetting to enable experimental sidebar
* user preference to enable experimental sidebar
* `experimental_sidebar_enabled` attribute for current user
* Empty glimmer component for Sidebar
2022-04-28 15:27:06 +08:00
Martin Brennan c841e34b62
FIX: Check if bookmarkable column exists before adding (#16497)
Because in 8040b95e8c we removed
a previous post migrate file, some people may not have had those
original polymorphic bookmark columns removed, and the migration
from this PR running will cause duplicate column errors.

cf. https://meta.discourse.org/t/duplicatecolumn-and-multisite-migrate-failed/224480
2022-04-19 12:01:18 +10:00
Roman Rizzi 6f76a12e0a
FEATURE: Let sites add a sitemap.xml file. (#16357)
* FEATURE: Let sites add a sitemap.xml file.

This PR adds the same features discourse-sitemap provides to core. Sitemaps are only added to the robots.txt file if the `enable_sitemap` setting is enabled and `login_required` disabled.

After merging discourse/discourse-sitemap#34, this change will take priority over the sitemap plugin because it will disable itself. We're also using the same sitemaps table, so our migration won't try to create it
again using `if_not_exists: true`.
2022-04-12 10:33:59 -03:00
David Taylor 1e436f242e
DEV: Move tag_group cleanup from post_migrate -> migrate (#16434)
There is no need to wait until after the deploy for this cleanup. In fact, running it later will mean there could be a window of a few minutes during which the site is broken.

The only requirement is that it runs after the broken `20220401130745_create_category_required_tag_groups` migration.

Followup to 39a6de3d73
2022-04-11 13:28:24 +01:00
Roman Rizzi 39a6de3d73
FIX: Clean required category tag groups with invalid tag_group ids (#16414)
The category table's required_tag_group_id contained references to deleted tag groups, which we copied to the new table. The new serializer tries to get the associated tag group name but fails because the tag group is nil.

This PR adds an inner join in the original migration to make sure tag groups still exist and adds a new post-migration to fix already migrated sites.
2022-04-07 17:02:11 -04:00
David Taylor 68c74e9b93
FEATURE: Allow multiple required tag groups for a category (#16381)
Previously we only supported a single 'required tag group' for a category. This commit allows admins to specify multiple required tag groups, each with their own minimum tag count.

A new category_required_tag_groups database table replaces the existing columns on the categories table. Data is automatically migrated.
2022-04-06 14:08:06 +01:00
Alan Guo Xiang Tan d0c2eb3359
DEV: Drop `enable_long_polling` and `long_polling_interval` settings (#16323)
Scheduled to drop in 2.9.

Co-authored-by: Loïc Guitaut <loic@discourse.org>
2022-03-30 16:32:40 +08:00
Martin Brennan 230e82e948
DEV: Make Bookmark#post_id column nullable (#16287)
As we are gradually moving to having a polymorphic
bookmarkable relationship on the Bookmark table,
we need to make the post_id column nullable to be
able to develop and test the new columns, and
for cutover/migration purposes later as well.
2022-03-28 13:09:13 +10:00
David Taylor f3aab19829
DEV: Promote historic post_deploy migrations (#16288)
This commit promotes all post_deploy migrations which existed in Discourse v2.7.13 (timestamp <= 20210328233843)

This reduces the likelihood of issues relating to migration run order

Also fixes a couple of typos in `script/promote_migrations`
2022-03-25 15:48:20 +00:00
Martin Brennan 8040b95e8c
DEV: Re-add polymorphic bookmark columns (#16261)
This commit is a redo of2f1ddadff7dd47f824070c8a3f633f00a27aacde
which we reverted because it blew up an internal CI check. I looked
into it, and it happened because the old migration to add the bookmark
columns still existed, and those columns were dropped in a post migrate,
so the two migrations to add the columns were conflicting before
the post migrate was run.

------

This commit only includes the creation of the new columns and index,
and does not add any triggers, backfilling, or new data.

A backfill will be done in the final PR when we switch this over.
Intermediate PRs will look something like this:

Add an experimental site setting for using polymorphic bookmarks,
and make sure in the places where bookmarks are created or updated
we fill in the columns. This setting will be used in subsequent
PRs as well.
Listing and searching bookmarks based on polymorphic associations
Creating post and topic bookmarks using polymorphic associations,
and changing special for_topic logic to just rely on the Topic
bookmarkable_type
Querying bookmark reminders based on polymorphic associations
Make sure various other areas like importers, bookmark guardian,
and others all rely on the associations
Prepare plugins that rely on the Bookmark model to use polymorphic
associations
The final core PR will remove all the setting gates and switch over
to using the polymorphic associations, backfill the bookmarks
table columns, and ignore the old post_id and for_topic colummns.
Then it will just be a matter of dropping the old columns down the
line.
2022-03-23 12:43:08 +10:00
Alan Guo Xiang Tan 4e6f7a1a6e
Revert "DEV: Re-add polymorphic bookmark columns (#16245)" (#16246)
This reverts commit 2f1ddadff7.
2022-03-22 13:13:42 +08:00
Martin Brennan 2f1ddadff7
DEV: Re-add polymorphic bookmark columns (#16245)
This commit is a redo of e21c640a3c
which we reverted to not include half-done work in a release.

This commit is slightly different though, in that it only includes
the creation of the new columns and index, and does not add any
triggers, backfilling, or new data.

A backfill will be done in the final PR when we switch this over.
Intermediate PRs will look something like this:

1. Add an experimental site setting for using polymorphic bookmarks,
   and make sure in the places where bookmarks are created or updated
   we fill in the columns. This setting will be used in subsequent
   PRs as well.
2. Listing and searching bookmarks based on polymorphic associations
3. Creating post and topic bookmarks using polymorphic associations,
   and changing special for_topic logic to just rely on the Topic
   bookmarkable_type
4. Querying bookmark reminders based on polymorphic associations
5. Make sure various other areas like importers, bookmark guardian,
   and others all rely on the associations
6. Prepare plugins that rely on the Bookmark model to use polymorphic
   associations

The final core PR will remove all the setting gates and switch over
to using the polymorphic associations, backfill the bookmarks
table columns, and ignore the old post_id and for_topic colummns.
Then it will just be a matter of dropping the old columns down the
line.
2022-03-22 14:26:13 +10:00
Bianca Nenciu 30f3e78834
FIX: Reset last sent for existent bookmarks (#16202)
The meaning of reminder_at and reminder_last_sent_at changed after
commit 6d422a8033. A bookmark reminder
will fire only if reminder_last_sent_at is null, but before that it
fired everytime reminder_at was set. This is no longer true because
sometimes reminder_at continues to exist even after a reminder fired.
2022-03-18 16:31:35 +02:00
Bianca Nenciu 6d422a8033
FEATURE: Highlight expired bookmark reminders (#15317)
The user can select what happens with a bookamrk after it expires. New
option allow bookmark's reminder to be kept even after it has expired.
After a bookmark's reminder notification is created, the reminder date
will be highlighted in red until the user resets the reminder date.
User can do that using the new Clear Reminder button from the dropdown.
2022-03-08 19:44:18 +02:00
Bianca Nenciu 34b4b53bac
FEATURE: Use Postgres unaccent to ignore accents (#16100)
The search_ignore_accents site setting can be used to make the search
indexer remove the accents before indexing the content. The unaccent
function from PostgreSQL is better than Ruby's unicode_normalize(:nfkd).
2022-03-07 23:03:10 +02:00
David Taylor 241e34e6cf
FIX: Update and rebake uses of the old centralized avatar service (#16086)
This URL was originally updated in 89cb537fae. However, some sites are not using the proxy, and have configured their forum to hotlink images directly to avatars.discourse.org.

We intend to shut down this domain in favor of `avatars.discourse-cdn.com`, so this migration will re-write any matching site setting values and queue affected posts for rebaking.
2022-03-07 09:45:20 +00:00
Jarek Radosz 2fc70c5572
DEV: Correctly tag heredocs (#16061)
This allows text editors to use correct syntax coloring for the heredoc sections.

Heredoc tag names we use:

languages: SQL, JS, RUBY, LUA, HTML, CSS, SCSS, SH, HBS, XML, YAML/YML, MF, ICS
other: MD, TEXT/TXT, RAW, EMAIL
2022-02-28 20:50:55 +01:00
Alan Guo Xiang Tan e613aef419
DEV: Remove use of Discourse application code in migration. (#16068)
This makes the migration less brittle to changes in the future.

Follow-up to 21a356e3af
2022-02-28 10:48:46 +08:00
Jeff Wong d1bdb6c65d
FEATURE: upload an avatar option for uploading avatars with selectable avatars (#15878)
* FEATURE: upload an avatar option for uploading avatars with selectable avatars

Allow staff or users at or above a trust level to upload avatars even when the site
has selectable avatars enabled.

Everyone can still pick from the list of avatars. The option to upload is shown
below the selectable avatar list.

refactored boolean site setting into an enum with the following values:

disabled: No selectable avatars enabled (default)
everyone: Show selectable avatars, and allow everyone to upload custom avatars
tl1: Show selectable avatars, but require tl1+ and staff to upload custom avatars
tl2: Show selectable avatars, but require tl2+ and staff to upload custom avatars
tl3: Show selectable avatars, but require tl3+ and staff to upload custom avatars
tl4: Show selectable avatars, but require tl4 and staff to upload custom avatars
staff: Show selectable avatars, but only allow staff to upload custom avatars
no_one: Show selectable avatars. No users can upload custom avatars

Co-authored-by: Régis Hanol <regis@hanol.fr>
2022-02-24 12:57:39 -08:00
Joffrey JAFFEUX f5ec32bc8c
FEATURE: adds the user_promoted event to webhooks (#15996) 2022-02-22 10:57:18 +01:00
Martin Brennan fa0c796baf
DEV: Fix SMTP bounce regexp (#16019)
Never trust me with regexp. Follow up to
01ef1d08fc,
which did not take into account codes in
the format X.X.XX (with the 2 digits on the end)
2022-02-22 08:54:01 +10:00
Martin Brennan 01ef1d08fc
FIX: Conform EmailLog#bounce_error_code to RFC (#16010)
This commit makes sure that the email log's bounce_error_code
conforms to the SMTP error code RFC on save, so that
it is always in the format X.X.X or XXX without any
additional string details. Also included is a migration
to fix this issue for past records.
2022-02-21 11:26:39 +10:00
Martin Brennan f9ec2b90a0
DEV: Drop user_stats count column constraints (#15949)
We added this constraint in 5bd55acf83
but it is causing problems in hosted sites and is catching the
issue too far down the line. This commit removes the constraint
for now, and also fixes an issue found with PostDestroyer
which wasn't using the UserStatCountUpdater when updating post_count
and thus was causing negative numbers to occur.
2022-02-16 12:49:11 +11:00
Martin Brennan 4086ee551e
DEV: Add bounce_error_code to EmailLog (#15948)
Whenever we got a bounced email in the Email::Receiver we
previously would just set bounced: true on the EmailLog and
discard the status/diagnostic code. This commit changes this
flow to store the bounce error code (defined in the RFC at
https://www.iana.org/assignments/smtp-enhanced-status-codes/smtp-enhanced-status-codes.xhtml)
not just in the Email::Receiver, but also via webhook events
from other mail services and from SNS.

This commit does not surface the bounce error in the UI,
we can do that later if necessary.
2022-02-15 14:17:26 +10:00
Martin Brennan ad9e791ba7
FIX: Post migration to fix custom emojis with secure URL (#15944)
Follow up to 88a8584348. Sets
the baked version of all posts with custom emoji and a secure
media URL in the cooked content to 0. Then our periodic rebake
posts job will rebake them to apply the fix in the linked
commit. This only matters on sites with secure media enabled.
2022-02-15 10:37:16 +10:00