Discource-C
/
discourse
mirror of https://github.com/discourse/discourse.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
discourse/app/jobs/regular
History
David Taylor 93c33e02f0
PERF: Avoid full `posts` table scans during anonymisation (#21081) 
2e78045a fixed the anonymization job so that it correctly updated self-mentions, which are not logged in the post_actions table. The solution was to scan the entire `posts` table with an `raw ILIKE` query. On sites with many posts, this can take a very long time.

This commit updates the job to take a two-pass approach:

First, we update posts based on the post_actions table. This is much more efficient than a full table scan, and takes care of all 'non-self' mentions.

Then, we make a second pass using the `raw ILIKE` approach. Since we already took care of most posts, we can scope this down to self-mentions only. By filtering the query to a specific posts.user_id, it is significantly more performant than a full table scan.
 		2023-04-12 18:39:10 +01:00
..
admin_confirmation_email.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
anonymize_user.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
automatic_group_membership.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
backfill_sidebar_site_settings.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
backup_chunks_merger.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
bulk_grant_trust_level.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
bulk_invite.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
bulk_user_title_update.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
bump_topic.rb DEV: Change Topic Timer from enqueue_at scheduled jobs to incrementally executed jobs (#11698) 2021-01-19 13:30:58 +10:00
clear_slow_mode.rb DEV: Change Topic Timer from enqueue_at scheduled jobs to incrementally executed jobs (#11698) 2021-01-19 13:30:58 +10:00
close_topic.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
confirm_sns_subscription.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
crawl_topic_link.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
create_avatar_thumbnails.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
create_backup.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
create_linked_topic.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
create_user_reviewable.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
critical_user_email.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
delete_inaccessible_notifications.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
delete_replies.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
delete_topic.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
download_avatar_from_url.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
download_backup_email.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
download_profile_background_from_url.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
emit_web_hook_event.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
enable_bootstrap_mode.rb FEATURE: Move bootstrap mode indicator to header (#20663) 2023-03-24 15:59:03 +02:00
export_csv_file.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
export_user_archive.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
feature_topic_users.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
generate_topic_thumbnails.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
group_pm_alert.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
group_pm_update_summary.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
group_smtp_email.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
index_category_for_search.rb FIX: don't error-index category job when missing category 2020-12-01 09:30:53 +08:00
invite_email.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
invite_password_instructions_email.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
make_embedded_topic_visible.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
mass_award_badge.rb FEATURE: Add option to grant badge multiple times to users using Bulk Award (#13571) 2021-07-15 05:53:26 +03:00
merge_user.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
notify_category_change.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
notify_mailing_list_subscribers.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
notify_moved_posts.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
notify_post_revision.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
notify_reviewable.rb DEV: reduce amount of errors logged when notifying on flags (#20472) 2023-03-01 08:58:32 +11:00
notify_tag_change.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
open_topic.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
post_alert.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
post_update_topic_tracking_state.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
process_bulk_invite_emails.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
process_email.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
process_post.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
process_sns_notification.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
publish_group_membership_updates.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
publish_topic_to_category.rb DEV: Change Topic Timer from enqueue_at scheduled jobs to incrementally executed jobs (#11698) 2021-01-19 13:30:58 +10:00
pull_hotlinked_images.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
pull_user_profile_hotlinked_images.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
push_notification.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
rebake_custom_emoji_posts.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
rebake_posts_for_upload.rb FIX: Mark secure media upload insecure automatically if used for theme component (#8413) 2019-11-28 07:32:17 +10:00
refresh_users_reviewable_counts.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
remove_banner.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
retrieve_topic.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
run_heartbeat.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
send_push_notification.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
send_system_message.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
suspicious_login.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
sync_acls_for_uploads.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
sync_topic_user_bookmarked.rb FEATURE: Promote polymorphic bookmarks to default and migrate (#16729) 2022-05-23 10:07:15 +10:00
toggle_topic_closed.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
topic_action_converter.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
topic_timer_base.rb DEV: Change Topic Timer from enqueue_at scheduled jobs to incrementally executed jobs (#11698) 2021-01-19 13:30:58 +10:00
truncate_user_flag_stats.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
unpin_topic.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
update_gravatar.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
update_group_mentions.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
update_hotlinked_raw.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
update_post_uploads_secure_status.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
update_s3_inventory.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
update_top_redirection.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
update_topic_upload_security.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
update_username.rb PERF: Avoid full `posts` table scans during anonymisation (#21081) 2023-04-12 18:39:10 +01:00
user_email.rb FIX: Don’t assume post is available in UserEmail job (#21054) 2023-04-12 12:34:22 +10:00