Discource-C
/
discourse
mirror of https://github.com/discourse/discourse.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
discourse/app/jobs/scheduled
History
Alan Guo Xiang Tan 9812407f76
FIX: Redo Sidekiq monitoring to restart stuck sidekiq processes (#30198) 
This commit reimplements how we monitor Sidekiq processes that are
forked from the Unicorn master process. Prior to this change, we rely on
`Jobs::Heartbeat` to enqueue a `Jobs::RunHeartbeat` job every 3 minutes.
The `Jobs::RunHeartbeat` job then sets a Redis key with a timestamp. In
the Unicorn master process, we then fetch the timestamp that has been set
by the job from Redis every 30 minutes. If the timestamp has not been
updated for more than 30 minutes, we restart the Sidekiq process. The
fundamental flaw with this approach is that it fails to consider
deployments with multiple hosts and multiple Sidekiq processes. A
sidekiq process on a host may be in a bad state but the heartbeat check
will not restart the process because the `Jobs::RunHeartbeat` job is
still being executed by the working Sidekiq processes on other hosts.

In order to properly ensure that stuck Sidekiq processs are restarted,
we now rely on the [Sidekiq::ProcessSet](https://github.com/sidekiq/sidekiq/wiki/API#processes)
API that is supported by Sidekiq. The API provides us with "near real-time (updated every 5 sec)
info about the current set of Sidekiq processes running". The API
provides useful information like the hostname, pid and also when Sidekiq
last did its own heartbeat check. With that information, we can easily
determine if a Sidekiq process needs to be restarted from the Unicorn
master process.
 		2024-12-18 12:48:50 +08:00
..
about_stats.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
activation_reminder_emails.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
admin_problems.rb FEATURE: Add new notification for admin problems (#21287) 2023-05-03 19:35:22 +03:00
aggregate_web_hooks_events.rb FEATURE: Add WebHookEventsDailyAggregate (#27542) 2024-06-25 13:56:47 -03:00
auto_queue_handler.rb DEV: replace .each with .find_each for paginated queries (#27159) 2024-05-23 15:42:21 -07:00
backfill_dominant_colors.rb PERF: Move dominant color calculation to separate job (#18501) 2022-10-06 13:26:08 +01:00
badge_grant.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
bookmark_reminder_notifications.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
call_discourse_hub.rb FEATURE: call hub API to update Discourse discover enrollment. (#25634) 2024-02-23 11:42:28 +05:30
category_stats.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
check_new_features.rb REVERT: Check for features sooner (#29746) 2024-11-13 11:30:34 -08:00
check_out_of_date_themes.rb FIX: Continue if checking themes if updating one fails (#19665) 2023-01-09 11:56:43 -06:00
check_translation_overrides.rb FIX: Ensure JsLocaleHelper to not output deprecated translations (#28037) 2024-07-29 15:21:25 +08:00
clean_dismissed_topic_users.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
clean_up_api_keys_max_life.rb FEATURE: site settings to revoke api keys older than a number of days (#23595) 2023-09-15 16:31:29 -03:00
clean_up_associated_accounts.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
clean_up_associated_groups.rb FEATURE: Experimental support for group membership via google auth (#14835) 2021-12-09 12:30:27 +00:00
clean_up_crawler_stats.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
clean_up_email_change_requests.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
clean_up_email_logs.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
clean_up_email_tokens.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
clean_up_exports.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
clean_up_inactive_users.rb FIX: consider users.created_at for inactive cleanup (#21688) 2023-05-23 13:41:23 +05:30
clean_up_post_reply_keys.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
clean_up_search_logs.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
clean_up_stylesheet_cache.rb PERF: Add scheduled job to delete old stylesheet cache rows (#13747) 2021-07-16 10:58:01 -04:00
clean_up_tags.rb FIX: log for CleanUpTags job (#23964) 2023-10-18 03:24:14 +00:00
clean_up_unmatched_emails.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
clean_up_unmatched_ips.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
clean_up_unsubscribe_keys.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
clean_up_unused_api_keys.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
clean_up_unused_registered_user_api_key_clients.rb Require permitted scopes when registering a client (#29718) 2024-11-19 15:28:04 -05:00
clean_up_unused_staged_users.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
clean_up_unused_user_api_keys.rb FEATURE: site settings to revoke api keys older than a number of days (#23595) 2023-09-15 16:31:29 -03:00
clean_up_uploads.rb DEV: update syntax tree to latest (#24623) 2023-11-29 16:38:07 +11:00
clean_up_user_api_keys_max_life.rb FEATURE: site settings to revoke api keys older than a number of days (#23595) 2023-09-15 16:31:29 -03:00
cleanup_imap_sync_log.rb DEV: IMAP log to database (#10435) 2020-08-14 12:01:31 +10:00
cleanup_redelivering_web_hook_events.rb FEATURE: Add Mechanism to redeliver all failed webhook events (#27609) 2024-07-08 15:43:16 -05:00
create_missing_avatars.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
create_recent_post_search_indexes.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
destroy_old_deletion_stubs.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
destroy_old_hidden_posts.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
directory_refresh_daily.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
directory_refresh_older.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
disable_bootstrap_mode.rb UX: make first admin a moderator to review user approvals. (#26588) 2024-04-10 20:59:03 +05:30
drop_backup_schema.rb FEATURE: Drop "backup" schema 7 days after restore 2020-01-16 17:48:47 +01:00
enqueue_digest_emails.rb FIX: send activity summaries based on "last seen" (#27035) 2024-05-22 10:23:03 +02:00
enqueue_onceoffs.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
enqueue_suspect_users.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
ensure_db_consistency.rb DEV: ensure rebaking works even when some users have inconsistent data (#30261) 2024-12-16 19:48:25 +08:00
ensure_s3_uploads_existence.rb FIX: `Jobs::EnsureS3UploadsExistence` broken for multisite (#27401) 2024-06-10 16:26:39 +08:00
fix_user_usernames_and_groups_names_clash.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
grant_anniversary_badges.rb DEV: extract anniversary badge query (#19716) 2023-01-16 11:55:00 +01:00
grant_new_user_of_the_month_badges.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
heartbeat.rb FIX: Redo Sidekiq monitoring to restart stuck sidekiq processes (#30198) 2024-12-18 12:48:50 +08:00
ignored_users_summary.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
invalidate_inactive_admins.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
migrate_upload_scheme.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
old_keys_reminder.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
pending_queued_posts_reminder.rb DEV: Enable `unless` cops 2023-02-21 10:30:48 +01:00
pending_reviewables_reminder.rb FIX: rename notify_about_flags_after to notify_about_reviewable_item_after (#21320) 2023-05-02 08:08:22 +10:00
pending_users_reminder.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
periodical_updates.rb DEV: ensure rebaking works even when some users have inconsistent data (#30261) 2024-12-16 19:48:25 +08:00
poll_mailbox.rb DEV: Database backed admin notices (#26192) 2024-05-23 09:29:08 +08:00
presence_channel_auto_leave.rb DEV: Introduce PresenceChannel API for core and plugin use 2021-08-27 16:26:06 +01:00
process_badge_backlog.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
process_shelved_notifications.rb DEV: Replace 'processed' column on notifications with new table (#11864) 2021-01-27 10:29:24 -06:00
process_user_notification_schedules.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
purge_deleted_uploads.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
purge_expired_ignored_users.rb DEV: Cleanup ignored user logic (#11107) 2020-11-03 12:38:54 +00:00
purge_old_web_hook_events.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
purge_old_web_hook_events_daily_aggregate.rb FEATURE: Add WebHookEventsDailyAggregate (#27542) 2024-06-25 13:56:47 -03:00
purge_unactivated.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
redeliver_web_hook_events.rb FEATURE: Add Mechanism to redeliver all failed webhook events (#27609) 2024-07-08 15:43:16 -05:00
regenerate_sitemaps.rb FEATURE: Let sites add a sitemap.xml file. (#16357) 2022-04-12 10:33:59 -03:00
reindex_search.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
reviewable_priorities.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
run_problem_checks.rb FIX: Appropriately handle uninstalled problem checks (#28771) 2024-09-18 10:11:52 +08:00
schedule_backup.rb FIX: refactor calling of timed backup deletion 2024-01-08 15:41:00 -05:00
tl3_promotions.rb DEV: Add tl3_custom_promotions plugin modifier to tl3_promotions.rb (#29834) 2024-11-22 15:28:43 -05:00
top_refresh_older.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
top_refresh_today.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
topic_timer_enqueuer.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
unsilence_users.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00
update_animated_uploads.rb DEV: Remove logical OR assignment of constants (#29201) 2024-10-16 10:09:07 +08:00
update_heat_settings.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
update_topic_hot_scores.rb FIX: muted tags breaking hot page when filtered to tags (#25824) 2024-02-23 17:11:39 +11:00
weekly.rb DEV: Apply syntax_tree formatting to `app/*` 2023-01-09 14:14:59 +00:00