Discource-C
/
discourse
mirror of https://github.com/discourse/discourse.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
discourse/spec/jobs
History
Alan Guo Xiang Tan 9812407f76
FIX: Redo Sidekiq monitoring to restart stuck sidekiq processes (#30198) 
This commit reimplements how we monitor Sidekiq processes that are
forked from the Unicorn master process. Prior to this change, we rely on
`Jobs::Heartbeat` to enqueue a `Jobs::RunHeartbeat` job every 3 minutes.
The `Jobs::RunHeartbeat` job then sets a Redis key with a timestamp. In
the Unicorn master process, we then fetch the timestamp that has been set
by the job from Redis every 30 minutes. If the timestamp has not been
updated for more than 30 minutes, we restart the Sidekiq process. The
fundamental flaw with this approach is that it fails to consider
deployments with multiple hosts and multiple Sidekiq processes. A
sidekiq process on a host may be in a bad state but the heartbeat check
will not restart the process because the `Jobs::RunHeartbeat` job is
still being executed by the working Sidekiq processes on other hosts.

In order to properly ensure that stuck Sidekiq processs are restarted,
we now rely on the [Sidekiq::ProcessSet](https://github.com/sidekiq/sidekiq/wiki/API#processes)
API that is supported by Sidekiq. The API provides us with "near real-time (updated every 5 sec)
info about the current set of Sidekiq processes running". The API
provides useful information like the hostname, pid and also when Sidekiq
last did its own heartbeat check. With that information, we can easily
determine if a Sidekiq process needs to be restarted from the Unicorn
master process.
 		2024-12-18 12:48:50 +08:00
..
regular DEV: Remove summarization code (#27373) 2024-07-02 08:51:47 -07:00
about_stats_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
activation_reminder_emails_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
admin_problems_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
auto_queue_handler_spec.rb DEV: Convert min_trust_to_flag_posts setting to groups (#24864) 2023-12-13 17:18:42 +08:00
automatic_group_membership_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
backfill_sidebar_site_settings_spec.rb FIX: Set `Jobs::BackfillSidebarSiteSettings` cluster concurrency to 1 (#22316) 2023-06-28 13:07:46 +08:00
bookmark_reminder_notifications_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
bulk_grant_trust_level_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
bulk_invite_spec.rb DEV: Update bulk-invite logs and PM template (#27057) 2024-05-17 12:21:21 -06:00
bump_topic_spec.rb DEV: Fix various rubocop lints (#24749) 2023-12-06 23:25:00 +01:00
call_discourse_hub_spec.rb FEATURE: call hub API to update Discourse discover enrollment. (#25634) 2024-02-23 11:42:28 +05:30
change_display_name_spec.rb SECURITY: escape display names 2023-11-09 13:39:06 +11:00
check_new_features_spec.rb DEV: Allow new_features URL to be configurable (#24306) 2023-11-09 13:50:21 +10:00
check_translation_overrides_spec.rb DEV: Catch missing translations during test runs (#26258) 2024-05-24 22:15:53 +08:00
clean_dismissed_topic_users_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
clean_up_associated_accounts_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
clean_up_crawler_stats_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
clean_up_email_change_requests_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
clean_up_email_logs_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
clean_up_inactive_users_spec.rb FIX: consider users.created_at for inactive cleanup (#21688) 2023-05-23 13:41:23 +05:30
clean_up_post_reply_keys_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
clean_up_tags_spec.rb DEV: Remove unnecessary rails_helper requiring (#26364) 2024-03-26 11:32:01 +01:00
clean_up_unused_staged_users_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
clean_up_unused_user_api_key_clients_spec.rb Require permitted scopes when registering a client (#29718) 2024-11-19 15:28:04 -05:00
clean_up_unused_user_api_keys_spec.rb FEATURE: site settings to revoke api keys older than a number of days (#23595) 2023-09-15 16:31:29 -03:00
clean_up_uploads_spec.rb DEV: Improve external upload debugging (#28627) 2024-08-30 10:25:04 +10:00
clean_up_user_api_keys_max_life_spec.rb FEATURE: site settings to revoke api keys older than a number of days (#23595) 2023-09-15 16:31:29 -03:00
clean_up_user_export_topics_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
cleanup_imap_sync_log_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
cleanup_redelivering_web_hook_events_spec.rb FEATURE: Add Mechanism to redeliver all failed webhook events (#27609) 2024-07-08 15:43:16 -05:00
close_topic_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
correct_missing_dualstack_urls_spec.rb DEV: Use Discourse::SYSTEM_USER_ID in fixtures/009_users (#24743) 2023-12-07 09:04:45 +10:00
crawl_topic_link_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
create_linked_topic_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
create_missing_avatars_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
create_recent_post_search_indexes_spec.rb DEV: Fix random typos (#24756) 2023-12-06 22:25:26 +01:00
create_user_reviewable_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
delete_replies_spec.rb DEV: Fix various rubocop lints (#24749) 2023-12-06 23:25:00 +01:00
delete_topic_spec.rb DEV: Fix various rubocop lints (#24749) 2023-12-06 23:25:00 +01:00
disable_bootstrap_mode_spec.rb UX: make first admin a moderator to review user approvals. (#26588) 2024-04-10 20:59:03 +05:30
download_avatar_from_url_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
download_backup_email_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
download_profile_background_from_url_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
emit_web_hook_event_spec.rb DEV: Add modifier for webhook event header generation (#27054) 2024-05-17 09:33:39 -05:00
enable_bootstrap_mode_spec.rb DEV: don't send moderator welcome message to first admin. (#26719) 2024-04-24 00:20:14 +05:30
enqueue_digest_emails_spec.rb FIX: send activity summaries based on "last seen" (#27035) 2024-05-22 10:23:03 +02:00
enqueue_suspect_users_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
ensure_s3_uploads_existence_multisite_spec.rb FIX: `Jobs::EnsureS3UploadsExistence` broken for multisite (#27401) 2024-06-10 16:26:39 +08:00
ensure_s3_uploads_existence_spec.rb DEV: Introduce hidden `s3_inventory_bucket` site setting (#27304) 2024-06-10 13:16:00 +08:00
export_csv_file_spec.rb FIX: export csv file failed message (#25443) 2024-01-26 11:16:02 -07:00
export_user_archive_spec.rb DEV: Convert min_trust_level_to_tag_topics to groups (#25273) 2024-01-26 13:25:03 +08:00
feature_topic_users_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
fix_out_of_sync_user_uploaded_avatar_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
fix_primary_emails_for_staged_users_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
fix_s3_etags_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
fix_user_usernames_and_groups_names_clash_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
grant_anniversary_badges_spec.rb DEV: extract anniversary badge query (#19716) 2023-01-16 11:55:00 +01:00
grant_new_user_of_the_month_badges_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
ignored_users_summary_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
index_user_fields_for_search_spec.rb FIX: Move user reindexing into a job (#26753) 2024-04-25 20:58:34 +08:00
invalidate_inactive_admins_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
invite_email_spec.rb DEV: Fix various rubocop lints (#24749) 2023-12-06 23:25:00 +01:00
jobs_base_spec.rb DEV: Fix broken sidekiq logging due to eeb01ea0de (#30199) 2024-12-10 17:01:25 +08:00
jobs_spec.rb FIX: Redo Sidekiq monitoring to restart stuck sidekiq processes (#30198) 2024-12-18 12:48:50 +08:00
mass_award_badge_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
notify_category_change_spec.rb DEV: Fix linting in notify_category_change_spec (#29175) 2024-10-11 19:55:33 +02:00
notify_mailing_list_subscribers_spec.rb DEV: Fix various spec linting issues (#24672) 2023-12-04 13:45:19 +01:00
notify_moved_posts_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
notify_reviewable_spec.rb FEATURE: Support designating multiple groups as mods on category (#28655) 2024-09-04 04:38:46 +03:00
notify_tag_change_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
old_keys_reminder_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
open_topic_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
pending_queued_posts_reminder_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
pending_reviewables_reminder_spec.rb DEV: Convert min_trust_to_flag_posts setting to groups (#24864) 2023-12-13 17:18:42 +08:00
pending_users_reminder_spec.rb DEV: Remove full group refreshes from tests (#25414) 2024-01-25 14:28:26 +08:00
periodical_updates_spec.rb DEV: ensure rebaking works even when some users have inconsistent data (#30261) 2024-12-16 19:48:25 +08:00
poll_mailbox_spec.rb DEV: Database backed admin notices (#26192) 2024-05-23 09:29:08 +08:00
post_update_topic_tracking_state_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
post_uploads_recovery_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
process_bulk_invite_emails_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
process_email_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
process_post_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
process_shelved_notifications_spec.rb DEV: Fix various rubocop lints (#24749) 2023-12-06 23:25:00 +01:00
publish_topic_to_category_spec.rb FIX: display validation errors when converting topics (#27064) 2024-05-17 16:36:25 +02:00
pull_hotlinked_images_spec.rb DEV: Move min_trust_to_post_embedded_media to group setting (#25238) 2024-01-25 09:50:59 +10:00
pull_user_profile_hotlinked_images_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
purge_expired_ignored_users_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
push_notification_spec.rb DEV: Fix undefined variable (#29876) 2024-11-21 20:23:20 +02:00
rebake_custom_emoji_posts_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
redeliver_web_hook_events_spec.rb DEV: Specs for redeliver_web_hook_events job (#27779) 2024-07-09 10:35:10 -05:00
refresh_users_reviewable_counts_spec.rb FEATURE: Support designating multiple groups as mods on category (#28655) 2024-09-04 04:38:46 +03:00
reindex_search_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
remove_banner_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
reviewable_priorities_spec.rb DEV: Split slow test in multiple smaller tests (#28646) 2024-08-30 14:47:29 +10:00
run_problem_check_spec.rb DEV: Remove old problem check system - Part 1 (#28772) 2024-09-06 17:00:25 +08:00
run_problem_checks_spec.rb FIX: Appropriately handle uninstalled problem checks (#28771) 2024-09-18 10:11:52 +08:00
send_push_notification_spec.rb DEV: refactor live notifications setting in user preferences (#28145) 2024-08-02 17:25:15 +04:00
send_system_message_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
suspicious_login_spec.rb DEV: Apply syntax_tree formatting to `spec/*` 2023-01-09 11:49:28 +00:00
sync_acls_for_uploads_spec.rb DEV: Remove unnecessary rails_helper requiring (#26364) 2024-03-26 11:32:01 +01:00
sync_topic_user_bookmarked_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
tl3_promotions_spec.rb DEV: Add tl3_custom_promotions plugin modifier to tl3_promotions.rb (#29834) 2024-11-22 15:28:43 -05:00
toggle_topic_closed_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
topic_timer_enqueuer_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
truncate_user_flag_stats_spec.rb DEV: Convert min_trust_to_flag_posts setting to groups (#24864) 2023-12-13 17:18:42 +08:00
unsilence_users_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
update_animated_uploads_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
update_gravatar_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
update_topic_hot_scores_spec.rb DEV: Call Discourse.redis.flushdb after the end of each test (#29117) 2024-10-09 07:19:31 +08:00
update_username_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
user_email_spec.rb FIX: send activity summaries based on "last seen" (#27035) 2024-05-22 10:23:03 +02:00