Discource-C
/
discourse
mirror of https://github.com/discourse/discourse.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
discourse/spec/jobs
History
Alan Guo Xiang Tan 043ba1d179
DEV: Fix job cluster concurrency spec timing out (#25035) 
Why this change?

On CI, we have been seeing the "handles job concurrency" job timing out
on CI after 45 seconds. Upon closer inspection of `Jobs::Base#perform`
when cluster concurrency has been set, we see that a thread is spun up
to extend the expiring of a redis key by 120 seconds every 60 seconds
while the job is still being executed. The thread looks like this before
the fix:

```
keepalive_thread =
  Thread.new do
    while parent_thread.alive? && !finished
      Discourse.redis.without_namespace.expire(cluster_concurrency_redis_key, 120)
      sleep 60
    end
  end
```

In an ensure block of `Jobs::Base#perform`, the thread is stop by doing
something like this:

```
finished = true
keepalive_thread.wakeup
keepalive_thread.join
```

If the thread is sleeping, `keepalive_thread.wakeup` will stop the
`sleep` method and run the next iteration causing the thread to
complete. However, there is a timing issue at play here. If
`keepalive_thread.wakeup` is called at a time when the thread is not
sleeping, it will have no effect and the thread may end up sleeping for
60 seconds which is longer than our timeout on CI of 45 seconds.

What does this change do?

1. Change `sleep 60` to sleep in intervals of 1 second checking if the
   job has been finished each time.

2. Add `use_redis_snapshotting` to `Jobs::Base` spec since Redis is
   involved in scheduling and we want to ensure we don't leak Redis
keys.

3. Add `ConcurrentJob.stop!` and `thread.join` to `ensure` block in "handles job concurrency"
   test since a failing expectation will cause us to not clean up the
thread we created in the test.
 		2023-12-26 14:47:03 +08:00
..
regular DEV: Fix various spec linting issues (#24672) 2023-12-04 13:45:19 +01:00
about_stats_spec.rb
…
activation_reminder_emails_spec.rb
…
admin_problems_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
auto_queue_handler_spec.rb DEV: Convert min_trust_to_flag_posts setting to groups (#24864) 2023-12-13 17:18:42 +08:00
automatic_group_membership_spec.rb
…
backfill_sidebar_site_settings_spec.rb FIX: Set `Jobs::BackfillSidebarSiteSettings` cluster concurrency to 1 (#22316) 2023-06-28 13:07:46 +08:00
bookmark_reminder_notifications_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
bulk_grant_trust_level_spec.rb
…
bulk_invite_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
bump_topic_spec.rb DEV: Fix various rubocop lints (#24749) 2023-12-06 23:25:00 +01:00
change_display_name_spec.rb SECURITY: escape display names 2023-11-09 13:39:06 +11:00
check_new_features_spec.rb DEV: Allow new_features URL to be configurable (#24306) 2023-11-09 13:50:21 +10:00
check_translation_overrides_spec.rb FIX: Don't show admin warnings about deleted translation overrides (#22614) 2023-07-14 16:52:39 +08:00
clean_dismissed_topic_users_spec.rb
…
clean_up_associated_accounts_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
clean_up_crawler_stats_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
clean_up_email_change_requests_spec.rb
…
clean_up_email_logs_spec.rb
…
clean_up_inactive_users_spec.rb FIX: consider users.created_at for inactive cleanup (#21688) 2023-05-23 13:41:23 +05:30
clean_up_post_reply_keys_spec.rb
…
clean_up_tags_spec.rb FIX: log for CleanUpTags job (#23964) 2023-10-18 03:24:14 +00:00
clean_up_unused_staged_users_spec.rb
…
clean_up_unused_user_api_keys_spec.rb FEATURE: site settings to revoke api keys older than a number of days (#23595) 2023-09-15 16:31:29 -03:00
clean_up_uploads_spec.rb DEV: Housekeeping for CleanUpUploads job (#24361) 2023-11-20 09:50:09 +10:00
clean_up_user_api_keys_max_life_spec.rb FEATURE: site settings to revoke api keys older than a number of days (#23595) 2023-09-15 16:31:29 -03:00
clean_up_user_export_topics_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
cleanup_imap_sync_log_spec.rb
…
close_topic_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
correct_missing_dualstack_urls_spec.rb DEV: Use Discourse::SYSTEM_USER_ID in fixtures/009_users (#24743) 2023-12-07 09:04:45 +10:00
crawl_topic_link_spec.rb
…
create_linked_topic_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
create_missing_avatars_spec.rb
…
create_recent_post_search_indexes_spec.rb DEV: Fix random typos (#24756) 2023-12-06 22:25:26 +01:00
create_user_reviewable_spec.rb
…
delete_replies_spec.rb DEV: Fix various rubocop lints (#24749) 2023-12-06 23:25:00 +01:00
delete_topic_spec.rb DEV: Fix various rubocop lints (#24749) 2023-12-06 23:25:00 +01:00
disable_bootstrap_mode_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
download_avatar_from_url_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
download_backup_email_spec.rb
…
download_profile_background_from_url_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
emit_web_hook_event_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
enable_bootstrap_mode_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
enqueue_digest_emails_spec.rb DEV: Fix various rubocop lints (#24749) 2023-12-06 23:25:00 +01:00
enqueue_suspect_users_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
ensure_s3_uploads_existence_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
export_csv_file_spec.rb DEV: there is no need anymore to wrap export methods into enumerators (#22567) 2023-08-17 22:09:58 +04:00
export_user_archive_spec.rb DEV: Convert min_trust_to_flag_posts setting to groups (#24864) 2023-12-13 17:18:42 +08:00
feature_topic_users_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
fix_out_of_sync_user_uploaded_avatar_spec.rb
…
fix_primary_emails_for_staged_users_spec.rb
…
fix_s3_etags_spec.rb
…
fix_user_usernames_and_groups_names_clash_spec.rb
…
grant_anniversary_badges_spec.rb
…
grant_new_user_of_the_month_badges_spec.rb
…
heartbeat_spec.rb
…
ignored_users_summary_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
invalidate_inactive_admins_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
invite_email_spec.rb DEV: Fix various rubocop lints (#24749) 2023-12-06 23:25:00 +01:00
jobs_base_spec.rb DEV: Fix job cluster concurrency spec timing out (#25035) 2023-12-26 14:47:03 +08:00
jobs_spec.rb
…
mass_award_badge_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
migrate_badge_image_to_uploads_spec.rb
…
notify_category_change_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
notify_mailing_list_subscribers_spec.rb DEV: Fix various spec linting issues (#24672) 2023-12-04 13:45:19 +01:00
notify_moved_posts_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
notify_reviewable_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
notify_tag_change_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
old_keys_reminder_spec.rb
…
open_topic_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
pending_queued_posts_reminder_spec.rb
…
pending_reviewables_reminder_spec.rb DEV: Convert min_trust_to_flag_posts setting to groups (#24864) 2023-12-13 17:18:42 +08:00
pending_users_reminder_spec.rb
…
periodical_updates_spec.rb
…
poll_mailbox_spec.rb FEATURE: Add hooks for email poller plugins (#21384) 2023-06-26 13:16:03 +08:00
post_update_topic_tracking_state_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
post_uploads_recovery_spec.rb
…
problem_check_spec.rb DEV: Fix various rubocop lints (#24749) 2023-12-06 23:25:00 +01:00
problem_checks_spec.rb DEV: Add support for custom retries for scheduled admin checks (#24224) 2023-11-06 08:57:02 +08:00
process_bulk_invite_emails_spec.rb
…
process_email_spec.rb
…
process_post_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
process_shelved_notifications_spec.rb DEV: Fix various rubocop lints (#24749) 2023-12-06 23:25:00 +01:00
publish_topic_to_category_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
pull_hotlinked_images_spec.rb DEV: Convert min_trust_to_create_topic to groups (#24740) 2023-12-13 14:50:13 +11:00
pull_user_profile_hotlinked_images_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
purge_expired_ignored_users_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
rebake_custom_emoji_posts_spec.rb
…
refresh_users_reviewable_counts_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
reindex_search_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
remove_banner_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
reviewable_priorities_spec.rb
…
send_push_notification_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
send_system_message_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
suspicious_login_spec.rb
…
sync_acls_for_uploads_spec.rb
…
sync_topic_user_bookmarked_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
tl3_promotions_spec.rb
…
toggle_topic_closed_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
topic_timer_enqueuer_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
truncate_user_flag_stats_spec.rb DEV: Convert min_trust_to_flag_posts setting to groups (#24864) 2023-12-13 17:18:42 +08:00
unsilence_users_spec.rb
…
update_animated_uploads_spec.rb
…
update_gravatar_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
update_s3_inventory_spec.rb
…
update_username_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
user_email_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00