PERF: Avoid using `ObjectSpace.each_object` in `Jobs::Onceoff.enqueue_all` (#28072)

We are investigating a memory leak in Sidekiq and saw the following line
when comparing heap dumps over time.

`Allocated IMEMO 14775 objects of size 591000/7389528 (in bytes) at:
/var/www/discourse/app/jobs/onceoff/onceoff.rb:36`

That line in question was doing a `.select { |klass| klass < self  }` on
`ObjectSpace.each_object(Class)`. This for some reason is allocating a
whole bunch of `IMEMO` objects which are instruction sequence objects.

Instead of diving deeper into why this might be leaking, we can just
save our time by switching to an implementation that is more efficient
and does not require looping through a ton of objects.
This commit is contained in:
Alan Guo Xiang Tan 2024-07-25 13:30:56 +08:00 committed by GitHub
parent 3838514d4e
commit f4d06f195d
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 14 additions and 11 deletions

View File

@ -3,6 +3,15 @@
class Jobs::Onceoff < ::Jobs::Base
sidekiq_options retry: false
class << self
attr_reader :onceoff_job_klasses
def inherited(klass)
@onceoff_job_klasses ||= Set.new
@onceoff_job_klasses << klass
end
end
def self.name_for(klass)
klass.name.sub(/\AJobs\:\:/, "")
end
@ -31,12 +40,9 @@ class Jobs::Onceoff < ::Jobs::Base
def self.enqueue_all
previously_ran = OnceoffLog.pluck(:job_name).uniq
ObjectSpace
.each_object(Class)
.select { |klass| klass < self }
.each do |klass|
job_name = name_for(klass)
Jobs.enqueue(job_name.underscore.to_sym) if previously_ran.exclude?(job_name)
end
self.onceoff_job_klasses.each do |klass|
job_name = name_for(klass)
Jobs.enqueue(job_name.underscore.to_sym) if previously_ran.exclude?(job_name)
end
end
end

View File

@ -7,9 +7,6 @@ RSpec.describe ::Jobs::Onceoff do
require_relative "../../app/jobs/onceoff/" + File.basename(f)
end
ObjectSpace
.each_object(Class)
.select { |klass| klass.superclass == ::Jobs::Onceoff }
.each { |job| job.new.execute_onceoff(nil) }
described_class.onceoff_job_klasses.each { |job| job.new.execute_onceoff(nil) }
end
end