Commit Graph

47 Commits

Author SHA1 Message Date
Dan Ungureanu 086b46051c
FIX: Zeitwerk-related fixes for jobs. (#8187) 2019-10-14 13:03:22 +03:00
Krzysztof Kotlarek 427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
Gerhard Schlager b788948985 FEATURE: English locale with international date formats
Makes en_US the new default locale
2019-05-20 13:47:20 +02:00
Sam Saffron 30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Robin Ward b58867b6e9 FEATURE: New 'Reviewable' model to make reviewable items generic
Includes support for flags, reviewable users and queued posts, with REST API
backwards compatibility.

Co-Authored-By: romanrizzi <romanalejandro@gmail.com>
Co-Authored-By: jjaffeux <j.jaffeux@gmail.com>
2019-03-28 12:45:10 -04:00
Robin Ward fa5a158683 REFACTOR: Move `queue_jobs` out of `SiteSetting`
It is not a setting, and only relevant in specs. The new API is:

```
Jobs.run_later!        # jobs will be thrown on the queue
Jobs.run_immediately!  # jobs will run right away, avoid the queue
```
2019-03-14 10:47:38 -04:00
David Taylor 0a4562253e DEV: Add 'starting' event to sidekiq log when interval logging enabled 2019-03-08 10:56:36 +00:00
David Taylor e2510d79cc DEV: Improve thread-safety of sidekiq logging 2019-03-08 10:31:49 +00:00
David Taylor 9db05a895a DEV: Add job_id to sidkiq log
This makes it easier to correlate 'pending' logs from the same job
2019-03-08 09:16:13 +00:00
David Taylor df474bceee DEV: Further sidekiq logging stability improvements
- Open the log file in "append" mode. This avoids issues if the file does not exist (and matches standard rails log behavior)
- Correctly parse the interval logging environment variable
2019-03-06 12:50:15 +00:00
David Taylor fe62de68dd DEV: Correct sidekiq logging to avoid thread leak 2019-03-06 10:11:31 +00:00
David Taylor 8b30ed5b7a DEV: Serialize the job parameters in sidekiq logs
Otherwise this can lead to some very large data structures when processing the logs later
2019-03-05 17:44:49 +00:00
David Taylor 8963f1af30
FEATURE: Optional detailed performance logging for Sidekiq jobs (#7091)
By default, this does nothing. Two environment variables are available:

- `DISCOURSE_LOG_SIDEKIQ`

  Set to `"1"` to enable logging. This will log all completed jobs to `log/rails/sidekiq.log`, along with various db/redis/network statistics. This is useful to track down poorly performing jobs.

- `DISCOURSE_LOG_SIDEKIQ_INTERVAL`

  (seconds) Check running jobs periodically, and log their current duration. They will appear in the logs with `status:pending`. This is useful to track down jobs which take a long time, then crash sidekiq before completing.
2019-03-05 11:19:11 +00:00
Sam e08a3f719c FEATURE: push post rebake regular task to low priority queue
This allows us to run regular rebakes without starving the normal queue.

It additionally adds the ability to specify queue with `Jobs.enqueue` so
we can specifically queue a job with lower priority using the `queue` arg.
2019-01-09 08:57:20 +11:00
Sam e4498d2a8a FIX: keep db and job correctly in multisite logs
This ensures we report job and db correctly, previously we were
only reporting this on default
2018-09-04 16:05:44 +10:00
Sam 44cf3cf975 FIX: queue heartbeats in readonly modes
If sidekiq is paused or Discourse is in readonly continue to queue
heartbeats

If we do not do that then a master process can end up reaping sidekiq
workers and causing various badness

This also impacts restore which can do weird stuff TM in cases like this
2018-08-29 12:36:59 +10:00
Sam 1f17b84b63 FEATURE: more context for error reporting on jobs fails 2018-08-16 12:38:49 +10:00
Neil Lalonde 4ad7ce70ce REFACTOR: extract scheduler to the mini_scheduler gem 2018-07-31 17:12:55 -04:00
Sam 7c5a71e929 DEV: allow queue_jobs = false in dev
your mileage may vary
2017-10-31 13:50:58 +11:00
Guo Xiang Tan 9dcb11f553 Fix the build. 2017-10-11 17:45:19 +08:00
Guo Xiang Tan 09721090a3 FIX: Ensure that we revert back to default connection after running jobs. 2017-10-11 17:17:03 +08:00
Guo Xiang Tan 59aeb0bc56 FIX: Sidekiq hot reloading wasn't working in dev.
https://meta.discourse.org/t/webhooks-sidekiq-issue-on-dev-instance/71129

* Remove code that is no longer required as well.
2017-10-09 18:23:25 +08:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Régis Hanol 7db2083d45 FIX: 'cancel_scheduled_job' was deleting all jobs in multisite 2016-08-12 13:10:52 +02:00
Régis Hanol 5dcdfb9777 ensure default locale is 'en' instead of nil 2016-06-30 17:37:00 +02:00
Régis Hanol d224966a0e FIX: retry sending an email in 1 hour when SMTP server is busy 2016-05-09 20:37:33 +02:00
Régis Hanol 8fcd359e2a Revert "FIX: Jobs.cancel_scheduled_job wasn't working anymore due to our move to using multiple queues"
This reverts commit b7c16991f7.
2016-04-13 18:30:25 +02:00
Régis Hanol b7c16991f7 FIX: Jobs.cancel_scheduled_job wasn't working anymore due to our move to using multiple queues
FIX: Don't queue more than 1 'update_gravatar' job per user
2016-04-13 18:12:28 +02:00
Robin Ward 526573074c Add one off job to grant the emoji badge retroactively 2016-04-07 13:49:44 -04:00
Sam 8ec7fd84fd FEATURE: prioritize sidekiq jobs
This commit introduces 3 queues for sidekiq

"critical" for urgent jobs (weighted at 4x weight)
"default" for standard jobs(weighted at 2x weight)
"low" for less important jobs


"critical jobs"

Reset Password emails has been seperated to its own job
Heartbeat which is required to keep sidekiq running
Test email which needs to return real quick


"low priority jobs"

Notify mailing list
Pull hotlinked images
Update gravatar

"default"

All the rest

Note: for people running sidekiq from command line use

bin/sidekiq -q critical,4 -q default,2 -q low
2016-04-07 12:56:43 +10:00
Régis Hanol 737c606710 FIX: 'cancel_scheduled_job' wasn't working due to sidekiq upgrade 2016-01-13 09:08:26 +01:00
Régis Hanol fbacaab2fc FIX: disable scheduled jobs when in readonly mode 2016-01-11 18:31:28 +01:00
Sam Saffron 426299d261 FEATURE: upgrade to Sidekiq 4 2016-01-01 15:40:31 +11:00
Robin Ward 3720783c1b Refactor to our own Discourse I18n backend
This removes some monkey patches and makes testing easier.
It will also support database backed I18n changes.
2015-11-13 16:35:02 -05:00
Gerhard Schlager 619d5b1bc1 FIX: Load fallback locales in Sidekiq jobs 2015-09-18 18:21:27 +02:00
Régis Hanol faf4f44776 FEATURE: make pin expiration mandatory 2015-07-29 16:34:21 +02:00
Tan Le c55fdc7a75 A shorter and more concise version of select..size 2015-04-12 22:26:48 +10:00
Sam bb20f64cb2 use standard error so its easier to catch 2015-03-23 12:20:50 +11:00
riking 5657006aca Rename handle_exception to handle_job_exception 2015-02-09 12:47:46 -08:00
Régis Hanol d7ef4f1edd remove useless 'puts' 2014-11-26 16:40:03 +01:00
Sam 8699c929db FIX: report db when we get an exception in multisite 2014-08-19 17:24:00 +10:00
riking d90404e830 Change 'code' to 'message' 2014-07-17 15:19:58 -07:00
riking 12cb682548 Start passing more context to Discourse.handle_exception 2014-07-17 14:11:56 -07:00
Sam Saffron 451598c511 BUGFIX: regresses correct job execution 2014-02-21 16:05:19 +11:00
Sam Saffron 9a3af8997b BUGFIX: handle partial job failure in multisite
log all failures
2014-02-21 15:31:15 +11:00
Sam e1f293ad66 FEATURE: new scheduler
Removed sidetiq, introduced new scheduler

- add basic UI
- add schedule discover
- add scheduling in initializer
2014-02-06 10:26:16 +11:00
Sam f0a122a66c move job files so they live underneath app/ and not in lib/
introduce new setting email_always, that will force emails to send to users regardless of presence on site
2013-10-01 17:04:02 +10:00