Recurring automations are triggered by a scheduled job that runs every minute and checks for due automations, runs them and then marks as them as completed (by deleting the `PendingAutomation` record). However, the job is currently subject to a race condition where a recurring automation can be executed more than once at its due date if it takes more than a minute to finish.
This commit adds a mutex around the code that triggers the recurring automation so that no concurrent executions can happen for a single automation.
Meta topic: https://meta.discourse.org/t/daily-summary-9pm-utc/291850/119?u=osama.
It's currently possible to setup multiple automation rules that trigger each other resulting in an infinite loop. To prevent that, this commit adds a global "circuit breaker" that prevents all automations from triggering while an automation rule is executing.
Internal topic: t/124365.
Prior to this fix, any change to an automation would reset `pending_automations`, now we only do it if any value related to recurrence (start_date, interval, frequency, execute_at...) has been changed.
It means that any trigger creating `pending_automations` now needs to manage them in the `on_update` callback.
This commit adds a new option to the `user_updated` trigger of the automation plugin to only trigger an automation for new users that join after the automation is enabled.
Internal topic: t/125829/9.
Osama Sayegh
Joffrey JAFFEUX