discourse/script/import_scripts
Alan Guo Xiang Tan 7c321d3aad
PERF: Update `Group#user_count` counter cache outside DB transaction (#19256)
While load testing our user creation code path in production, we
identified that executing the DB statement to update the `Group#user_count` column within a
transaction is creating a bottleneck for us. This is because the
creation of a user and addition of the user to the relevant groups are
done in a transaction. When we execute the DB statement to update
`Group#user_count` for the relevant group, a row level lock is held
until the transaction completes. This row level lock acts like a global
lock when the server is creating users that will be added to the same
group in quick succession.

Instead of updating the counter cache within a transaction which the
default ActiveRecord `counter_cache` option does, we simply update the
counter cache outside of the committing transaction.

Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>

Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>
2022-11-30 11:52:08 -03:00
..
base DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
mbox DEV: mbox importer improvements (#16557) 2022-04-29 13:24:29 -03:00
nodebb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
phpbb3 FIX: the phpbbb import script was not parsing youtube tags (#17787) 2022-08-05 15:20:32 -04:00
quandora DEV: Correct typos and spelling mistakes (#12812) 2021-05-21 11:43:47 +10:00
settings FEATURE: Import script for jForum 2020-06-21 12:12:42 +02:00
socialcast DEV: Correct typos and spelling mistakes (#12812) 2021-05-21 11:43:47 +10:00
support DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
answerbase.rb Make import scripts work with frozen strings 2019-05-30 22:22:24 +02:00
answerhub.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
askbot.rb DEV: Correct typos and spelling mistakes (#12812) 2021-05-21 11:43:47 +10:00
base.rb FEATURE: Promote polymorphic bookmarks to default and migrate (#16729) 2022-05-23 10:07:15 +10:00
bbpress.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
bespoke_1.rb Make import scripts work with frozen strings 2019-05-30 22:22:24 +02:00
csv_importer.rb FEATURE: Adding a simple CSV importer (#17993) 2022-08-19 13:09:30 -04:00
csv_restore_staged_users.rb DEV: Add a CSV importer for restoring deleted users (#12147) 2021-02-19 13:46:54 -07:00
discuz_x.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
disqus.rb DEV: Apply Rubocop redundant return style 2019-11-14 15:10:51 -05:00
drupal-6.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
drupal.rb DEV: pull email address validation out to a new EmailAddressValidator 2022-02-17 21:49:22 -05:00
drupal_json.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
drupal_qa.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
elgg.rb FEATURE: Add import script for Elgg (#19140) 2022-11-28 16:28:08 +01:00
flarum_import.rb DEV: fix Flarum import script (#13385) 2021-06-15 19:08:55 +05:30
fluxbb.rb Fix syntax error in fluxbb.rb (#9727) 2020-05-11 11:07:57 -04:00
friendsmegplus.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
getsatisfaction.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
google_groups.rb DEV: Correctly tag heredocs (#16061) 2022-02-28 20:50:55 +01:00
higher_logic.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
ipboard.rb DEV: Correctly tag heredocs (#16061) 2022-02-28 20:50:55 +01:00
ipboard3.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
jforum.rb DEV: Fix rubocop issues (#14715) 2021-10-27 11:39:28 +03:00
jive.rb FEATURE: Nokogumbo (#9577) 2020-05-05 13:46:57 +10:00
jive_api.rb FEATURE: Polymorphic bookmarks pt. 3 (reminders, imports, exports, refactors) (#16591) 2022-05-09 09:37:23 +10:00
json_generic.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
kunena.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
kunena3.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
lithium.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
mbox.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
modx.rb DEV: Correctly tag heredocs (#16061) 2022-02-28 20:50:55 +01:00
muut.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
mybb.rb FEATURE: my_bb import supports avatars (#17617) 2022-07-25 15:22:25 +02:00
mybbru.rb FEATURE: add mybb.ru import script (#8609) 2019-12-20 11:10:18 -05:00
mylittleforum.rb DEV: Correctly tag heredocs (#16061) 2022-02-28 20:50:55 +01:00
nabble.rb Make import scripts work with frozen strings 2019-05-30 22:22:24 +02:00
ning.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
phorum.rb DEV: Apply rubocop to script/import_scripts/phorum.rb (#14727) 2021-10-26 19:16:52 +01:00
phpbb3.rb Fix comment to include phpbb 3.3, which is now supported (#18006) 2022-08-19 16:42:32 -04:00
punbb.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
question2answer.rb DEV: Improve q2a import script (#10346) 2020-07-31 12:04:03 -05:00
sfn.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
simplepress.rb fix invalid byte sequence in UTF-8 (ArgumentError) (#9077) 2020-02-28 10:26:18 -05:00
smf1.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
smf2.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
sourceforge.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
stack_overflow.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
telligent.rb FEATURE: Add last visit indication to topic view page. (#13471) 2021-07-05 14:17:31 +08:00
vanilla.rb FIX: Vanilla importer fixes (#14699) 2021-10-27 14:22:37 +02:00
vanilla_body_parser.rb Improve Vanilla import script. (#11701) 2021-01-13 23:10:00 +05:30
vanilla_mysql.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
vbulletin.rb PERF: Update `Group#user_count` counter cache outside DB transaction (#19256) 2022-11-30 11:52:08 -03:00
vbulletin5.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
xenforo.rb DEV: Xenforo importer improvements (#18457) 2022-11-28 16:42:39 +01:00
yahoogroup.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
zendesk.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
zendesk_api.rb DEV: Fix rubocop issues (#14715) 2021-10-27 11:39:28 +03:00
zoho.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00