discourse/db/migrate/20210922064213_alter_bumped...

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

28 lines
701 B
Ruby
Raw Permalink Normal View History

PERF: Improve database query perf when loading topics for a category. (#14416) * PERF: Improve database query perf when loading topics for a category. Instead of left joining the `topics` table against `categories` by filtering with `categories.id`, we can improve the query plan by filtering against `topics.category_id` first before joining which helps to reduce the number of rows in the topics table that has to be joined against the other tables and also make better use of our existing index. The following is a before and after of the query plan for a category with many subcategories. Before: ``` QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- Limit (cost=1.28..747.09 rows=30 width=12) (actual time=85.502..2453.727 rows=30 loops=1) -> Nested Loop Left Join (cost=1.28..566518.36 rows=22788 width=12) (actual time=85.501..2453.722 rows=30 loops=1) Join Filter: (category_users.category_id = topics.category_id) Filter: ((topics.category_id = 11) OR (COALESCE(category_users.notification_level, 1) <> 0) OR (tu.notification_level > 1)) -> Nested Loop Left Join (cost=1.00..566001.58 rows=22866 width=20) (actual time=85.494..2453.702 rows=30 loops=1) Filter: ((COALESCE(tu.notification_level, 1) > 0) AND ((topics.category_id <> 11) OR (topics.pinned_at IS NULL) OR ((t opics.pinned_at <= tu.cleared_pinned_at) AND (tu.cleared_pinned_at IS NOT NULL)))) Rows Removed by Filter: 1 -> Nested Loop (cost=0.57..528561.75 rows=68606 width=24) (actual time=85.472..2453.562 rows=31 loops=1) Join Filter: ((topics.category_id = categories.id) AND ((categories.topic_id <> topics.id) OR (categories.id = 1 1))) Rows Removed by Join Filter: 13938306 -> Index Scan using index_topics_on_bumped_at on topics (cost=0.42..100480.05 rows=715549 width=24) (actual ti me=0.010..633.015 rows=464623 loops=1) Filter: ((deleted_at IS NULL) AND ((archetype)::text <> 'private_message'::text)) Rows Removed by Filter: 105321 -> Materialize (cost=0.14..36.04 rows=30 width=8) (actual time=0.000..0.002 rows=30 loops=464623) -> Index Scan using categories_pkey on categories (cost=0.14..35.89 rows=30 width=8) (actual time=0.006. .0.040 rows=30 loops=1) Index Cond: (id = ANY ('{11,53,57,55,54,56,112,94,107,115,116,117,97,95,102,103,101,105,99,114,106,1 13,104,98,100,96,108,109,110,111}'::integer[])) -> Index Scan using index_topic_users_on_topic_id_and_user_id on topic_users tu (cost=0.43..0.53 rows=1 width=16) (a ctual time=0.004..0.004 rows=0 loops=31) Index Cond: ((topic_id = topics.id) AND (user_id = 1103877)) -> Materialize (cost=0.28..2.30 rows=1 width=8) (actual time=0.000..0.000 rows=0 loops=30) -> Index Scan using index_category_users_on_user_id_and_last_seen_at on category_users (cost=0.28..2.29 rows=1 width =8) (actual time=0.004..0.004 rows=0 loops=1) Index Cond: (user_id = 1103877) Planning Time: 1.359 ms Execution Time: 2453.765 ms (23 rows) ``` After: ``` QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Limit (cost=1.28..438.55 rows=30 width=12) (actual time=38.297..657.215 rows=30 loops=1) -> Nested Loop Left Join (cost=1.28..195944.68 rows=13443 width=12) (actual time=38.296..657.211 rows=30 loops=1) Filter: ((categories.topic_id <> topics.id) OR (topics.category_id = 11)) Rows Removed by Filter: 29 -> Nested Loop Left Join (cost=1.13..193462.59 rows=13443 width=16) (actual time=38.289..657.092 rows=59 loops=1) Join Filter: (category_users.category_id = topics.category_id) Filter: ((topics.category_id = 11) OR (COALESCE(category_users.notification_level, 1) <> 0) OR (tu.notification_level > 1)) -> Nested Loop Left Join (cost=0.85..193156.79 rows=13489 width=20) (actual time=38.282..657.059 rows=59 loops=1) Filter: ((COALESCE(tu.notification_level, 1) > 0) AND ((topics.category_id <> 11) OR (topics.pinned_at IS NULL) OR ((topics.pinned_at <= tu.cleared_pinned_at) AND (tu.cleared_pinned_at IS NOT NULL)))) Rows Removed by Filter: 1 -> Index Scan using index_topics_on_bumped_at on topics (cost=0.42..134521.06 rows=40470 width=24) (actual time=38.267..656.850 rows=60 loops=1) Filter: ((deleted_at IS NULL) AND ((archetype)::text <> 'private_message'::text) AND (category_id = ANY ('{11,53,57,55,54,56,112,94,107,115,116,117,97,95,102,103,101,105,99,114,106,113,104,98,100,96,108,109,110,111}'::integer[]))) Rows Removed by Filter: 569895 -> Index Scan using index_topic_users_on_topic_id_and_user_id on topic_users tu (cost=0.43..1.43 rows=1 width=16) (actual time=0.003..0.003 rows=0 loops=60) Index Cond: ((topic_id = topics.id) AND (user_id = 1103877)) -> Materialize (cost=0.28..2.30 rows=1 width=8) (actual time=0.000..0.000 rows=0 loops=59) -> Index Scan using index_category_users_on_user_id_and_last_seen_at on category_users (cost=0.28..2.29 rows=1 width=8) (actual time=0.004..0.004 rows=0 loops=1) Index Cond: (user_id = 1103877) -> Index Scan using categories_pkey on categories (cost=0.14..0.17 rows=1 width=8) (actual time=0.001..0.001 rows=1 loops=59) Index Cond: (id = topics.category_id) Planning Time: 1.633 ms Execution Time: 657.255 ms (22 rows) ``` * PERF: Optimize index on topics bumped_at. Replace `index_topics_on_bumped_at` index with a partial index on `Topic#bumped_at` filtered by archetype since there is already another index that covers private topics.
2021-09-27 22:05:00 -04:00
# frozen_string_literal: true
class AlterBumpedAtIndexesOnTopics < ActiveRecord::Migration[6.1]
disable_ddl_transaction!
def up
execute(<<~SQL)
CREATE INDEX CONCURRENTLY IF NOT EXISTS index_topics_on_bumped_at_public
ON topics (bumped_at)
WHERE ((deleted_at IS NULL) AND ((archetype)::text <> 'private_message'::text));
SQL
execute(<<~SQL)
DROP INDEX IF EXISTS index_topics_on_bumped_at;
SQL
# The following index is known to have not been properly renamed. Drop it if
# exists just in case.
execute(<<~SQL)
DROP INDEX IF EXISTS index_forum_threads_on_bumped_at;
SQL
end
def down
raise ActiveRecord::IrreversibleMigration
end
end