mirror of
https://github.com/discourse/discourse-ai.git
synced 2025-02-19 09:54:57 +00:00
791fad1e6a
On very large sites, the rare cache misses for Related Topics can take around 200ms, which affects our p99 metric on the topic page. In order to mitigate this impact, we now have several tools at our disposal. First, one is to migrate the index embedding type from halfvec to bit and change the related topic query to leverage the new bit index by changing the search algorithm from inner product to Hamming distance. This will reduce our index sizes by 90%, severely reducing the impact of embeddings on our storage. By making the related query a bit smarter, we can have zero impact on recall by using the index to over-capture N*2 results, then re-ordering those N*2 using the full halfvec vectors and taking the top N. The expected impact is to go from 200ms to <20ms for cache misses and from a 2.5GB index to a 250MB index on a large site. Another tool is migrating our index type from IVFFLAT to HNSW, which can increase the cache misses performance even further, eventually putting us in the under 5ms territory. Co-authored-by: Roman Rizzi <roman@discourse.org>
38 lines
1.7 KiB
Ruby
38 lines
1.7 KiB
Ruby
# frozen_string_literal: true
|
|
class DropOldEmbeddingsIndexes < ActiveRecord::Migration[7.1]
|
|
def up
|
|
execute <<~SQL
|
|
DROP INDEX IF EXISTS ai_topic_embeddings_1_1_search;
|
|
DROP INDEX IF EXISTS ai_topic_embeddings_2_1_search;
|
|
DROP INDEX IF EXISTS ai_topic_embeddings_3_1_search;
|
|
DROP INDEX IF EXISTS ai_topic_embeddings_4_1_search;
|
|
DROP INDEX IF EXISTS ai_topic_embeddings_5_1_search;
|
|
DROP INDEX IF EXISTS ai_topic_embeddings_6_1_search;
|
|
DROP INDEX IF EXISTS ai_topic_embeddings_7_1_search;
|
|
DROP INDEX IF EXISTS ai_topic_embeddings_8_1_search;
|
|
|
|
DROP INDEX IF EXISTS ai_post_embeddings_1_1_search;
|
|
DROP INDEX IF EXISTS ai_post_embeddings_2_1_search;
|
|
DROP INDEX IF EXISTS ai_post_embeddings_3_1_search;
|
|
DROP INDEX IF EXISTS ai_post_embeddings_4_1_search;
|
|
DROP INDEX IF EXISTS ai_post_embeddings_5_1_search;
|
|
DROP INDEX IF EXISTS ai_post_embeddings_6_1_search;
|
|
DROP INDEX IF EXISTS ai_post_embeddings_7_1_search;
|
|
DROP INDEX IF EXISTS ai_post_embeddings_8_1_search;
|
|
|
|
DROP INDEX IF EXISTS ai_document_fragment_embeddings_1_1_search;
|
|
DROP INDEX IF EXISTS ai_document_fragment_embeddings_2_1_search;
|
|
DROP INDEX IF EXISTS ai_document_fragment_embeddings_3_1_search;
|
|
DROP INDEX IF EXISTS ai_document_fragment_embeddings_4_1_search;
|
|
DROP INDEX IF EXISTS ai_document_fragment_embeddings_5_1_search;
|
|
DROP INDEX IF EXISTS ai_document_fragment_embeddings_6_1_search;
|
|
DROP INDEX IF EXISTS ai_document_fragment_embeddings_7_1_search;
|
|
DROP INDEX IF EXISTS ai_document_fragment_embeddings_8_1_search;
|
|
SQL
|
|
end
|
|
|
|
def down
|
|
raise ActiveRecord::IrreversibleMigration
|
|
end
|
|
end
|