Commit Graph

13 Commits

Author SHA1 Message Date
Rafael dos Santos Silva 791fad1e6a
FEATURE: Index embeddings using bit vectors ()
On very large sites, the rare cache misses for Related Topics can take around 200ms, which affects our p99 metric on the topic page. In order to mitigate this impact, we now have several tools at our disposal.

First, one is to migrate the index embedding type from halfvec to bit and change the related topic query to leverage the new bit index by changing the search algorithm from inner product to Hamming distance. This will reduce our index sizes by 90%, severely reducing the impact of embeddings on our storage. By making the related query a bit smarter, we can have zero impact on recall by using the index to over-capture N*2 results, then re-ordering those N*2 using the full halfvec vectors and taking the top N. The expected impact is to go from 200ms to <20ms for cache misses and from a 2.5GB index to a 250MB index on a large site.

Another tool is migrating our index type from IVFFLAT to HNSW, which can increase the cache misses performance even further, eventually putting us in the under 5ms territory. 

Co-authored-by: Roman Rizzi <roman@discourse.org>
2024-10-14 13:26:03 -03:00
Rafael dos Santos Silva eb93b21769
FEATURE: Add BGE-M3 embeddings support ()
BAAI/bge-m3 is an interesting model, that is multilingual and with a
context size of 8192. Even with a 16x larger context, it's only 4x slower
to compute it's embeddings on the worst case scenario.

Also includes a minor refactor of the rake task, including setting model
and concurrency levels when running the backfill task.
2024-04-10 17:24:01 -03:00
Rafael dos Santos Silva 140359c2ef
FEATURE: Per post embeddings () 2023-12-29 12:28:45 -03:00
Rafael dos Santos Silva 818b20fb6f
FEATURE: Make embeddings turn-key ()
To ease the administrative burden of enabling the embeddings model, this change introduces automatic backfill when the setting is enabled. It also moves the topic visit embedding creation to a lower priority queue in sidekiq and adds an option to skip embedding computation and persistence when we match on the digest.
2023-10-26 12:07:37 -03:00
Rafael dos Santos Silva 453928e7bb
FIX: Improvment to embeddings index task () 2023-10-02 16:37:13 -03:00
Sam 615eb8b440
FEATURE: add semantic search with hyde bot ()
In specific scenarios (no special filters or limits) we will also
always include 5 semantic results (at least) with every query.

This effectively means that all very wide queries will always return
20 results, regardless of how complex they are.

Also: 

FIX: embedding backfill rake task not working
We renamed internals, this corrects the implementation
2023-09-07 13:25:26 +10:00
Rafael dos Santos Silva 2c0f535bab
FEATURE: HyDE-powered semantic search. ()
* FEATURE: HyDE-powered semantic search.

It relies on the new outlet added on  to display semantic search results in an unobtrusive way.

We'll use a HyDE-backed approach for semantic search, which consists on generating an hypothetical document from a given keywords, which gets transformed into a vector and used in a asymmetric similarity topic search.

This PR also reorganizes the internals to have less moving parts, maintaining one hierarchy of DAOish classes for vector-related operations like transformations and querying.

Completions and vectors created by HyDE will remain cached on Redis for now, but we could later use Postgres instead.

* Missing translation and rate limiting

---------

Co-authored-by: Roman Rizzi <rizziromanalejandro@gmail.com>
2023-09-05 11:08:23 -03:00
Rafael dos Santos Silva 703762a7a9
PERF: .find_each instead of .find to save us from memory allocation peaks
also Fix embeddings rake task for new db structure
2023-07-13 18:59:25 -03:00
Rafael dos Santos Silva 739b314312
Fixes for embeddings and truncate () 2023-05-18 09:21:28 +10:00
Rafael dos Santos Silva f1133f66a6
Updates to embedding rake tasks ()
- Creates embeddings in topic ID order, so it's easier to stop and
restart from where we stopped

- Update index parameters with current best practices
2023-05-09 13:45:16 -03:00
Roman Rizzi 4e05763a99
FEATURE: Semantic assymetric full-page search ()
Depends on 

Hooks to the full-page-search component using an experimental API and performs an assymetric similarity search using our embeddings database.
2023-03-31 15:29:56 -03:00
Rafael dos Santos Silva 6bdbc0e32d
FIX: Proper flow when a topic doesn't have embeddings () 2023-03-20 16:44:55 -03:00
Rafael dos Santos Silva 80d662e9e8
FEATURE: Semantic Suggested Topics () 2023-03-15 17:21:45 -03:00