discourse-ai/lib/embeddings/vector_representations
Rafael dos Santos Silva eb93b21769
FEATURE: Add BGE-M3 embeddings support (#569)
BAAI/bge-m3 is an interesting model, that is multilingual and with a
context size of 8192. Even with a 16x larger context, it's only 4x slower
to compute it's embeddings on the worst case scenario.

Also includes a minor refactor of the rake task, including setting model
and concurrency levels when running the backfill task.
2024-04-10 17:24:01 -03:00
..
all_mpnet_base_v2.rb FEATURE: AI Quick Semantic Search (#501) 2024-03-08 13:02:50 -03:00
base.rb FEATURE: Add BGE-M3 embeddings support (#569) 2024-04-10 17:24:01 -03:00
bge_large_en.rb FEATURE: AI Quick Semantic Search (#501) 2024-03-08 13:02:50 -03:00
bge_m3.rb FEATURE: Add BGE-M3 embeddings support (#569) 2024-04-10 17:24:01 -03:00
gemini.rb FEATURE: AI Quick Semantic Search (#501) 2024-03-08 13:02:50 -03:00
multilingual_e5_large.rb FEATURE: AI Quick Semantic Search (#501) 2024-03-08 13:02:50 -03:00
text_embedding_3_large.rb FEATURE: AI Quick Semantic Search (#501) 2024-03-08 13:02:50 -03:00
text_embedding_3_small.rb FEATURE: AI Quick Semantic Search (#501) 2024-03-08 13:02:50 -03:00
text_embedding_ada_002.rb FEATURE: AI Quick Semantic Search (#501) 2024-03-08 13:02:50 -03:00