discourse-ai

mirror of https://github.com/discourse/discourse-ai.git synced 2025-02-07 03:58:18 +00:00

History

Rafael dos Santos Silva eb93b21769

FEATURE: Add BGE-M3 embeddings support (#569 )

BAAI/bge-m3 is an interesting model, that is multilingual and with a
context size of 8192. Even with a 16x larger context, it's only 4x slower
to compute it's embeddings on the worst case scenario.

Also includes a minor refactor of the rake task, including setting model
and concurrency levels when running the backfill task.

2024-04-10 17:24:01 -03:00

all-mpnet-base-v2.json

FIX: Disable truncation and padding in all-mpnet-base-v2 tokenizer (#105 )

2023-07-13 21:09:46 -03:00

Apache License

Refinements to embeddings and tokenizers (#61 )

2023-05-15 15:10:42 -03:00

bert-base-uncased.json

FEATURE: Add a basic tokenizer API (#37 )

2023-04-19 11:55:59 -03:00

bge-large-en.json

FEATURE: Bge-large-en embeddings via Cloudflare Workers AI API (#241 )