Roman Rizzi ddf2bf7034 DEV: Backfill embeddings concurrently. (#941) ... We are adding a new method for generating and storing embeddings in bulk, which relies on `Concurrent::Promises::Future`. Generating an embedding consists of three steps: Prepare text HTTP call to retrieve the vector Save to DB. Each one is independently executed on whatever thread the pool gives us. We are bringing a custom thread pool instead of the global executor since we want control over how many threads we spawn to limit concurrency. We also avoid firing thousands of HTTP requests when working with large batches.		2024-11-26 14:12:32 -03:00
..
ai_bot	FEATURE: LLM mentions and auto silence (#949)	2024-11-26 07:19:56 +11:00
ai_helper	FEATURE: Truncate AI Captions to a reasonable max size (#907)	2024-11-12 15:52:46 -03:00
automation	FEATURE: LLM mentions and auto silence (#949)	2024-11-26 07:19:56 +11:00
completions	FIX: cancel functionality regressed (#938)	2024-11-21 17:51:45 +11:00
configuration	FEATURE: improve visibility of AI usage in LLM page (#845)	2024-10-22 11:16:02 +11:00
database	DEV: port directory structure to Zeitwerk (#319)	2023-11-29 15:17:46 +11:00
discord/bot	FEATURE: Discord Bot integration (#831)	2024-10-16 12:41:18 -03:00
embeddings	DEV: Backfill embeddings concurrently. (#941)	2024-11-26 14:12:32 -03:00
inference	REFACTOR: Tidy-up embedding endpoints config. (#937)	2024-11-25 13:12:43 -03:00
nsfw	REFACTOR: Tidy-up embedding endpoints config. (#937)	2024-11-25 13:12:43 -03:00
sentiment	FEATURE: Refinements to Emotion in dashboard (#947)	2024-11-25 11:31:51 -03:00
summarization	FEATURE: Calculate gists from non hot topics too (#958)	2024-11-26 13:44:12 -03:00
tasks/modules	FEATURE: Index embeddings using bit vectors (#824)	2024-10-14 13:26:03 -03:00
tokenizer	FIX/REFACTOR: FoldContent revamp (#866)	2024-10-25 11:51:17 -03:00
toxicity	REFACTOR: Tidy-up embedding endpoints config. (#937)	2024-11-25 13:12:43 -03:00
utils	FEATURE: Add basic connection check to DNS SRV resources (#563)	2024-04-12 10:39:19 -03:00
automation.rb	FEATURE: LLM mentions and auto silence (#949)	2024-11-26 07:19:56 +11:00
chat_message_classificator.rb	DEV: port directory structure to Zeitwerk (#319)	2023-11-29 15:17:46 +11:00
classificator.rb	DEV: port directory structure to Zeitwerk (#319)	2023-11-29 15:17:46 +11:00
engine.rb	DEV: port directory structure to Zeitwerk (#319)	2023-11-29 15:17:46 +11:00
guardian_extensions.rb	FEATURE: Calculate gists from non hot topics too (#958)	2024-11-26 13:44:12 -03:00
multisite_hash.rb	FIX: properly cache user locale (#593)	2024-04-26 09:28:35 -03:00
post_classificator.rb	DEV: port directory structure to Zeitwerk (#319)	2023-11-29 15:17:46 +11:00
summarization.rb	FEATURE: Generate topic gists for the hot topics list. (#837)	2024-10-18 18:01:39 -03:00
topic_extensions.rb	PERF: Preload only gists when including summaries in topic list (#948)	2024-11-25 12:24:02 -03:00