Commit Graph

202 Commits

Author SHA1 Message Date
Kris 18ecc843e5
UX: move templates to main LLM config tab, restyle (#813)
Restructures LLM config page so it is far clearer. 

Also corrects bugs around adding LLMs and having LLMs not editable post addition 
---------

Co-authored-by: Sam Saffron <sam.saffron@gmail.com>
2024-09-30 17:15:11 +10:00
Hoa Nguyen 1002dc877d
DEV: remove ignore column syntax for the removed provider column in completion prompt model (#810) 2024-09-30 08:57:23 +10:00
Sam 03eccbe392
FEATURE: Make tool support polymorphic (#798)
Polymorphic RAG means that we will be able to access RAG fragments both from AiPersona and AiCustomTool

In turn this gives us support for richer RAG implementations.
2024-09-16 08:17:17 +10:00
Sam 5b9add0ac8
FEATURE: add a SambaNova LLM provider (#797)
Note, at the moment the context window is quite small, it is
mainly useful as a helper backend or hyde generator
2024-09-12 11:28:08 +10:00
Sam cabecb801e
FEATURE: disable rate limiting when skipping hyde (#793)
Embedding search is rate limited due to potentially expensive
hyde operation (which require LLM access).

Embedding generally is very cheap compared to it. (usually 100x cheaper)

This raises the limit to 100 per minute for embedding searches,
while keeping the old 4 per minute for HyDE powered search.
2024-09-04 15:51:01 +10:00
Sam a48acc894a
FEATURE: more accurate and faster titles (#791)
Previously we waited 1 minute before automatically titling PMs

The new change introduces adding a title immediately after the the
llm replies

Prompt was also modified to include the LLM reply in title suggestion.

This helps situation like:

user: tell me a joke
llm: a very funy joke about horses

Then the title would be "A Funny Horse Joke"

Specs already covered some auto title logic, amended to also
catch the new message bus message we have been sending.
2024-09-03 15:52:20 +10:00
Roman Rizzi e408cd080c
FIX: coerce value before downcasing the hyde param (#787) 2024-08-30 12:13:29 -03:00
Sam 584753cf60
FIX: we were never reindexing old content (#786)
* FIX: we were never reindexing old content

Embedding backfill contains logic for searching for old content
change and then backfilling.

Unfortunately it was excluding all topics that had embedding
unconditionally, leading to no backfill ever happening.


This change adds a test and ensures we backfill.

* over select results, this ensures we will be more likely to find
ai results when filtered
2024-08-30 14:37:55 +10:00
Keegan George fdadfa029e
FEATURE: smooth streaming animation for summarization (#778) 2024-08-29 15:07:07 -07:00
Keegan George 94f6c632bf
DEV: Publish AI Bot PM title update to message bus channel (#781) 2024-08-29 14:48:44 -07:00
Rafael dos Santos Silva a08d168740
FEATURE: Initial support for seeded LLMs (#756) 2024-08-28 15:57:58 -03:00
Sam 0687ec75c3
FEATURE: allow embedding based search without hyde (#777)
This allows callers of embedding based search to bypass hyde.

Hyde will expand the search term using an LLM, but if an LLM is
performing the search we can skip this expansion.

It also introduced some tests for the controller which we did not have
2024-08-28 14:17:34 +10:00
Roman Rizzi 64641b6175
FEATURE: LLM Triage support for systemless models. (#757)
* FEATURE: LLM Triage support for systemless models.

This change adds support for OSS models without support for system messages. LlmTriage's system message field is no longer mandatory. We now send the post contents in a separate user message.

* Models using Ollama can also disable system prompts
2024-08-21 11:41:55 -03:00
Sam 14443bf890
FIX: more robust summary implementation (#750)
When navigating between topic we were not correctly resetting
internal state for summarization. This leads to a situation where
incorrect summaries can be displayed to users and wrong summaries
can be displayed.

Additionally our controller for grabbing summaries was always
streaming results via message bus, which could be delayed when
sidekiq is overloaded. We now will return the cached summary
right away if it is available direct from REST endpoint.
2024-08-13 08:47:47 -03:00
Keegan George f72ab12761
DEV: Clearly separate post/composer helper settings (#747) 2024-08-12 15:40:23 -07:00
Keegan George 1d6a6c9f8f
FEATURE: Stream other post helper options (#745) 2024-08-08 11:32:39 -07:00
Roman Rizzi 20efc9285e
FIX: Correctly save provider-specific params for new models. (#744)
Creating a new model, either manually or from presets, doesn't initialize the `provider_params` object, meaning their custom params won't persist.

Additionally, this change adds some validations for Bedrock params, which are mandatory, and a clear message when a completion fails because we cannot build the URL.
2024-08-07 16:08:56 -03:00
Roman Rizzi 7b4c099673
FIX: LlmModel validations. (#742)
- Validate fields to reduce the chance of breaking features by a misconfigured model.
- Fixed a bug where the URL might get deleted during an update.
- Display a warning when a model is currently in use.
2024-08-06 14:35:35 -03:00
Rafael dos Santos Silva 9d887ad4ac
FIX: Use correct date for cached summary (#733) 2024-07-31 15:21:26 -03:00
Sam c16c622b53
FIX: properly pass errors to client (#731)
render_json_error expects a AR model not a serializer, using a
serializer eats up the error message
2024-07-31 17:53:18 +10:00
Roman Rizzi bed044448c
DEV: Remove old code now that features rely on LlmModels. (#729)
* DEV: Remove old code now that features rely on LlmModels.

* Hide old settings and migrate persona llm overrides

* Remove shadowing special URL + seeding code. Use srv:// prefix instead.
2024-07-30 13:44:57 -03:00
Roman Rizzi 5c196bca89
FEATURE: Track if a model can do vision in the llm_models table (#725)
* FEATURE: Track if a model can do vision in the llm_models table

* Data migration
2024-07-24 16:29:47 -03:00
Roman Rizzi f328b81c78
FIX: Make sure custom tool enums follow json-schema. (#718)
Enums didn't work as expected because we the dialect couldn't translate
them correctly. It doesn't understand what "enum_values" is.
2024-07-16 14:23:17 -03:00
Roman Rizzi 5cb91217bd
FIX: Flaky SRV-backed model seeding. (#708)
* Seeding the SRV-backed model should happen inside an initializer.
* Keep the model up to date when the hidden setting changes.
* Use the correct Mixtral model name and fix previous data migration.
* URL validation should trigger only when we attempt to update it.
2024-07-08 18:47:10 -03:00
Keegan George eab2f74b58
DEV: Use site locale for composer helper translations (#698) 2024-07-04 08:23:37 -07:00
Sam 38153608f8
FIX: repair id sequence identity on summary table (#701)
1. Repairs the identity on the summary table, we migrated data without resetting it.
2. Adds an index into ai_summary table to match expected retrieval pattern
2024-07-04 12:23:46 +10:00
Sam 1320eed9b2
FEATURE: move summary to use llm_model (#699)
This allows summary to use the new LLM models and migrates of API key based model selection

Claude 3.5 etc... all work now. 

---------

Co-authored-by: Roman Rizzi <rizziromanalejandro@gmail.com>
2024-07-04 10:48:18 +10:00
Keegan George 1b0ba9197c
DEV: Add summarization logic from core (#658) 2024-07-02 08:51:59 -07:00
Jarek Radosz a5a39dd2ee
DEV: Clean up after #677 (#694)
Follow up to b863ddc94b

Ruby:
* Validate `summary` (the column is `not null`)
* Fix `name` validation (the column has `max_length` 100)
* Fix table annotations
* Accept missing `parameter` attributes (`required, `enum`, `enum_values`)

JS:
* Use native classes
* Don't use ember's array extensions
* Add explicit service injections
* Correct class names
* Use `||=` operator
* Use `store` service to create records
* Remove unused service injections
* Extract consts
* Group actions together
* Use `async`/`await`
* Use `withEventValue`
* Sort html attributes
* Use DButtons `@label` arg
* Use `input` elements instead of Ember's `Input` component (same w/ textarea)
* Remove `btn-default` class (automatically applied by DButton)
* Don't mix `I18n.t` and `i18n` in the same template
* Don't track props that aren't used in a template
* Correct invalid `target.value` code
* Remove unused/invalid `this.parameter`/`onChange` code
* Whitespace
* Use the new service import `inject as service` -> `service`
* Use `Object.entries()`
* Add missing i18n strings
* Fix an error in `addEnumValue` (calling `pushObject` on `undefined`)
* Use `TrackedArray`/`TrackedObject`
* Transform tool `parameters` keys (`enumValues` -> `enum_values`)
2024-06-28 08:59:51 +10:00
Jan Cernik 8e83c091a2
DEV: Use explicit serializers for all models (#691) 2024-06-27 10:43:00 -03:00
Sam b863ddc94b
FEATURE: custom user defined tools (#677)
Introduces custom AI tools functionality. 

1. Why it was added:
   The PR adds the ability to create, manage, and use custom AI tools within the Discourse AI system. This feature allows for more flexibility and extensibility in the AI capabilities of the platform.

2. What it does:
   - Introduces a new `AiTool` model for storing custom AI tools
   - Adds CRUD (Create, Read, Update, Delete) operations for AI tools
   - Implements a tool runner system for executing custom tool scripts
   - Integrates custom tools with existing AI personas
   - Provides a user interface for managing custom tools in the admin panel

3. Possible use cases:
   - Creating custom tools for specific tasks or integrations (stock quotes, currency conversion etc...)
   - Allowing administrators to add new functionalities to AI assistants without modifying core code
   - Implementing domain-specific tools for particular communities or industries

4. Code structure:
   The PR introduces several new files and modifies existing ones:

   a. Models:
      - `app/models/ai_tool.rb`: Defines the AiTool model
      - `app/serializers/ai_custom_tool_serializer.rb`: Serializer for AI tools

   b. Controllers:
      - `app/controllers/discourse_ai/admin/ai_tools_controller.rb`: Handles CRUD operations for AI tools

   c. Views and Components:
      - New Ember.js components for tool management in the admin interface
      - Updates to existing AI persona management components to support custom tools 

   d. Core functionality:
      - `lib/ai_bot/tool_runner.rb`: Implements the custom tool execution system
      - `lib/ai_bot/tools/custom.rb`: Defines the custom tool class

   e. Routes and configurations:
      - Updates to route configurations to include new AI tool management pages

   f. Migrations:
      - `db/migrate/20240618080148_create_ai_tools.rb`: Creates the ai_tools table

   g. Tests:
      - New test files for AI tool functionality and integration

The PR integrates the custom tools system with the existing AI persona framework, allowing personas to use both built-in and custom tools. It also includes safety measures such as timeouts and HTTP request limits to prevent misuse of custom tools.

Overall, this PR significantly enhances the flexibility and extensibility of the Discourse AI system by allowing administrators to create and manage custom AI tools tailored to their specific needs.

Co-authored-by: Martin Brennan <martin@discourse.org>
2024-06-27 17:27:40 +10:00
Roman Rizzi e39e0bdb4a
FIX: Move the bot user toggling to the controller. (#688)
Having this as a callback prevents deploys of sites with a vLLM SRV configured and pending migrations. Additionally, this fixes a bug where we didn't delete/deactivate the companion user after deleting an LLM.
2024-06-25 12:45:19 -03:00
Roman Rizzi f622e2644f
FEATURE: Store provider-specific parameters. (#686)
Previously, we stored request parameters like the OpenAI organization and Bedrock's access key and region as site settings. This change stores them in the `llm_models` table instead, letting us drop more settings while also becoming more flexible.
2024-06-25 08:26:30 +10:00
Sam 1d5fa0ce6c
FIX: when creating an llm we were not creating user (#685)
This meant that if you toggle ai user early it surprisingly did
not work.

Also remove safety settings from gemini, it is overly cautious
2024-06-24 09:59:42 +10:00
Sam e04a7be122
FEATURE: LLM presets for model creation (#681)
* FEATURE: LLM presets for model creation

Previous to this users needed to look up complicated settings
when setting up models.

This introduces and extensible preset system with Google/OpenAI/Anthropic
presets.

This will cover all the most common LLMs, we can always add more as
we go.

Additionally:

- Proper support for Anthropic Claude Sonnet 3.5
- Stop blurring api keys when navigating away - this made it very complex to reuse keys
2024-06-21 17:32:15 +10:00
Roman Rizzi 8849caf136
DEV: Transition "Select model" settings to only use LlmModels (#675)
We no longer support the "provider:model" format in the "ai_helper_model" and
"ai_embeddings_semantic_search_hyde_model" settings. We'll migrate existing
values and work with our new data-driven LLM configs from now on.
2024-06-19 18:01:35 -03:00
Roman Rizzi ed3d5521a8
UX: QoL impromevements to the admin LLM models page. (#674)
API Key value is secret by default, and we include a link to the AI bot user.
2024-06-19 11:21:21 -03:00
Sam 0d6d9a6ef5
FEATURE: allow access to private topics if tool permits (#673)
Previously read tool only had access to public topics, this allows
access to all topics user has access to, if admin opts for the option
Also

- Fixes VLLM migration
- Display which llms have bot enabled
2024-06-19 15:49:36 +10:00
Roman Rizzi 8d5f901a67
DEV: Rewire AI bot internals to use LlmModel (#638)
* DRAFT: Create AI Bot users dynamically and support custom LlmModels

* Get user associated to llm_model

* Track enabled bots with attribute

* Don't store bot username. Minor touches to migrate default values in settings

* Handle scenario where vLLM uses a SRV record

* Made 3.5-turbo-16k the default version so we can remove hack
2024-06-18 14:32:14 -03:00
Sam 52a7dd2a4b
FEATURE: optional tool detail blocks (#662)
This is a rather huge refactor with 1 new feature (tool details can
be suppressed)

Previously we use the name "Command" to describe "Tools", this unifies
all the internal language and simplifies the code.

We also amended the persona UI to use less DToggles which aligns
with our design guidelines.

Co-authored-by: Martin Brennan <martin@discourse.org>
2024-06-11 18:14:14 +10:00
Rafael dos Santos Silva 92108452f2
FIX: AI Bot Shared Conversation didn't respect CDN / Subfolder (#657) 2024-06-06 15:13:51 -03:00
Sam 13840f68b3
FEATURE: restrict public sharing on login required sites (#649)
Initial implementation allowed internet wide sharing of
AI conversations, on sites that require login.

This feature can be an anti feature for private sites cause they
can not share conversations internally.

For now we are removing support for public sharing on login required
sites, if the community need the feature we can consider adding a
setting.
2024-05-29 11:04:47 +10:00
Sam b487de933d
FEATURE: add support for all vision models (#646)
Previoulsy on GPT-4-vision was supported, change introduces support
for Google/Anthropic and new OpenAI models

Additionally this makes vision work properly in dev environments
cause we sent the encoded payload via prompt vs sending urls
2024-05-28 10:31:15 -03:00
Loïc Guitaut dd4e305ff7
DEV: Update rubocop-discourse to version 3.8.0 (#641) 2024-05-28 11:15:42 +02:00
Roman Rizzi 333b331eb9
FEATURE: Allow deleting custom LLMs. (#643)
This change allows us to delete custom models. It checks if there is no module using them.

It also fixes a bug where the after-create transition wasn't working. While this prevents a model from being saved multiple times, endpoint validations are still needed (will be added in a separate PR).:
2024-05-27 16:44:08 -03:00
Keegan George a1c649965f
FEATURE: Auto image captions (#637) 2024-05-27 10:49:24 -07:00
Sam baf88e7cfc
FEATURE: improve logging by including llm name (#640)
Log the language model name when logging api requests
2024-05-27 16:46:01 +10:00
Ted Johansson d8a0f44fed
FIX: Amend incorrect translation keys (#639)
I am enabling config.i18n.raise_on_missing_translations in core. This revealed a couple of broken translations in the plugin.
2024-05-24 20:00:36 +08:00
Roman Rizzi 3a9080dd14
FEATURE: Test LLM configuration (#634) 2024-05-21 13:35:50 -03:00
Sam d4116ecfac
FEATURE: Add support for contextualizing a DM to a bot (#627)
This brings the context of the current topic on screen into chat
2024-05-21 17:17:02 +10:00
Roman Rizzi 1d786fbaaf
FEATURE: Set endpoint credentials directly from LlmModel. (#625)
* FEATURE: Set endpoint credentials directly from LlmModel.

Drop Llama2Tokenizer since we no longer use it.

* Allow http for custom LLMs

---------

Co-authored-by: Rafael Silva <xfalcox@gmail.com>
2024-05-16 09:50:22 -03:00
Sam 8eee6893d6
FEATURE: GPT4o support and better auditing (#618)
- Introduce new support for GPT4o (automation / bot / summary / helper)
- Properly account for token counts on OpenAI models
- Track feature that was used when generating AI completions
- Remove custom llm support for summarization as we need better interfaces to control registration and de-registration
2024-05-14 13:28:46 +10:00
Roman Rizzi e22194f321
HACK: Llama3 support for summarization/AI helper. (#616)
There are still some limitations to which models we can support with the `LlmModel` class. This will enable support for Llama3 while we sort those out.
2024-05-13 15:54:42 -03:00
Roman Rizzi 62fc7d6ed0
FEATURE: Configurable LLMs. (#606)
This PR introduces the concept of "LlmModel" as a new way to quickly add new LLM models without making any code changes. We are releasing this first version and will add incremental improvements, so expect changes.

The AI Bot can't fully take advantage of this feature as users are hard-coded. We'll fix this in a separate PR.s
2024-05-13 12:46:42 -03:00
Bianca Nenciu 5861418e9d
FIX: Load categories for AI search results (#614)
Categories should be preloaded when "lazy load categories" is enabled.
2024-05-13 16:47:37 +03:00
Roman Rizzi 4f1a3effe0
REFACTOR: Migrate Vllm/TGI-served models to the OpenAI format. (#588)
Both endpoints provide OpenAI-compatible servers. The only difference is that Vllm doesn't support passing tools as a separate parameter. Even if the tool param is supported, it ultimately relies on the model's ability to handle native functions, which is not the case with the models we have today.

As a part of this change, we are dropping support for StableBeluga/Llama2 models. They don't have a chat_template, meaning the new API can translate them.

These changes let us remove some of our existing dialects and are a first step in our plan to support any LLM by defining them as data-driven concepts.

 I rewrote the "translate" method to use a template method and extracted the tool support strategies into its classes to simplify the code.

Finally, these changes bring support for Ollama when running in dev mode. It only works with Mistral for now, but it will change soon..
2024-05-07 10:02:16 -03:00
Sam e4b326c711
FEATURE: support Chat with AI Persona via a DM (#488)
Add support for chat with AI personas

- Allow enabling chat for AI personas that have an associated user
- Add new setting `allow_chat` to AI persona to enable/disable chat
- When a message is created in a DM channel with an allowed AI persona user, schedule a reply job
- AI replies to chat messages using the persona's `max_context_posts` setting to determine context
- Store tool calls and custom prompts used to generate a chat reply on the `ChatMessageCustomPrompt` table
- Add tests for AI chat replies with tools and context

At the moment unlike posts we do not carry tool calls in the context.

No @mention support yet for ai personas in channels, this is future work
2024-05-06 09:49:02 +10:00
Sam 32b3004ce9
FEATURE: Add Question Consolidator for robust Upload support in Personas (#596)
This commit introduces a new feature for AI Personas called the "Question Consolidator LLM". The purpose of the Question Consolidator is to consolidate a user's latest question into a self-contained, context-rich question before querying the vector database for relevant fragments. This helps improve the quality and relevance of the retrieved fragments.

Previous to this change we used the last 10 interactions, this is not ideal cause the RAG would "lock on" to an answer. 

EG:

- User: how many cars are there in europe
- Model: detailed answer about cars in europe including the term car and vehicle many times
- User: Nice, what about trains are there in the US

In the above example "trains" and "US" becomes very low signal given there are pages and pages talking about cars and europe. This mean retrieval is sub optimal. 

Instead, we pass the history to the "question consolidator", it would simply consolidate the question to "How many trains are there in the United States", which would make it fare easier for the vector db to find relevant content. 

The llm used for question consolidator can often be less powerful than the model you are talking to, we recommend using lighter weight and fast models cause the task is very simple. This is configurable from the persona ui.

This PR also removes support for {uploads} placeholder, this is too complicated to get right and we want freedom to shift RAG implementation. 

Key changes:

1. Added a new `question_consolidator_llm` column to the `ai_personas` table to store the LLM model used for question consolidation.

2. Implemented the `QuestionConsolidator` module which handles the logic for consolidating the user's latest question. It extracts the relevant user and model messages from the conversation history, truncates them if needed to fit within the token limit, and generates a consolidated question prompt.

3. Updated the `Persona` class to use the Question Consolidator LLM (if configured) when crafting the RAG fragments prompt. It passes the conversation context to the consolidator to generate a self-contained question.

4. Added UI elements in the AI Persona editor to allow selecting the Question Consolidator LLM. Also made some UI tweaks to conditionally show/hide certain options based on persona configuration.

5. Wrote unit tests for the QuestionConsolidator module and updated existing persona tests to cover the new functionality.

This feature enables AI Personas to better understand the context and intent behind a user's question by consolidating the conversation history into a single, focused question. This can lead to more relevant and accurate responses from the AI assistant.
2024-04-30 13:49:21 +10:00
Sam 85734fef52
FIX: properly cache user locale (#593)
This blob is localized according to user locale, so we can end up
bleeding incorrect data in the cache
2024-04-26 09:28:35 -03:00
Roman Rizzi 283445cf81
FIX: RAG uploader must support multi-file indexing. (#592)
Updating the editing model's rag_uploads in the editor component broke multi-file uploading. Instead, we'll keep the uploads in the uploader and update the model when we finish.

This PR also fast-tracks the initial update so we can show feedback to the user quickly, and allows uploading MD files.

Bug reported on https://meta.discourse.org/t/discourse-ai-persona-upload-support/304049/11
2024-04-25 10:48:55 -03:00
Sam a5e4ab2825
FIX: blank metadata leading to errors (#578)
blank metadata block in RAG was leading to an error, this handles the edge case
2024-04-17 13:46:40 +10:00
Sam 4a29f8ed1c
FEATURE: Enhance AI debugging capabilities and improve interface adjustments (#577)
* FIX: various RAG edge cases

- Nicer text to describe RAG, avoids the word RAG
- Do not attempt to save persona when removing uploads and it is not created
- Remove old code that avoided touching rag params on create

* FIX: Missing pause button for persona users

* Feature: allow specific users to debug ai request / response chains

This can help users easily tune RAG and figure out what is going
on with requests.

* discourse helper so it does not explode

* fix test

* simplify implementation
2024-04-15 23:22:06 +10:00
Sam f6ac5cd0a8
FEATURE: allow tuning of RAG generation (#565)
* FEATURE: allow tuning of RAG generation

- change chunking to be token based vs char based (which is more accurate)
- allow control over overlap / tokens per chunk and conversation snippets inserted
- UI to control new settings

* improve ui a bit

* fix various reindex issues

* reduce concurrency

* try ultra low queue ... concurrency 1 is too slow.
2024-04-12 10:32:46 -03:00
Martin Brennan bab5e52e38
FIX: Secure/unsecure uploads when sharing AI conversations (#554)
This commit uses a new plugin modifier introduced in https://github.com/discourse/discourse/pull/26508
to mark all uploads as _not_ secure in shared PM AI conversations.
This is so images created by the AI bot (or uploaded by the user)
do not end up as broken URLs because of the security requirements
around them.

This relies on the UpdateTopicUploadSecurity job in core as well,
which is fired when an AI conversation is shared or deleted.
2024-04-11 10:00:41 +10:00
Sam 7f16d3ad43
FEATURE: Cohere Command R support (#558)
- Added Cohere Command models (Command, Command Light, Command R, Command R Plus) to the available model list
- Added a new site setting `ai_cohere_api_key` for configuring the Cohere API key
- Implemented a new `DiscourseAi::Completions::Endpoints::Cohere` class to handle interactions with the Cohere API, including:
   - Translating request parameters to the Cohere API format
   - Parsing Cohere API responses 
   - Supporting streaming and non-streaming completions
   - Supporting "tools" which allow the model to call back to discourse to lookup additional information
- Implemented a new `DiscourseAi::Completions::Dialects::Command` class to translate between the generic Discourse AI prompt format and the Cohere Command format
- Added specs covering the new Cohere endpoint and dialect classes
- Updated `DiscourseAi::AiBot::Bot.guess_model` to map the new Cohere model to the appropriate bot user

In summary, this PR adds support for using the Cohere Command family of models with the Discourse AI plugin. It handles configuring API keys, making requests to the Cohere API, and translating between Discourse's generic prompt format and Cohere's specific format. Thorough test coverage was added for the new functionality.
2024-04-11 07:24:17 +10:00
Roman Rizzi aa8918911d
UX: Display the indexing progress for RAG uploads (#557) 2024-04-09 11:03:07 -03:00
Sam 830cc26075
FEATURE: Add metadata support for RAG (#553)
* FEATURE: Add metadata support for RAG

You may include non indexed metadata in the RAG document by using

[[metadata ....]]

This information is attached to all the text below and provided to
the retriever.

This allows for RAG to operate within a rich amount of contexts
without getting lost

Also:

- re-implemented chunking algorithm so it streams
- moved indexing to background low priority queue

* Baran gem no longer required.

* tokenizers is on 4.4 ... upgrade it ...
2024-04-04 11:02:16 -03:00
Roman Rizzi 1f1c94e5c6
FEATURE: AI Bot RAG support. (#537)
This PR lets you associate uploads to an AI persona, which we'll split and generate embeddings from. When building the system prompt to get a bot reply, we'll do a similarity search followed by a re-ranking (if available). This will let us find the most relevant fragments from the body of knowledge you associated with the persona, resulting in better, more informed responses.

For now, we'll only allow plain-text files, but this will change in the future.

Commits:

* FEATURE: RAG embeddings for the AI Bot

This first commit introduces a UI where admins can upload text files, which we'll store, split into fragments,
and generate embeddings of. In a next commit, we'll use those to give the bot additional information during
conversations.

* Basic asymmetric similarity search to provide guidance in system prompt

* Fix tests and lint

* Apply reranker to fragments

* Uploads filter, css adjustments and file validations

* Add placeholder for rag fragments

* Update annotations
2024-04-01 13:43:34 -03:00
Sam 61e4c56e1a
FEATURE: Add vision support to AI personas (Claude 3) (#546)
This commit adds the ability to enable vision for AI personas, allowing them to understand images that are posted in the conversation.

For personas with vision enabled, any images the user has posted will be resized to be within the configured max_pixels limit, base64 encoded and included in the prompt sent to the AI provider.

The persona editor allows enabling/disabling vision and has a dropdown to select the max supported image size (low, medium, high). Vision is disabled by default.

This initial vision support has been tested and implemented with Anthropic's claude-3 models which accept images in a special format as part of the prompt.

Other integrations will need to be updated to support images.

Several specs were added to test the new functionality at the persona, prompt building and API layers.

 - Gemini is omitted, pending API support for Gemini 1.5. Current Gemini bot is not performing well, adding images is unlikely to make it perform any better.

 - Open AI is omitted, vision support on GPT-4 it limited in that the API has no tool support when images are enabled so we would need to full back to a different prompting technique, something that would add lots of complexity


---------

Co-authored-by: Martin Brennan <martin@discourse.org>
2024-03-27 14:30:11 +11:00
David Taylor dfc13fc631
UX: Add title suffix to shared AI pages (#531) 2024-03-15 10:31:58 +00:00
David Taylor 8b0e9984cf
FIX: Syntax highlighting for shared-ai conversions with CSP enabled (#532)
- Adds a nonce to both script tags
- Removes the `onload=` inline script, and moves the tags to the end of the `<body>` instead. This provides the same UX (page will load and render, then hljs will be applied when ready)
2024-03-15 13:48:24 +11:00
Kris fc6c3c81e7
UX: add support for dark mode (#529) 2024-03-13 13:16:07 -04:00
Sam ea5c38686e
FIX: expire assets when CSS changes FIX: missing translation on share page (#528)
* FIX: expire assets when CSS changes
FIX: missing translation on share page

Also add trivial image styling

* oops not needed
2024-03-13 20:52:33 +11:00
Kris 1c90ad81c2
UX: update styles and markup for share feature (#525) 2024-03-12 18:25:33 -04:00
Sam a03bc6ddec
FEATURE: Share conversations with AI via a URL (#521)
This allows users to share a static page of an AI conversation with
the rest of the world.

By default this feature is disabled, it is enabled by turning on
ai_bot_allow_public_sharing via site settings

Precautions are taken when sharing

1. We make a carbonite copy
2. We minimize work generating page
3. We limit to 100 interactions
4. Many security checks - including disallowing if there is a mix
of users in the PM.

* Bonus commit, large PRs like this PR did not work with github tool
large objects would destroy context


Co-authored-by: Martin Brennan <martin@discourse.org>
2024-03-12 16:51:41 +11:00
Keegan George b515b4f66d
FEATURE: AI Quick Semantic Search (#501)
This PR adds AI semantic search to the search pop available on every page.

It depends on several new and optional settings, like per post embeddings and a reranker model, so this is an experimental endeavour.


---------

Co-authored-by: Rafael Silva <xfalcox@gmail.com>
2024-03-08 13:02:50 -03:00
Loïc Guitaut 6ae4218a96 DEV: Fix new Rubocop offenses 2024-03-06 15:23:29 +01:00
Sam 484fd1435b
DEV: improve internal design of ai persona and bug fix (#495)
* DEV: improve internal design of ai persona and bug fix

- Fixes bug where OpenAI could not describe images
- Fixes bug where mentionable personas could not be mentioned unless overarching bot was enabled
- Improves internal design of playground and bot to allow better for non "bot" users
- Allow PMs directly to persona users (previously bot user would also have to be in PM)
- Simplify internal code


Co-authored-by: Martin Brennan <martin@discourse.org>
2024-02-28 16:46:32 +11:00
Keegan George 97f3cba603
DEV: Add attribution to AI captioned images (#483) 2024-02-21 10:10:22 -08:00
Martin Brennan 0c1aad7850
DEV: Cleanup caption endpoint and account for secure uploads (#478)
Utilizes the check for secure upload permissions from core PR
https://github.com/discourse/discourse/pull/25758 and cleans up
controller codes and spec code to reuse existing code and better
reflect reality.
2024-02-19 23:43:39 -03:00
Rafael dos Santos Silva cf19ce0d72
FEATURE: Handle secure uploads in image caption (#476) 2024-02-19 18:08:19 -03:00
Keegan George a9b2d6a30a
FEATURE: AI image caption (#470)
This PR adds a new feature where you can generate captions for images in the composer using AI.

---------

Co-authored-by: Rafael Silva <xfalcox@gmail.com>
2024-02-19 14:56:28 -03:00
Sam 1f74a77e17
DEV: correct flaky spec (#475)
We were not properly expiring prompt cache
2024-02-19 15:21:55 +11:00
Keegan George d66915ecc1
DEV: Make prompts available on `CurrentUserSerializer` (#472) 2024-02-16 10:57:14 -08:00
Sam 3a8d95f6b2
FEATURE: mentionable personas and random picker tool, context limits (#466)
1. Personas are now optionally mentionable, meaning that you can mention them either from public topics or PMs
       - Mentioning from PMs helps "switch" persona mid conversation, meaning if you want to look up sites setting you can invoke the site setting bot, or if you want to generate an image you can invoke dall e
        - Mentioning outside of PMs allows you to inject a bot reply in a topic trivially
     - We also add the support for max_context_posts this allow you to limit the amount of context you feed in, which can help control costs

2. Add support for a "random picker" tool that can be used to pick random numbers 

3. Clean up routing ai_personas -> ai-personas

4. Add Max Context Posts so users can control how much history a persona can consume (this is important for mentionable personas) 

Co-authored-by: Martin Brennan <martin@discourse.org>
2024-02-15 16:37:59 +11:00
Sam a3c827efcc
FEATURE: allow personas to supply top_p and temperature params (#459)
* FEATURE: allow personas to supply top_p and temperature params

Code assistance generally are more focused at a lower temperature
This amends it so SQL Helper runs at 0.2 temperature vs the more
common default across LLMs of 1.0.

Reduced temperature leads to more focused, concise and predictable
answers for the SQL Helper

* fix tests

* This is not perfect, but far better than what we do today

Instead of fishing for

1. Draft sequence
2. Draft body

We skip (2), this means the composer "only" needs 1 http request to
open, we also want to eliminate (1) but it is a bit of a trickier
core change, may figure out how to pull it off (defer it to first draft save)

Value of bot drafts < value of opening bot conversations really fast
2024-02-03 07:09:34 +11:00
Rafael dos Santos Silva fd6fcfdb61
DEV: Increase embeddings backfill job frequency (#453)
The idea is to increase the frequency so we can run with smaller batch sizes.
Big batches cause problems when running backups, so it's better to have shorter but
more frequent jobs.
2024-01-31 15:09:39 -03:00
Sam dcafc8032f
FIX: improve embedding generation (#452)
1. on failure we were queuing a job to generate embeddings, it had the wrong params. This is both fixed and covered in a test.
2. backfill embedding in the order of bumped_at, so newest content is embedded first, cover with a test
3. add a safeguard for hidden site setting that only allows batches of 50k in an embedding job run

Previously old embeddings were updated in a random order, this changes it so we update in a consistent order
2024-01-31 10:38:47 -03:00
Rafael dos Santos Silva 04bc402aae
FEATURE: Setting to control per post embeddings (#439)
* FEATURE: Setting to control per post embeddings
2024-01-23 22:09:27 -03:00
Rafael dos Santos Silva d4e23e0df6
FIX: Don't try to generate embeddings of posts in deleted topics (#433) 2024-01-18 16:10:25 -03:00
Roman Rizzi 04eae76f68
REFACTOR: Represent generic prompts with an Object. (#416)
* REFACTOR: Represent generic prompts with an Object.

* Adds a bit more validation for clarity

* Rewrite bot title prompt and fix quirk handling

---------

Co-authored-by: Sam Saffron <sam.saffron@gmail.com>
2024-01-12 14:36:44 -03:00
Martin Brennan 37b957dbbb
DEV: Fix SemanticRelated module load error (#419)
Followup 2636efcd1b,
whenever ruby code was changed locally this would break
module loading, giving an "uninitialized constant
DiscourseAi::Embeddings::EntryPoint::SemanticRelated" error.
2024-01-11 13:52:50 +10:00
Roman Rizzi f9d7d7f5f0
DEV: AI bot migration to the Llm pattern. (#343)
* DEV: AI bot migration to the Llm pattern.

We added tool and conversation context support to the Llm service in discourse-ai#366, meaning we met all the conditions to migrate this module.

This PR migrates to the new pattern, meaning adding a new bot now requires minimal effort as long as the service supports it. On top of this, we introduce the concept of a "Playground" to separate the PM-specific bits from the completion, allowing us to use the bot in other contexts like chat in the future. Commands are called tools, and we simplified all the placeholder logic to perform updates in a single place, making the flow more one-wayish.

* Followup fixes based on testing

* Cleanup unused inference code

* FIX: text-based tools could be in the middle of a sentence

* GPT-4-turbo support

* Use new LLM API
2024-01-04 10:44:07 -03:00
Sam 03fc94684b
FIX: AI helper not working correctly with mixtral (#399)
* FIX: AI helper not working correctly with mixtral

This PR introduces a new function on the generic llm called #generate

This will replace the implementation of completion!

#generate introduces a new way to pass temperature, max_tokens and stop_sequences

Then LLM implementers need to implement #normalize_model_params to
ensure the generic names match the LLM specific endpoint

This also adds temperature and stop_sequences to completion_prompts
this allows for much more robust completion prompts

* port everything over to #generate

* Fix translation

- On anthropic this no longer throws random "This is your translation:"
- On mixtral this actually works

* fix markdown table generation as well
2024-01-04 09:53:47 -03:00
Jan Cernik d9c052f8e7
FIX: 500 error when reviewable has a missing message (#397) 2024-01-03 11:49:47 -03:00
Rafael dos Santos Silva c778592da4
FIX: Corner cases on post embedding and crawler related (#391)
* FIX: Simplify markup for crawler related

* FIX: Handle title-less topics when truncating for embeddings

* fix
2023-12-29 14:05:02 -03:00
Rafael dos Santos Silva 140359c2ef
FEATURE: Per post embeddings (#387) 2023-12-29 12:28:45 -03:00
Natalie Tay 73c58155f8
FIX: Get topic's category instead of array's (#388)
Currently we're seeing 500s when related_topics are getting rendered. We should get the topic's category rather than on the array.
```
ActionView::Template::Error (undefined method `category' for [#<Topic id ... ]
```
2023-12-29 16:52:48 +08:00
Rafael dos Santos Silva 2636efcd1b
FEATURE: Render Related Topics for Crawlers (#386) 2023-12-28 15:32:03 -03:00
Rafael dos Santos Silva 5db7bf6e68
Mixtral (#376)
Add both Mistral and Mixtral support. Also includes vLLM-openAI inference support.

Co-authored-by: Roman Rizzi <rizziromanalejandro@gmail.com>
2023-12-26 14:49:55 -03:00