Commit Graph

287 Commits

Author SHA1 Message Date
Kris a9afa04329
UX: update gist toggle styles (#926) 2024-11-19 15:33:34 -05:00
Roman Rizzi fb80d776d8
FEATURE: Enable gists on all topic lists (#922) 2024-11-19 11:04:34 -03:00
Rafael dos Santos Silva 48d08dedd4
FEATURE: Emotion activity metrics table (#916) 2024-11-19 10:01:10 -03:00
David Taylor 6b9c66054c
DEV: Update eslint config (#917)
* DEV: Update eslint config

* fixup

* pnpm upgrade
2024-11-19 11:57:40 +01:00
Sam 0d7f353284
FEATURE: AI artifacts (#898)
This is a significant PR that introduces AI Artifacts functionality to the discourse-ai plugin along with several other improvements. Here are the key changes:

1. AI Artifacts System:
   - Adds a new `AiArtifact` model and database migration
   - Allows creation of web artifacts with HTML, CSS, and JavaScript content
   - Introduces security settings (`strict`, `lax`, `disabled`) for controlling artifact execution
   - Implements artifact rendering in iframes with sandbox protection
   - New `CreateArtifact` tool for AI to generate interactive content

2. Tool System Improvements:
   - Adds support for partial tool calls, allowing incremental updates during generation
   - Better handling of tool call states and progress tracking
   - Improved XML tool processing with CDATA support
   - Fixes for tool parameter handling and duplicate invocations

3. LLM Provider Updates:
   - Updates for Anthropic Claude models with correct token limits
   - Adds support for native/XML tool modes in Gemini integration
   - Adds new model configurations including Llama 3.1 models
   - Improvements to streaming response handling

4. UI Enhancements:
   - New artifact viewer component with expand/collapse functionality
   - Security controls for artifact execution (click-to-run in strict mode)
   - Improved dialog and response handling
   - Better error management for tool execution

5. Security Improvements:
   - Sandbox controls for artifact execution
   - Public/private artifact sharing controls
   - Security settings to control artifact behavior
   - CSP and frame-options handling for artifacts

6. Technical Improvements:
   - Better post streaming implementation
   - Improved error handling in completions
   - Better memory management for partial tool calls
   - Enhanced testing coverage

7. Configuration:
   - New site settings for artifact security
   - Extended LLM model configurations
   - Additional tool configuration options

This PR significantly enhances the plugin's capabilities for generating and displaying interactive content while maintaining security and providing flexible configuration options for administrators.
2024-11-19 09:22:39 +11:00
Keegan George f75b13c4fa
FIX: results not being reset when appending to query param (#912)
This PR fixes an issue where the AI search results were not being reset when you append your search to an existing query param (typically when you've come from quick search). This is because `handleSearch()` doesn't get called in this situation. So here we explicitly check for query params, trigger a reset and search for those occasions.
2024-11-13 07:19:34 -08:00
Sérgio Saquetim 9583964676
DEV: Added compatibility with the Glimmer Post Menu (#887) 2024-11-12 15:46:17 -03:00
Sam e817b7dc11
FEATURE: improve tool support (#904)
This re-implements tool support in DiscourseAi::Completions::Llm #generate

Previously tool support was always returned via XML and it would be the responsibility of the caller to parse XML

New implementation has the endpoints return ToolCall objects.

Additionally this simplifies the Llm endpoint interface and gives it more clarity. Llms must implement

decode, decode_chunk (for streaming)

It is the implementers responsibility to figure out how to decode chunks, base no longer implements. To make this easy we ship a flexible json decoder which is easy to wire up.

Also (new)

    Better debugging for PMs, we now have a next / previous button to see all the Llm messages associated with a PM
    Token accounting is fixed for vllm (we were not correctly counting tokens)
2024-11-12 08:14:30 +11:00
Keegan George 644141ff08
FIX: Regenerate summary button still shows cached summary (#903)
This PR fixes an issue where clicking to regenerate a summary was still showing the cached summary. To resolve this we call resetSummary() to reset all the summarization related properties before creating a new request.
2024-11-07 16:01:18 -08:00
Keegan George c421f713a3
DEV: Handle streaming animation within `AiSummaryBox` (#901)
This PR further decouples the streaming animation by completely handling the streaming animation directly in the `AiSummaryBox` component. Previously, handling the streaming animation by calling methods in the `ai-streamer` API was leading to timing issues making things out-of-sync. This results in some issues such as the last update of streamed text not being shown. Handling streaming directly in the component should simplify things drastically and prevent any issues.
2024-11-07 08:08:32 -08:00
Kris 893fa624e4
UX: increase gist size, and adjust surrounding elements to accommodate (#900) 2024-11-07 08:35:05 -05:00
Kris 1ad5321c09
UX: add sparkle icon to related topics for anons (#897) 2024-11-05 17:15:20 -05:00
Osama Sayegh da7c97d294
FIX: Specify type for Rag upload (#895)
Specifying `type` when using `UppyUpload` will be required as of https://github.com/discourse/discourse/pull/29600.
2024-11-05 22:10:54 +03:00
Jarek Radosz fa7ca8bc31
DEV: Use the new more-topics API (#885)
See: https://github.com/discourse/discourse/pull/29143
2024-11-04 17:42:50 +01:00
Kris 05790a6a40
UX: convert AI gist disclosure to a toggle (#878) 2024-10-29 11:59:41 -04:00
Roman Rizzi 37b6461d68
FIX: Make sure that topic gists are displayed ONLY on the hot list. (#873) 2024-10-28 15:15:53 -03:00
jbrw c479b177a7
DEV: Check for presence of currentRoute.attributes (#870) 2024-10-25 13:30:29 -04:00
Sam 12869f2146
FIX: testing tool was not showing rag results (#867)
This changeset contains 4 fixes:

1. We were allowing running tests on unsaved tools,
this is problematic cause uploads are not yet associated or indexed
leading to confusing results. We now only show the test button when
tool is saved.


2. We were not properly scoping rag document fragements, this
meant that personas and ai tools could get results from other
unrelated tools, just to be filtered out later


3. index.search showed options as "optional" but implementation
required the second option

4. When testing tools searching through document fragments was
not working at all cause we did not properly load the tool
2024-10-25 16:01:25 +11:00
Sam 4923837165
FIX: Llm selector / forced tools / search tool (#862)
* FIX: Llm selector / forced tools / search tool


This fixes a few issues:

1. When search was not finding any semantic results we would break the tool
2. Gemin / Anthropic models did not implement forced tools previously despite it being an API option
3. Mechanics around displaying llm selector were not right. If you disabled LLM selector server side persona PM did not work correctly.
4. Disabling native tools for anthropic model moved out of a site setting. This deliberately does not migrate cause this feature is really rare to need now, people who had it set probably did not need it.
5. Updates anthropic model names to latest release

* linting

* fix a couple of tests I missed

* clean up conditional
2024-10-25 06:24:53 +11:00
David Taylor c1fa84ad29
DEV: Update form-template-field/upload uppy usage (#863) 2024-10-24 15:21:13 +01:00
Keegan George 9e8608b070
UX: Hide AI bot in seeded LLM (#858)
AI bot won't be turned on for seeded LLMs so it makes no sense to expose it here. This will cleanup the template and avoid the double `{{#unless}}` check.
2024-10-23 16:36:17 -07:00
Kris 72111a10ae
UX: add disclosure for topic list gists (#861) 2024-10-23 19:32:22 -04:00
Keegan George 9af0c2e719
UX: Improve seeded LLM edit page (#856) 2024-10-23 13:58:27 -07:00
Kris 0aa2789437
UX: switch gist outlet to avoid badges (#855) 2024-10-23 15:10:41 -04:00
Kris 657d103919
UX: adjust gist position, darken unread color (#854) 2024-10-23 13:00:52 -04:00
Keegan George eae7716177
DEV: Improve `ai-streamer` API (#851)
In preparation for applying the streaming animation elsewhere, we want to better improve the organization of folder structure and methods used in the `ai-streamer`
2024-10-22 10:55:35 -07:00
Sam a1f859a415
FEATURE: improve visibility of AI usage in LLM page (#845)
This changeset: 

1. Corrects some issues with "force_default_llm" not applying
2. Expands the LLM list page to show LLM usage
3. Clarifies better what "enabling a bot" on an llm means (you get it in the selector)
2024-10-22 11:16:02 +11:00
Keegan George 712a07c39b
DEV: `IMAGE_MARKDOWN_REGEX` import available in core (#847) 2024-10-21 14:48:10 -07:00
Keegan George 37c2930fbf
FIX: Decouple DOM manipulation from SummaryStreamer (#844)
Previously, when we added smooth streaming animation to summarization (https://github.com/discourse/discourse-ai/pull/778) we used the same logic and lib we did for AI Bot. However, since `AiSummaryBox` is an Ember component, the direct DOM manipulation done in the streamer (`SummaryUpdater`) would often result in issues with summarization where sometimes summarization updates would hang, especially on the last result. This is likely due to the DOM manipulation being done in the streamer being incongruent with Ember's way of rendering.

In this PR, we remove the direct DOM manipulation done in the lib `SummaryUpdater` in favour of directly updating the properties in `AiSummaryBox` using the `componentContext`. Instead of messing with Ember's rendered DOM, passing the updates and allowing the component to render the updates directly should likely prevent further issues with summarization.

The bug itself is quite difficult to repro and also difficult to test, so no tests have been added to this PR. But I will be manually testing and assessing for any potential issues.
2024-10-21 09:15:25 -07:00
Roman Rizzi 27b5542357
FEATURE: Generate topic gists for the hot topics list. (#837)
* Display gists in the hot topics list

* Adjust hot topics gist strategy and add a job to generate gists

* Replace setting with a configurable batch size

* Avoid loading summaries for other topic lists

* Tweak gist prompt to focus on latest posts in the context of the OP

* Remove serializer hack and rely on core change from discourse/discourse#29291

* Update lib/summarization/strategies/hot_topic_gists.rb

Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>

---------

Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>
2024-10-18 18:01:39 -03:00
Sam bdf3b6268b
FEATURE: smarter persona tethering (#832)
Splits persona permissions so you can allow a persona on:

- chat dms
- personal messages
- topic mentions
- chat channels

(any combination is allowed)

Previously we did not have this flexibility.

Additionally, adds the ability to "tether" a language model to a persona so it will always be used by the persona. This allows people to use a cheaper language model for one group of people and more expensive one for other people
2024-10-16 07:20:31 +11:00
Sam 6c4c96e83c
FEATURE: allow persona to only force tool calls on limited replies (#827)
This introduces another configuration that allows operators to
limit the amount of interactions with forced tool usage.

Forced tools are very handy in initial llm interactions, but as
conversation progresses they can hinder by slowing down stuff
and adding confusion.
2024-10-11 07:23:42 +11:00
Mark VanLandingham 51494db236
REVERT: "DEV: Convert related-topics to gjs (#822)" (#825)
This reverts commit a3c6938cb3.
2024-10-09 10:10:03 -05:00
Jarek Radosz a3c6938cb3
DEV: Convert related-topics to gjs (#822) 2024-10-08 14:16:08 +02:00
Sam 545500b329
FEATURE: allows forced LLM tool use (#818)
* FEATURE: allows forced LLM tool use

Sometimes we need to force LLMs to use tools, for example in RAG
like use cases we may want to force an unconditional search.

The new framework allows you backend to force tool usage.

Front end commit to follow

* UI for forcing tools now works, but it does not react right

* fix bugs

* fix tests, this is now ready for review
2024-10-05 09:46:57 +10:00
Keegan George 110a1629aa
DEV: Update rate limits for image captioning (#816)
This PR updates the rate limits for AI helper so that image caption follows a specific rate limit of 20 requests per minute. This should help when uploading multiple files that need to be captioned. This PR also updates the UI so that it shows toast message with the extracted error message instead of having a blocking `popupAjaxError` error dialog.
---------

Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>
Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
2024-10-02 10:36:35 -07:00
Martin Brennan 7325fb21ab
DEV: Use section landing components for LLMs templates (#817)
Relies on https://github.com/discourse/discourse/pull/28477,
uses AdminSectionLandingWrapper and AdminSectionLandingItem
for the section items on the LLM page which are used to create
a new LLM config from a template.
2024-10-02 15:31:48 +10:00
Kris 62ba2fa4d7
UX: update icon and text for copying message (#815) 2024-10-01 18:38:57 -04:00
Sam 5cbc9190eb
FEATURE: RAG search within tools (#802)
This allows custom tools access to uploads and sophisticated searches using embedding.

It introduces:

 - A shared front end for listing and uploading files (shared with personas)
 -  Backend implementation of index.search function within a custom tool.

Custom tools now may search through uploaded files

function invoke(params) {
   return index.search(params.query)
}

This means that RAG implementers now may preload tools with knowledge and have high fidelity over
the search.

The search function support

    specifying max results
    specifying a subset of files to search (from uploads)

Also

 - Improved documentation for tools (when creating a tool a preamble explains all the functionality)
  - uploads were a bit finicky, fixed an edge case where the UI would not show them as updated
2024-09-30 17:27:50 +10:00
Kris 18ecc843e5
UX: move templates to main LLM config tab, restyle (#813)
Restructures LLM config page so it is far clearer. 

Also corrects bugs around adding LLMs and having LLMs not editable post addition 
---------

Co-authored-by: Sam Saffron <sam.saffron@gmail.com>
2024-09-30 17:15:11 +10:00
Keegan George 95f80325e5
DEV: Prevent close of summary from outside clicks (#808)
Often it is helpful to have the summary box open while composing a reply to the topic. However, the summary box currently gets closed each time you click outside the box. In this PR we add `closeOnClickOutside: false` attribute to the `DMenu` options for summary box to prevent that from occurring.
2024-09-18 10:36:42 -07:00
Keegan George e666266473
DEV: Make indicator wave a reusable component (#807)
Previously we had some hardcoded markup with scss making a loading indicator wave. This code was being duplicated and used in both semantic search and summarization. We want to add the indicator wave to the AI helper diff modal as well and have the text flashing instead of the loading spinner. To ensure we do not repeat ourselves, in this PR we turn the summary indicator wave into a reusable template only component called: `AiIndicatorWave`. We then apply the usage of that component to semantic search, summarization, and the composer helper modal.
2024-09-18 09:53:54 -07:00
Keegan George 493d65af1f
FIX: Diff modal closing along with composer menu on mobile (#803)
The `DiffModal` is triggered after selecting an option in the composer helper menu. After selecting an option, we should close the composer helper menu and only show the diff modal. On mobile, there was an edge-case where `this.args.close()` for was causing the closing of both the `DiffModal` and the `AiComposerHelperMenu`. This PR resolves that by ensuring the menu is closed _first_ asynchronously, followed by opening the relevant modal.
2024-09-16 14:00:41 -07:00
Keegan George b16390ae2a
UX: Improve toast message location (#800) 2024-09-14 09:19:13 +10:00
Keegan George 9374cd7ac1
FIX: Keyboard shortcut should be platform specific (#801) 2024-09-14 09:18:07 +10:00
Keegan George 9cd14b0003
DEV: Move composer AI helper to toolbar (#796)
Previously we had moved the AI helper from the options menu to a selection menu that appears when selecting text in the composer. This had the benefit of making the AI helper a more discoverable feature. Now that some time has passed and the AI helper is more recognized, we will be moving it back to the composer toolbar.

This is better because:
- It consistent with other behavior and ways of accessing tools in the composer
- It has an improved mobile experience
- It reduces unnecessary code and keeps things easier to migrate when we have composer V2.
- It allows for easily triggering AI helper for all content by clicking the button instead of having to select everything.
2024-09-13 11:59:30 -07:00
chapoi 22d1e71dc9
UX: AI post helper DMenu styling (#770) 2024-09-11 05:45:48 +02:00
Roman Rizzi c4c9dc2034
FIX: Display cached summaries with our new streamer. (#792)
Make sure the summary box is in the DOM before attempting to
display a cached summary.:
2024-09-03 18:45:28 -03:00
Kris 4a1874265e
UX: replace "share" with "share-alt" icon (#784) 2024-08-30 12:13:09 -03:00
Keegan George fdadfa029e
FEATURE: smooth streaming animation for summarization (#778) 2024-08-29 15:07:07 -07:00