Commit Graph

951 Commits

Author SHA1 Message Date
Rafael dos Santos Silva 0ac18d157b
FEATURE: Adjustments to gist summaries (#988)
- makes visible to everyone by default
- backfills gists before full summaries
- adds configurable max age setting to backfill job
2024-12-02 15:22:35 -03:00
Keegan George 50ad5415ff
UX: `no-text` style being incorrectly added to tag suggester (#987)
In the tag suggester menu we use `DButton` as a wrapper element and use the `discourseTag` helper to render the text inside the element. So visually there is text content inside the button. However, since `DButton` assumes that no `label`/`translatedLabel` inside an element means `.no-text` CSS style should be applied to the button's element, it was resulting in some incorrect styling being applied to this menu. This PR resolves that by programmatically adding the tag as a `translatedLabel` and then visually hiding it with CSS.
2024-12-02 10:11:04 -08:00
Tobias Eigen c53363406d
changed "automatic auto" to "auto" in client.en.yml (#986)
The description of the automatic captions dialog prompt included "Would you like to enable automatic auto captions on image uploads." I removed the second "auto" at the suggestion of translators.
2024-12-02 09:32:08 -08:00
Rafael dos Santos Silva 3828370679
DEV: Cleanup deprecations (#952) 2024-12-02 14:18:03 -03:00
Roman Rizzi 0abd4b1244
FIX: Sentiment classification results needs to be transformed before saving (#983) 2024-11-29 17:31:56 -03:00
David Taylor 120a20c5cd
DEV: Convert to native class syntax (#981) 2024-11-29 16:41:09 +01:00
Sam 0cb2c413ba
FEATURE: exclude muted categories from category suggester (#979)
The logic here is that users do not particularly care about
topics in the category so we can exclude them from tag
and category suggestions
2024-11-29 12:17:28 +11:00
Rafael dos Santos Silva 80adefa1c1
FIX: Move count logic to the end for tag suggestions (#978) 2024-11-28 16:27:38 -03:00
Sam bc0657f478
FEATURE: AI Usage page (#964)
- Added a new admin interface to track AI usage metrics, including tokens, features, and models.
- Introduced a new route `/admin/plugins/discourse-ai/ai-usage` and supporting API endpoint in `AiUsageController`.
- Implemented `AiUsageSerializer` for structuring AI usage data.
- Integrated CSS stylings for charts and tables under `stylesheets/modules/llms/common/usage.scss`.
- Enhanced backend with `AiApiAuditLog` model changes: added `cached_tokens` column  (implemented with OpenAI for now) with relevant DB migration and indexing.
- Created `Report` module for efficient aggregation and filtering of AI usage metrics.
- Updated AI Bot title generation logic to log correctly to user vs bot
- Extended test coverage for the new tracking features, ensuring data consistency and access controls.
2024-11-29 06:26:48 +11:00
Roman Rizzi c980c34d77
REFACTOR: Simplify sentiment classification (#977)
This change adds a simpler class for sentiment classification, replacing the soon-to-be removed `Classificator` hierarchy. Additionally, it adds a method for classifying concurrently, speeding up the backfill rake task.
2024-11-28 15:38:23 -03:00
Rafael dos Santos Silva 6456a4f44a
FIX: Keep related topics data cached when a user scrolls up (#976)
See https://meta.discourse.org/t/related-topics-missing-after-rereading-a-topic-with-more-than-20-posts/306147/6
2024-11-28 14:25:15 -03:00
Rafael dos Santos Silva eacc4ed945
UX: Fix styling of AI Helper on quick edit (#975) 2024-11-28 13:06:48 -03:00
David Taylor 4a2369356f
DEV: Update linting (#974) 2024-11-28 16:07:42 +01:00
Rafael dos Santos Silva 4980a4b2f7
FIX: Multiple concurrent summaries could result in pg index errors (#973) 2024-11-28 11:53:04 -03:00
Keegan George 54ebbbe769
UX: Ensure suggestions are left aligned (#971)
This PR ensures that suggestions are aligned to the left. By default, they are center aligned which looks odd when there are long lines of text.
2024-11-27 20:53:30 -08:00
Keegan George 8b33ba3d19
FIX: Suggestion menus not showing in mobile (#970)
The suggestion menu z-index was not enough for mobile so it isn't being shown when it's used in `modalForMobile` bottom menus.
2024-11-27 19:51:03 -08:00
Keegan George f1c7ee8624
DEV: Better control what prompts can appear in post/composer (#969)
This PR updates the logic for the location map so it permits only the desired prompts through to the composer/post menu. Anything else won't be shown by default.

This PR also adds relevant tests to prevent regression.
2024-11-27 16:14:21 -08:00
Keegan George 4da033c667
FIX: Double render error with thumbnail suggestions (#968)
This PR fixes a bug where a double render error appears in the logs when thumbnails are suggested
2024-11-27 15:12:27 -08:00
Keegan George dabef02919
DEV: Prevent `detect_text_locale` from appearing in menus (#967)
### 🔍 Overview
With the recent changes to allow DiscourseAi in the translator plugin, `detect_text_locale` was needed as a CompletionPrompt. However, it is leaking into composer/post helper menus. This PR ensures we don't not show it in those menus.
2024-11-28 09:27:08 +11:00
Sam fbcc8e493a
FEATURE: Skip PM scanning in LLM triage by default (#966)
Usually people do not want to scan personal messages. Sometimes
they may. In that case they can enable triage on personal messages.
2024-11-28 09:25:29 +11:00
Keegan George 6b7d7c1179
REFACTOR: Helper suggestions (#914)
This PR adds some updates to the Helper suggestions to improve it's functionality and modernize some of the codebase.
2024-11-27 12:21:03 -08:00
Rafael dos Santos Silva 0d3e6b2726
FIX: Fix ordering of random post embeddings backfill (#965)
* FIX: Fix ordering of random post embeddings backfill

* fix annotations

---------

Co-authored-by: Roman Rizzi <rizziromanalejandro@gmail.com>
2024-11-27 17:01:54 -03:00
Martin Brennan be05e28679
UX: Add table style for LLM list (#963)
Followup 2f7895bb91,
I forgot to commit this minor change
2024-11-27 13:45:01 +10:00
Martin Brennan 2f7895bb91
UX: Applying more admin UI guidelines (#956)
This commit applies further admin UI guidelines, now that they have been more
fleshed out in core, to the AI admin UI:

* Tools
* LLMs
* Personas

The changes include but are not limited to:

* Applying the table CSS classes, for desktop and mobile
* Adding a description and learn more link for each tab
* Adding an empty list placeholder with CTA using `AdminConfigAreaEmptyList`
* Replacing custom headings with `AdminPageSubheader`
2024-11-27 13:34:56 +10:00
Discourse Translator Bot 7a094cde18
Update translations (#962) 2024-11-26 23:15:10 +01:00
Roman Rizzi 251628bfa1
FIX: Shutdown embeddings thread pool after processing (#961) 2024-11-26 18:12:03 -03:00
Roman Rizzi ef07fcb308
FIX: Skip records without content to classify (#960) 2024-11-26 15:54:20 -03:00
Roman Rizzi ddf2bf7034
DEV: Backfill embeddings concurrently. (#941)
We are adding a new method for generating and storing embeddings in bulk, which relies on `Concurrent::Promises::Future`. Generating an embedding consists of three steps:

Prepare text
HTTP call to retrieve the vector
Save to DB.
Each one is independently executed on whatever thread the pool gives us.

We are bringing a custom thread pool instead of the global executor since we want control over how many threads we spawn to limit concurrency. We also avoid firing thousands of HTTP requests when working with large batches.
2024-11-26 14:12:32 -03:00
Rafael dos Santos Silva 23193ee6f2
FEATURE: Calculate gists from non hot topics too (#958)
Also renames some settings to remove 'hot' references.
2024-11-26 13:44:12 -03:00
Sam 54f2d34ccb
DEV: skip flakey spec (#955)
This spec fails inconsistently with:

-fragment-n14
       +You are a helpful Discourse assistant.
       +You _understand_ and **generate** Discourse Markdown.
       +You live in a Discourse Forum Message.
       +
       +You live in the forum with the URL: http://test.localhost
       +The title of your site: test site title
       +The description is: test site description
       +The participants in this conversation are: joe, jane
       +The date now is: 2024-11-25 20:23:02 UTC, much has changed since you were trained.
       +
       +You were trained on OLD data, lean on search to get up to date information about this forum
       +When searching try to SIMPLIFY search terms
       +Discourse search joins all terms with AND. Reduce and simplify terms to find more results.<guidance>
       +The following texts will give you additional guidance for your response.
       +We included them because we believe they are relevant to this conversation topic.
       +
       +Texts:
       +
       +fragment-n10
       +fragment-n9
       +fragment-n8
       +fragment-n7
       +fragment-n6
       +fragment-n5
       +fragment-n4
       +fragment-n3
       +fragment-n2
       +fragment-n1
       +</guidance>
2024-11-26 07:49:33 +11:00
Sam 616b990894
FEATURE: LLM mentions and auto silence (#949)
* FEATURE: allow mentioning an LLM mid conversation to switch

This is a edgecase feature that allow you to start a conversation
in a PM with LLM1 and then use LLM2 to evaluation or continue
the conversation

* FEATURE: allow auto silencing of spam accounts

New rule can also allow for silencing an account automatically

This can prevent spammers from creating additional posts.
2024-11-26 07:19:56 +11:00
Rafael dos Santos Silva 6c25718a7f
FEATURE: Add links to filtered emotion view on emotion dashboard table (#953) 2024-11-25 15:51:01 -03:00
Roman Rizzi 79021252e9
REFACTOR: Tidy-up embedding endpoints config. (#937)
Two changes worth mentioning:

`#instance` returns a fully configured embedding endpoint ready to use.
All endpoints respond to the same method and have the same signature - `perform!(text)`

This makes it easier to reuse them when generating embeddings in bulk.
2024-11-25 13:12:43 -03:00
Rafael dos Santos Silva aeba72441d
FIX: Switch up start/end dates in emotion dashboard report (#951) 2024-11-25 12:25:48 -03:00
Roman Rizzi 95762723de
PERF: Preload only gists when including summaries in topic list (#948)
* PERF: Preload only gists when including summaries in topic list

* Add unique index on summaries and dedup existing records

* Make hot topics batch size setting hidden
2024-11-25 12:24:02 -03:00
Rafael dos Santos Silva 5fb1177f7b
FEATURE: Refinements to Emotion in dashboard (#947)
* FEATURE: Refinements to Emotion in dashboard

- Added descriptions to individual reports
- Made reports work with data older than 60 days
2024-11-25 11:31:51 -03:00
David Taylor ea24328415
FIX: Correctly register 'info' icon (#950)
This icon is not part of the core set. Previously, d-ai was relying on other installed plugins (e.g. data-explorer) to include the icon
2024-11-25 10:20:17 -03:00
Natalie Tay f8231d259b
FEATURE: Add locale detection prompt from translator (#946) 2024-11-25 08:33:54 +11:00
Roman Rizzi e54f2da1a5
FIX: Unnecessary complex preloading accidentally filters some topics. (#945)
The `topic_query_create_list_topics` modifier we append was always meant to avoid an N+1 situation when serializing gists. However, I tried to be too smart and only preload these, which resulted in some topics with *only* regular summaries getting removed from the list. This issue became apparent now we are adding gists to other lists besides hot.

Let's simplify the preloading, which still solves the N+1 issue, and let the serializer get the needed summary.
2024-11-22 12:07:27 -03:00
Rafael dos Santos Silva 1921e4e90d
UX: Add emojis to the admin dash emotion table (#944) 2024-11-22 12:02:18 -03:00
Joffrey JAFFEUX 2cc8115b48
FIX: disables temporarily ai_summaries filtering (#943) 2024-11-22 08:34:54 +01:00
Sam 86cf4ccba7
FIX: automatically bust cache for share ai assets (#942)
* FIX: automatically bust cache for share ai assets

CDNs can be configured to strip query params in Discourse
hosting. This is generally safe, but in this case we had
no way of busting the cache using the path.

New design properly caches and properly breaks busts the
cache if asset changes so we don't need to worry about versions

* one day I will set up conditional lint on save :)
2024-11-22 11:23:15 +11:00
Rafael dos Santos Silva 2c78961bed
FIX: Properly capture period for /filter emotion ordering (#940) 2024-11-21 16:54:30 -03:00
Rafael dos Santos Silva 8e00e036e1
FEATURE: Make emotion /filter ordering match the dashboard table (#939)
* FEATURE: Make emotion /filter ordering match the dashboard table

This change makes the /filter endpoint use the same criteria we use
in the dashboard table for emotion, so it is not confusing for users.
It means that only posts made in the period with the emotion shall be
shown in the /filter, and the order is simply a count of posts that
match the emotion in the period.

It also uses a trick to extract the filter period, and apply it to
the CTE clause that calculates post emotion count on the period, making
it a bit more efficient. Downside is that /filter filters are evaluated
from left to right, so it will only get the speed-up if the emotion
order is last. As we do this on the dashboard table, it should cover
most uses of the ordering, kicking the need for materialized views
down the road.

* Remove zero score in filter

* add table tooltip

* lint
2024-11-21 15:18:31 -03:00
Sam d56ed53eb1
FIX: cancel functionality regressed (#938)
The cancel messaging was not floating correctly to the HTTP call leading to impossible to cancel completions 

This is now fully tested as well.
2024-11-21 17:51:45 +11:00
Kris d83248cf68
UX: force long words to break in topic list gists (#936) 2024-11-20 15:49:20 -05:00
David Taylor 872e43e849
DEV: Update linting (#935) 2024-11-20 18:50:50 +01:00
David Taylor 2900d2c87d
DEV: Update linting (#933) 2024-11-20 15:43:28 +01:00
Kris b68a4ca718
UX: ensure gists occupy full row (#934) 2024-11-20 09:33:49 -05:00
David Taylor 1a10680818
DEV: Replace DSection with body-class helper (#924) 2024-11-20 14:11:15 +00:00