Commit Graph

418 Commits

Author SHA1 Message Date
Roman Rizzi 3bc010b686
FIX: call the right method to summarize with truncation (#328) 2023-12-01 10:17:24 -03:00
Sam 7f36175ad3
FIX: allow selection of persona when list gets too long (#327) 2023-12-01 08:29:10 +11:00
Jarek Radosz f973fafc33
DEV: Update linting (#326) 2023-11-29 23:01:48 +01:00
Discourse Translator Bot 4ccb98fdcd
Update translations (#320) 2023-11-29 10:41:32 +01:00
Sam a0b9fb9721
FIX: explicitly load embedding strategies (#325)
If not, sometimes during tests these constants may not be loaded
leading to flaky tests
2023-11-29 16:36:56 +11:00
Sam 6ddc17fd61
DEV: port directory structure to Zeitwerk (#319)
Previous to this change we relied on explicit loading for a files in Discourse AI.

This had a few downsides:

- Busywork whenever you add a file (an extra require relative)
- We were not keeping to conventions internally ... some places were OpenAI others are OpenAi
- Autoloader did not work which lead to lots of full application broken reloads when developing.

This moves all of DiscourseAI into a Zeitwerk compatible structure.

It also leaves some minimal amount of manual loading (automation - which is loading into an existing namespace that may or may not be there)

To avoid needing /lib/discourse_ai/... we mount a namespace thus we are able to keep /lib pointed at ::DiscourseAi

Various files were renamed to get around zeitwerk rules and minimize usage of custom inflections

Though we can get custom inflections to work it is not worth it, will require a Discourse core patch which means we create a hard dependency.
2023-11-29 15:17:46 +11:00
Kris 0b9947771c
FIX: adjust tag composer input to avoid wrapping (#324) 2023-11-28 17:38:29 -05:00
Rafael dos Santos Silva fd0fb58eca
FEATURE: HuggingFace Text Embeddings Inference compatibility (#323)
* FEATURE: HuggingFace Text Embeddings Inference compatibility

* lint
2023-11-28 17:05:26 -03:00
Roman Rizzi f26adf2cf6
FIX: Use XML tags in generate_titles prompt. (#322)
We must ensure we can isolate titles, and the models sometimes ignore the example we give them.

Additionally, anons can generate HyDE posts, so we need to check if user is nil when attempting to log requests.
2023-11-28 12:52:22 -03:00
Rafael dos Santos Silva 11e531b099
FEATURE: Backfill task for sentiment module (#316)
* FEATURE: Backfill task for sentiment module

* fix join clause
2023-11-28 12:28:36 -03:00
Roman Rizzi 2e7c5f047d
DEV: Don't attempt to update log if completion request fails. (#321)
We already log the request failure when we raise the exception.
2023-11-28 11:15:12 -03:00
Keegan George 02f7b368a1
FIX: Too many requests from single search (#318) 2023-11-27 17:51:42 -08:00
Keegan George c7665e891b
A11Y: Add title attribute to sparkles icon for AI search results (#317) 2023-11-27 14:25:33 -08:00
Keegan George d493880c0a
FIX: Incorrect sort order label appearing when it should not (#315) 2023-11-27 17:27:22 -03:00
Roman Rizzi 775610b1c2
FIX: Chat titler was still using the old code after LLM migration (#314) 2023-11-27 13:03:24 -03:00
Roman Rizzi 54a8dd9556
REFACTOR: Use LLM abstraction in the AI Helper. (#312)
It also removes the need for multiple versions of our seeded prompts per model, further simplifying the code.
2023-11-27 09:33:31 -03:00
Sam 5a4598a7b4
FEATURE: Azure OpenAI support for DALL*E 3 (#313)
* FEATURE: Azure OpenAI support for DALL*E 3

Previous to this there was no way to add an inference endpoint for
DALL*E on Azure cause it requires custom URLs

Also:

- On save, when editing a persona it would revert priority and enabled
- More forgiving parsing in command framework for array function calls
- By default generate HD images - they tend to be a bit better
- Improve DALL*E prompt which was getting very annoying and always echoing what it is about to do
- Add a bit of a sleep between retries on image generation
- Fix error handling in image_command
2023-11-27 13:01:05 +11:00
Sam dff9f33a97
FEATURE: DALL-E-3 persona for image generation (#311)
* FIX: no selected persona should pick first prioritized one

Previously we were looking at `.personaId` but there is only an
id attribute so it failed

* FEATURE: new DALL-E-3 persona

This persona generates images using DALL-E-3 API and is enabled
by default

Keep in mind that we are still waiting on seeds/gen_id so we can
not retain style consistently between turns.

This will change as soon as a new Open AI API provides the missing
parameters

Co-authored-by: Martin Brennan <martin@discourse.org>
2023-11-24 18:08:08 +11:00
Sam 6282b6d21f
FIX: implement tools framework for Anthropic (#307)
Previous to this changeset we used a custom system for tools/command
support for Anthropic.

We defined commands by using !command as a signal to execute it

Following Anthropic Claude 2.1, there is an official supported syntax (beta)
for tools execution.

eg:

```
+      <function_calls>
+      <invoke>
+      <tool_name>image</tool_name>
+      <parameters>
+      <prompts>
+      [
+      "an oil painting",
+      "a cute fluffy orange",
+      "3 apple's",
+      "a cat"
+      ]
+      </prompts>
+      </parameters>
+      </invoke>
+      </function_calls>
```

This implements the spec per Anthropic, it should be stable enough
to also work on other LLMs.

Keep in mind that OpenAI is not impacted here at all, as it has its
own custom system for function calls.

Additionally:

- Fixes the title system prompt so it works with latest Anthropic
- Uses new spec for "system" messages by Anthropic
- Tweak forum helper persona to guide Anthropic a tiny be better

Overall results are pretty awesome and Anthropic Claude performs
really well now on Discourse
2023-11-24 06:39:56 +11:00
Roman Rizzi 419c43592a
FIX: Make summaries more cohesive by tweaking prompt. (#310)
Other changes:

- Don't use Bedrock for non claude models if credentials are set.
- Remove extra sentence from HyDE prompt.
2023-11-23 16:33:37 -03:00
Keegan George df8804afcd
DEV: Only allow semantic search on "Relevance" sort mode (#306) 2023-11-23 11:30:17 -08:00
Roman Rizzi 02efca162e
FIX: Bedrock uses slightly different model names
* Revert "FIX: We don't need to prepend anthropic. to bedrock models (#308)"

This reverts commit 8a01751991.

* FIX: Bedrock uses slightly different model names
2023-11-23 15:49:24 -03:00
Roman Rizzi 8a01751991
FIX: We don't need to prepend anthropic. to bedrock models (#308) 2023-11-23 14:39:21 -03:00
Roman Rizzi 3064d4c288
REFACTOR: Summarization and HyDE now use an LLM abstraction. (#297)
* DEV: One LLM abstraction to rule them all

* REFACTOR: HyDE search uses new LLM abstraction

* REFACTOR: Summarization uses the LLM abstraction

* Updated documentation and made small fixes. Remove Bedrock claude-2 restriction
2023-11-23 12:58:54 -03:00
Jarek Radosz 53b7f031ba
DEV: Remove `gem "aws-eventstream"` entry (#305)
Core already includes aws-eventstream through aws-sdk-s3 and aws-sdk-sns.

Specifying a dependency in a plugin was blocking an upgrade in core (https://github.com/discourse/discourse/pull/24518)
2023-11-23 10:00:25 +01:00
Keegan George 787cd1bf17
FIX: Error 500 from search with only filters (#304) 2023-11-22 11:05:56 -08:00
Keegan George 432590f8d9
DEV: Run AI search before submit if query params present (#303) 2023-11-21 16:51:08 -08:00
Keegan George c55014839f
FIX: more results not appearing on scroll (#302) 2023-11-21 09:46:37 -08:00
Roman Rizzi e0691e70e8
DEV: Updates to the summarization strategy API (#301)
Introduced by discourse/discourse#24489

In the future, this change will let us log who requested the summary in the `AiApiAuditLog`.:
2023-11-21 13:27:35 -03:00
Discourse Translator Bot 493b48477a
Update translations (#300) 2023-11-21 14:36:22 +01:00
Sam 98c89953d3
FEATURE: remember previously selected persona (#299)
People tend to keep to 1 persona when working with the bot,
this adds local browser memory for the last persona you interacted
with so you do not need to select it over and over again.

This is per browser, not per user memory.

Also... clean up tests so they do not need to require stubs which
were breaking the build

---------

Co-authored-by: Martin Brennan <martin@discourse.org>
2023-11-21 17:02:27 +10:00
Sam 5b5edb22c6
FEATURE: UI to update ai personas on admin page (#290)
Introduces a UI to manage customizable personas (admin only feature)

Part of the change was some extensive internal refactoring:

- AIBot now has a persona set in the constructor, once set it never changes
- Command now takes in bot as a constructor param, so it has the correct persona and is not generating AIBot objects on the fly
- Added a .prettierignore file, due to the way ALE is configured in nvim it is a pre-req for prettier to work
- Adds a bunch of validations on the AIPersona model, system personas (artist/creative etc...) are all seeded. We now ensure
- name uniqueness, and only allow certain properties to be touched for system personas.
- (JS note) the client side design takes advantage of nested routes, the parent route for personas gets all the personas via this.store.findAll("ai-persona") then child routes simply reach into this model to find a particular persona.
- (JS note) data is sideloaded into the ai-persona model the meta property supplied from the controller, resultSetMeta
- This removes ai_bot_enabled_personas and ai_bot_enabled_chat_commands, both should be controlled from the UI on a per persona basis
- Fixes a long standing bug in token accounting ... we were doing to_json.length instead of to_json.to_s.length
- Amended it so {commands} are always inserted at the end unconditionally, no need to add it to the template of the system message as it just confuses things
- Adds a concept of required_commands to stock personas, these are commands that must be configured for this stock persona to show up.
- Refactored tests so we stop requiring inference_stubs, it was very confusing to need it, added to plugin.rb for now which at least is clearer
- Migrates the persona selector to gjs

---------

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
Co-authored-by: Martin Brennan <martin@discourse.org>
2023-11-21 16:56:43 +11:00
Keegan George 491111e5c8
FIX: Un-clickable AI Results (#298) 2023-11-20 11:37:00 -08:00
Keegan George f7277d244e
DEV: Mix semantic search results with normal results (#278) 2023-11-17 12:46:59 -08:00
Keegan George d1f21c78f1
UX: Add copy button to generated suggestion (#296) 2023-11-17 09:25:41 -08:00
David Taylor e1a7d07a91
DEV: Update tests for core change (#293)
https://github.com/discourse/discourse/pull/23967
2023-11-15 09:23:27 +00:00
Discourse Translator Bot c6902c40ce
Update translations (#294) 2023-11-14 14:30:22 +01:00
Roman Rizzi 1588ab99a6
FIX: ChatChannel no longer inherits from RestModel. (#292)
Trying to hydrate using the store will fail. We need to use the model create function instead.
2023-11-13 14:45:20 -03:00
Roman Rizzi 244a338558
UX: Graph surprise next to joy in post_emotion chart (#291) 2023-11-10 14:41:50 -03:00
Sam a4f419f54f
FEATURE: basic infrastructure for custom personas (#288)
- New AiPersona model which can store custom personas
- Persona are restricted via group security
- They can contain custom system messages
- They can support a list of commands optionally

To avoid expensive DB calls in the serializer a Multisite friendly Hash was introduced (which can be expired on transaction commit)
2023-11-10 11:39:49 +11:00
Roman Rizzi d0198c5c5b
FIX: Changes to the sentiment reports. (#289)
This PR aims to clarify sentiment reports by replacing averages with a count of posts that have one of their values above a threshold (60), meaning we have some level of confidence they are, in fact, positive or negative.

Same thing happen with post emotions, with the difference that a post can have multiple values above it (30). Additionally, we dropped the "Neutral" axis.

We also reworded the tooltip next to each report title, and added an early return to signal we have no data available instead of displaying an empty chart.
2023-11-09 17:23:25 -03:00
Martin Brennan 0c83963a28
DEV: Fix plugin.rb wording (#287)
This was not a complete sentence before
2023-11-09 11:23:43 +10:00
Roman Rizzi 458e66aef9
FIX: Filter classification type using the correct column (#286) 2023-11-08 14:58:35 -03:00
Roman Rizzi 231cf91cc2
FIX: Don't divide by zero if there is no emotion data for TL group (#285) 2023-11-08 13:05:36 -03:00
Roman Rizzi b172ef11c4
FEATURE: Expose sentiment classifications via the admin dashboard. (#284)
This PR adds new reports for displaying information about post sentiments grouped by date and emotions group by TL.

Depends on discourse/discourse#24274
2023-11-08 10:50:37 -03:00
Martin Brennan 30821badf2
Update plugin.rb metadata (#279)
Co-authored-by: AlexDev <104522507+alexdevlaeminck@users.noreply.github.com>
2023-11-07 15:45:02 -08:00
Ty Correll 3df6cb1ec9
UX: prevent suggestion button from wrapping (#282)
* UX: prevent suggestion button from wrapping

This PR addresses the visual bug where the suggestion button wraps on any screen size.

* UX: edge case screen size

This addition fixes screen size issues at specific screen sizes without affecting larger screen sizes, no media query necessary.
2023-11-04 12:52:51 -05:00
Keegan George f6996a6ef0
DEV: Add wrapping classes and remove `has()` usage (#281) 2023-11-03 11:41:57 -07:00
David Taylor 0902f74af5
DEV: Update linting configs (#280) 2023-11-03 11:30:09 +00:00
Discourse Translator Bot e30082dd20
Update translations (#277) 2023-11-01 12:27:29 -03:00