Commit Graph

153 Commits

Author SHA1 Message Date
Sam 8b382d6098
FEATURE: support for claude opus and sonnet (#508)
This provides new support for messages API from Claude.

It is required for latest model access.

Also corrects implementation of function calls.

* Fix message interleving

* fix broken spec

* add new models to automation
2024-03-06 06:04:37 +11:00
Sam b7a96e3bcb
FIX: avoid all bot feedback loops (#507)
We need to ensure that under no circumstances feedback loops between
bots will emerge cause this can eat up a lot of tokens
2024-03-05 10:02:49 +11:00
Sam 77cf9e2cff
FIX: system persona non English save, missing bot pms
- FIX: only update system attributes when updating system persona
- FIX: update participant count by hand so bot messages show in inbox
 

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2024-03-04 09:56:59 +11:00
Sam c02794cf2e
FIX: support multiple tool calls (#502)
* FIX: support multiple tool calls

Prior to this change we had a hard limit of 1 tool call per llm
round trip. This meant you could not google multiple things at
once or perform searches across two tools.

Also:

- Hint when Google stops working
- Log topic_id / post_id when performing completions

* Also track id for title
2024-03-02 07:53:21 +11:00
Sam 59bab2bba3
FIX: stream messages when directly PMing a persona (#500)
previous to this fix we did not consider personas a bot in the
front end
2024-03-01 07:53:42 +11:00
Rafael dos Santos Silva 1b72a00d2c
FEATURE: Option for AI triage to send a post to the review queue (#498)
Option for AI triage to send a post to the review queue
2024-02-29 12:33:28 +11:00
Sam 484fd1435b
DEV: improve internal design of ai persona and bug fix (#495)
* DEV: improve internal design of ai persona and bug fix

- Fixes bug where OpenAI could not describe images
- Fixes bug where mentionable personas could not be mentioned unless overarching bot was enabled
- Improves internal design of playground and bot to allow better for non "bot" users
- Allow PMs directly to persona users (previously bot user would also have to be in PM)
- Simplify internal code


Co-authored-by: Martin Brennan <martin@discourse.org>
2024-02-28 16:46:32 +11:00
Sam d036f3fb8e
FEATURE: AI helper support in non English languages (#489)
* FEATURE: AI helper support in non English languages

This attempts some prompt engineering to coerce AI helper to answer
in the appropriate language.

Note mileage will vary, in testing GPT-4 produces the best results
GPT-3.5 can return OKish results.

* Extend non english support for GPT-4V image caption

* Update db/fixtures/ai_helper/603_completion_prompts.rb

---------

Co-authored-by: Rafael Silva <xfalcox@gmail.com>
2024-02-27 16:31:51 -03:00
Sam 1f74a77e17
DEV: correct flaky spec (#475)
We were not properly expiring prompt cache
2024-02-19 15:21:55 +11:00
Sam 0fb87b00e2
FEATURE: new Discourse Helper persona (#473)
This persona searches Discourse Meta for help with Discourse and
points users at relevant posts.

It is somewhat similar to using "Forum Helper" on meta, with the
notable difference that we can not lean on semantic search so using
some prompt engineering we try to keep it simple.
2024-02-19 14:52:12 +11:00
Keegan George d66915ecc1
DEV: Make prompts available on `CurrentUserSerializer` (#472) 2024-02-16 10:57:14 -08:00
Sam 3a8d95f6b2
FEATURE: mentionable personas and random picker tool, context limits (#466)
1. Personas are now optionally mentionable, meaning that you can mention them either from public topics or PMs
       - Mentioning from PMs helps "switch" persona mid conversation, meaning if you want to look up sites setting you can invoke the site setting bot, or if you want to generate an image you can invoke dall e
        - Mentioning outside of PMs allows you to inject a bot reply in a topic trivially
     - We also add the support for max_context_posts this allow you to limit the amount of context you feed in, which can help control costs

2. Add support for a "random picker" tool that can be used to pick random numbers 

3. Clean up routing ai_personas -> ai-personas

4. Add Max Context Posts so users can control how much history a persona can consume (this is important for mentionable personas) 

Co-authored-by: Martin Brennan <martin@discourse.org>
2024-02-15 16:37:59 +11:00
Rafael dos Santos Silva 0dba6623a0
FIX: Better AI chat thread titles (#467)
* FIX: Better AI chat thread titles

- Fix quote removal when multi-line

- Use XML tags for better LLM output parsing

- Use stop_sequences for faster and less wasteful LLM calls

- Adds truncation as the last line of defense
2024-02-09 14:49:28 -03:00
Rafael dos Santos Silva bccb7efdd6
FIX: Use a dedicated prompt for thread titles (#464) 2024-02-07 15:05:50 -03:00
Sam ba3c3951cf
FIX: typo causing text_embedding_3_large to fail (#460) 2024-02-05 11:16:36 +11:00
Sam a3c827efcc
FEATURE: allow personas to supply top_p and temperature params (#459)
* FEATURE: allow personas to supply top_p and temperature params

Code assistance generally are more focused at a lower temperature
This amends it so SQL Helper runs at 0.2 temperature vs the more
common default across LLMs of 1.0.

Reduced temperature leads to more focused, concise and predictable
answers for the SQL Helper

* fix tests

* This is not perfect, but far better than what we do today

Instead of fishing for

1. Draft sequence
2. Draft body

We skip (2), this means the composer "only" needs 1 http request to
open, we also want to eliminate (1) but it is a bit of a trickier
core change, may figure out how to pull it off (defer it to first draft save)

Value of bot drafts < value of opening bot conversations really fast
2024-02-03 07:09:34 +11:00
Roman Rizzi fba9c1bf2c
UX: Re-introduce embedding settings validations (#457)
* Revert "Revert "UX: Validate embeddings settings (#455)" (#456)"

This reverts commit 392e2e8aef.

* Resstore previous default
2024-02-01 16:54:09 -03:00
Roman Rizzi 392e2e8aef
Revert "UX: Validate embeddings settings (#455)" (#456)
This reverts commit 85fca89e01.
2024-02-01 14:06:51 -03:00
Roman Rizzi 85fca89e01
UX: Validate embeddings settings (#455) 2024-02-01 13:05:38 -03:00
Sam dcafc8032f
FIX: improve embedding generation (#452)
1. on failure we were queuing a job to generate embeddings, it had the wrong params. This is both fixed and covered in a test.
2. backfill embedding in the order of bumped_at, so newest content is embedded first, cover with a test
3. add a safeguard for hidden site setting that only allows batches of 50k in an embedding job run

Previously old embeddings were updated in a random order, this changes it so we update in a consistent order
2024-01-31 10:38:47 -03:00
Sam ab7e9e31aa
FEATURE: allow excluding tags and categories from LLM report (#447)
Also

- Better diagnostics, output model being used
- Prompt LLM that true content is being injected in <context> tag
2024-01-30 15:55:05 +11:00
Roman Rizzi bae71eb047
FIX: Include provider in automation models (#446) 2024-01-29 18:07:29 -03:00
Roman Rizzi 0634b85a81
UX: Validations to LLM-backed features (except AI Bot) (#436)
* UX: Validations to Llm-backed features (except AI Bot)

This change is part of an ongoing effort to prevent enabling a broken feature due to lack of configuration. We also want to explicit which provider we are going to use. For example, Claude models are available through AWS Bedrock and Anthropic, but the configuration differs.

Validations are:

* You must choose a model before enabling the feature.
* You must turn off the feature before setting the model to blank.
* You must configure each model settings before being able to select it.

* Add provider name to summarization options

* vLLM can technically support same models as HF

* Check we can talk to the selected model

* Check for Bedrock instead of anthropic as a site could have both creds setup
2024-01-29 16:04:25 -03:00
Rafael dos Santos Silva 04bc402aae
FEATURE: Setting to control per post embeddings (#439)
* FEATURE: Setting to control per post embeddings
2024-01-23 22:09:27 -03:00
Jarek Radosz 5802cd1a0c
DEV: Fix various typos (#434) 2024-01-19 12:51:26 +01:00
Sam 825f01cfb2
FEATURE: even smoother streaming (#420)
Account properly for function calls, don't stream through <details> blocks
- Rush cooked content back to client
- Wait longer (up to 60 seconds) before giving up on streaming
- Clean up message bus channels so we don't have leftover data
- Make ai streamer much more reusable and much easier to read
- If buffer grows quickly, rush update so you are not artificially waiting
- Refine prompt interface
- Fix lost system message when prompt gets long
2024-01-15 18:51:14 +11:00
Jarek Radosz 6b8a57d957
DEV: Update linting (#423)
Co-authored-by: Keegan George <kgeorge13@gmail.com>
2024-01-13 00:28:06 +01:00
Roman Rizzi 04eae76f68
REFACTOR: Represent generic prompts with an Object. (#416)
* REFACTOR: Represent generic prompts with an Object.

* Adds a bit more validation for clarity

* Rewrite bot title prompt and fix quirk handling

---------

Co-authored-by: Sam Saffron <sam.saffron@gmail.com>
2024-01-12 14:36:44 -03:00
Rafael dos Santos Silva 3be76ebd7a
FEATURE: Move the default embeddings model to bge-large-en (#417) 2024-01-11 14:16:25 -03:00
Sam 05f7808057
FEATURE: more elegant progress (#409)
Previous to this change it was very hard to tell if completion was
stuck or not.

This introduces a "dot" that follows the completion and starts
flashing after 5 seconds.
2024-01-09 09:20:28 -03:00
Sam 17cc09ec9c
FIX: don't include <details> in context (#406)
* FIX: don't include <details> in context

We need to be careful adding <details> into context of conversations
it can cause LLMs to hallucinate results

* Fix Gemini multi-turn ctx flattening

---------

Co-authored-by: Roman Rizzi <rizziromanalejandro@gmail.com>
2024-01-05 15:21:14 -03:00
Keegan George 7201d482d5
FEATURE: Add DallE support to AI helper's illustrate post (#404) 2024-01-05 09:03:23 -08:00
Sam dd42a4e47b
FIX: array arguments not parsed correctly (#405)
DALL E command accepts an Array as a tool argument, this was not
parsed correctly by the invoker leading to errors generating
images with DALL E

Side quest ... don't use update! it calls validations and will now
fail due to email validation
2024-01-05 14:39:32 +11:00
Roman Rizzi 971e03bdf2
FEATURE: AI Bot Gemini support. (#402)
It also corrects the syntax around tool support, which was wrong.

Gemini doesn't want us to include messages about previous tool invocations, so I had to shuffle around some code to send the response it generated from those invocations instead. For this, I created the "multi_turn" context, which bundles all the context involved in the interaction.
2024-01-04 18:15:34 -03:00
Roman Rizzi f9d7d7f5f0
DEV: AI bot migration to the Llm pattern. (#343)
* DEV: AI bot migration to the Llm pattern.

We added tool and conversation context support to the Llm service in discourse-ai#366, meaning we met all the conditions to migrate this module.

This PR migrates to the new pattern, meaning adding a new bot now requires minimal effort as long as the service supports it. On top of this, we introduce the concept of a "Playground" to separate the PM-specific bits from the completion, allowing us to use the bot in other contexts like chat in the future. Commands are called tools, and we simplified all the placeholder logic to perform updates in a single place, making the flow more one-wayish.

* Followup fixes based on testing

* Cleanup unused inference code

* FIX: text-based tools could be in the middle of a sentence

* GPT-4-turbo support

* Use new LLM API
2024-01-04 10:44:07 -03:00
Keegan George 1a5985134a
FIX: Show illustrate post only if stability API key present (#395) 2024-01-02 11:24:16 -08:00
Rafael dos Santos Silva 140359c2ef
FEATURE: Per post embeddings (#387) 2023-12-29 12:28:45 -03:00
Sam 37dd98c937
FIX: exclude non visible topics from report context (#375)
Generally non visible topics are not that interesting, do not add
this noise to the report context
2023-12-21 19:08:36 +11:00
Keegan George 7b4710d5c9
FEATURE: Generate post illustrations (#367) 2023-12-19 11:17:34 -08:00
Sam 529703b5ec
FEATURE: support sending AI report to an email address (#368)
Support emailing the AI report to any arbitrary email
2023-12-19 17:51:49 +11:00
Sam d0f54443ae
FEATURE: LLM based peroidical summary report (#357)
Introduce a Discourse Automation based periodical report. Depends on Discourse Automation.

Report works best with very large context language models such as GPT-4-Turbo and Claude 2.

- Introduces final_insts to generic llm format, for claude to work best it is better to guide the last assistant message (we should add this to other spots as well)
- Adds GPT-4 turbo support to generic llm interface
2023-12-19 12:04:15 +11:00
Krzysztof Kotlarek 6de9b9c274
DEV: Replace deprecated min_trust_to_create_post (#356)
In https://github.com/discourse/discourse/pull/24740, `min_trust_to_create_topic` site setting was replaced by `create_topic_allowed_groups`. This PR replaces the former, deprecated one, with the latter.
2023-12-14 14:07:28 +11:00
Sam 605445831f
FEATURE: try including views/username/likes in search results (#349)
This is somewhat experimental, but the context of likes/view/username
can help the llm find out what content is more important or even
common users that produce great content

This inflates the amount of tokens somewhat, but given it is all numbers
and search columns titles are only included once this is not severe
2023-12-12 12:22:28 +11:00
Sam a66b1042cc
FEATURE: scale up result count for search depending on model (#346)
We were limiting to 20 results unconditionally cause we had to make
sure search always fit in an 8k context window.

Models such as GPT 3.5 Turbo (16k) and GPT 4 Turbo / Claude 2.1 (over 150k)
allow us to return a lot more results.

This means we have a much richer understanding cause context is far
larger.

This also allows a persona to tweak this number, in some cases admin
may want to be conservative and save on tokens by limiting results

This also tweaks the `limit` param which GPT-4 liked to set to tell
model only to use it when it needs to (and describes default behavior)
2023-12-11 16:54:16 +11:00
Sam 6380ebd829
FEATURE: allow personas to provide command options (#331)
Personas now support providing options for commands.

This PR introduces a single option "base_query" for the SearchCommand. When supplied all searches the persona will perform will also include the pre-supplied filter.

This can allow personas to search a subset of the forum (such as documentation)

This system is extensible we can add options to any command trivially.
2023-12-08 08:42:56 +11:00
Roman Rizzi 450ec915d8
FIX: Make FoldContent strategy more resilient when using models with low token count. (#341)
We'll recursively summarize  the content into smaller chunks until we are sure we can concatenate
them without going over the token limit.
2023-12-06 19:00:24 -03:00
Martin Brennan 24370a9ca6
Revert "FIX: Use Guardian.basic_user instead of new (anon) (#332)" (#337)
This reverts commit a3a1285dc5.

c.f. https://github.com/discourse/discourse/pull/24742
2023-12-06 16:26:43 +10:00
Martin Brennan a3a1285dc5
FIX: Use Guardian.basic_user instead of new (anon) (#332)
c.f. de983796e1b66aa2ab039a4fb6e32cec8a65a098

There will soon be additional login_required checks
for Guardian, and the intent of many checks by automated
systems is better fulfilled by using BasicUser, which
simulates a logged in TL0 forum user, rather than an
anon user.
2023-12-06 12:01:41 +10:00
Roman Rizzi 3bc010b686
FIX: call the right method to summarize with truncation (#328) 2023-12-01 10:17:24 -03:00
Sam 6ddc17fd61
DEV: port directory structure to Zeitwerk (#319)
Previous to this change we relied on explicit loading for a files in Discourse AI.

This had a few downsides:

- Busywork whenever you add a file (an extra require relative)
- We were not keeping to conventions internally ... some places were OpenAI others are OpenAi
- Autoloader did not work which lead to lots of full application broken reloads when developing.

This moves all of DiscourseAI into a Zeitwerk compatible structure.

It also leaves some minimal amount of manual loading (automation - which is loading into an existing namespace that may or may not be there)

To avoid needing /lib/discourse_ai/... we mount a namespace thus we are able to keep /lib pointed at ::DiscourseAi

Various files were renamed to get around zeitwerk rules and minimize usage of custom inflections

Though we can get custom inflections to work it is not worth it, will require a Discourse core patch which means we create a hard dependency.
2023-11-29 15:17:46 +11:00