Commit Graph

128 Commits

Author SHA1 Message Date
Sam 7eedbf29e0
FIX: refine image and read command (#131)
- Attempt to hint reading is done by sending complete:true
- Do not include post_number in result unless it was sent in
- Rush visual feedback when a command is run (ensure we always revise)
- Include hyperlink in read command description
- Stop round tripping to GPT after image generation (speeds up images by a lot)
- Add a test for image command
2023-08-09 16:01:48 +10:00
Sam 958dfc360e
FEATURE: experimental read command for bot (#129)
This command is useful for reading a topics content. It allows us to perform
critical analysis or suggest answers.

Given 8k token limit in GPT-4 I hardcoded reading to 1500 tokens, but we can
follow up and allow larger windows on models that support more tokens.

On local testing even in this limited form this can be very useful.
2023-08-09 07:19:56 +10:00
Rafael dos Santos Silva 3e7c99de89
FEATURE: Support for locally infered embeddings in 100 languages (#115)
* FEATURE: Support for locally infered embeddings in 100 languages

* add table
2023-07-27 15:50:03 -03:00
Rafael dos Santos Silva b25daed60b
FEATURE: Llama2 for summarization (#116) 2023-07-27 13:55:32 -03:00
Sam a028309cbd
FEATURE: add ai_bot_enabled_chat commands and tune search (#94)
* FEATURE: add ai_bot_enabled_chat commands and tune search

This allows admins to disable/enable GPT command integrations.

Also hones search results which were looping cause the result did not denote
the failure properly (it lost context)

* include more context for google command
include more context for time command

* type
2023-06-21 17:10:30 +10:00
Sam d1ab79e82f
FEATURE: Add Azure cognitive service support (#93)
The new site settings:

ai_openai_gpt35_url : distribution for GPT 16k
ai_openai_gpt4_url: distribution for GPT 4
ai_openai_embeddings_url: distribution for ada2

If untouched we will simply use OpenAI endpoints.

Azure requires 1 URL per model, OpenAI allows a single URL to serve multiple models. Hence the new settings.
2023-06-21 10:39:51 +10:00
Sam 30778d8af8
FIX: avoid storing corrupt prompts (#92)
```
prompt << build_message(bot_user.username, reply)
```

Would store a "cooked" prompt which is invalid, instead just store the raw
values which are later passed to build_message

Additionally:

1. Disable summary command which needs honing
2. Stop storing decorations (searched for X) in prompt which leads to straying
3. Ship username directly to model, avoiding "user: content" in prompts. This
 was causing GPT to stray
2023-06-20 15:44:03 +10:00
Roman Rizzi 3364fec425
DEV: Remove the summarization feature (#83)
* DEV: Remove the summarization feature

Instead, we'll register summarization implementations for OpenAI, Anthropic, and Discourse AI using the API defined in discourse/discourse#21813.

Core and chat will implement features on top of these implementations instead of this plugin extending them.

* Register instances that contain the model, requiring less site settings
2023-06-13 14:32:26 -03:00
Sam 92fb84e24d
iterate commands (#73)
* FEATURE: introduce a more efficient formatter

Previous formatting style was space inefficient given JSON consumes lots
of tokens, the new format is now used consistently across commands

Also fixes

- search limited to 10
- search breaking on limit: non existent directive

* Slight improvement to summarizer
Stop blowing up context with custom prompts

* ensure we include the guiding message

* correct spec

* langchain style summarizer ...

much more accurate (albeit more expensive)

* lint
2023-05-22 12:09:14 +10:00
Sam d59ed1091b
FEATURE: add support for GPT <-> Forum integration
This change-set connects GPT based chat with the forum it runs on. Allowing it to perform search, lookup tags and categories and summarize topics. 

The integration is currently restricted to public portions of the forum. 

Changes made:

- Do not run ai reply job for small actions
- Improved composable system prompt
- Trivial summarizer for topics
- Image generator 
- Google command for searching via Google
- Corrected trimming of posts raw (was replacing with numbers) 
- Bypass of problem specs

The feature works best with GPT-4


---------

Co-authored-by: Roman Rizzi <rizziromanalejandro@gmail.com>
2023-05-20 17:45:54 +10:00
Roman Rizzi 362f6167d1
FEATURE: Less friction for starting a conversation with an AI bot. (#63)
* FEATURE: Less friction for starting a conversation with an AI bot.

This PR adds a new header icon as a shortcut to start a conversation with one of our AI Bots. After clicking and selecting one from the dropdown menu, we'll open the composer with some fields already filled (recipients and title).

If you leave the title as is, we'll queue a job after five minutes to update it using a bot suggestion.

* Update assets/javascripts/initializers/ai-bot-replies.js

Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>

* Update assets/javascripts/initializers/ai-bot-replies.js

Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>

---------

Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>
2023-05-16 14:38:21 -03:00
Roman Rizzi 7e3cb0ea16
FEATURE: Multi-model support for the AI Bot module. (#56)
We'll create one bot user for each available model. When listed in the `ai_bot_enabled_chat_bots` setting, they will reply.

This PR lets us use Claude-v1 in stream mode.
2023-05-11 10:03:03 -03:00
Roman Rizzi 71b105a1bb
FEATURE: Introduce the ai-bot module (#52)
This module lets you chat with our GPT bot inside a PM. The bot only replies to members of the groups listed on the ai_bot_allowed_groups setting and only if you invite it to participate in the PM.
2023-05-05 15:28:31 -03:00
Rafael dos Santos Silva 5549e4d5b3
FEATURE: Chat channel summarization. (#32)
* start summary module

* chat channel summarization

* FEATURE: modal for channel summarization

---------

Co-authored-by: Roman Rizzi <rizziromanalejandro@gmail.com>
2023-04-04 11:24:09 -03:00
Roman Rizzi 333cb8f212
FIX: Don't try to use pg headlines for semantic search. (#36)
Depends on discourse/discourse#20939.
2023-04-03 11:48:38 -03:00
Roman Rizzi 4e05763a99
FEATURE: Semantic assymetric full-page search (#34)
Depends on discourse/discourse#20915

Hooks to the full-page-search component using an experimental API and performs an assymetric similarity search using our embeddings database.
2023-03-31 15:29:56 -03:00
Sam 0d80d9ec49
FEATURE: allow limiting results in related topics section (#30)
Also:

- Normalizes behavior between logged in and anon,
 we only show related topics in the related topic section

- Renames "suggested" to "related" given this only exists in related section
- Adds a spec section to ensure anon does not regress
- Adds `ai_embeddings_semantic_related_topics` to limit related topics

Renamed settings:

ai_embeddings_semantic_suggested_model -> ai_embeddings_semantic_related_model
ai_embeddings_semantic_suggested_topics_enabled -> ai_embeddings_semantic_related_topics_enabled

Plugins is still in an experimental phase and not much is overidden hence
avoiding adding site setting migrations.


Co-authored-by: Krzysztof Kotlarek <kotlarek.krzysztof@gmail.com>
2023-03-31 11:04:34 +11:00
Sam 1d097b9d82
FEATURE: attempt to include related topics above suggested (#28)
Allows related topics to show up for logged on users

- Introduces a new "Related Topics" block above suggested when related topics exist
- Renames `ai_embeddings_semantic_suggested_topics_anons_enabled` -> `ai_embeddings_semantic_suggested_topics_enabled` (given it is only deployed on 1 site not bothering with a migration)
- Adds an integration test to ensure data arrives correctly on the client
2023-03-31 09:07:22 +11:00
Rafael dos Santos Silva b942a18298
FEATURE: Support for GPT-4 in AI Helper module (#29) 2023-03-28 23:22:34 -03:00
Roman Rizzi 4c960970fa
DEV: Log information about errors from the completions OpenAI API (#26) 2023-03-22 16:00:28 -03:00
Sam 1d14f7ffaf
FEATURE: Add a markdown table AI helper (#25) 2023-03-22 13:16:29 -03:00
Roman Rizzi 39f7f1f29e
FEATURE: Prompts can consist of multiple messages. (#21)
A prompt with multiple messages leads to better results, as the AI can learn for given examples. Alongside this change, we provide a better default proofreading prompt.
2023-03-21 12:04:59 -03:00
Roman Rizzi 320ac6e84b
REFACTOR: Store prompts in a dedicated table. (#14)
This change makes it easier to add new prompts to our AI helper. We don't have a UI for it yet. You'll have to do it through a console.
2023-03-17 15:14:19 -03:00
Rafael dos Santos Silva 80d662e9e8
FEATURE: Semantic Suggested Topics (#10) 2023-03-15 17:21:45 -03:00
Roman Rizzi f99fe7e1ed
FEATURE: Composer AI helper (#8)
* FEATURE: Composer AI helper

This change introduces a new composer button for the group members listed in the `ai_helper_allowed_groups` site setting.

Users can use chatGPT to review, improve, or translate their posts to English.

* Add a safeguard for PMs and don't rely on parentView
2023-03-15 17:02:20 -03:00
Roman Rizzi a838116cd5
FEATURE: Use dedicated reviewables for AI flags. (#4)
This change adds two new reviewable types: ReviewableAIPost and ReviewableAIChatMessage. They have the same actions as their existing counterparts: ReviewableFlaggedPost and ReviewableChatMessage.

We'll display the model used and their accuracy when showing these flags in the review queue and adjust the latter after staff performs an action, tracking a global accuracy per existing model in a separate table.


* FEATURE: Dedicated reviewables for AI flags

* Store and adjust model accuracy

* Display accuracy in reviewable templates
2023-03-07 15:39:28 -03:00
Rafael dos Santos Silva 6cf411ec90
add toxicity and sentiment modules 2023-02-22 20:46:53 -03:00
Roman Rizzi 2b319d9077
Initial commit 2023-02-17 11:33:47 -03:00