Commit Graph

504 Commits

Author SHA1 Message Date
Sam 057fbe1ce6
FEATURE: add internal support for streaming mode (#42)
Also adds some tests around completions and supports additional params
such as top_p, temperature and max_tokens

This also migrates off Faraday to using Net::HTTP directly
2023-04-21 16:54:25 +10:00
Meghna 14b21b4f4d
UX: add a custom sparkles icon for AI action buttons (#44) 2023-04-20 20:41:24 +05:30
Roman Rizzi 38e007a3a5
FEATURE: Topic summarization (#41)
* FEATURE: Topic summarization

Summarize topics using the TopicView's "summary" filter. The UI is similar to what we do for chat, but we don't allow the user to select a timeframe.


Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>
2023-04-19 17:57:31 -03:00
Rafael dos Santos Silva 9783e3b025
FEATURE: Add a basic tokenizer API (#37)
* FEATURE: Add a basic tokenizer API

* Add tests

* lint
2023-04-19 11:55:59 -03:00
Rafael dos Santos Silva 5a78b6a953
DEV: Fix test (#43)
* DEV: Fix test

* lint
2023-04-18 15:43:10 -03:00
Rafael dos Santos Silva 4368ef29d8
FIX: Sometimes Claude sends all titles suggestions in a single ai tag (#40) 2023-04-10 16:02:44 -03:00
Rafael dos Santos Silva bb0b829634
FEATURE: Anthropic Claude for AIHelper and Summarization modules (#39) 2023-04-10 11:04:42 -03:00
Roman Rizzi f2e52f7f24
FIX: Check for chat channel permissions (#38) 2023-04-06 09:07:31 -03:00
Rafael dos Santos Silva 5549e4d5b3
FEATURE: Chat channel summarization. (#32)
* start summary module

* chat channel summarization

* FEATURE: modal for channel summarization

---------

Co-authored-by: Roman Rizzi <rizziromanalejandro@gmail.com>
2023-04-04 11:24:09 -03:00
Roman Rizzi 333cb8f212
FIX: Don't try to use pg headlines for semantic search. (#36)
Depends on discourse/discourse#20939.
2023-04-03 11:48:38 -03:00
Roman Rizzi 7a54455cf6
FIX: Use correct variable and method for embeddings (#35) 2023-03-31 16:15:10 -03:00
Roman Rizzi 4e05763a99
FEATURE: Semantic assymetric full-page search (#34)
Depends on discourse/discourse#20915

Hooks to the full-page-search component using an experimental API and performs an assymetric similarity search using our embeddings database.
2023-03-31 15:29:56 -03:00
Rafael dos Santos Silva 99886fb64d
Update default thresholds for Toxicity (#33) 2023-03-31 13:03:44 -03:00
Sam 6543c50758
FIX: stop returning self as a candidate for related topics (#31) 2023-03-31 11:04:17 +10:00
Sam 0d80d9ec49
FEATURE: allow limiting results in related topics section (#30)
Also:

- Normalizes behavior between logged in and anon,
 we only show related topics in the related topic section

- Renames "suggested" to "related" given this only exists in related section
- Adds a spec section to ensure anon does not regress
- Adds `ai_embeddings_semantic_related_topics` to limit related topics

Renamed settings:

ai_embeddings_semantic_suggested_model -> ai_embeddings_semantic_related_model
ai_embeddings_semantic_suggested_topics_enabled -> ai_embeddings_semantic_related_topics_enabled

Plugins is still in an experimental phase and not much is overidden hence
avoiding adding site setting migrations.


Co-authored-by: Krzysztof Kotlarek <kotlarek.krzysztof@gmail.com>
2023-03-31 11:04:34 +11:00
Sam 1d097b9d82
FEATURE: attempt to include related topics above suggested (#28)
Allows related topics to show up for logged on users

- Introduces a new "Related Topics" block above suggested when related topics exist
- Renames `ai_embeddings_semantic_suggested_topics_anons_enabled` -> `ai_embeddings_semantic_suggested_topics_enabled` (given it is only deployed on 1 site not bothering with a migration)
- Adds an integration test to ensure data arrives correctly on the client
2023-03-31 09:07:22 +11:00
Rafael dos Santos Silva b942a18298
FEATURE: Support for GPT-4 in AI Helper module (#29) 2023-03-28 23:22:34 -03:00
Rafael dos Santos Silva 45950f1bb4
FIX: Only show public visible topics as suggested for anons (#27)
* FIX: Only show public visible topics as suggested for anons

* DEV: Add tests for embeddings

* Update spec/lib/modules/embeddings/semantic_suggested_spec.rb

Co-authored-by: Bianca Nenciu <nbianca@users.noreply.github.com>

* Update spec/lib/modules/embeddings/semantic_suggested_spec.rb

Co-authored-by: Bianca Nenciu <nbianca@users.noreply.github.com>

* move to top

---------

Co-authored-by: Bianca Nenciu <nbianca@users.noreply.github.com>
2023-03-23 17:28:01 -03:00
Roman Rizzi 4c960970fa
DEV: Log information about errors from the completions OpenAI API (#26) 2023-03-22 16:00:28 -03:00
Sam 1d14f7ffaf
FEATURE: Add a markdown table AI helper (#25) 2023-03-22 13:16:29 -03:00
Roman Rizzi 1916bafca0
FIX: Remove `null: false` from dropped column (#24)
* FIX: Remove `null: false` from dropped column

It's causing an issue when attempting to seed data in fresh installs.
The column was dropped in a post-migration.

* Update db/migrate/20230322142028_make_dropped_value_column_nullable.rb

Co-authored-by: David Taylor <david@taylorhq.com>

---------

Co-authored-by: David Taylor <david@taylorhq.com>
2023-03-22 11:40:20 -03:00
Rafael dos Santos Silva bd342f538d
FEATURE: Try to generate embeddings for a topic when those aren't found (#23) 2023-03-21 18:20:46 -03:00
Roman Rizzi 55182ca0f5
FIX: Allow null messages to migrate existing rows (#22) 2023-03-21 12:33:30 -03:00
Roman Rizzi 39f7f1f29e
FEATURE: Prompts can consist of multiple messages. (#21)
A prompt with multiple messages leads to better results, as the AI can learn for given examples. Alongside this change, we provide a better default proofreading prompt.
2023-03-21 12:04:59 -03:00
Rafael dos Santos Silva 6bdbc0e32d
FIX: Proper flow when a topic doesn't have embeddings (#20) 2023-03-20 16:44:55 -03:00
Roman Rizzi fea9041ee1
DEV: Use 10s timeout when using the completions API (#19) 2023-03-20 16:43:51 -03:00
Roman Rizzi 54a1694637
FIX: Remove old seeded prompts. (#17)
We switched to negative integers for seeded prompts, but we also
need to delete the old ones using IDs 1/2/3 if they exist,
 or deploys will fail due to the unique index on the name column.
2023-03-20 12:55:38 -03:00
Roman Rizzi 95d7692ae8
FIX: Fix reviewable title translations (#18) 2023-03-20 10:09:20 -03:00
Roman Rizzi 258eb7e78f
DEV: Use negative ids for seeded prompts. (#16)
This makes easier to add new defaults in the feature without clashing with user created ones.
2023-03-17 15:44:45 -03:00
Roman Rizzi 01a08f41cf
FIX: Change gen titles prompt to only return a list (#15) 2023-03-17 15:36:38 -03:00
Roman Rizzi 320ac6e84b
REFACTOR: Store prompts in a dedicated table. (#14)
This change makes it easier to add new prompts to our AI helper. We don't have a UI for it yet. You'll have to do it through a console.
2023-03-17 15:14:19 -03:00
Joffrey JAFFEUX edfdc6dfae
DEV: applies chat namespacing (#12) 2023-03-17 15:15:38 +01:00
Roman Rizzi 75aa595105
FIX: Use cooked suggestion when generating a diff (#13) 2023-03-16 11:09:28 -03:00
Roman Rizzi 5571babb63
DEV: Fix AiHelper flaky system spec (#11) 2023-03-15 18:43:28 -03:00
Rafael dos Santos Silva 80d662e9e8
FEATURE: Semantic Suggested Topics (#10) 2023-03-15 17:21:45 -03:00
Roman Rizzi f99fe7e1ed
FEATURE: Composer AI helper (#8)
* FEATURE: Composer AI helper

This change introduces a new composer button for the group members listed in the `ai_helper_allowed_groups` site setting.

Users can use chatGPT to review, improve, or translate their posts to English.

* Add a safeguard for PMs and don't rely on parentView
2023-03-15 17:02:20 -03:00
Roman Rizzi aa2fca6086
DEV: DiscourseAI -> DiscourseAi rename to have consistent folders and files (#9) 2023-03-14 16:03:50 -03:00
Roman Rizzi cbaa40edc5
FIX: Do not inherit from classes defined by plugins (#6) 2023-03-08 12:39:03 -03:00
Rafael dos Santos Silva 510c6487e3
DEV: Preparation work for multiple inference providers (#5) 2023-03-07 16:14:39 -03:00
Roman Rizzi a838116cd5
FEATURE: Use dedicated reviewables for AI flags. (#4)
This change adds two new reviewable types: ReviewableAIPost and ReviewableAIChatMessage. They have the same actions as their existing counterparts: ReviewableFlaggedPost and ReviewableChatMessage.

We'll display the model used and their accuracy when showing these flags in the review queue and adjust the latter after staff performs an action, tracking a global accuracy per existing model in a separate table.


* FEATURE: Dedicated reviewables for AI flags

* Store and adjust model accuracy

* Display accuracy in reviewable templates
2023-03-07 15:39:28 -03:00
Roman Rizzi 676d3ce6b2
DEV: Rename XClassification --> XClassificator to make it more obvious (#3) 2023-02-28 11:17:03 -03:00
Rafael dos Santos Silva a5f1b003d9
Set default endpoints for all modules so plugins works out of the box (#2) 2023-02-27 16:44:11 -03:00
Roman Rizzi b9a650fde4
DEV: Dedicated table for saving classification results (#1) 2023-02-27 16:21:40 -03:00
Roman Rizzi 5f9597474c
REFACTOR: Streamline flag and classification process 2023-02-24 13:25:02 -03:00
Roman Rizzi 85768cfb1c
FEATURE: Classify posts looking for NSFW images 2023-02-24 09:11:58 -03:00
Roman Rizzi 94933f3c58
DEV: Add missing specs for the toxicity module 2023-02-24 07:53:43 -03:00
Roman Rizzi e8bffcdd64
DEV: Add tests for the sentiment module 2023-02-23 15:50:10 -03:00
Roman Rizzi ef6c785aca
DEV: Move jobs undear each module lib directory 2023-02-23 14:09:52 -03:00
Roman Rizzi 1afa274b99
DEV: Reorganize files and add an entry point for each module 2023-02-23 12:25:00 -03:00
Rafael dos Santos Silva a73931c151
Refactoring of nsfw and flagger 2023-02-23 12:13:26 -03:00