Commit Graph

70 Commits

Author SHA1 Message Date
Sam Saffron 6e6ced4554
Work in progress... post embeddings and version
Feel free to continue any work here, will pick it up again next week
2023-05-12 15:28:12 +10:00
Sam 93d9d9ea91
FEATURE: hide various api keys in site settings (#59)
Also... ensures that plugin has a link to more info and proper description
2023-05-12 12:54:25 +10:00
Rafael dos Santos Silva 97124b30de
FEATURE: Update summarization token count and add Claude 100k (#58) 2023-05-11 15:35:58 -03:00
Rafael dos Santos Silva 66bf4c74c6
FEATURE: Handle invalid media in NSFW module (#57)
* FEATURE: Handle invalid media in NSFW module

* fix lint
2023-05-11 15:35:39 -03:00
Roman Rizzi 7e3cb0ea16
FEATURE: Multi-model support for the AI Bot module. (#56)
We'll create one bot user for each available model. When listed in the `ai_bot_enabled_chat_bots` setting, they will reply.

This PR lets us use Claude-v1 in stream mode.
2023-05-11 10:03:03 -03:00
Rafael dos Santos Silva e5537d4c77
FEATURE: Allow excluding closed topics from semantic related (#55) 2023-05-09 15:30:50 -03:00
Rafael dos Santos Silva f1133f66a6
Updates to embedding rake tasks (#54)
- Creates embeddings in topic ID order, so it's easier to stop and
restart from where we stopped

- Update index parameters with current best practices
2023-05-09 13:45:16 -03:00
Sam e76fc77189
fixes (#53)
* Minor... use username suggester in case username already exists

* FIX: ensure we truncate long prompts

Previously we

1. Used raw length instead of token counts for counting length
2. We totally dropped a prompt if it was too long

New implementation will truncate "raw" if it gets too long maintaining
meaning.
2023-05-06 07:31:53 -03:00
Roman Rizzi 71b105a1bb
FEATURE: Introduce the ai-bot module (#52)
This module lets you chat with our GPT bot inside a PM. The bot only replies to members of the groups listed on the ai_bot_allowed_groups setting and only if you invite it to participate in the PM.
2023-05-05 15:28:31 -03:00
Rafael dos Santos Silva 29b22243de
FIX: Toxicity automatic flagging should be disabled by default (#51) 2023-05-03 16:44:33 -03:00
Rafael dos Santos Silva c96edc8a72
FIX: Pass correct API Key to summarization service (#50) 2023-05-02 21:41:11 -03:00
Rafael dos Santos Silva 89ac5d720a
FIX: Only send supported image types for classification (#49)
* FIX: Only send supported image types for classification
2023-04-27 17:52:20 -03:00
Sam 2cd60a4b3b
FEATURE: add a table to audit OpenAI usage (#45)
Still need to build a job to purge logs
2023-04-26 11:44:29 +10:00
Jarek Radosz f6c30e8df9
DEV: Fix minor js issues (#48)
1. Add missing `pluginId` to `modifyClass` calls
2. Replace deprecated `site-settings:main` lookups with `service:site-settings`
3. Use "kebab-case" for initializer names
2023-04-25 13:19:57 +02:00
Joffrey JAFFEUX daacd2ffe3
FIX: chat-composer is now using glimmer (#47)
As a result of this change the channel is now accessible through `this.args.channel` and not `this.chatChannel`.

Longer term we want a better solution here, but this should fix the issue for now.
2023-04-25 11:25:07 +02:00
David Taylor a0542d1859
DEV: Resolve add_to_serializer deprecations (#46)
26b7f8a63b
2023-04-24 16:07:17 +01:00
Sam 057fbe1ce6
FEATURE: add internal support for streaming mode (#42)
Also adds some tests around completions and supports additional params
such as top_p, temperature and max_tokens

This also migrates off Faraday to using Net::HTTP directly
2023-04-21 16:54:25 +10:00
Meghna 14b21b4f4d
UX: add a custom sparkles icon for AI action buttons (#44) 2023-04-20 20:41:24 +05:30
Roman Rizzi 38e007a3a5
FEATURE: Topic summarization (#41)
* FEATURE: Topic summarization

Summarize topics using the TopicView's "summary" filter. The UI is similar to what we do for chat, but we don't allow the user to select a timeframe.


Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>
2023-04-19 17:57:31 -03:00
Rafael dos Santos Silva 9783e3b025
FEATURE: Add a basic tokenizer API (#37)
* FEATURE: Add a basic tokenizer API

* Add tests

* lint
2023-04-19 11:55:59 -03:00
Rafael dos Santos Silva 5a78b6a953
DEV: Fix test (#43)
* DEV: Fix test

* lint
2023-04-18 15:43:10 -03:00
Rafael dos Santos Silva 4368ef29d8
FIX: Sometimes Claude sends all titles suggestions in a single ai tag (#40) 2023-04-10 16:02:44 -03:00
Rafael dos Santos Silva bb0b829634
FEATURE: Anthropic Claude for AIHelper and Summarization modules (#39) 2023-04-10 11:04:42 -03:00
Roman Rizzi f2e52f7f24
FIX: Check for chat channel permissions (#38) 2023-04-06 09:07:31 -03:00
Rafael dos Santos Silva 5549e4d5b3
FEATURE: Chat channel summarization. (#32)
* start summary module

* chat channel summarization

* FEATURE: modal for channel summarization

---------

Co-authored-by: Roman Rizzi <rizziromanalejandro@gmail.com>
2023-04-04 11:24:09 -03:00
Roman Rizzi 333cb8f212
FIX: Don't try to use pg headlines for semantic search. (#36)
Depends on discourse/discourse#20939.
2023-04-03 11:48:38 -03:00
Roman Rizzi 7a54455cf6
FIX: Use correct variable and method for embeddings (#35) 2023-03-31 16:15:10 -03:00
Roman Rizzi 4e05763a99
FEATURE: Semantic assymetric full-page search (#34)
Depends on discourse/discourse#20915

Hooks to the full-page-search component using an experimental API and performs an assymetric similarity search using our embeddings database.
2023-03-31 15:29:56 -03:00
Rafael dos Santos Silva 99886fb64d
Update default thresholds for Toxicity (#33) 2023-03-31 13:03:44 -03:00
Sam 6543c50758
FIX: stop returning self as a candidate for related topics (#31) 2023-03-31 11:04:17 +10:00
Sam 0d80d9ec49
FEATURE: allow limiting results in related topics section (#30)
Also:

- Normalizes behavior between logged in and anon,
 we only show related topics in the related topic section

- Renames "suggested" to "related" given this only exists in related section
- Adds a spec section to ensure anon does not regress
- Adds `ai_embeddings_semantic_related_topics` to limit related topics

Renamed settings:

ai_embeddings_semantic_suggested_model -> ai_embeddings_semantic_related_model
ai_embeddings_semantic_suggested_topics_enabled -> ai_embeddings_semantic_related_topics_enabled

Plugins is still in an experimental phase and not much is overidden hence
avoiding adding site setting migrations.


Co-authored-by: Krzysztof Kotlarek <kotlarek.krzysztof@gmail.com>
2023-03-31 11:04:34 +11:00
Sam 1d097b9d82
FEATURE: attempt to include related topics above suggested (#28)
Allows related topics to show up for logged on users

- Introduces a new "Related Topics" block above suggested when related topics exist
- Renames `ai_embeddings_semantic_suggested_topics_anons_enabled` -> `ai_embeddings_semantic_suggested_topics_enabled` (given it is only deployed on 1 site not bothering with a migration)
- Adds an integration test to ensure data arrives correctly on the client
2023-03-31 09:07:22 +11:00
Rafael dos Santos Silva b942a18298
FEATURE: Support for GPT-4 in AI Helper module (#29) 2023-03-28 23:22:34 -03:00
Rafael dos Santos Silva 45950f1bb4
FIX: Only show public visible topics as suggested for anons (#27)
* FIX: Only show public visible topics as suggested for anons

* DEV: Add tests for embeddings

* Update spec/lib/modules/embeddings/semantic_suggested_spec.rb

Co-authored-by: Bianca Nenciu <nbianca@users.noreply.github.com>

* Update spec/lib/modules/embeddings/semantic_suggested_spec.rb

Co-authored-by: Bianca Nenciu <nbianca@users.noreply.github.com>

* move to top

---------

Co-authored-by: Bianca Nenciu <nbianca@users.noreply.github.com>
2023-03-23 17:28:01 -03:00
Roman Rizzi 4c960970fa
DEV: Log information about errors from the completions OpenAI API (#26) 2023-03-22 16:00:28 -03:00
Sam 1d14f7ffaf
FEATURE: Add a markdown table AI helper (#25) 2023-03-22 13:16:29 -03:00
Roman Rizzi 1916bafca0
FIX: Remove `null: false` from dropped column (#24)
* FIX: Remove `null: false` from dropped column

It's causing an issue when attempting to seed data in fresh installs.
The column was dropped in a post-migration.

* Update db/migrate/20230322142028_make_dropped_value_column_nullable.rb

Co-authored-by: David Taylor <david@taylorhq.com>

---------

Co-authored-by: David Taylor <david@taylorhq.com>
2023-03-22 11:40:20 -03:00
Rafael dos Santos Silva bd342f538d
FEATURE: Try to generate embeddings for a topic when those aren't found (#23) 2023-03-21 18:20:46 -03:00
Roman Rizzi 55182ca0f5
FIX: Allow null messages to migrate existing rows (#22) 2023-03-21 12:33:30 -03:00
Roman Rizzi 39f7f1f29e
FEATURE: Prompts can consist of multiple messages. (#21)
A prompt with multiple messages leads to better results, as the AI can learn for given examples. Alongside this change, we provide a better default proofreading prompt.
2023-03-21 12:04:59 -03:00
Rafael dos Santos Silva 6bdbc0e32d
FIX: Proper flow when a topic doesn't have embeddings (#20) 2023-03-20 16:44:55 -03:00
Roman Rizzi fea9041ee1
DEV: Use 10s timeout when using the completions API (#19) 2023-03-20 16:43:51 -03:00
Roman Rizzi 54a1694637
FIX: Remove old seeded prompts. (#17)
We switched to negative integers for seeded prompts, but we also
need to delete the old ones using IDs 1/2/3 if they exist,
 or deploys will fail due to the unique index on the name column.
2023-03-20 12:55:38 -03:00
Roman Rizzi 95d7692ae8
FIX: Fix reviewable title translations (#18) 2023-03-20 10:09:20 -03:00
Roman Rizzi 258eb7e78f
DEV: Use negative ids for seeded prompts. (#16)
This makes easier to add new defaults in the feature without clashing with user created ones.
2023-03-17 15:44:45 -03:00
Roman Rizzi 01a08f41cf
FIX: Change gen titles prompt to only return a list (#15) 2023-03-17 15:36:38 -03:00
Roman Rizzi 320ac6e84b
REFACTOR: Store prompts in a dedicated table. (#14)
This change makes it easier to add new prompts to our AI helper. We don't have a UI for it yet. You'll have to do it through a console.
2023-03-17 15:14:19 -03:00
Joffrey JAFFEUX edfdc6dfae
DEV: applies chat namespacing (#12) 2023-03-17 15:15:38 +01:00
Roman Rizzi 75aa595105
FIX: Use cooked suggestion when generating a diff (#13) 2023-03-16 11:09:28 -03:00
Roman Rizzi 5571babb63
DEV: Fix AiHelper flaky system spec (#11) 2023-03-15 18:43:28 -03:00