Go to file
Rafael dos Santos Silva 3b8f900486
FIX: Handle unicode on tokenizer (#515)
* FIX: Handle unicode on tokenizer

Our fast track code broke when strings had characters who are longer in tokens than
in UTF-8.

Admins can set `DISCOURSE_AI_STRICT_TOKEN_COUNTING: true` in app.yml to ensure token counting is strict, even if slower.


Co-authored-by: wozulong <sidle.pax_0e@icloud.com>
2024-03-14 17:33:30 -03:00
.github/workflows Initial commit 2023-02-17 11:33:47 -03:00
app UX: add support for dark mode (#529) 2024-03-13 13:16:07 -04:00
assets FIX: don't show share conversation incorrectly (#526) 2024-03-13 11:24:22 +11:00
config FIX: Handle unicode on tokenizer (#515) 2024-03-14 17:33:30 -03:00
db FEATURE: Share conversations with AI via a URL (#521) 2024-03-12 16:51:41 +11:00
discourse_automation DEV: Fix new Rubocop offenses 2024-03-06 15:23:29 +01:00
lib FIX: Handle unicode on tokenizer (#515) 2024-03-14 17:33:30 -03:00
public/ai-share UX: add support for dark mode (#529) 2024-03-13 13:16:07 -04:00
spec FIX: Handle unicode on tokenizer (#515) 2024-03-14 17:33:30 -03:00
test/javascripts FEATURE: mentionable personas and random picker tool, context limits (#466) 2024-02-15 16:37:59 +11:00
tokenizers Mixtral (#376) 2023-12-26 14:49:55 -03:00
.discourse-compatibility DEV: Lower pinned version to fix unsupported API error on stable (#514) 2024-03-07 14:30:05 +00:00
.eslintrc.cjs DEV: Update linting configs (#280) 2023-11-03 11:30:09 +00:00
.gitignore Initial commit 2023-02-17 11:33:47 -03:00
.prettierignore FEATURE: UI to update ai personas on admin page (#290) 2023-11-21 16:56:43 +11:00
.prettierrc.cjs DEV: Update linting configs (#280) 2023-11-03 11:30:09 +00:00
.rubocop.yml DEV: Update linting configs (#280) 2023-11-03 11:30:09 +00:00
.streerc DEV: Update linting configs (#280) 2023-11-03 11:30:09 +00:00
.template-lintrc.cjs DEV: Update linting (#326) 2023-11-29 23:01:48 +01:00
Gemfile DEV: Update linting configs (#280) 2023-11-03 11:30:09 +00:00
Gemfile.lock DEV: Fix new Rubocop offenses 2024-03-06 15:23:29 +01:00
LICENSE Initial commit 2023-02-17 11:33:47 -03:00
README.md CHORE: Update Readme (#185) 2023-09-04 15:46:35 -03:00
package.json DEV: Update linting (#423) 2024-01-13 00:28:06 +01:00
plugin.rb FEATURE: Share conversations with AI via a URL (#521) 2024-03-12 16:51:41 +11:00
translator.yml DEV: Add Crowdin support (#108) 2023-07-15 00:56:15 +02:00
yarn.lock DEV: Update linting (#423) 2024-01-13 00:28:06 +01:00

README.md

Discourse AI Plugin

Plugin Summary

For more information, please see: https://meta.discourse.org/t/discourse-ai/259214?u=falco