mirror of
https://github.com/discourse/discourse-ai.git
synced 2025-06-26 01:22:20 +00:00
* FIX: Handle unicode on tokenizer Our fast track code broke when strings had characters who are longer in tokens than in UTF-8. Admins can set `DISCOURSE_AI_STRICT_TOKEN_COUNTING: true` in app.yml to ensure token counting is strict, even if slower. Co-authored-by: wozulong <sidle.pax_0e@icloud.com>
Discourse AI Plugin
Plugin Summary
For more information, please see: https://meta.discourse.org/t/discourse-ai/259214?u=falco
Languages
Ruby
81.3%
JavaScript
15.5%
SCSS
2.2%
CSS
0.4%
HTML
0.4%
Other
0.2%