mirror of
https://github.com/discourse/discourse-ai.git
synced 2025-07-31 02:13:29 +00:00
* FIX: OpenAI Tokenizer was failing to truncate mid emojis * Update spec/shared/tokenizer.rb Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com> --------- Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>