mirror of
https://github.com/discourse/discourse-ai.git
synced 2025-08-04 04:03:29 +00:00
Previous to this changeset we used a custom system for tools/command support for Anthropic. We defined commands by using !command as a signal to execute it Following Anthropic Claude 2.1, there is an official supported syntax (beta) for tools execution. eg: ``` + <function_calls> + <invoke> + <tool_name>image</tool_name> + <parameters> + <prompts> + [ + "an oil painting", + "a cute fluffy orange", + "3 apple's", + "a cat" + ] + </prompts> + </parameters> + </invoke> + </function_calls> ``` This implements the spec per Anthropic, it should be stable enough to also work on other LLMs. Keep in mind that OpenAI is not impacted here at all, as it has its own custom system for function calls. Additionally: - Fixes the title system prompt so it works with latest Anthropic - Uses new spec for "system" messages by Anthropic - Tweak forum helper persona to guide Anthropic a tiny be better Overall results are pretty awesome and Anthropic Claude performs really well now on Discourse
75 lines
1.9 KiB
Ruby
75 lines
1.9 KiB
Ruby
# frozen_string_literal: true
|
|
|
|
module DiscourseAi
|
|
module AiBot
|
|
class AnthropicBot < Bot
|
|
def self.can_reply_as?(bot_user)
|
|
bot_user.id == DiscourseAi::AiBot::EntryPoint::CLAUDE_V2_ID
|
|
end
|
|
|
|
def bot_prompt_with_topic_context(post, allow_commands:)
|
|
super(post, allow_commands: allow_commands).join("\n\n") + "\n\nAssistant:"
|
|
end
|
|
|
|
def prompt_limit(allow_commands: true)
|
|
# no side channel for commands, so we can ignore allow commands
|
|
50_000 # https://console.anthropic.com/docs/prompt-design#what-is-a-prompt
|
|
end
|
|
|
|
def title_prompt(post)
|
|
super(post).join("\n\n") + "\n\nAssistant:"
|
|
end
|
|
|
|
def get_delta(partial, context)
|
|
completion = partial[:completion]
|
|
if completion&.start_with?(" ") && !context[:processed_first]
|
|
completion = completion[1..-1]
|
|
context[:processed_first] = true
|
|
end
|
|
completion
|
|
end
|
|
|
|
def tokenizer
|
|
DiscourseAi::Tokenizer::AnthropicTokenizer
|
|
end
|
|
|
|
private
|
|
|
|
def build_message(poster_username, content, system: false, function: nil)
|
|
role = poster_username == bot_user.username ? "Assistant" : "Human"
|
|
|
|
if system || function
|
|
content
|
|
else
|
|
"#{role}: #{content}"
|
|
end
|
|
end
|
|
|
|
def model_for
|
|
"claude-2"
|
|
end
|
|
|
|
def get_updated_title(prompt)
|
|
DiscourseAi::Inference::AnthropicCompletions.perform!(
|
|
prompt,
|
|
model_for,
|
|
temperature: 0.4,
|
|
max_tokens: 40,
|
|
).dig(:completion)
|
|
end
|
|
|
|
def submit_prompt(prompt, post: nil, prefer_low_cost: false, &blk)
|
|
DiscourseAi::Inference::AnthropicCompletions.perform!(
|
|
prompt,
|
|
model_for,
|
|
temperature: 0.4,
|
|
max_tokens: 3000,
|
|
post: post,
|
|
stop_sequences: ["\n\nHuman:", "</function_calls>"],
|
|
&blk
|
|
)
|
|
end
|
|
end
|
|
end
|
|
end
|