Discource-C
/
discourse-ai
mirror of https://github.com/discourse/discourse-ai.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
discourse-ai/discourse_automation/llm_report.rb

107 lines
3.8 KiB
Ruby
Raw Normal View History

FEATURE: LLM based peroidical summary report (#357) Introduce a Discourse Automation based periodical report. Depends on Discourse Automation. Report works best with very large context language models such as GPT-4-Turbo and Claude 2. - Introduces final_insts to generic llm format, for claude to work best it is better to guide the last assistant message (we should add this to other spots as well) - Adds GPT-4 turbo support to generic llm interface
2023-12-18 20:04:15 -05:00
# frozen_string_literal: true
if defined?(DiscourseAutomation)
module DiscourseAutomation::LlmReport
end
DEV: Fix new Rubocop offenses
2024-03-05 10:48:28 -05:00
DiscourseAutomation::Scriptable.add("llm_report") do
FEATURE: LLM based peroidical summary report (#357) Introduce a Discourse Automation based periodical report. Depends on Discourse Automation. Report works best with very large context language models such as GPT-4-Turbo and Claude 2. - Introduces final_insts to generic llm format, for claude to work best it is better to guide the last assistant message (we should add this to other spots as well) - Adds GPT-4 turbo support to generic llm interface
2023-12-18 20:04:15 -05:00
version 1
triggerables %i[recurring]
field :sender, component: :user, required: true
FEATURE: allow sending AI based report to a topic (#377) This makes the reporting far more flexible cause it can target a far wider audience by pointing it at a topic in a secure category or an existing PM
2023-12-21 19:46:23 -05:00
field :receivers, component: :users
field :topic_id, component: :text
field :title, component: :text
FEATURE: LLM based peroidical summary report (#357) Introduce a Discourse Automation based periodical report. Depends on Discourse Automation. Report works best with very large context language models such as GPT-4-Turbo and Claude 2. - Introduces final_insts to generic llm format, for claude to work best it is better to guide the last assistant message (we should add this to other spots as well) - Adds GPT-4 turbo support to generic llm interface
2023-12-18 20:04:15 -05:00
field :days, component: :text, required: true, default_value: 7
field :offset, component: :text, required: true, default_value: 0
field :instructions,
component: :message,
required: true,
default_value: DiscourseAi::Automation::ReportRunner.default_instructions
field :sample_size, component: :text, required: true, default_value: 100
field :tokens_per_post, component: :text, required: true, default_value: 150
field :model,
component: :choices,
required: true,
extra: {
DEV: Use LlmModels as options in automation rules (#676)
2024-06-20 18:07:17 -04:00
content: DiscourseAi::Automation.available_models,
FEATURE: LLM based peroidical summary report (#357) Introduce a Discourse Automation based periodical report. Depends on Discourse Automation. Report works best with very large context language models such as GPT-4-Turbo and Claude 2. - Introduces final_insts to generic llm format, for claude to work best it is better to guide the last assistant message (we should add this to other spots as well) - Adds GPT-4 turbo support to generic llm interface
2023-12-18 20:04:15 -05:00
}
field :priority_group, component: :group
field :categories, component: :categories
field :tags, component: :tags
FEATURE: allow excluding tags and categories from LLM report (#447) Also - Better diagnostics, output model being used - Prompt LLM that true content is being injected in <context> tag
2024-01-29 23:55:05 -05:00
field :exclude_categories, component: :categories
field :exclude_tags, component: :tags
FEATURE: LLM based peroidical summary report (#357) Introduce a Discourse Automation based periodical report. Depends on Discourse Automation. Report works best with very large context language models such as GPT-4-Turbo and Claude 2. - Introduces final_insts to generic llm format, for claude to work best it is better to guide the last assistant message (we should add this to other spots as well) - Adds GPT-4 turbo support to generic llm interface
2023-12-18 20:04:15 -05:00
field :allow_secure_categories, component: :boolean
FEATURE: fine tune llm report to follow instructions more closely (#451) - Allow users to supply top_p and temperature values, which means people can fine tune randomness - Fix bad localization string - Fix bad remapping of max tokens in gemini - Add support for top_p as a general param to llms - Amend system prompt so persona stops treating a user as an adversary
2024-01-30 17:58:25 -05:00
field :top_p, component: :text, required: true, default_value: 0.1
field :temperature, component: :text, required: true, default_value: 0.2
FEATURE: allow suppression of notifications from report generation (#533) * FEATURE: allow suppression of notifications from report generation Previously we needed to do this by hand, unfortunately this uses up too many tokens and is very hard to discover. New option means that we can trivially disable notifications without needing any prompt engineering. * URI.parse is safer, use it
2024-03-15 17:05:03 -04:00
field :suppress_notifications, component: :boolean
FEATURE: LLM based peroidical summary report (#357) Introduce a Discourse Automation based periodical report. Depends on Discourse Automation. Report works best with very large context language models such as GPT-4-Turbo and Claude 2. - Introduces final_insts to generic llm format, for claude to work best it is better to guide the last assistant message (we should add this to other spots as well) - Adds GPT-4 turbo support to generic llm interface
2023-12-18 20:04:15 -05:00
field :debug_mode, component: :boolean
script do |context, fields, automation|
begin
sender = fields.dig("sender", "value")
receivers = fields.dig("receivers", "value")
FEATURE: allow sending AI based report to a topic (#377) This makes the reporting far more flexible cause it can target a far wider audience by pointing it at a topic in a secure category or an existing PM
2023-12-21 19:46:23 -05:00
topic_id = fields.dig("topic_id", "value")
FEATURE: LLM based peroidical summary report (#357) Introduce a Discourse Automation based periodical report. Depends on Discourse Automation. Report works best with very large context language models such as GPT-4-Turbo and Claude 2. - Introduces final_insts to generic llm format, for claude to work best it is better to guide the last assistant message (we should add this to other spots as well) - Adds GPT-4 turbo support to generic llm interface
2023-12-18 20:04:15 -05:00
title = fields.dig("title", "value")
model = fields.dig("model", "value")
category_ids = fields.dig("categories", "value")
tags = fields.dig("tags", "value")
allow_secure_categories = !!fields.dig("allow_secure_categories", "value")
debug_mode = !!fields.dig("debug_mode", "value")
sample_size = fields.dig("sample_size", "value")
instructions = fields.dig("instructions", "value")
days = fields.dig("days", "value")
offset = fields.dig("offset", "value").to_i
priority_group = fields.dig("priority_group", "value")
tokens_per_post = fields.dig("tokens_per_post", "value")
FEATURE: allow excluding tags and categories from LLM report (#447) Also - Better diagnostics, output model being used - Prompt LLM that true content is being injected in <context> tag
2024-01-29 23:55:05 -05:00
exclude_category_ids = fields.dig("exclude_categories", "value")
exclude_tags = fields.dig("exclude_tags", "value")
FEATURE: fine tune llm report to follow instructions more closely (#451) - Allow users to supply top_p and temperature values, which means people can fine tune randomness - Fix bad localization string - Fix bad remapping of max tokens in gemini - Add support for top_p as a general param to llms - Amend system prompt so persona stops treating a user as an adversary
2024-01-30 17:58:25 -05:00
# set defaults in code to support easy migration for old rules
top_p = 0.1
top_p = fields.dig("top_p", "value").to_f if fields.dig("top_p", "value")
temperature = 0.2
temperature = fields.dig("temperature", "value").to_f if fields.dig("temperature", "value")
FEATURE: allow suppression of notifications from report generation (#533) * FEATURE: allow suppression of notifications from report generation Previously we needed to do this by hand, unfortunately this uses up too many tokens and is very hard to discover. New option means that we can trivially disable notifications without needing any prompt engineering. * URI.parse is safer, use it
2024-03-15 17:05:03 -04:00
suppress_notifications = !!fields.dig("suppress_notifications", "value")
FEATURE: LLM based peroidical summary report (#357) Introduce a Discourse Automation based periodical report. Depends on Discourse Automation. Report works best with very large context language models such as GPT-4-Turbo and Claude 2. - Introduces final_insts to generic llm format, for claude to work best it is better to guide the last assistant message (we should add this to other spots as well) - Adds GPT-4 turbo support to generic llm interface
2023-12-18 20:04:15 -05:00
DiscourseAi::Automation::ReportRunner.run!(
sender_username: sender,
receivers: receivers,
FEATURE: allow sending AI based report to a topic (#377) This makes the reporting far more flexible cause it can target a far wider audience by pointing it at a topic in a secure category or an existing PM
2023-12-21 19:46:23 -05:00
topic_id: topic_id,
FEATURE: LLM based peroidical summary report (#357) Introduce a Discourse Automation based periodical report. Depends on Discourse Automation. Report works best with very large context language models such as GPT-4-Turbo and Claude 2. - Introduces final_insts to generic llm format, for claude to work best it is better to guide the last assistant message (we should add this to other spots as well) - Adds GPT-4 turbo support to generic llm interface
2023-12-18 20:04:15 -05:00
title: title,
model: model,
category_ids: category_ids,
tags: tags,
allow_secure_categories: allow_secure_categories,
debug_mode: debug_mode,
sample_size: sample_size,
instructions: instructions,
days: days,
offset: offset,
priority_group_id: priority_group,
tokens_per_post: tokens_per_post,
FEATURE: allow excluding tags and categories from LLM report (#447) Also - Better diagnostics, output model being used - Prompt LLM that true content is being injected in <context> tag
2024-01-29 23:55:05 -05:00
exclude_category_ids: exclude_category_ids,
exclude_tags: exclude_tags,
FEATURE: fine tune llm report to follow instructions more closely (#451) - Allow users to supply top_p and temperature values, which means people can fine tune randomness - Fix bad localization string - Fix bad remapping of max tokens in gemini - Add support for top_p as a general param to llms - Amend system prompt so persona stops treating a user as an adversary
2024-01-30 17:58:25 -05:00
temperature: temperature,
top_p: top_p,
FEATURE: allow suppression of notifications from report generation (#533) * FEATURE: allow suppression of notifications from report generation Previously we needed to do this by hand, unfortunately this uses up too many tokens and is very hard to discover. New option means that we can trivially disable notifications without needing any prompt engineering. * URI.parse is safer, use it
2024-03-15 17:05:03 -04:00
suppress_notifications: suppress_notifications,
FEATURE: LLM based peroidical summary report (#357) Introduce a Discourse Automation based periodical report. Depends on Discourse Automation. Report works best with very large context language models such as GPT-4-Turbo and Claude 2. - Introduces final_insts to generic llm format, for claude to work best it is better to guide the last assistant message (we should add this to other spots as well) - Adds GPT-4 turbo support to generic llm interface
2023-12-18 20:04:15 -05:00
)
rescue => e
Discourse.warn_exception e, message: "Error running LLM report!"
if Rails.env.development?
p e
puts e.backtrace
end
end
end
end
end