mirror of
https://github.com/discourse/discourse-ai.git
synced 2025-07-09 07:33:30 +00:00
17f04c76d8
This commit enhances the AI image generation functionality by adding support for: 1. OpenAI's GPT-based image generation model (gpt-image-1) 2. Image editing capabilities through the OpenAI API 3. A new "Designer" persona specialized in image generation and editing 4. Two new AI tools: CreateImage and EditImage Technical changes include: - Renaming `ai_openai_dall_e_3_url` to `ai_openai_image_generation_url` with a migration - Adding `ai_openai_image_edit_url` setting for the image edit API endpoint - Refactoring image generation code to handle both DALL-E and the newer GPT models - Supporting multipart/form-data for image editing requests * wild guess but maybe quantization is breaking the test sometimes this increases distance * Update lib/personas/designer.rb Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com> * simplify and de-flake code * fix, in chat we need enough context so we know exactly what uploads a user uploaded. * Update lib/personas/tools/edit_image.rb Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com> * cleanup downloaded files right away * fix implementation --------- Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
98 lines
2.3 KiB
Ruby
98 lines
2.3 KiB
Ruby
# frozen_string_literal: true
|
|
|
|
module DiscourseAi
|
|
module Personas
|
|
module Tools
|
|
class DallE < Tool
|
|
def self.signature
|
|
{
|
|
name: name,
|
|
description: "Renders images from supplied descriptions",
|
|
parameters: [
|
|
{
|
|
name: "prompts",
|
|
description:
|
|
"The prompts used to generate or create or draw the image (5000 chars or less, be creative) up to 4 prompts",
|
|
type: "array",
|
|
item_type: "string",
|
|
required: true,
|
|
},
|
|
{
|
|
name: "aspect_ratio",
|
|
description: "The aspect ratio (optional, square by default)",
|
|
type: "string",
|
|
required: false,
|
|
enum: %w[tall square wide],
|
|
},
|
|
],
|
|
}
|
|
end
|
|
|
|
def self.name
|
|
"dall_e"
|
|
end
|
|
|
|
def prompts
|
|
parameters[:prompts]
|
|
end
|
|
|
|
def aspect_ratio
|
|
parameters[:aspect_ratio]
|
|
end
|
|
|
|
def chain_next_response?
|
|
false
|
|
end
|
|
|
|
def invoke
|
|
# max 4 prompts
|
|
max_prompts = prompts.take(4)
|
|
progress = prompts.first
|
|
|
|
yield(progress)
|
|
|
|
results = nil
|
|
|
|
size = "1024x1024"
|
|
if aspect_ratio == "tall"
|
|
size = "1024x1792"
|
|
elsif aspect_ratio == "wide"
|
|
size = "1792x1024"
|
|
end
|
|
|
|
results =
|
|
DiscourseAi::Inference::OpenAiImageGenerator.create_uploads!(
|
|
max_prompts,
|
|
model: "dall-e-3",
|
|
size: size,
|
|
user_id: bot_user.id,
|
|
)
|
|
|
|
if results.blank?
|
|
return { prompts: max_prompts, error: "Something went wrong, could not generate image" }
|
|
end
|
|
|
|
self.custom_raw = <<~RAW
|
|
|
|
[grid]
|
|
#{
|
|
results
|
|
.map { |item| "![#{item[:prompt].gsub(/\|\'\"/, "")}](#{item[:upload].short_url})" }
|
|
.join(" ")
|
|
}
|
|
[/grid]
|
|
RAW
|
|
|
|
{ prompts: results.map { |item| item[:prompt] } }
|
|
end
|
|
|
|
protected
|
|
|
|
def description_args
|
|
{ prompt: prompts.first }
|
|
end
|
|
end
|
|
end
|
|
end
|
|
end
|