FEATURE: custom user defined tools (#677)

Introduces custom AI tools functionality. 

1. Why it was added:
   The PR adds the ability to create, manage, and use custom AI tools within the Discourse AI system. This feature allows for more flexibility and extensibility in the AI capabilities of the platform.

2. What it does:
   - Introduces a new `AiTool` model for storing custom AI tools
   - Adds CRUD (Create, Read, Update, Delete) operations for AI tools
   - Implements a tool runner system for executing custom tool scripts
   - Integrates custom tools with existing AI personas
   - Provides a user interface for managing custom tools in the admin panel

3. Possible use cases:
   - Creating custom tools for specific tasks or integrations (stock quotes, currency conversion etc...)
   - Allowing administrators to add new functionalities to AI assistants without modifying core code
   - Implementing domain-specific tools for particular communities or industries

4. Code structure:
   The PR introduces several new files and modifies existing ones:

   a. Models:
      - `app/models/ai_tool.rb`: Defines the AiTool model
      - `app/serializers/ai_custom_tool_serializer.rb`: Serializer for AI tools

   b. Controllers:
      - `app/controllers/discourse_ai/admin/ai_tools_controller.rb`: Handles CRUD operations for AI tools

   c. Views and Components:
      - New Ember.js components for tool management in the admin interface
      - Updates to existing AI persona management components to support custom tools 

   d. Core functionality:
      - `lib/ai_bot/tool_runner.rb`: Implements the custom tool execution system
      - `lib/ai_bot/tools/custom.rb`: Defines the custom tool class

   e. Routes and configurations:
      - Updates to route configurations to include new AI tool management pages

   f. Migrations:
      - `db/migrate/20240618080148_create_ai_tools.rb`: Creates the ai_tools table

   g. Tests:
      - New test files for AI tool functionality and integration

The PR integrates the custom tools system with the existing AI persona framework, allowing personas to use both built-in and custom tools. It also includes safety measures such as timeouts and HTTP request limits to prevent misuse of custom tools.

Overall, this PR significantly enhances the flexibility and extensibility of the Discourse AI system by allowing administrators to create and manage custom AI tools tailored to their specific needs.

Co-authored-by: Martin Brennan <martin@discourse.org>

admin/assets/javascripts/discourse/routes/admin-plugins-show-discourse-ai-tools-new.js

@@ -0,0 +1,16 @@
+import DiscourseRoute from "discourse/routes/discourse";
+
+export default DiscourseRoute.extend({
+  async model() {
+    const record = this.store.createRecord("ai-tool");
+    return record;
+  },
+
+  setupController(controller, model) {
+    this._super(controller, model);
+    const toolsModel = this.modelFor("adminPlugins.show.discourse-ai-tools");
+
+    controller.set("allTools", toolsModel);
+    controller.set("presets", toolsModel.resultSetMeta.presets);
+  },
+});

admin/assets/javascripts/discourse/routes/admin-plugins-show-discourse-ai-tools-show.js

@@ -0,0 +1,17 @@
+import DiscourseRoute from "discourse/routes/discourse";
+
+export default DiscourseRoute.extend({
+  async model(params) {
+    const allTools = this.modelFor("adminPlugins.show.discourse-ai-tools");
+    const id = parseInt(params.id, 10);
+    return allTools.findBy("id", id);
+  },
+
+  setupController(controller, model) {
+    this._super(controller, model);
+    const toolsModel = this.modelFor("adminPlugins.show.discourse-ai-tools");
+
+    controller.set("allTools", toolsModel);
+    controller.set("presets", toolsModel.resultSetMeta.presets);
+  },
+});

admin/assets/javascripts/discourse/routes/admin-plugins-show-discourse-ai-tools.js

@@ -0,0 +1,7 @@
+import DiscourseRoute from "discourse/routes/discourse";
+
+export default class DiscourseAiToolsRoute extends DiscourseRoute {
+  model() {
+    return this.store.findAll("ai-tool");
+  }
+}

admin/assets/javascripts/discourse/templates/admin-plugins/show/discourse-ai-tools/index.hbs

@@ -0,0 +1 @@
+<AiToolListEditor @tools={{this.model}} />

admin/assets/javascripts/discourse/templates/admin-plugins/show/discourse-ai-tools/new.hbs

@@ -0,0 +1,5 @@
+<AiToolEditor
+  @tools={{this.allTools}}
+  @model={{this.model}}
+  @presets={{this.presets}}
+/>

admin/assets/javascripts/discourse/templates/admin-plugins/show/discourse-ai-tools/show.hbs

@@ -0,0 +1,5 @@
+<AiToolEditor
+  @tools={{this.allTools}}
+  @model={{this.model}}
+  @presets={{this.presets}}
+/>

app/controllers/discourse_ai/admin/ai_personas_controller.rb

@@ -19,6 +19,14 @@ module DiscourseAi
           DiscourseAi::AiBot::Personas::Persona.all_available_tools.map do |tool|
             AiToolSerializer.new(tool, root: false)
           end
+        AiTool
+          .where(enabled: true)
+          .each do |tool|
+            tools << {
+              id: "custom-#{tool.id}",
+              name: I18n.t("discourse_ai.tools.custom_name", name: tool.name.capitalize),
+            }
+          end
         llms =
           DiscourseAi::Configuration::LlmEnumerator.values.map do |hash|
             { id: hash[:value], name: hash[:name] }

app/controllers/discourse_ai/admin/ai_tools_controller.rb

@@ -0,0 +1,89 @@
+# frozen_string_literal: true
+
+module DiscourseAi
+  module Admin
+    class AiToolsController < ::Admin::AdminController
+      requires_plugin ::DiscourseAi::PLUGIN_NAME
+
+      before_action :find_ai_tool, only: %i[show update destroy]
+
+      def index
+        ai_tools = AiTool.all
+        render_serialized({ ai_tools: ai_tools }, AiCustomToolListSerializer, root: false)
+      end
+
+      def show
+        render_serialized(@ai_tool, AiCustomToolSerializer)
+      end
+
+      def create
+        ai_tool = AiTool.new(ai_tool_params)
+        ai_tool.created_by_id = current_user.id
+
+        if ai_tool.save
+          render_serialized(ai_tool, AiCustomToolSerializer, status: :created)
+        else
+          render_json_error ai_tool
+        end
+      end
+
+      def update
+        if @ai_tool.update(ai_tool_params)
+          render_serialized(@ai_tool, AiCustomToolSerializer)
+        else
+          render_json_error @ai_tool
+        end
+      end
+
+      def destroy
+        if @ai_tool.destroy
+          head :no_content
+        else
+          render_json_error @ai_tool
+        end
+      end
+
+      def test
+        if params[:id].present?
+          ai_tool = AiTool.find(params[:id])
+        else
+          ai_tool = AiTool.new(ai_tool_params)
+        end
+
+        parameters = params[:parameters].to_unsafe_h
+
+        # we need an llm so we have a tokenizer
+        # but will do without if none is available
+        llm = LlmModel.first&.to_llm
+        runner = ai_tool.runner(parameters, llm: llm, bot_user: current_user, context: {})
+        result = runner.invoke
+
+        if result.is_a?(Hash) && result[:error]
+          render_json_error result[:error]
+        else
+          render json: { output: result }
+        end
+      rescue ActiveRecord::RecordNotFound => e
+        render_json_error e.message, status: 400
+      rescue => e
+        render_json_error "Error executing the tool: #{e.message}", status: 400
+      end
+
+      private
+
+      def find_ai_tool
+        @ai_tool = AiTool.find(params[:id])
+      end
+
+      def ai_tool_params
+        params.require(:ai_tool).permit(
+          :name,
+          :description,
+          :script,
+          :summary,
+          parameters: %i[name type description],
+        )
+      end
+    end
+  end
+end

app/models/ai_persona.rb

@@ -142,16 +142,25 @@ class AiPersona < ActiveRecord::Base
     options = {}
     tools =
       self.tools.filter_map do |element|
-        inner_name, current_options = element.is_a?(Array) ? element : [element, nil]
-        inner_name = inner_name.gsub("Tool", "")
-        inner_name = "List#{inner_name}" if %w[Categories Tags].include?(inner_name)
+        klass = nil
+
+        if element.is_a?(String) && element.start_with?("custom-")
+          custom_tool_id = element.split("-", 2).last.to_i
+          if AiTool.exists?(id: custom_tool_id, enabled: true)
+            klass = DiscourseAi::AiBot::Tools::Custom.class_instance(custom_tool_id)
+          end
+        else
+          inner_name, current_options = element.is_a?(Array) ? element : [element, nil]
+          inner_name = inner_name.gsub("Tool", "")
+          inner_name = "List#{inner_name}" if %w[Categories Tags].include?(inner_name)
+
+          begin
+            klass = "DiscourseAi::AiBot::Tools::#{inner_name}".constantize
+            options[klass] = current_options if current_options
+          rescue StandardError
+          end
 
-        begin
-          klass = "DiscourseAi::AiBot::Tools::#{inner_name}".constantize
-          options[klass] = current_options if current_options
           klass
-        rescue StandardError
-          nil
         end
       end
 

app/models/ai_tool.rb

@@ -0,0 +1,183 @@
+# frozen_string_literal: true
+
+class AiTool < ActiveRecord::Base
+  validates :name, presence: true, length: { maximum: 255 }
+  validates :description, presence: true, length: { maximum: 1000 }
+  validates :script, presence: true, length: { maximum: 100_000 }
+  validates :created_by_id, presence: true
+  belongs_to :created_by, class_name: "User"
+
+  def signature
+    { name: name, description: description, parameters: parameters.map(&:symbolize_keys) }
+  end
+
+  def runner(parameters, llm:, bot_user:, context: {})
+    DiscourseAi::AiBot::ToolRunner.new(
+      parameters: parameters,
+      llm: llm,
+      bot_user: bot_user,
+      context: context,
+      tool: self,
+    )
+  end
+
+  after_commit :bump_persona_cache
+
+  def bump_persona_cache
+    AiPersona.persona_cache.flush!
+  end
+
+  def self.presets
+    [
+      {
+        preset_id: "browse_web_jina",
+        name: "browse_web",
+        description: "Browse the web as a markdown document",
+        parameters: [
+          { name: "url", type: "string", required: true, description: "The URL to browse" },
+        ],
+        script: <<~SCRIPT,
+          let url;
+          function invoke(p) {
+              url = p.url;
+              result = http.get(`https://r.jina.ai/${url}`);
+              // truncates to 15000 tokens
+              return llm.truncate(result.body, 15000);
+          }
+          function details() {
+            return "Read: " + url
+          }
+        SCRIPT
+      },
+      {
+        preset_id: "exchange_rate",
+        name: "exchange_rate",
+        description: "Get current exchange rates for various currencies",
+        parameters: [
+          {
+            name: "base_currency",
+            type: "string",
+            required: true,
+            description: "The base currency code (e.g., USD, EUR)",
+          },
+          {
+            name: "target_currency",
+            type: "string",
+            required: true,
+            description: "The target currency code (e.g., EUR, JPY)",
+          },
+          { name: "amount", type: "number", description: "Amount to convert eg: 123.45" },
+        ],
+        script: <<~SCRIPT,
+        // note: this script uses the open.er-api.com service, it is only updated
+        // once every 24 hours, for more up to date rates see: https://www.exchangerate-api.com
+        function invoke(params) {
+          const url = `https://open.er-api.com/v6/latest/${params.base_currency}`;
+          const result = http.get(url);
+          if (result.status !== 200) {
+            return { error: "Failed to fetch exchange rates" };
+          }
+          const data = JSON.parse(result.body);
+          const rate = data.rates[params.target_currency];
+          if (!rate) {
+            return { error: "Target currency not found" };
+          }
+
+          const rval = {
+            base_currency: params.base_currency,
+            target_currency: params.target_currency,
+            exchange_rate: rate,
+            last_updated: data.time_last_update_utc
+          };
+
+          if (params.amount) {
+            rval.original_amount = params.amount;
+            rval.converted_amount = params.amount * rate;
+          }
+
+          return rval;
+        }
+
+        function details() {
+          return "<a href='https://www.exchangerate-api.com'>Rates By Exchange Rate API</a>";
+        }
+      SCRIPT
+        summary: "Get current exchange rates between two currencies",
+      },
+      {
+        preset_id: "stock_quote",
+        name: "stock_quote",
+        description: "Get real-time stock quote information using AlphaVantage API",
+        parameters: [
+          {
+            name: "symbol",
+            type: "string",
+            required: true,
+            description: "The stock symbol (e.g., AAPL, GOOGL)",
+          },
+        ],
+        script: <<~SCRIPT,
+        function invoke(params) {
+          const apiKey = 'YOUR_ALPHAVANTAGE_API_KEY'; // Replace with your actual API key
+          const url = `https://www.alphavantage.co/query?function=GLOBAL_QUOTE&symbol=${params.symbol}&apikey=${apiKey}`;
+
+          const result = http.get(url);
+          if (result.status !== 200) {
+            return { error: "Failed to fetch stock quote" };
+          }
+
+          const data = JSON.parse(result.body);
+          if (data['Error Message']) {
+            return { error: data['Error Message'] };
+          }
+
+          const quote = data['Global Quote'];
+          if (!quote || Object.keys(quote).length === 0) {
+            return { error: "No data found for the given symbol" };
+          }
+
+          return {
+            symbol: quote['01. symbol'],
+            price: parseFloat(quote['05. price']),
+            change: parseFloat(quote['09. change']),
+            change_percent: quote['10. change percent'],
+            volume: parseInt(quote['06. volume']),
+            latest_trading_day: quote['07. latest trading day']
+          };
+        }
+
+        function details() {
+          return "<a href='https://www.alphavantage.co'>Stock data provided by AlphaVantage</a>";
+        }
+      SCRIPT
+        summary: "Get real-time stock quotes using AlphaVantage API",
+      },
+      { preset_id: "empty_tool", script: <<~SCRIPT },
+          function invoke(params) {
+            // logic here
+            return params;
+          }
+          function details() {
+            return "Details about this tool";
+          }
+        SCRIPT
+    ].map do |preset|
+      preset[:preset_name] = I18n.t("discourse_ai.tools.presets.#{preset[:preset_id]}.name")
+      preset
+    end
+  end
+end
+
+# == Schema Information
+#
+# Table name: ai_tools
+#
+#  id            :bigint           not null, primary key
+#  name          :string           not null
+#  description   :text             not null
+#  parameters    :jsonb            not null
+#  script        :text             not null
+#  created_by_id :integer          not null
+#  created_at    :datetime         not null
+#  updated_at    :datetime         not null
+#

app/models/llm_model.rb

@@ -41,6 +41,10 @@ class LlmModel < ActiveRecord::Base
     }
   end
 
+  def to_llm
+    DiscourseAi::Completions::Llm.proxy_from_obj(self)
+  end
+
   def toggle_companion_user
     return if name == "fake" && Rails.env.production?
 

app/serializers/ai_custom_tool_list_serializer.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+class AiCustomToolListSerializer < ApplicationSerializer
+  attributes :meta
+
+  has_many :ai_tools, serializer: AiCustomToolSerializer, embed: :objects
+
+  def meta
+    { presets: AiTool.presets }
+  end
+
+  def ai_tools
+    object[:ai_tools]
+  end
+end

app/serializers/ai_custom_tool_serializer.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+class AiCustomToolSerializer < ApplicationSerializer
+  attributes :id,
+             :name,
+             :description,
+             :summary,
+             :parameters,
+             :script,
+             :created_by_id,
+             :created_at,
+             :updated_at
+
+  self.root = "ai_tool"
+end

assets/javascripts/discourse/admin-discourse-ai-plugin-route-map.js

@@ -13,5 +13,10 @@ export default {
       this.route("new");
       this.route("show", { path: "/:id" });
     });
+
+    this.route("discourse-ai-tools", { path: "ai-tools" }, function () {
+      this.route("new");
+      this.route("show", { path: "/:id" });
+    });
   },
 };

assets/javascripts/discourse/admin/adapters/ai-tool.js

@@ -0,0 +1,21 @@
+import RestAdapter from "discourse/adapters/rest";
+
+export default class Adapter extends RestAdapter {
+  jsonMode = true;
+
+  basePath() {
+    return "/admin/plugins/discourse-ai/";
+  }
+
+  pathFor(store, type, findArgs) {
+    // removes underscores which are implemented in base
+    let path =
+      this.basePath(store, type, findArgs) +
+      store.pluralize(this.apiNameFor(type));
+    return this.appendQueryParams(path, findArgs);
+  }
+
+  apiNameFor() {
+    return "ai-tool";
+  }
+}

assets/javascripts/discourse/admin/models/ai-tool.js

@@ -0,0 +1,27 @@
+import RestModel from "discourse/models/rest";
+
+const CREATE_ATTRIBUTES = [
+  "id",
+  "name",
+  "description",
+  "parameters",
+  "script",
+  "summary",
+  "enabled",
+];
+
+export default class AiTool extends RestModel {
+  createProperties() {
+    return this.getProperties(CREATE_ATTRIBUTES);
+  }
+
+  updateProperties() {
+    return this.getProperties(CREATE_ATTRIBUTES);
+  }
+
+  workingCopy() {
+    let attrs = this.getProperties(CREATE_ATTRIBUTES);
+    attrs.parameters = attrs.parameters || [];
+    return AiTool.create(attrs);
+  }
+}

assets/javascripts/discourse/components/ai-tool-editor.gjs

@@ -0,0 +1,223 @@
+import Component from "@glimmer/component";
+import { tracked } from "@glimmer/tracking";
+import { Input } from "@ember/component";
+import { action } from "@ember/object";
+import didInsert from "@ember/render-modifiers/modifiers/did-insert";
+import didUpdate from "@ember/render-modifiers/modifiers/did-update";
+import { inject as service } from "@ember/service";
+import BackButton from "discourse/components/back-button";
+import DButton from "discourse/components/d-button";
+import Textarea from "discourse/components/d-textarea";
+import DTooltip from "discourse/components/d-tooltip";
+import { popupAjaxError } from "discourse/lib/ajax-error";
+import I18n from "discourse-i18n";
+import AceEditor from "admin/components/ace-editor";
+import ComboBox from "select-kit/components/combo-box";
+import AiToolParameterEditor from "./ai-tool-parameter-editor";
+import AiToolTestModal from "./modal/ai-tool-test-modal";
+
+export default class AiToolEditor extends Component {
+  @service router;
+  @service store;
+  @service dialog;
+  @service modal;
+  @service toasts;
+
+  @tracked isSaving = false;
+  @tracked editingModel = null;
+  @tracked showDelete = false;
+
+  @tracked selectedPreset = null;
+
+  aceEditorMode = "javascript";
+  aceEditorTheme = "chrome";
+
+  @action
+  updateModel() {
+    this.editingModel = this.args.model.workingCopy();
+    this.showDelete = !this.args.model.isNew;
+  }
+
+  get presets() {
+    return this.args.presets.map((preset) => {
+      return {
+        name: preset.preset_name,
+        id: preset.preset_id,
+      };
+    });
+  }
+
+  get showPresets() {
+    return !this.selectedPreset && this.args.model.isNew;
+  }
+
+  @action
+  configurePreset() {
+    this.selectedPreset = this.args.presets.findBy("preset_id", this.presetId);
+    this.editingModel = this.args.model.workingCopy();
+    this.editingModel.setProperties(this.selectedPreset);
+    this.showDelete = false;
+  }
+
+  @action
+  async save() {
+    this.isSaving = true;
+
+    try {
+      await this.args.model.save(
+        this.editingModel.getProperties(
+          "name",
+          "description",
+          "parameters",
+          "script",
+          "summary"
+        )
+      );
+
+      this.toasts.success({
+        data: { message: I18n.t("discourse_ai.tools.saved") },
+        duration: 2000,
+      });
+      if (!this.args.tools.any((tool) => tool.id === this.args.model.id)) {
+        this.args.tools.pushObject(this.args.model);
+      }
+
+      this.router.transitionTo(
+        "adminPlugins.show.discourse-ai-tools.show",
+        this.args.model
+      );
+    } catch (e) {
+      popupAjaxError(e);
+    } finally {
+      this.isSaving = false;
+    }
+  }
+
+  @action
+  delete() {
+    return this.dialog.confirm({
+      message: I18n.t("discourse_ai.tools.confirm_delete"),
+      didConfirm: () => {
+        return this.args.model.destroyRecord().then(() => {
+          this.args.tools.removeObject(this.args.model);
+          this.router.transitionTo(
+            "adminPlugins.show.discourse-ai-tools.index"
+          );
+        });
+      },
+    });
+  }
+
+  @action
+  updateScript(script) {
+    this.editingModel.script = script;
+  }
+
+  @action
+  openTestModal() {
+    this.modal.show(AiToolTestModal, {
+      model: {
+        tool: this.editingModel,
+      },
+    });
+  }
+
+  <template>
+    <BackButton
+      @route="adminPlugins.show.discourse-ai-tools"
+      @label="discourse_ai.ai_tool.back"
+    />
+
+    <form
+      class="form-horizontal ai-tool-editor"
+      {{didUpdate this.updateModel @model.id}}
+      {{didInsert this.updateModel @model.id}}
+    >
+      {{#if this.showPresets}}
+        <div class="control-group">
+          <label>{{I18n.t "discourse_ai.tools.presets"}}</label>
+          <ComboBox
+            @value={{this.presetId}}
+            @content={{this.presets}}
+            class="ai-tool-editor__presets"
+          />
+        </div>
+
+        <div class="control-group ai-llm-editor__action_panel">
+          <DButton
+            class="ai-tool-editor__next"
+            @action={{this.configurePreset}}
+          >
+            {{I18n.t "discourse_ai.tools.next.title"}}
+          </DButton>
+        </div>
+      {{else}}
+        <div class="control-group">
+          <label>{{I18n.t "discourse_ai.tools.name"}}</label>
+          <Input
+            @type="text"
+            @value={{this.editingModel.name}}
+            class="ai-tool-editor__name"
+          />
+          <DTooltip
+            @icon="question-circle"
+            @content={{I18n.t "discourse_ai.tools.name_help"}}
+          />
+        </div>
+        <div class="control-group">
+          <label>{{I18n.t "discourse_ai.tools.description"}}</label>
+          <Textarea
+            @value={{this.editingModel.description}}
+            class="ai-tool-editor__description input-xxlarge"
+            placeholder={{I18n.t "discourse_ai.tools.description_help"}}
+          />
+        </div>
+        <div class="control-group">
+          <label>{{I18n.t "discourse_ai.tools.summary"}}</label>
+          <Input
+            @type="text"
+            @value={{this.editingModel.summary}}
+            class="ai-tool-editor__summary input-xxlarge"
+          />
+          <DTooltip
+            @icon="question-circle"
+            @content={{I18n.t "discourse_ai.tools.summary_help"}}
+          />
+        </div>
+        <div class="control-group">
+          <label>{{I18n.t "discourse_ai.tools.parameters"}}</label>
+          <AiToolParameterEditor @parameters={{this.editingModel.parameters}} />
+        </div>
+        <div class="control-group">
+          <label>{{I18n.t "discourse_ai.tools.script"}}</label>
+          <AceEditor
+            @content={{this.editingModel.script}}
+            @mode={{this.aceEditorMode}}
+            @theme={{this.aceEditorTheme}}
+            @onChange={{this.updateScript}}
+            @editorId="ai-tool-script-editor"
+          />
+        </div>
+        <div class="control-group ai-tool-editor__action_panel">
+          <DButton
+            @action={{this.openTestModal}}
+            class="btn-default ai-tool-editor__test-button"
+          >{{I18n.t "discourse_ai.tools.test"}}</DButton>
+          <DButton
+            class="btn-primary ai-tool-editor__save"
+            @action={{this.save}}
+            @disabled={{this.isSaving}}
+          >{{I18n.t "discourse_ai.tools.save"}}</DButton>
+          {{#if this.showDelete}}
+            <DButton
+              @action={{this.delete}}
+              class="btn-danger ai-tool-editor__delete"
+            >
+              {{I18n.t "discourse_ai.tools.delete"}}
+            </DButton>
+          {{/if}}
+        </div>
+      {{/if}}
+    </form>
+  </template>
+}

assets/javascripts/discourse/components/ai-tool-list-editor.gjs

@@ -0,0 +1,47 @@
+import { LinkTo } from "@ember/routing";
+import icon from "discourse-common/helpers/d-icon";
+import i18n from "discourse-common/helpers/i18n";
+import I18n from "discourse-i18n";
+
+<template>
+  <section class="ai-tool-list-editor__current admin-detail pull-left">
+    <div class="ai-tool-list-editor__header">
+      <h3>{{i18n "discourse_ai.tools.short_title"}}</h3>
+      <LinkTo
+        @route="adminPlugins.show.discourse-ai-tools.new"
+        class="btn btn-small btn-primary ai-tool-list-editor__new-button"
+      >
+        {{icon "plus"}}
+        <span>{{I18n.t "discourse_ai.tools.new"}}</span>
+      </LinkTo>
+    </div>
+
+    <table class="content-list ai-tool-list-editor">
+      <tbody>
+        {{#each @tools as |tool|}}
+          <tr data-tool-id={{tool.id}} class="ai-tool-list__row">
+            <td>
+              <div class="ai-tool-list__name-with-description">
+                <div class="ai-tool-list__name">
+                  <strong>
+                    {{tool.name}}
+                  </strong>
+                </div>
+                <div class="ai-tool-list__description">
+                  {{tool.description}}
+                </div>
+              </div>
+            </td>
+            <td>
+              <LinkTo
+                @route="adminPlugins.show.discourse-ai-tools.show"
+                @model={{tool}}
+                class="btn btn-text btn-small"
+              >{{i18n "discourse_ai.tools.edit"}} </LinkTo>
+            </td>
+          </tr>
+        {{/each}}
+      </tbody>
+    </table>
+  </section>
+</template>

assets/javascripts/discourse/components/ai-tool-parameter-editor.gjs

@@ -0,0 +1,160 @@
+import Component from "@glimmer/component";
+import { tracked } from "@glimmer/tracking";
+import { Input } from "@ember/component";
+import { fn } from "@ember/helper";
+import { on } from "@ember/modifier";
+import { action } from "@ember/object";
+import DButton from "discourse/components/d-button";
+import I18n from "discourse-i18n";
+import ComboBox from "select-kit/components/combo-box";
+
+const PARAMETER_TYPES = [
+  { name: "string", id: "string" },
+  { name: "number", id: "number" },
+  { name: "boolean", id: "boolean" },
+  { name: "array", id: "array" },
+];
+
+export default class AiToolParameterEditor extends Component {
+  @tracked parameters = [];
+
+  @action
+  addParameter() {
+    this.args.parameters.pushObject({
+      name: "",
+      description: "",
+      type: "string",
+      required: false,
+      enum: false,
+      enumValues: [],
+    });
+  }
+
+  @action
+  removeParameter(parameter) {
+    this.args.parameters.removeObject(parameter);
+  }
+
+  @action
+  updateParameter(parameter, field, value) {
+    parameter[field] = value;
+  }
+
+  @action
+  toggleEnum(parameter) {
+    parameter.enum = !parameter.enum;
+    if (!parameter.enum) {
+      parameter.enumValues = [];
+    }
+    this.args.onChange(this.parameters);
+  }
+
+  @action
+  addEnumValue(parameter) {
+    parameter.enumValues.pushObject("");
+  }
+
+  @action
+  removeEnumValue(parameter, index) {
+    parameter.enumValues.removeAt(index);
+  }
+
+  <template>
+    {{#each @parameters as |parameter|}}
+      <div class="ai-tool-parameter">
+        <div class="parameter-row">
+          <Input
+            @type="text"
+            @value={{parameter.name}}
+            placeholder={{I18n.t "discourse_ai.tools.parameter_name"}}
+          />
+          <ComboBox @value={{parameter.type}} @content={{PARAMETER_TYPES}} />
+        </div>
+        <div class="parameter-row">
+          <Input
+            @type="text"
+            @value={{parameter.description}}
+            placeholder={{I18n.t "discourse_ai.tools.parameter_description"}}
+            {{on
+              "input"
+              (fn
+                this.updateParameter
+                parameter
+                "description"
+                value="target.value"
+              )
+            }}
+          />
+        </div>
+        <div class="parameter-row">
+          <label>
+            <Input
+              @type="checkbox"
+              @checked={{parameter.required}}
+              {{on
+                "change"
+                (fn
+                  this.updateParameter
+                  parameter
+                  "required"
+                  value="target.checked"
+                )
+              }}
+            />
+            {{I18n.t "discourse_ai.tools.parameter_required"}}
+          </label>
+          <label>
+            <Input
+              @type="checkbox"
+              @checked={{parameter.enum}}
+              {{on "change" (fn this.toggleEnum parameter)}}
+            />
+            {{I18n.t "discourse_ai.tools.parameter_enum"}}
+          </label>
+          <DButton
+            @icon="trash-alt"
+            @action={{fn this.removeParameter parameter}}
+            class="btn-danger"
+          />
+        </div>
+        {{#if parameter.enum}}
+          <div class="parameter-enum-values">
+            {{#each parameter.enumValues as |enumValue enumIndex|}}
+              <div class="enum-value-row">
+                <Input
+                  @type="text"
+                  @value={{enumValue}}
+                  placeholder={{I18n.t "discourse_ai.tools.enum_value"}}
+                  {{on
+                    "input"
+                    (fn
+                      this.updateParameter
+                      parameter.enumValues
+                      enumIndex
+                      value="target.value"
+                    )
+                  }}
+                />
+                <DButton
+                  @icon="trash-alt"
+                  @action={{fn this.removeEnumValue parameter enumIndex}}
+                  class="btn-danger"
+                />
+              </div>
+            {{/each}}
+            <DButton
+              @icon="plus"
+              @action={{fn this.addEnumValue parameter}}
+              @label="discourse_ai.tools.add_enum_value"
+            />
+          </div>
+        {{/if}}
+      </div>
+    {{/each}}
+    <DButton
+      @icon="plus"
+      @action={{this.addParameter}}
+      @label="discourse_ai.tools.add_parameter"
+    />
+  </template>
+}

assets/javascripts/discourse/components/modal/ai-tool-test-modal.gjs

@@ -0,0 +1,83 @@
+import Component from "@glimmer/component";
+import { tracked } from "@glimmer/tracking";
+import { fn } from "@ember/helper";
+import { on } from "@ember/modifier";
+import { action } from "@ember/object";
+import DButton from "discourse/components/d-button";
+import DModal from "discourse/components/d-modal";
+import { ajax } from "discourse/lib/ajax";
+import { popupAjaxError } from "discourse/lib/ajax-error";
+import I18n from "discourse-i18n";
+import { jsonToHtml } from "../../lib/utilities";
+
+export default class AiToolTestModal extends Component {
+  @tracked testResult = null;
+  @tracked isLoading = false;
+  @tracked parameterValues = {};
+
+  @action
+  updateParameter(name, event) {
+    this.parameterValues[name] = event.target.value;
+  }
+
+  @action
+  async runTest() {
+    this.isLoading = true;
+    try {
+      const data = JSON.stringify({
+        ai_tool: this.args.model.tool,
+        parameters: this.parameterValues,
+      }); // required given this is a POST
+
+      const response = await ajax(
+        "/admin/plugins/discourse-ai/ai-tools/test.json",
+        {
+          type: "POST",
+          data,
+          contentType: "application/json",
+        }
+      );
+      this.testResult = jsonToHtml(response.output);
+    } catch (error) {
+      popupAjaxError(error);
+    } finally {
+      this.isLoading = false;
+    }
+  }
+
+  <template>
+    <DModal
+      @title={{I18n.t "discourse_ai.tools.test_modal.title"}}
+      @closeModal={{@closeModal}}
+      class="ai-tool-test-modal"
+      @bodyClass="ai-tool-test-modal__body"
+    >
+      <:body>
+        {{#each @model.tool.parameters as |param|}}
+          <div class="control-group">
+            <label>{{param.name}}</label>
+            <input
+              type="text"
+              name={{param.name}}
+              {{on "input" (fn this.updateParameter param.name)}}
+            />
+          </div>
+        {{/each}}
+        {{#if this.testResult}}
+          <div class="ai-tool-test-modal__test-result">
+            <h3>{{I18n.t "discourse_ai.tools.test_modal.result"}}</h3>
+            <div>{{this.testResult}}</div>
+          </div>
+        {{/if}}
+      </:body>
+      <:footer>
+        <DButton
+          @action={{this.runTest}}
+          @label="discourse_ai.tools.test_modal.run"
+          @disabled={{this.isLoading}}
+          class="btn-primary ai-tool-test-modal__run-button"
+        />
+      </:footer>
+    </DModal>
+  </template>
+}

assets/javascripts/discourse/components/modal/debug-ai-modal.gjs

@@ -11,6 +11,7 @@ import { clipboardCopy, escapeExpression } from "discourse/lib/utilities";
 import i18n from "discourse-common/helpers/i18n";
 import discourseLater from "discourse-common/lib/later";
 import I18n from "discourse-i18n";
+import { jsonToHtml } from "../../lib/utilities";
 
 export default class DebugAiModal extends Component {
   @tracked info = null;
@@ -41,7 +42,7 @@ export default class DebugAiModal extends Component {
       return this.info.raw_request_payload;
     }
 
-    return htmlSafe(this.jsonToHtml(parsed));
+    return jsonToHtml(parsed);
   }
 
   formattedResponse(response) {
@@ -52,35 +53,6 @@ export default class DebugAiModal extends Component {
     return htmlSafe(safe);
   }
 
-  jsonToHtml(json) {
-    let html = "<ul>";
-    for (let key in json) {
-      if (!json.hasOwnProperty(key)) {
-        continue;
-      }
-      html += "<li>";
-      if (typeof json[key] === "object" && Array.isArray(json[key])) {
-        html += `<strong>${escapeExpression(key)}:</strong> ${this.jsonToHtml(
-          json[key]
-        )}`;
-      } else if (typeof json[key] === "object") {
-        html += `<strong>${escapeExpression(
-          key
-        )}:</strong> <ul><li>${this.jsonToHtml(json[key])}</li></ul>`;
-      } else {
-        let value = json[key];
-        if (typeof value === "string") {
-          value = escapeExpression(value);
-          value = value.replace(/\n/g, "<br>");
-        }
-        html += `<strong>${escapeExpression(key)}:</strong> ${value}`;
-      }
-      html += "</li>";
-    }
-    html += "</ul>";
-    return html;
-  }
-
   @action
   copyRequest() {
     this.copy(this.info.raw_request_payload);

assets/javascripts/discourse/lib/utilities.js

@@ -1,2 +1,37 @@
+import { htmlSafe } from "@ember/template";
+import { escapeExpression } from "discourse/lib/utilities";
+
 export const IMAGE_MARKDOWN_REGEX =
   /!\[(.*?)\|(\d{1,4}x\d{1,4})(,\s*\d{1,3}%)?(.*?)\]\((upload:\/\/.*?)\)(?!(.*`))/g;
+
+export function jsonToHtml(json) {
+  if (typeof json !== "object") {
+    return escapeExpression(json);
+  }
+  let html = "<ul>";
+  for (let key in json) {
+    if (!json.hasOwnProperty(key)) {
+      continue;
+    }
+    html += "<li>";
+    if (typeof json[key] === "object" && Array.isArray(json[key])) {
+      html += `<strong>${escapeExpression(key)}:</strong> ${jsonToHtml(
+        json[key]
+      )}`;
+    } else if (typeof json[key] === "object") {
+      html += `<strong>${escapeExpression(key)}:</strong> <ul><li>${jsonToHtml(
+        json[key]
+      )}</li></ul>`;
+    } else {
+      let value = json[key];
+      if (typeof value === "string") {
+        value = escapeExpression(value);
+        value = value.replace(/\n/g, "<br>");
+      }
+      html += `<strong>${escapeExpression(key)}:</strong> ${value}`;
+    }
+    html += "</li>";
+  }
+  html += "</ul>";
+  return htmlSafe(html);
+}

assets/javascripts/initializers/admin-plugin-configuration-nav.js

@@ -12,13 +12,17 @@ export default {
 
     withPluginApi("1.1.0", (api) => {
       api.addAdminPluginConfigurationNav("discourse-ai", PLUGIN_NAV_MODE_TOP, [
+        {
+          label: "discourse_ai.llms.short_title",
+          route: "adminPlugins.show.discourse-ai-llms",
+        },
         {
           label: "discourse_ai.ai_persona.short_title",
           route: "adminPlugins.show.discourse-ai-personas",
         },
         {
-          label: "discourse_ai.llms.short_title",
-          route: "adminPlugins.show.discourse-ai-llms",
+          label: "discourse_ai.tools.short_title",
+          route: "adminPlugins.show.discourse-ai-tools",
         },
       ]);
     });

assets/stylesheets/modules/ai-bot/common/ai-tools.scss

@@ -0,0 +1,79 @@
+.ai-tool-parameter {
+  margin-bottom: 2em;
+  padding: 1.5em;
+  border: 1px solid var(--primary-low);
+  border-radius: 3px;
+  background-color: var(--secondary-very-low);
+
+  .parameter-row {
+    display: flex;
+    align-items: center;
+    margin-bottom: 1em;
+
+    input[type="text"] {
+      flex-grow: 1;
+      margin-right: 1em;
+    }
+
+    label {
+      margin-right: 1em;
+      white-space: nowrap;
+    }
+  }
+
+  .parameter-enum-values {
+    margin-top: 0.5em;
+
+    .enum-value-row {
+      display: flex;
+      align-items: center;
+      margin-bottom: 0.5em;
+
+      input[type="text"] {
+        flex-grow: 1;
+        margin-right: 0.5em;
+      }
+    }
+  }
+}
+
+.ai-tool-editor {
+  max-width: 80%;
+  position: relative;
+  .ace-wrapper {
+    border: 1px solid var(--primary-low);
+    max-width: 100%;
+    position: relative;
+    width: 100%;
+    height: 100%;
+    min-height: 500px;
+    .ace_editor {
+      position: absolute;
+      left: 0;
+      right: 0;
+      top: 0;
+      bottom: 0;
+    }
+  }
+}
+
+.ai-tool-test-modal {
+  &__test-result div {
+    ul {
+      padding-left: 1em;
+    }
+  }
+}
+
+.ai-tool-list-editor {
+  &__header {
+    display: flex;
+    justify-content: space-between;
+    align-items: center;
+    margin: 0 0 1em 0;
+
+    h3 {
+      margin: 0;
+    }
+  }
+}

config/locales/client.en.yml

@@ -183,6 +183,36 @@ en:
           uploading: "Uploading..."
           remove: "Remove upload"
 
+      tools:
+        short_title: "Tools"
+        new: "New Tool"
+        name: "Name"
+        name_help: "The unique name of the tool as used by the language model"
+        description: "Description"
+        description_help: "A clear description of the tool's purpose for the language model"
+        summary: "Summary"
+        summary_help: "Summary of tools purpose to be displayed to end users"
+        script: "Script"
+        parameters: "Parameters"
+        save: "Save"
+        add_parameter: "Add Parameter"
+        parameter_required: "Required"
+        parameter_enum: "Enum"
+        parameter_name: "Parameter Name"
+        parameter_description: "Parameter Description"
+        edit: "Edit"
+        test: "Run Test"
+        delete: "Delete"
+        saved: "Tool saved"
+        presets: "Select a preset..."
+        confirm_delete: Are you sure you want to delete this tool?
+        next:
+          title: "Next"
+        test_modal:
+          title: "Test AI Tool"
+          run: "Run Test"
+          result: "Test Result"
+
       llms:
         short_title: "LLMs"
         no_llms: "No LLMs yet"

config/locales/server.en.yml

@@ -142,6 +142,18 @@ en:
   discourse_ai:
     unknown_model: "Unknown AI model"
 
+    tools:
+      custom_name: "%{name} (custom)"
+      presets:
+        browse_web_jina:
+          name: "Browse web using jina.ai"
+        exchange_rate:
+          name: "Exchange rate"
+        stock_quote:
+          name: "Stock quote (AlphaVantage)"
+        empty_tool:
+          name: "Start from blank..."
+
     ai_helper:
       errors:
         completion_request_failed: "Something went wrong while trying to provide suggestions. Please, try again."

config/routes.rb

@@ -41,6 +41,13 @@ Discourse::Application.routes.draw do
               path: "ai-personas",
               controller: "discourse_ai/admin/ai_personas"
 
+    resources(
+      :ai_tools,
+      only: %i[index create show update destroy],
+      path: "ai-tools",
+      controller: "discourse_ai/admin/ai_tools",
+    ) { post :test, on: :collection }
+
     post "/ai-personas/:id/create-user", to: "discourse_ai/admin/ai_personas#create_user"
     post "/ai-personas/files/upload", to: "discourse_ai/admin/ai_personas#upload_file"
     put "/ai-personas/:id/files/remove", to: "discourse_ai/admin/ai_personas#remove_file"

db/migrate/20240618080148_create_ai_tools.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+class CreateAiTools < ActiveRecord::Migration[7.0]
+  def change
+    create_table :ai_tools do |t|
+      t.string :name, null: false, max_length: 100, unique: true
+      t.string :description, null: false, max_length: 1000
+
+      t.string :summary, null: false, max_length: 255
+
+      t.jsonb :parameters, null: false, default: {}
+      t.text :script, null: false, max_length: 100_000
+      t.integer :created_by_id, null: false
+
+      t.boolean :enabled, null: false, default: true
+      t.timestamps
+    end
+  end
+end

lib/ai_bot/personas/persona.rb

@@ -130,7 +130,11 @@ module DiscourseAi
         end
 
         def available_tools
-          self.class.all_available_tools.filter { |tool| tools.include?(tool) }
+          self
+            .class
+            .all_available_tools
+            .filter { |tool| tools.include?(tool) }
+            .concat(tools.filter(&:custom?))
         end
 
         def craft_prompt(context, llm: nil)

lib/ai_bot/tool_runner.rb

@@ -0,0 +1,174 @@
+# frozen_string_literal: true
+
+module DiscourseAi
+  module AiBot
+    class ToolRunner
+      attr_reader :tool, :parameters, :llm
+      attr_accessor :running_attached_function, :timeout
+
+      TooManyRequestsError = Class.new(StandardError)
+
+      DEFAULT_TIMEOUT = 2000
+      MAX_MEMORY = 10_000_000
+      MARSHAL_STACK_DEPTH = 20
+      MAX_HTTP_REQUESTS = 20
+
+      def initialize(parameters:, llm:, bot_user:, context: {}, tool:, timeout: nil)
+        @parameters = parameters
+        @llm = llm
+        @bot_user = bot_user
+        @context = context
+        @tool = tool
+        @timeout = timeout || DEFAULT_TIMEOUT
+        @running_attached_function = false
+
+        @http_requests_made = 0
+      end
+
+      def mini_racer_context
+        @mini_racer_context ||=
+          begin
+            ctx =
+              MiniRacer::Context.new(
+                max_memory: MAX_MEMORY,
+                marshal_stack_depth: MARSHAL_STACK_DEPTH,
+              )
+            attach_truncate(ctx)
+            attach_http(ctx)
+            ctx.eval(framework_script)
+            ctx
+          end
+      end
+
+      def framework_script
+        <<~JS
+        const http = {
+          get: function(url, options) { return _http_get(url, options) },
+          post: function(url, options) { return _http_post(url, options) },
+        };
+
+        const llm = {
+          truncate: _llm_truncate,
+        };
+        function details() { return ""; };
+      JS
+      end
+
+      def details
+        eval_with_timeout("details()")
+      end
+
+      def eval_with_timeout(script, timeout: nil)
+        timeout ||= @timeout
+        mutex = Mutex.new
+        done = false
+        elapsed = 0
+
+        t =
+          Thread.new do
+            begin
+              while !done
+                # this is not accurate. but reasonable enough for a timeout
+                sleep(0.001)
+                elapsed += 1 if !self.running_attached_function
+                if elapsed > timeout
+                  mutex.synchronize { mini_racer_context.stop unless done }
+                  break
+                end
+              end
+            rescue => e
+              STDERR.puts e
+              STDERR.puts "FAILED TO TERMINATE DUE TO TIMEOUT"
+            end
+          end
+
+        rval = mini_racer_context.eval(script)
+
+        mutex.synchronize { done = true }
+
+        # ensure we do not leak a thread in state
+        t.join
+        t = nil
+
+        rval
+      ensure
+        # exceptions need to be handled
+        t&.join
+      end
+
+      def invoke
+        mini_racer_context.eval(tool.script)
+        eval_with_timeout("invoke(#{JSON.generate(parameters)})")
+      rescue MiniRacer::ScriptTerminatedError
+        { error: "Script terminated due to timeout" }
+      end
+
+      private
+
+      def attach_truncate(mini_racer_context)
+        mini_racer_context.attach(
+          "_llm_truncate",
+          ->(text, length) { @llm.tokenizer.truncate(text, length) },
+        )
+      end
+
+      def attach_http(mini_racer_context)
+        mini_racer_context.attach(
+          "_http_get",
+          ->(url, options) do
+            begin
+              @http_requests_made += 1
+              if @http_requests_made > MAX_HTTP_REQUESTS
+                raise TooManyRequestsError.new("Tool made too many HTTP requests")
+              end
+
+              self.running_attached_function = true
+              headers = (options && options["headers"]) || {}
+
+              result = {}
+              DiscourseAi::AiBot::Tools::Tool.send_http_request(url, headers: headers) do |response|
+                result[:body] = response.body
+                result[:status] = response.code.to_i
+              end
+
+              result
+            ensure
+              self.running_attached_function = false
+            end
+          end,
+        )
+
+        mini_racer_context.attach(
+          "_http_post",
+          ->(url, options) do
+            begin
+              @http_requests_made += 1
+              if @http_requests_made > MAX_HTTP_REQUESTS
+                raise TooManyRequestsError.new("Tool made too many HTTP requests")
+              end
+
+              self.running_attached_function = true
+              headers = (options && options["headers"]) || {}
+              body = options && options["body"]
+
+              result = {}
+              DiscourseAi::AiBot::Tools::Tool.send_http_request(
+                url,
+                method: :post,
+                headers: headers,
+                body: body,
+              ) do |response|
+                result[:body] = response.body
+                result[:status] = response.code.to_i
+              end
+
+              result
+            ensure
+              self.running_attached_function = false
+            end
+          end,
+        )
+      end
+    end
+  end
+end

lib/ai_bot/tools/custom.rb

@@ -0,0 +1,60 @@
+# frozen_string_literal: true
+
+module DiscourseAi
+  module AiBot
+    module Tools
+      class Custom < Tool
+        def self.class_instance(tool_id)
+          klass = Class.new(self)
+          klass.tool_id = tool_id
+          klass
+        end
+
+        def self.custom?
+          true
+        end
+
+        def self.tool_id
+          @tool_id
+        end
+
+        def self.tool_id=(tool_id)
+          @tool_id = tool_id
+        end
+
+        def self.signature
+          AiTool.find(tool_id).signature
+        end
+
+        def self.name
+          AiTool.where(id: tool_id).pluck(:name).first
+        end
+
+        def invoke
+          runner.invoke
+        end
+
+        def runner
+          @runner ||= ai_tool.runner(parameters, llm: llm, bot_user: bot_user, context: context)
+        end
+
+        def ai_tool
+          @ai_tool ||= AiTool.find(self.class.tool_id)
+        end
+
+        def summary
+          ai_tool.summary
+        end
+
+        def details
+          runner.details
+        end
+
+        def help
+          # I do not think this is called, but lets make sure
+          raise "Not implemented"
+        end
+      end
+    end
+  end
+end

lib/ai_bot/tools/tool.rb

@@ -19,6 +19,10 @@ module DiscourseAi
             raise NotImplemented
           end
 
+          def custom?
+            false
+          end
+
           def accepted_options
             []
           end
@@ -124,7 +128,34 @@ module DiscourseAi
           response_code == "200" ? repo_data["default_branch"] : "main"
         end
 
-        def send_http_request(url, headers: {}, authenticate_github: false, follow_redirects: false)
+        def send_http_request(
+          url,
+          headers: {},
+          authenticate_github: false,
+          follow_redirects: false,
+          method: :get,
+          body: nil,
+          &blk
+        )
+          self.class.send_http_request(
+            url,
+            headers: headers,
+            authenticate_github: authenticate_github,
+            follow_redirects: follow_redirects,
+            method: method,
+            body: body,
+            &blk
+          )
+        end
+
+        def self.send_http_request(
+          url,
+          headers: {},
+          authenticate_github: false,
+          follow_redirects: false,
+          method: :get,
+          body: nil
+        )
           raise "Expecting caller to use a block" if !block_given?
 
           uri = nil
@@ -152,7 +183,17 @@ module DiscourseAi
 
           return if uri.blank?
 
-          request = FinalDestination::HTTP::Get.new(uri)
+          request = nil
+          if method == :get
+            request = FinalDestination::HTTP::Get.new(uri)
+          elsif method == :post
+            request = FinalDestination::HTTP::Post.new(uri)
+          end
+
+          raise ArgumentError, "Invalid method: #{method}" if !request
+
+          request.body = body if body
+
           request["User-Agent"] = DiscourseAi::AiBot::USER_AGENT
           headers.each { |k, v| request[k] = v }
           if authenticate_github && SiteSetting.ai_bot_github_access_token.present?

plugin.rb

@@ -28,6 +28,8 @@ register_asset "stylesheets/modules/sentiment/mobile/dashboard.scss", :mobile
 
 register_asset "stylesheets/modules/llms/common/ai-llms-editor.scss"
 
+register_asset "stylesheets/modules/ai-bot/common/ai-tools.scss"
+
 module ::DiscourseAi
   PLUGIN_NAME = "discourse-ai"
 end

spec/fabricators/llm_model_fabricator.rb

@@ -4,6 +4,6 @@ Fabricator(:llm_model) do
   display_name "A good model"
   name "gpt-4-turbo"
   provider "open_ai"
-  tokenizer "DiscourseAi::Tokenizers::OpenAi"
+  tokenizer "DiscourseAi::Tokenizer::OpenAiTokenizer"
   max_prompt_tokens 32_000
 end

spec/lib/modules/ai_bot/playground_spec.rb

@@ -63,6 +63,88 @@ RSpec.describe DiscourseAi::AiBot::Playground do
     end
   end
 
+  describe "custom tool integration" do
+    let!(:custom_tool) do
+      AiTool.create!(
+        name: "search",
+        summary: "searching for things",
+        description: "A test custom tool",
+        parameters: [{ name: "query", type: "string", description: "Input for the custom tool" }],
+        script:
+          "function invoke(params) { return 'Custom tool result: ' + params.query; }; function details() { return 'did stuff'; }",
+        created_by: user,
+      )
+    end
+
+    let!(:ai_persona) { Fabricate(:ai_persona, tools: ["custom-#{custom_tool.id}"]) }
+
+    it "uses custom tool in conversation" do
+      persona_klass = AiPersona.all_personas.find { |p| p.name == ai_persona.name }
+      bot = DiscourseAi::AiBot::Bot.as(bot_user, persona: persona_klass.new)
+      playground = DiscourseAi::AiBot::Playground.new(bot)
+
+      function_call = (<<~XML).strip
+        <function_calls>
+          <invoke>
+            <tool_name>search</tool_name>
+            <tool_id>666</tool_id>
+            <parameters>
+              <query>Can you use the custom tool</query>
+            </parameters>
+          </invoke>
+        </function_calls>",
+      XML
+
+      responses = [function_call, "custom tool did stuff (maybe)"]
+
+      reply_post = nil
+
+      DiscourseAi::Completions::Llm.with_prepared_responses(responses) do |_, _, _prompt|
+        new_post = Fabricate(:post, raw: "Can you use the custom tool?")
+        reply_post = playground.reply_to(new_post)
+      end
+
+      expected = <<~TXT.strip
+        <details>
+          <summary>searching for things</summary>
+          <p>did stuff</p>
+        </details>
+        <span></span>
+
+        custom tool did stuff (maybe)
+      TXT
+      expect(reply_post.raw).to eq(expected)
+
+      custom_prompt = PostCustomPrompt.find_by(post_id: reply_post.id).custom_prompt
+      expected_prompt = [
+        [
+          "{\"arguments\":{\"query\":\"Can you use the custom tool\"}}",
+          "666",
+          "tool_call",
+          "search",
+        ],
+        ["\"Custom tool result: Can you use the custom tool\"", "666", "tool", "search"],
+        ["custom tool did stuff (maybe)", "claude-2"],
+      ]
+
+      expect(custom_prompt).to eq(expected_prompt)
+
+      custom_tool.update!(enabled: false)
+      # so we pick up new cache
+      persona_klass = AiPersona.all_personas.find { |p| p.name == ai_persona.name }
+      bot = DiscourseAi::AiBot::Bot.as(bot_user, persona: persona_klass.new)
+      playground = DiscourseAi::AiBot::Playground.new(bot)
+
+      # lets ensure tool does not run...
+      DiscourseAi::Completions::Llm.with_prepared_responses(responses) do |_, _, _prompt|
+        new_post = Fabricate(:post, raw: "Can you use the custom tool?")
+        reply_post = playground.reply_to(new_post)
+      end
+
+      expect(reply_post.raw.strip).to eq(function_call)
+    end
+  end
+
   describe "image support" do
     before do
       Jobs.run_immediately!
@@ -459,7 +541,6 @@ RSpec.describe DiscourseAi::AiBot::Playground do
 
     it "picks the correct llm for persona in PMs" do
       gpt_35_turbo = Fabricate(:llm_model, name: "gpt-3.5-turbo")
-      gpt_35_turbo_16k = Fabricate(:llm_model, name: "gpt-3.5-turbo-16k")
 
       # If you start a PM with GPT 3.5 bot, replies should come from it, not from Claude
       SiteSetting.ai_bot_enabled = true

spec/models/ai_tool_spec.rb

@@ -0,0 +1,196 @@
+# frozen_string_literal: true
+
+RSpec.describe AiTool do
+  fab!(:llm_model) { Fabricate(:llm_model, name: "claude-2") }
+  let(:llm) { DiscourseAi::Completions::Llm.proxy_from_obj(llm_model) }
+
+  def create_tool(parameters: nil, script: nil)
+    AiTool.create!(
+      name: "test",
+      description: "test",
+      parameters: parameters || [{ name: "query", type: "string", desciption: "perform a search" }],
+      script: script || "function invoke(params) { return params; }",
+      created_by_id: 1,
+      summary: "Test tool summary",
+    )
+  end
+
+  it "it can run a basic tool" do
+    tool = create_tool
+
+    expect(tool.signature).to eq(
+      {
+        name: "test",
+        description: "test",
+        parameters: [{ name: "query", type: "string", desciption: "perform a search" }],
+      },
+    )
+
+    runner = tool.runner({ "query" => "test" }, llm: nil, bot_user: nil, context: {})
+
+    expect(runner.invoke).to eq("query" => "test")
+  end
+
+  it "can perform POST HTTP requests" do
+    script = <<~JS
+    function invoke(params) {
+      result = http.post("https://example.com/api",
+        {
+          headers: { TestHeader: "TestValue" },
+          body: JSON.stringify({ data: params.data })
+        }
+      );
+
+      return result.body;
+    }
+  JS
+
+    tool = create_tool(script: script)
+    runner = tool.runner({ "data" => "test data" }, llm: nil, bot_user: nil, context: {})
+
+    stub_request(:post, "https://example.com/api").with(
+      body: "{\"data\":\"test data\"}",
+      headers: {
+        "Accept" => "*/*",
+        "Testheader" => "TestValue",
+        "User-Agent" => "Discourse AI Bot 1.0 (https://www.discourse.org)",
+      },
+    ).to_return(status: 200, body: "Success", headers: {})
+
+    result = runner.invoke
+
+    expect(result).to eq("Success")
+  end
+
+  it "can perform GET HTTP requests, with 1 param" do
+    script = <<~JS
+      function invoke(params) {
+        result = http.get("https://example.com/" + params.query);
+        return result.body;
+      }
+    JS
+
+    tool = create_tool(script: script)
+    runner = tool.runner({ "query" => "test" }, llm: nil, bot_user: nil, context: {})
+
+    stub_request(:get, "https://example.com/test").with(
+      headers: {
+        "Accept" => "*/*",
+        "User-Agent" => "Discourse AI Bot 1.0 (https://www.discourse.org)",
+      },
+    ).to_return(status: 200, body: "Hello World", headers: {})
+
+    result = runner.invoke
+
+    expect(result).to eq("Hello World")
+  end
+
+  it "is limited to MAX http requests" do
+    script = <<~JS
+      function invoke(params) {
+        let i = 0;
+        while (i < 21) {
+          http.get("https://example.com/");
+          i += 1;
+        }
+        return "will not happen";
+      }
+      JS
+
+    tool = create_tool(script: script)
+    runner = tool.runner({}, llm: nil, bot_user: nil, context: {})
+
+    stub_request(:get, "https://example.com/").to_return(
+      status: 200,
+      body: "Hello World",
+      headers: {
+      },
+    )
+
+    expect { runner.invoke }.to raise_error(DiscourseAi::AiBot::ToolRunner::TooManyRequestsError)
+  end
+
+  it "can perform GET HTTP requests" do
+    script = <<~JS
+      function invoke(params) {
+        result = http.get("https://example.com/" + params.query,
+          { headers: { TestHeader: "TestValue" } }
+        );
+
+        return result.body;
+      }
+    JS
+
+    tool = create_tool(script: script)
+    runner = tool.runner({ "query" => "test" }, llm: nil, bot_user: nil, context: {})
+
+    stub_request(:get, "https://example.com/test").with(
+      headers: {
+        "Accept" => "*/*",
+        "Testheader" => "TestValue",
+        "User-Agent" => "Discourse AI Bot 1.0 (https://www.discourse.org)",
+      },
+    ).to_return(status: 200, body: "Hello World", headers: {})
+
+    result = runner.invoke
+
+    expect(result).to eq("Hello World")
+  end
+
+  it "will not timeout on slow HTTP reqs" do
+    script = <<~JS
+      function invoke(params) {
+        result = http.get("https://example.com/" + params.query,
+          { headers: { TestHeader: "TestValue" } }
+        );
+
+        return result.body;
+      }
+    JS
+
+    tool = create_tool(script: script)
+    runner = tool.runner({ "query" => "test" }, llm: nil, bot_user: nil, context: {})
+
+    stub_request(:get, "https://example.com/test").to_return do
+      sleep 0.01
+      { status: 200, body: "Hello World", headers: {} }
+    end
+
+    runner.timeout = 5
+
+    result = runner.invoke
+
+    expect(result).to eq("Hello World")
+  end
+
+  it "has access to llm truncation tools" do
+    script = <<~JS
+      function invoke(params) {
+        return llm.truncate("Hello World", 1);
+      }
+    JS
+
+    tool = create_tool(script: script)
+
+    runner = tool.runner({}, llm: llm, bot_user: nil, context: {})
+    result = runner.invoke
+
+    expect(result).to eq("Hello")
+  end
+
+  it "can timeout slow JS" do
+    script = <<~JS
+      function invoke(params) {
+        while (true) {}
+      }
+    JS
+
+    tool = create_tool(script: script)
+    runner = tool.runner({ "query" => "test" }, llm: nil, bot_user: nil, context: {})
+
+    runner.timeout = 5
+
+    result = runner.invoke
+    expect(result[:error]).to eq("Script terminated due to timeout")
+  end
+end

spec/requests/admin/ai_tools_controller_spec.rb

@@ -0,0 +1,150 @@
+# frozen_string_literal: true
+
+RSpec.describe DiscourseAi::Admin::AiToolsController do
+  fab!(:admin)
+  fab!(:ai_tool) do
+    AiTool.create!(
+      name: "Test Tool",
+      description: "A test tool",
+      script: "function invoke(params) { return params; }",
+      parameters: [{ name: "query", type: "string", description: "perform a search" }],
+      summary: "Test tool summary",
+      created_by_id: -1,
+    )
+  end
+
+  before do
+    sign_in(admin)
+    SiteSetting.ai_embeddings_enabled = true
+  end
+
+  describe "GET #index" do
+    it "returns a success response" do
+      get "/admin/plugins/discourse-ai/ai-tools.json"
+      expect(response).to be_successful
+      expect(response.parsed_body["ai_tools"].length).to eq(AiTool.count)
+      expect(response.parsed_body["meta"]["presets"].length).to be > 0
+    end
+  end
+
+  describe "GET #show" do
+    it "returns a success response" do
+      get "/admin/plugins/discourse-ai/ai-tools/#{ai_tool.id}.json"
+      expect(response).to be_successful
+      expect(response.parsed_body["ai_tool"]["name"]).to eq(ai_tool.name)
+    end
+  end
+
+  describe "POST #create" do
+    let(:valid_attributes) do
+      {
+        name: "Test Tool",
+        description: "A test tool",
+        parameters: [{ name: "query", type: "string", description: "perform a search" }],
+        script: "function invoke(params) { return params; }",
+        summary: "Test tool summary",
+      }
+    end
+
+    it "creates a new AiTool" do
+      expect {
+        post "/admin/plugins/discourse-ai/ai-tools.json",
+             params: { ai_tool: valid_attributes }.to_json,
+             headers: {
+               "CONTENT_TYPE" => "application/json",
+             }
+      }.to change(AiTool, :count).by(1)
+
+      expect(response).to have_http_status(:created)
+      expect(response.parsed_body["ai_tool"]["name"]).to eq("Test Tool")
+    end
+  end
+
+  describe "PUT #update" do
+    it "updates the requested ai_tool" do
+      put "/admin/plugins/discourse-ai/ai-tools/#{ai_tool.id}.json",
+          params: {
+            ai_tool: {
+              name: "Updated Tool",
+            },
+          }
+
+      expect(response).to be_successful
+      expect(ai_tool.reload.name).to eq("Updated Tool")
+    end
+  end
+
+  describe "DELETE #destroy" do
+    it "destroys the requested ai_tool" do
+      expect { delete "/admin/plugins/discourse-ai/ai-tools/#{ai_tool.id}.json" }.to change(
+        AiTool,
+        :count,
+      ).by(-1)
+
+      expect(response).to have_http_status(:no_content)
+    end
+  end
+
+  describe "#test" do
+    it "runs an existing tool and returns the result" do
+      post "/admin/plugins/discourse-ai/ai-tools/test.json",
+           params: {
+             id: ai_tool.id,
+             parameters: {
+               input: "Hello, World!",
+             },
+           }
+
+      expect(response.status).to eq(200)
+      expect(response.parsed_body["output"]).to eq("input" => "Hello, World!")
+    end
+
+    it "runs a new unsaved tool and returns the result" do
+      post "/admin/plugins/discourse-ai/ai-tools/test.json",
+           params: {
+             ai_tool: {
+               name: "New Tool",
+               description: "A new test tool",
+               script: "function invoke(params) { return 'New test result: ' + params.input; }",
+               parameters: [
+                 { name: "input", type: "string", description: "Input for the new test tool" },
+               ],
+             },
+             parameters: {
+               input: "Test input",
+             },
+           }
+
+      expect(response.status).to eq(200)
+      expect(response.parsed_body["output"]).to eq("New test result: Test input")
+    end
+
+    it "returns an error for invalid tool_id" do
+      post "/admin/plugins/discourse-ai/ai-tools/test.json",
+           params: {
+             id: -1,
+             parameters: {
+               input: "Hello, World!",
+             },
+           }
+
+      expect(response.status).to eq(400)
+      expect(response.parsed_body["errors"]).to include("Couldn't find AiTool with 'id'=-1")
+    end
+
+    it "handles exceptions during tool execution" do
+      ai_tool.update!(script: "function invoke(params) { throw new Error('Test error'); }")
+
+      post "/admin/plugins/discourse-ai/ai-tools/test.json",
+           params: {
+             id: ai_tool.id,
+             parameters: {
+               input: "Hello, World!",
+             },
+           }
+
+      expect(response.status).to eq(400)
+      expect(response.parsed_body["errors"].to_s).to include("Error executing the tool")
+    end
+  end
+end

spec/system/ai_bot/tool_spec.rb

@@ -0,0 +1,61 @@
+# frozen_string_literal: true
+
+require "rails_helper"
+
+describe "AI Tool Management", type: :system do
+  fab!(:admin)
+
+  before do
+    SiteSetting.ai_embeddings_enabled = true
+    sign_in(admin)
+  end
+
+  it "allows admin to create a new AI tool from preset" do
+    visit "/admin/plugins/discourse-ai/ai-tools"
+
+    expect(page).to have_content("Tools")
+
+    find(".ai-tool-list-editor__new-button").click
+
+    select_kit = PageObjects::Components::SelectKit.new(".ai-tool-editor__presets")
+    select_kit.expand
+    select_kit.select_row_by_value("exchange_rate")
+
+    find(".ai-tool-editor__next").click
+    find(".ai-tool-editor__test-button").click
+
+    expect(page).not_to have_button(".ai-tool-editor__delete")
+
+    modal = PageObjects::Modals::AiToolTest.new
+    modal.base_currency = "USD"
+    modal.target_currency = "EUR"
+    modal.amount = "100"
+
+    stub_request(:get, %r{https://open\.er-api\.com/v6/latest/USD}).to_return(
+      status: 200,
+      body: '{"rates": {"EUR": 0.85}}',
+      headers: {
+        "Content-Type" => "application/json",
+      },
+    )
+    modal.run_test
+
+    expect(modal).to have_content("exchange_rate")
+    expect(modal).to have_content("0.85")
+
+    modal.close
+
+    find(".ai-tool-editor__save").click
+
+    expect(page).to have_content("Tool saved")
+
+    visit "/admin/plugins/discourse-ai/ai-personas/new"
+
+    tool_id = AiTool.order("id desc").limit(1).pluck(:id).first
+    tool_selector = PageObjects::Components::SelectKit.new(".ai-persona-editor__tools")
+    tool_selector.expand
+
+    tool_selector.select_row_by_value("custom-#{tool_id}")
+    expect(tool_selector).to have_selected_value("custom-#{tool_id}")
+  end
+end

spec/system/page_objects/modals/ai_tool_test_modal.rb

@@ -0,0 +1,26 @@
+# frozen_string_literal: true
+
+module PageObjects
+  module Modals
+    class AiToolTest < PageObjects::Modals::Base
+      BODY_SELECTOR = ".ai-tool-test-modal__body"
+      MODAL_SELECTOR = ".ai-tool-test-modal"
+
+      def base_currency=(value)
+        body.fill_in("base_currency", with: value)
+      end
+
+      def target_currency=(value)
+        body.fill_in("target_currency", with: value)
+      end
+
+      def amount=(value)
+        body.fill_in("amount", with: value)
+      end
+
+      def run_test
+        click_primary_button
+      end
+    end
+  end
+end