mirror of
https://github.com/discourse/discourse-ai.git
synced 2025-06-24 08:32:14 +00:00
b5d42f373b
Followup fe7e73a6a436795c053a611884431f663f4bdd27 and 1300cc8a36dc33870f0d3c81e1e1a21a93e1f1de Reintroduces proofreader specs and unskips ones skipped when streaming composer suggestions were added. We just need to make sure the jobs run immediately in specs because the job is what sends the MessageBus event to the UI. Also adds a pageobject method to confirm a diff is shown in the modal before pressing Confirm button.
Discourse AI Plugin
Plugin Summary
For more information, please see: https://meta.discourse.org/t/discourse-ai/259214?u=falco
Evals
The directory evals contains AI evals for the Discourse AI plugin.
You may create a local config by copying config/eval-llms.yml to config/eval-llms.local.yml and modifying the values.
To run them use:
cd evals ./run --help
Usage: evals/run [options]
-e, --eval NAME Name of the evaluation to run
--list-models List models
-m, --model NAME Model to evaluate (will eval all models if not specified)
-l, --list List evals
To run evals you will need to configure API keys in your environment:
OPENAI_API_KEY=your_openai_api_key ANTHROPIC_API_KEY=your_anthropic_api_key GEMINI_API_KEY=your_gemini_api_key