When checking `attach_csv` in the `Schedule a PM with Data Explorer results`
automation script, `csv` has to be added to the list of authorized extensions in the site settings. This will now raise an error if it's not the case.
Followup da1c99ec2d
We already had the functionality to convert results like
this:
```
|blah_url|
|--------|
|3,https://test.com|
```
To a link clientside, so in the UI it looks like this:
```
<a href="https://test.com">3</a>
```
With the addition of the recurring report to post automation,
and the existing report to PM automation, we also need to be
able to do this server-side, so reports don't come out malformed
if they have these type of count + URL columns.
This script is similar to the existing one that schedules
report results to to be sent to a PM on a regular basis,
but instead takes a topic ID and posts to that. This way
people can have report results sent to a public topic regularly
too and not have to deal with PM recipients and so on.
This commit uses GroupChooser as the input for param input of type
group_id. Meanwhile, it improves invalid group validation and semantic
error prompts.
The old float validation had several bugs. It will recognize strings
like "a1.2" and "3.4b" as valid doubles, but will not recognize integers
like "1234" as doubles. Also, since an empty string is not falsy in Ruby,
it will recognize "Inf" as -Infinity.
This commit fixes these issues
Validation of `user_id` parameter will throw a 500 error because
`User.find_by_username_or_email` does not throw
`ActiveRecord::RecordNotFound`, but silently returns `nil`.
This results in a `NoMethodError` in `object.id` on the next line
What's the problem?
===================
TL;DR: When the user enters an incorrect default value, its
corresponding param-input will disappear
When creating a parameter from SQL, we perform cast_to_ruby, which means
that if the default value given by the user is invalid, the entire
parameter will not be added to the param_list of the query.
This behavior is very confusing, and users will not understand why an
incorrect initial value will cause the param-input to disappear.
What's the fix?
================
The cast_to_ruby process is canceled in create_from_sql, so that
param-input with incorrect default value will still be displayed.
We have a simple validation process on the front end, which is enough to
prompt whether some default inputs are incorrect.
We used `with_deleted` incorrectly in the code. This method does not exist
on `User`, `Badge`, and `Group`. This will cause an error when entering a
numeric id in these three parameter input, forcing the user to enter the
name/username of these inputs.
See https://github.com/discourse/discourse-data-explorer/pull/307#issuecomment-2291017256
* FIX: Wrong type in category_id param input
We will dasherize category_id. The dasherize function accepts a string,
but we don't type-check it, so the default null may be passed in. This
will cause a type error and crash the front-end.
Since 67a8080e33
in core, the functionality to bookmark a report from the group
Reports tab has been broken. This commit fixes the issue and adds
system spec coverage to prevent regression.
The main change here is that we now send a single PM to a group rather than individual PMs to each group member.
This change also adds email recipients correctly as target_emails since support was added within Discourse Automation.
* Added can_see_bookmarkable? methods to BaseBookmarkable, implement
that in QueryGroupBookmarkable
* Update spec to check that Notification for reminder has
bookmarkable_id and bookmarkable_type
c.f. https://github.com/discourse/discourse/pull/25905
If a column is payload or contains _payload it will be assumed
it has JSON data in it, then we will show the truncated JSON in the
result column with a button to show the full-screen formatted
JSON using our full-screen code viewer. We also do the same if
the column is the `json` postgres data type.
This change includes the Discourse base url to allow for subfolder installs, it will also fix another potential issue where reports are sent via email (since we use the email_group_user component) that would require the base_url to know which domain to point to.
This PR fixes 2 issues that were picked up by users for the Scheduled Data Explorer Report automation script and a couple of small improvements to better match the format of manual data explorer query results.
The first issue is that within result_to_markdown the colrender contains null values, and we are currently checking length (previously treated as a packed array but it is actually a sparse array). Therefore we can check if the current index of the array is null rather than checking the size of the array.
The second issue addresses the blank query_params field. When the data explorer script does not require any params to be passed in via the automation script then it will have a nil value, however it should be defaulted to {} within the plugin.
To improve formatting the markdown table for PMs is now aligned to left and where values are substituted (for example user_id becomes username) we then include the id within brackets, for example:
user_id becomes username (user_id)
Data Explorer can run arbitrary SQL queries which can be costly for us if over-used. Because of that we want to add the ability to rate limit the query run endpoint, in particular when requested programmatically using API.
This commit introduces a rate limit to the `QueryController#run` endpoint. It heavily leans on the existing `RateLimiter` implementation, and the ability of `ApplicationController` to turn rate limit exceptions into nicely formatted JSON responses.
The rate limit (per 10 seconds) can be configured through the global setting `max_data_explorer_api_reqs_per_10_seconds`, and defaults to 2.
Handling can be configured through `max_data_explorer_api_req_mode`, and can be set to warn, block, or both warn and block. We will default to warn for now and monitor the logs for a while.
This feature enables admins to create reports automatically based on a recurring schedule.
It introduces a new automation script that includes the new email_group_user field added to discourse-automation, along with a query_id and query_params to pass in parameters to the existing data explorer query.
The output of the report will be sent via pm (as a markdown table) to the recipients entered within the automation script.
The automation (supports individual users, email addresses and groups).
This commit updates the plugin to the latest guidelines, as shown in
discourse-plugin-skeleton, which involves moving a lot of the code to
dedicated files, use proper namespaces, use the autoloader as much as
possible, etc.
Followup to 360d0dde650704a0f01fd6d8b525e933b1d7fcf2,
this causes other plugin tests to fail because
`DiscoursePluginRegistry.reset!` is
a shotgun. We can use the more surgical version
`DiscoursePluginRegistry.reset_register!(:bookmarkables)`
instead.
- Require query name is present
- Ensure all routes are treated by default as .json, so errors flow correctly
- Remove superflous save/cancel controls from group settings
- Remove group control when item is destroyed
- Disable editing of query when it is deleted
Co-authored-by: Osama Sayegh <asooomaasoooma90@gmail.com>
Before this fix, the use of PG template patterns containing ":" or the
use of "?" in comments in the SQL will result in an error being raised
because `DB.param_encoder.encode` calls ActiveRecord's `sanitize_sql_array` which is
meant for SQL fragments and not an entire SQL string.
Instead we change data-explorer to use `MiniSql::InlineParamEncoder`
instead which takes into account of template patterns and does not trip
on `?` which is a special param encoding character used by ActiveRecord.
* FIX: allow groups to access system queries (without having to run the query once first)
Bug is: Trying to allow a group to access a system query results in a Discourse::NotFound unless the query is run first.
Cause:
- System queries don't exist in the database by default
- update calls set_query before action
- set_query searches the database for the system query with Query.find_by(:id), which will not exist by default.
- running system queries first fixes this because Query.find is overridden to include system queries (Queries.default) in its results, avoiding the Discourse::NotFound.
Solution: use the overridden Query.find in set_query to include system queries in the search, instead of Query.find_by(:id)
* Added test for fixing allowing groups to access system query.
* Fixed test formatting.
Joffrey JAFFEUX
Jarek Radosz
Martin Brennan
Linca
Osama Sayegh
Natalie Tay
David Battersby
Penar Musaraj
Loïc Guitaut
Ted Johansson
Alan Guo Xiang Tan
Bianca Nenciu
Krzysztof Kotlarek
David Taylor
Sam
Frank