Data Explorer can run arbitrary SQL queries which can be costly for us if over-used. Because of that we want to add the ability to rate limit the query run endpoint, in particular when requested programmatically using API.
This commit introduces a rate limit to the `QueryController#run` endpoint. It heavily leans on the existing `RateLimiter` implementation, and the ability of `ApplicationController` to turn rate limit exceptions into nicely formatted JSON responses.
The rate limit (per 10 seconds) can be configured through the global setting `max_data_explorer_api_reqs_per_10_seconds`, and defaults to 2.
Handling can be configured through `max_data_explorer_api_req_mode`, and can be set to warn, block, or both warn and block. We will default to warn for now and monitor the logs for a while.
This feature enables admins to create reports automatically based on a recurring schedule.
It introduces a new automation script that includes the new email_group_user field added to discourse-automation, along with a query_id and query_params to pass in parameters to the existing data explorer query.
The output of the report will be sent via pm (as a markdown table) to the recipients entered within the automation script.
The automation (supports individual users, email addresses and groups).
This commit updates the plugin to the latest guidelines, as shown in
discourse-plugin-skeleton, which involves moving a lot of the code to
dedicated files, use proper namespaces, use the autoloader as much as
possible, etc.
# Context
Data explorer gives you the ability to use special attributes such as 👇
> SELECT TIMESTAMP 'yesterday' as reltime$time
# Problem
During the upgrade to ember octane these were neglected and did not work. This PR updates them to working condition.
# Additional
- Drop unused action of `saveDefaults`
Followup to 360d0dde650704a0f01fd6d8b525e933b1d7fcf2,
this causes other plugin tests to fail because
`DiscoursePluginRegistry.reset!` is
a shotgun. We can use the more surgical version
`DiscoursePluginRegistry.reset_register!(:bookmarkables)`
instead.
Prevent the table to have a fixed 1000px height with the horizontal scrollbar at the bottom of a mostly blank section when we have only a few results.
This change doesn't interfere with the sticky header.
- Drop `explorer-container` and move its logic to `admin-plugin-explorer` container
- Convert resizing of the query edit pane from jquery -> draggable modifier
We were not passing the correct arguments through in the group reports template causing the download url to default to an admin route. This made downloads unavailable to non-admin users. Fixed by passing the correct values through.
During the upgrade to Octane group reports did not have the necessary 'updateParams' function added to have the param input changes bubble up to the parent. This PR adds the missing function as well as a small test to check that params can be inputted as expected (inserting a param would error previously).
After running a query with a non-default query param (inserting the new param into the url) we want to have the same params available after reloading the page. To do this we need to pass the updated params back up to the parent due to Octane's one direction data stream. I went over this with @pmusaraj and we both agreed this was extremely difficult to test due to needing to reload the page in a test, so we opted to move forward without one. A system test could be helpful in this case... I will investigate in a follow up PR.
- Move param-input tests to a dedicated file
When there were no query results it would throw an error due to `this.resultCount` always passing as it is in the format of
```
"INTEGER - results returned"
```
so we need to grab the first index of the string and check if the integer is great than 0
- Require query name is present
- Ensure all routes are treated by default as .json, so errors flow correctly
- Remove superflous save/cancel controls from group settings
- Remove group control when item is destroyed
- Disable editing of query when it is deleted
Co-authored-by: Osama Sayegh <asooomaasoooma90@gmail.com>
Due to recent core changes for Ember in
0221855ba7
and 952b033165
the correct way of calling these actions must be observed,
otherwise clicking on a query did nothing.
This commit fixes the click and also makes scrollTop
work with the new {{on X}} Ember syntax.
Before this fix, the use of PG template patterns containing ":" or the
use of "?" in comments in the SQL will result in an error being raised
because `DB.param_encoder.encode` calls ActiveRecord's `sanitize_sql_array` which is
meant for SQL fragments and not an entire SQL string.
Instead we change data-explorer to use `MiniSql::InlineParamEncoder`
instead which takes into account of template patterns and does not trip
on `?` which is a special param encoding character used by ActiveRecord.
Ted Johansson
Isaac Janzen
David Battersby
Alan Guo Xiang Tan
Bianca Nenciu
Martin Brennan
Krzysztof Kotlarek
Discourse Translator Bot
Canapin
discoursebot
David Taylor
dependabot[bot]
Sam
Penar Musaraj