Commit Graph

66 Commits

Author SHA1 Message Date
锦心 6bf3ac7426
FIX: Fix `user_id` validation (#312)
Validation of `user_id` parameter will throw a 500 error because
`User.find_by_username_or_email` does not throw
`ActiveRecord::RecordNotFound`, but silently returns `nil`.
This results in a `NoMethodError` in `object.id` on the next line
2024-08-21 12:03:29 +08:00
锦心 760667d744
FIX: Param should be displayed when the default value is invalid (#313)
What's the problem?
===================

TL;DR: When the user enters an incorrect default value, its
corresponding param-input will disappear

When creating a parameter from SQL, we perform cast_to_ruby, which means
that if the default value given by the user is invalid, the entire
parameter will not be added to the param_list of the query.
This behavior is very confusing, and users will not understand why an
incorrect initial value will cause the param-input to disappear.

What's the fix?
================

The cast_to_ruby process is canceled in create_from_sql, so that
param-input with incorrect default value will still be displayed.
We have a simple validation process on the front end, which is enough to
prompt whether some default inputs are incorrect.
2024-08-21 12:03:11 +08:00
锦心 5080ce9b1f
UX: Rewrite param-input using FormKit (#307)
What does this PR do?
=====================

This PR refactors param-input to use FormKit. FormKit is a structured
form tool in the core. After the rewrite, we will be able to get semantic 
parameter error prompts, etc.

meta link: https://meta.discourse.org/t/wishlist-param-dropdown-for-data-explorer-query/253883/28?u=lhc_fl
2024-08-20 09:42:50 +08:00
锦心 b063db4ba4
FIX: Use with_deleted only in topic and post (#308)
We used `with_deleted` incorrectly in the code. This method does not exist
on `User`, `Badge`, and `Group`. This will cause an error when entering a
numeric id in these three parameter input, forcing the user to enter the
name/username of these inputs.

See https://github.com/discourse/discourse-data-explorer/pull/307#issuecomment-2291017256
2024-08-15 20:01:53 +08:00
锦心 902b8c7913
UX: Using CategoryChooser for param param_input (#306)
This change changes the category selector to use CategoryChooser instead
of a bare text input to improve the user experience.

Related meta link: https://meta.discourse.org/t/wishlist-param-dropdown-for-data-explorer-query/253883/28
2024-08-13 16:03:26 +08:00
锦心 41dfa217ca
FIX: Wrong type in category_id param input (#304)
* FIX: Wrong type in category_id param input

We will dasherize category_id. The dasherize function accepts a string,
but we don't type-check it, so the default null may be passed in. This
will cause a type error and crash the front-end.
2024-08-05 20:11:16 +08:00
Martin Brennan 87d7958db8
FIX: Bookmarking group reports (#291)
Since 67a8080e33
in core, the functionality to bookmark a report from the group
Reports tab has been broken. This commit fixes the issue and adds
system spec coverage to prevent regression.
2024-05-22 09:18:09 +10:00
Osama Sayegh 7f7d930559
DEV: Rename `Jobs::DiscourseAutomationTracker` -> `Jobs::DiscourseAutomation::Tracker` (#289)
The class is renamed in https://github.com/discourse/discourse/pull/26860.
2024-05-06 23:12:48 +03:00
Natalie Tay b9d875cc3e
FEATURE: Add ability to skip sending the PM if there are no results (#286) 2024-04-18 21:40:28 +08:00
David Battersby cde34fb316
FIX: send single report PM to groups (#284)
The main change here is that we now send a single PM to a group rather than individual PMs to each group member.

This change also adds email recipients correctly as target_emails since support was added within Discourse Automation.
2024-03-27 17:40:26 +08:00
Penar Musaraj 2f1044820c
FEATURE: Add group_list parameter type (#283)
This allows admins to select multiple groups from a dropdown list and
filter results using that parameter.
2024-03-12 12:06:01 -04:00
Loïc Guitaut abe51c2aba
DEV: Fix new Rubocop offenses (#282) 2024-02-29 18:15:57 +01:00
Martin Brennan f09e631636
DEV: Adapt to core bookmark changes (#280)
* Added can_see_bookmarkable? methods to BaseBookmarkable, implement
  that in QueryGroupBookmarkable
* Update spec to check that Notification for reminder has
  bookmarkable_id and bookmarkable_type

c.f. https://github.com/discourse/discourse/pull/25905
2024-02-29 09:11:40 +10:00
Martin Brennan 94e29e2143
DEV: Skip spec (#281)
Will reinstate after https://github.com/discourse/discourse/pull/25905
which is failing because of this spec not being compatible
with the changes.
2024-02-28 13:29:01 +10:00
Martin Brennan 48a4038809
FEATURE: Allow topic URL without post number for post_id param (#275)
This commit allows base topic URLs (e.g. https://meta.discourse.org/t/hide-text-in-text-select-popup-menu-feedback/287656)
to be used by the post_id parameter type. We just assume in this case
that the post_id for the param is 1.
2024-01-16 15:09:05 +10:00
Natalie Tay 66c54a35cd
FIX: Continue to send the PM to others when there is a group or user that does not exist (#274)
Don't error out  when there is a group or user that does not exist
and also allow admins to be sent
2024-01-03 14:05:03 +08:00
Jarek Radosz e067df0b43
DEV: Update linting (#267) 2023-12-01 16:47:54 +01:00
Martin Brennan 5776aa7fc9
FEATURE: Add JSON result type component (#260)
If a column is payload or contains _payload it will be assumed
it has JSON data in it, then we will show the truncated JSON in the
result column with a button to show the full-screen formatted
JSON using our full-screen code viewer. We also do the same if
the column is the `json` postgres data type.
2023-11-02 09:50:05 +10:00
David Battersby e4f8d3924a
FIX: update report url to work with subfolder installs (#255)
This change includes the Discourse base url to allow for subfolder installs, it will also fix another potential issue where reports are sent via email (since we use the email_group_user component) that would require the base_url to know which domain to point to.
2023-07-31 10:14:12 +08:00
David Battersby ee308c637c
FIX: Scheduled PM with Data Explorer Results not sending PM (#250)
This PR fixes 2 issues that were picked up by users for the Scheduled Data Explorer Report automation script and a couple of small improvements to better match the format of manual data explorer query results.

The first issue is that within result_to_markdown the colrender contains null values, and we are currently checking length (previously treated as a packed array but it is actually a sparse array). Therefore we can check if the current index of the array is null rather than checking the size of the array.

The second issue addresses the blank query_params field. When the data explorer script does not require any params to be passed in via the automation script then it will have a nil value, however it should be defaulted to {} within the plugin.

To improve formatting the markdown table for PMs is now aligned to left and where values are substituted (for example user_id becomes username) we then include the id within brackets, for example:
user_id becomes username (user_id)
2023-07-04 14:18:56 +08:00
Martin Brennan b011f3ef8a
DEV: Remove usages of RateLimiter.clear_all! (#247)
c.f. https://github.com/discourse/discourse/pull/22103
2023-06-15 09:39:08 +10:00
Joffrey JAFFEUX 97d60de92a
FIX: ensures explorer is using default values when provided (#240) 2023-04-17 09:06:27 -05:00
Ted Johansson bc02d030b6
DEV: Add configurable rate limit for Data Explorer API query runs (#238)
Data Explorer can run arbitrary SQL queries which can be costly for us if over-used. Because of that we want to add the ability to rate limit the query run endpoint, in particular when requested programmatically using API.

This commit introduces a rate limit to the `QueryController#run` endpoint. It heavily leans on the existing `RateLimiter` implementation, and the ability of `ApplicationController` to turn rate limit exceptions into nicely formatted JSON responses.

The rate limit (per 10 seconds) can be configured through the global setting `max_data_explorer_api_reqs_per_10_seconds`, and defaults to 2.

Handling can be configured through `max_data_explorer_api_req_mode`, and can be set to warn, block, or both warn and block. We will default to warn for now and monitor the logs for a while.
2023-04-03 13:46:35 +08:00
David Battersby f589864ce1
FIX: Flaky test fix for Data Explorer Automated PM Feature (#235) 2023-03-27 15:57:33 +08:00
Alan Guo Xiang Tan 2c418e7484
DEV: Skip flaky test (#234) 2023-03-27 12:37:19 +11:00
David Battersby 705753216c
FEATURE: Allow data explorer query result to be sent as recurring PM (#233)
This feature enables admins to create reports automatically based on a recurring schedule.

It introduces a new automation script that includes the new email_group_user field added to discourse-automation, along with a query_id and query_params to pass in parameters to the existing data explorer query.

The output of the report will be sent via pm (as a markdown table) to the recipients entered within the automation script.

The automation (supports individual users, email addresses and groups).
2023-03-24 16:38:42 +08:00
Bianca Nenciu 206d937a78
DEV: Update plugin to match latest guidelines (#229)
This commit updates the plugin to the latest guidelines, as shown in
discourse-plugin-skeleton, which involves moving a lot of the code to
dedicated files, use proper namespaces, use the autoloader as much as
possible, etc.
2023-03-22 23:29:08 +02:00
Martin Brennan 389b8e15c9
DEV: Fix broken plugin specs because of bookmarkable changes (#231)
Followup to 360d0dde650704a0f01fd6d8b525e933b1d7fcf2,
this causes other plugin tests to fail because
`DiscoursePluginRegistry.reset!` is
a shotgun. We can use the more surgical version
`DiscoursePluginRegistry.reset_register!(:bookmarkables)`
instead.
2023-03-08 10:39:43 +08:00
Martin Brennan a684897468
DEV: Change bookmarkable registration to use new plugin API (#230)
Relies on core PR https://github.com/discourse/discourse/pull/20556
to be merged first.
2023-03-08 10:39:20 +10:00
Bianca Nenciu 0f6b30c2d8
Revert "DEV: Update plugin to match latest guidelines (#227)" (#228)
This reverts commit 9210cf6fa2.
2023-02-23 19:10:46 +02:00
Bianca Nenciu 9210cf6fa2
DEV: Update plugin to match latest guidelines (#227)
- Define extension modules
- Use different files instead of plugin.rb
- Make sure plugin is disabled according to the setting
2023-02-23 15:29:13 +02:00
Krzysztof Kotlarek 466d4bb00c
FIX: clear state when reports are changed (#226)
When show report page is left, results should be cleared to not interfere with next report.
2023-02-22 16:25:33 +11:00
David Taylor 148d6c32a3
DEV: Introduce syntax_tree for ruby formatting (#208) 2022-12-29 13:31:29 +01:00
Sam 2313237a95
FIX: better handling of edge cases (#187)
- Require query name is present
- Ensure all routes are treated by default as .json, so errors flow correctly
- Remove superflous save/cancel controls from group settings
- Remove group control when item is destroyed
- Disable editing of query when it is deleted

Co-authored-by: Osama Sayegh <asooomaasoooma90@gmail.com>
2022-11-18 07:09:50 +11:00
Alan Guo Xiang Tan 4236689d27
FIX: Errors when running query due to PG template patterns or comments (#189)
Before this fix, the use of PG template patterns containing ":" or the
use of "?" in comments in the SQL will result in an error being raised
because `DB.param_encoder.encode` calls ActiveRecord's `sanitize_sql_array` which is
meant for SQL fragments and not an entire SQL string.

Instead we change data-explorer to use `MiniSql::InlineParamEncoder`
instead which takes into account of template patterns and does not trip
on `?` which is a special param encoding character used by ActiveRecord.
2022-10-10 09:03:37 +08:00
David Taylor 729e5a2add
DEV: Bump rubocop-discourse (#188)
* DEV: Bump rubocop-discourse

* Fix rubocop violations
2022-10-03 22:20:04 +02:00
Frank 16bb6a946c
FIX: Allow groups to access system queries (#185)
* FIX: allow groups to access system queries (without having to run the query once first)

Bug is: Trying to allow a group to access a system query results in a Discourse::NotFound unless the query is run first.

Cause:

 - System queries don't exist in the database by default
 - update calls set_query before action
 - set_query searches the database for the system query with Query.find_by(:id), which will not exist by default.
 - running system queries first fixes this because Query.find is overridden to include system queries (Queries.default) in its results, avoiding the Discourse::NotFound.

Solution: use the overridden Query.find in set_query to include system queries in the search, instead of Query.find_by(:id)

* Added test for fixing allowing groups to access system query.

* Fixed test formatting.
2022-08-19 00:14:07 +08:00
Martin Brennan a48b7d1ab5
DEV: Bookmarkable reminder refinement (#183)
Incorporates changes from core
 0ca1152c1c
2022-07-18 13:40:24 +10:00
Frank 45b6e7eb4f
FEATURE: Bookmarkable QueryGroups. (#177)
Adds the ability for non-Admin users to bookmark Queries from inside Group > Reports > Query view.
2022-06-14 23:07:02 +08:00
Jarek Radosz a9403561ef
DEV: Fix specs flakyness (#160) 2022-02-01 21:38:31 +01:00
Jarek Radosz bac43572cb
DEV: Clean up some specs code (#161) 2022-02-01 21:38:24 +01:00
Osama Sayegh 8464d517b2
DEV: Improve tests (#155) 2022-01-21 07:15:23 +03:00
Osama Sayegh ea66bcdc75
FEATURE: Add an API scope for running queries (#154) 2022-01-21 07:15:04 +03:00
Martin Brennan 70b973ea9a
FEATURE: Move query limit to hidden site setting (#153)
Previously the `QUERY_RESULT_DEFAULT_LIMIT` const was used
to limit the number of query results. This commit adds the
`data_explorer_query_result_limit` site setting which defaults
to 1000 and has a max of 10000 which matches the const
`QUERY_RESULT_MAX_LIMIT`.
2022-01-19 12:27:21 +10:00
David Taylor 216dff3ed9
DEV: Improve logic for showing/hiding the reports tab in group screens (#107)
Previously this was adding an extra AJAX request to check if the group had any queries available. Now a boolean is included in the group serializer, so there is no need for the extra request.

Removing this ajax request will also stop other plugin JS integration tests from failing when the data-explorer plugin is installed.

This commit also fixes the HTML markup of the tab, so that it doesn't have a <ul> nested inside the existing <ul>. Also adds an icon for good measure.
2021-04-08 17:47:44 +01:00
Arpit Jalan f70c95271a
FEATURE: destroy old hidden queries (#82)
This commit adds a scheduled job to hard delete queries that are hidden and

- were last run more than 7 days ago
- were updated more than 7 days ago
2020-11-25 22:09:05 +05:30
Krzysztof Kotlarek 8cc2d4e4e4
FIX: flaky fix_query_id_spec because id conflict (#73)
This rake task is responsible for fixing broken ids - move them back to the original values - ones before migration to a separate table.

Rake task can only fix queries with unique names.

In the test, we are creating duplicated queries and using `original_id` 4. Spec is ensuring that after rake tasks valuation, queries id is different than `4`

However, spec run in random order may create a query with id 4 and in that case, the spec would fail.

Therefore, when we create queries for test purpose, we should control what ids they are given to avoid conflict.
2020-10-13 12:31:51 +11:00
Roman Rizzi 51a047b60c
FIX: Store query groups in a temp table when fixing ids. (#68)
Reintroduces the migration removed in `cdfc5d4`, and fixes it to work with `query_groups`. Since the code is fairly complex, I moved into a rake task so it can be tested and make sure it works.
2020-09-28 12:23:53 -03:00
Guo Xiang Tan 353da45b6f
Fix failing test. 2020-08-27 09:36:51 +08:00
Krzysztof Kotlarek fe0806eb2a
FEATURE: introduce data-explorer tables (#61)
Instead of using `PluginStoreRow` we should use plugin-specific models like `DataExplorer::Query` and `DataExplorer::QueryGroup`
2020-08-27 10:29:57 +10:00