Commit Graph

205 Commits

Author SHA1 Message Date
David Taylor 03818e642a
FEATURE: Include optimized thumbnails for topics (#9215)
This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this:

- Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact

- A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake

- Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued

- Topic lists and topics now include a `thumbnails` key, which includes all the available images:
   ```
   "thumbnails": [
   {
     "max_width": null,
     "max_height": null,
     "url": "//example.com/original-image.png",
     "width": 1380,
     "height": 1840
   },
   {
     "max_width": 1024,
     "max_height": 1024,
     "url": "//example.com/optimized-image.png",
     "width": 768,
     "height": 1024
   }
   ]
  ```

- Themes can request additional thumbnail sizes by using a modifier in their `about.json` file:
   ```
    "modifiers": {
      "topic_thumbnail_sizes": [
        [200, 200],
        [800, 800]
      ],
      ...
  ```
  Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated

- Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 09:07:50 +01:00
Joffrey JAFFEUX 01f2819dde
DEV: enables and fixes multisite-spec (#9557) 2020-04-27 20:55:36 +02:00
Dan Ungureanu 0653750fbf
FEATURE: Hash user API keys in the database (#9344)
The 'key' column will be dropped in a future commit.
2020-04-07 16:42:52 +03:00
Blake Erickson d04ba4b3b2
DEPRECATION: Remove support for api creds in query params (#9106)
* DEPRECATION: Remove support for api creds in query params

This commit removes support for api credentials in query params except
for a few whitelisted routes like rss/json feeds and the handle_mail
route.

Several tests were written to valid these changes, but the bulk of the
spec changes are just switching them over to use header based auth so
that they will pass without changing what they were actually testing.

Original commit that notified admins this change was coming was created
over 3 months ago: 2db2003187

* fix tests

* Also allow iCalendar feeds

Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>
2020-04-06 16:55:44 -06:00
David Taylor 19814c5e81
FIX: Allow CSP to work correctly for non-default hostnames/schemes (#9180)
- Define the CSP based on the requested domain / scheme (respecting force_https)
- Update EnforceHostname middleware to allow secondary domains, add specs
- Add URL scheme to anon cache key so that CSP headers are cached correctly
2020-03-19 19:54:42 +00:00
Jarek Radosz 29b35aa64c
DEV: Improve flaky time-sensitive specs (#9141) 2020-03-10 22:13:17 +01:00
Bianca Nenciu 3ec2081059
UX: Include public groups in mentionable groups set (#8516) 2019-12-12 13:13:40 +02:00
Neil Lalonde 875f0d8fd8
FEATURE: Tag synonyms
This feature adds the ability to define synonyms for tags, and the ability to merge one tag into another while keeping it as a synonym. For example, tags named "js" and "java-script" can be synonyms of "javascript". When searching and creating topics using synonyms, they will be mapped to the base tag.

Along with this change is a new UI found on each tag's page (for example, `/tags/javascript`) where more information about the tag can be shown. It will list the synonyms, which categories it's restricted to (if any), and which tag groups it belongs to (if tag group names are public on the `/tags` page by enabling the "tags listed by group" setting). Staff users will be able to manage tags in this UI, merge tags, and add/remove synonyms.
2019-12-04 13:33:51 -05:00
David Taylor a6aada16bd DEV: Refactor API key specs to avoid hard-coding keys
By hard-coding keys, we are not testing the API key system end to end. This change also makes the specs more resilient to upcoming API key changes
2019-11-29 15:16:22 +00:00
Neil Lalonde 228c4814be FIX: errors when using tags with colons in their name 2019-11-18 13:20:37 -05:00
Neil Lalonde 7711df40e6
REFACTOR: redo DiscourseTagging.filter_allowed_tags (#8328)
This method had grown into a monster. Its query had bugs
that I couldn't fix, and new features would be hard to add.
Also I don't understand how it all works anymore...
Replace it with common table expressions that can be queried
to generate the results we need, instead of subtracting
results using lots of "NOT IN" clauses.

Fixed are bugs with tag schemas that use combinations of
tag groups, parent tags, and one-tag-per-topic restrictions.
For example: https://meta.discourse.org/t/130991/6
2019-11-12 14:28:44 -05:00
Neil Lalonde b6d140e4bd UX: tag input suggests required tags if none have been selected
This is a follow-up to the new feature that allows a category to
require a certain number of tags from a tag group. The tag input will
shows results from the required group if none have been chosen yet.
Once a require tag is selected, the tag input will include other
results as usual. Staff users can ignore this restriction, so the input
behaviour is unchanged for them.
2019-11-04 16:51:54 -05:00
Neil Lalonde cfe26eb301 FIX: backwards compatibility for uncompiled email style css 2019-10-23 19:22:33 -04:00
Neil Lalonde f061aee818 FEATURE: support SCSS in custom email style
In the CSS tab of Admin > Customize > Email Style, SCSS can now be used.
2019-10-23 15:42:37 -04:00
Neil Lalonde 5ef49692e0 FIX: tag cannot be used if it belongs to two tag groups with parent tag
If two tag groups exist with a mandatory parent tag, and one tag is
added to both tag groups, then the tag couldn't be used on any topics.
2019-10-16 14:28:04 -04:00
Krzysztof Kotlarek 427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
David Taylor 92f2202e4a SECURITY: Restrict message-bus access on login_required sites 2019-08-14 09:43:12 +01:00
Sam Saffron 1f47ed1ea3 PERF: message_bus will be deferred by server when flooded
The message_bus performs a fair amount of work prior to hijacking requests
this change ensures that if there is a situation where the server is flooded
message_bus will inform client to back off for 30 seconds + random(120 secs)

This back-off is ultra cheap and happens very early in the middleware.

It corrects a situation where a flood to message bus could cause the app
to become unresponsive

MessageBus update is here to ensure message_bus gem properly respects
Retry-After header and status 429.

Under normal state this code should never trigger, to disable raise the
value of DISCOURSE_REJECT_MESSAGE_BUS_QUEUE_SECONDS, default is to tell
message bus to go away if we are queueing for 100ms or longer
2019-08-09 17:48:01 +10:00
Neil Lalonde 9656a21fdb
FEATURE: customization of html emails (#7934)
This feature adds the ability to customize the HTML part of all emails using a custom HTML template and optionally some CSS to style it. The CSS will be parsed and converted into inline styles because CSS is poorly supported by email clients. When writing the custom HTML and CSS, be aware of what email clients support. Keep customizations very simple.

Customizations can be added and edited in Admin > Customize > Email Style.

Since the summary email is already heavily styled, there is a setting to disable custom styles for summary emails called "apply custom styles to digest" found in Admin > Settings > Email.

As part of this work, RTL locales are now rendered correctly for all emails.
2019-07-30 15:05:08 -04:00
Gerhard Schlager abe6202af9 DEV: Fix heisentest 2019-07-23 18:04:53 +02:00
Osama Sayegh f14c6d81f4
FEATURE: Watched words improvements (#7899)
This commit contains 3 features:

- FEATURE: Allow downloading watched words
This introduces a button that allows admins to download watched words per action in a `.txt` file.

- FEATURE: Allow clearing watched words in bulk
This adds a "Clear All" button that clears all deleted words per action (e.g. block, flag etc.)

- FEATURE: List all blocked words contained in the post when it's blocked
When a post is rejected because it contains one or more blocked words, the error message now lists all the blocked words contained in the post.

-------

This also changes the format of the file for importing watched words from `.csv` to `.txt` so it becomes inconsistent with the extension of the file when watched words are exported.
2019-07-22 14:59:56 +03:00
Guo Xiang Tan a4234e9be0 DEV: Minor tweaks to `Admin::WatchedWordsController`. 2019-07-15 10:22:46 +08:00
Sam Saffron bb8dfb124e DEV: change testing cache clearing so it is more deliberate
This test is flaky, this changes it so we always start tests with a clean
cache (concurrency and redis delays pending)
2019-06-03 10:21:38 +10:00
Robin Ward 89b84651c3 Migrate score settings to use sensitivities
We hide scores so these settings no longer made sense.
2019-05-24 15:44:24 -04:00
David Taylor b5dd4478e5
FIX: Blocked watched words should apply to staff (#7547)
Having different behavior for staff and regular users can make it confusing for admins to understand how their configuration changes affect regular users
2019-05-16 15:19:41 +01:00
Sam Saffron 88650a1259 PERF: avoid checking for consecutive replies in test
This check can issue up to 2 queries per post created, we have specific
tests for it so we can avoid.

This also rolls back #4da6ca4d
2019-05-09 13:28:53 +10:00
Daniel Waterworth 4da6ca4d9f PERF: Skip post validation by default when fabricating posts (#7508)
This speeds up tests by 10%
2019-05-09 10:15:38 +10:00
Daniel Waterworth e219588142 DEV: Prefabrication (test optimization) (#7414)
* Introduced fab!, a helper that creates database state for a group

It's almost identical to let_it_be, except:

 1. It creates a new object for each test by default,
 2. You can disable it using PREFABRICATION=0
2019-05-07 13:12:20 +10:00
Sam Saffron 4ea21fa2d0 DEV: use #frozen_string_literal: true on all spec
This change both speeds up specs (less strings to allocate) and helps catch
cases where methods in Discourse are mutating inputs.

Overall we will be migrating everything to use #frozen_string_literal: true
it will take a while, but this is the first and safest move in this direction
2019-04-30 10:27:42 +10:00
Sam Saffron 45285f1477 DEV: remove update_attributes which is deprecated in Rails 6
See: https://github.com/rails/rails/pull/31998

update_attributes is a relic of the past, it should no longer be used.
2019-04-29 17:32:25 +10:00
Daniel Waterworth ad44243a57 Removed unused let blocks (#7446)
The bodies of these blocks were never evaluated.
2019-04-29 15:08:56 +08:00
Neil Lalonde c2a8a2bc97 FIX: if mandatory parent tag is missing, add it
Previous behaviour was to silently remove tags that
belonged to a group with a parent tag that was missing.

The "required parent tag" feature is meant to guide people
to use the correct tags and show scoped results in the tag
input field, and to help create topic lists of related
tags. It isn't meant to be a strict requirement in the
composer that should trigger errors or restrictions.
2019-04-26 14:46:11 -04:00
Robin Ward 87de5d9e8f Record the reason that a post was put into the queue 2019-04-10 17:42:49 -04:00
Neil Lalonde a0f7953f0c remove extra lines 2019-04-04 17:03:57 -04:00
Neil Lalonde 47eefb9f37 remove wip comments 2019-04-04 16:47:02 -04:00
Neil Lalonde 83996fc8ea FEATURE: ability to restrict some tags to a category while allowing all others
A new checkbox has been added to the Tags tab of the category settings modal
which is used when some tags and/or tag groups are restricted to the category,
and all other unrestricted tags should also be allowed.
Default is the same as the previous behaviour: only allow the specified set of
tags and tag groups in the category.
2019-04-04 16:40:15 -04:00
Robin Ward c1ea63bdc1 FIX: Reviewables should not be created for users until they are active
Conversely, if a user is deactivated the reviewable should automatically
be rejected.

Before this fix, if a user was not active they'd still show in the
review queue but without an "Approve" button which was confusing.
2019-04-03 15:25:00 -04:00
Robin Ward b58867b6e9 FEATURE: New 'Reviewable' model to make reviewable items generic
Includes support for flags, reviewable users and queued posts, with REST API
backwards compatibility.

Co-Authored-By: romanrizzi <romanalejandro@gmail.com>
Co-Authored-By: jjaffeux <j.jaffeux@gmail.com>
2019-03-28 12:45:10 -04:00
Sam Saffron 9ebabc1de8 FEATURE: unconditionally update Topic updated_at when posts change in topic
Previously we would bypass touching `Topic.updated_at` for whispers and post
recovery / deletions.

This meant that certain types of caching can not be done where we rely on
this information for cache accuracy.

For example if we know we have zero unread topics as of yesterday and whisper
is made I need to bump this date so the cache remains accurate

This is only half of a larger change but provides the groundwork.

Confirmed none of our serializers leak out Topic.updated_at so this is safe
spot for this info

At the moment edits still do not change this but it is not relevant for the
unread cache.

This commit also cleans up some specs to use the new `eq_time` matcher for
millisecond fidelity comparison of times

Previously `freeze_time` would fudge this which is not that clean.
2019-03-28 17:28:01 +11:00
Neil Lalonde 1812a38f0a FIX: upload watched words should use UTF-8 2019-03-21 13:46:16 -04:00
Robin Ward fa5a158683 REFACTOR: Move `queue_jobs` out of `SiteSetting`
It is not a setting, and only relevant in specs. The new API is:

```
Jobs.run_later!        # jobs will be thrown on the queue
Jobs.run_immediately!  # jobs will run right away, avoid the queue
```
2019-03-14 10:47:38 -04:00
Robin Ward d1d9a4f128 Add new `run_jobs_synchronously!` helper for tests
Previously if you wanted to have jobs execute in test mode, you'd have
to do `SiteSetting.queue_jobs = false`, because the opposite of queue
is to execute.

I found this very confusing, so I created a test helper called
`run_jobs_synchronously!` which is much more clear about what it does.
2019-03-11 16:58:35 -04:00
Robin Ward bc3efab816 FIX: When disagreeing with a flag that silenced a user, unsilence them
Previously it would unhide their post but leave them silenced.

This fix also cleans up some of the helper classes to make it easier
to pass extra data to the silencing code (for example, a link to the
post that caused the user to be silenced.)

This patch also refactors the auto_silence specs to avoid using
stubs.
2019-02-08 08:50:50 -05:00
Bianca Nenciu 860c1c3dcd FEATURE: Automatically expire keys if not used for a configurable amount of time. (#6264) 2018-08-20 17:36:14 +02:00
Régis Hanol 93201d8dbe FIX: don't trigger 'flag_reviewed' when no flags were reviewed 2018-08-16 18:11:29 +02:00
Guo Xiang Tan 7fc8a36529 DEV: Take 2 Queue jobs in tests by default.
On my machine this cuts the time taken to run our test suite
from ~11mins to ~9mins.
2018-05-31 16:23:23 +08:00
Guo Xiang Tan 56e9ff6853 Revert "DEV: Queue jobs in tests by default."
Too risky for now

This reverts commit be28154d3b.
2018-05-31 15:34:46 +08:00
Guo Xiang Tan be28154d3b DEV: Queue jobs in tests by default. 2018-05-31 14:45:47 +08:00
Neil Lalonde 6ca71e1319 FIX: when creating a topic in a category that only allows tags from a tag group, don't allow creation of new tags 2018-03-19 11:42:10 -04:00
Neil Lalonde 58508e553d FIX: tag input should not include tags you've already chosen in the search results 2018-03-13 17:17:16 -04:00
Kyle Zhao f7bd05e534 FEATURE: set 'Retry-After' header for 429 responses (#5659) 2018-03-13 23:12:41 +08:00
Neil Lalonde baf1c385eb UX: when a post is blocked due to a watched word, message includes the word being blocked 2018-02-28 11:22:18 -05:00
Sam 38f4acd55a FIX: rate limiter text is confusing, should not say daily
Also, adds easily parseable JSON so users can figure out
how long to wait when the API is limited. ("extras" "wait_seconds")
2018-02-14 15:29:50 +11:00
Neil Lalonde 76c309fe6b PERF: a faster way to count tags used per category 2018-02-12 15:16:47 -05:00
Robin Ward 569e57f0a9 FIX: Delete the invalid auth cookie even if you hit the rate limit 2018-02-09 19:09:54 -05:00
Sam 49ed382c2a FIX: return 429 when admin api key is limited on admin route
This also handles a general case where exceptions leak out prior to being handled by the application controller
2018-01-12 14:15:26 +11:00
Sam 63bdc4056d More log suppressions for rate limit exceeded 2017-12-11 17:52:57 +11:00
Robin Ward 1f14350220 Rename "Blocked" to "Silenced" 2017-11-10 14:10:27 -05:00
Joffrey JAFFEUX 73aa7edb8b
Temporarily skip multisite spec 2017-11-09 15:38:34 -08:00
Sam 86e6732f78 FEATURE: update rails multisite
always allow /srv/status through even if host does not match
2017-11-09 15:40:56 +11:00
Neil Lalonde c53f41f4f2 FIX: don't flag watched words when rebaking posts 2017-10-12 15:34:22 -04:00
Guo Xiang Tan 77d4c4d8dc Fix all the errors to get our tests green on Rails 5.1. 2017-09-25 13:48:58 +08:00
Guo Xiang Tan 8779490ce4 Move new controller specs to reqeusts folder. 2017-08-24 12:01:11 +09:00
Guo Xiang Tan f7d3702454 FIX: Return 404 if API access is invalid. 2017-08-10 18:27:01 +09:00
Guo Xiang Tan 90d7dd1f05 FIX: Ensure that post action moderation post uses the site's default locale.
https://meta.discourse.org/t/a-post-in-looking-for-someone-to-customize-discourse-to-create-a-forum-site-requires-staff-attention/67468/5?u=tgxworld
2017-08-09 18:20:20 +09:00
Guo Xiang Tan a9613163b5 FEATURE: Force user to enter reason when requesting for group membership. 2017-08-09 15:45:28 +09:00
Guo Xiang Tan b6ce93a965 Add specs for 1c80c233cf 2017-08-02 20:58:35 +09:00
Guo Xiang Tan 4620dfe92d FEATURE: Add group settngs to allow users to leave a group freely.
https://meta.discourse.org/t/split-join-leave-freely-setting-on-groups/65565
2017-07-28 15:00:25 +09:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Neil Lalonde 68b3dd43ce fix intermittent failing tests, some watched word refactoring 2017-07-27 12:27:01 -04:00
Guo Xiang Tan 2442bba131 UX: Better group creation workflow.
* Owners and users can now be added to a group during creation.

https://meta.discourse.org/t/you-cannot-allow-membership-requests-without-any-owners/64760/3
2017-07-27 16:12:42 +09:00
Neil Lalonde 24cb950432 FEATURE: Watched Words: when posts contain words, do one of flag, require approval, censor, or block 2017-07-26 11:01:09 -04:00
Sam Saffron 045a2abcec FEATURE: remove the timecop gem
We should only have one way of mocking time, misuse of timecop
was causing build stability issues
2017-07-24 12:11:10 -04:00
Guo Xiang Tan 1b0750d7ef Merge pull request #4983 from tgxworld/group_owners_can_invite_users_to_groups
Group owners can invite users to groups
2017-07-24 16:21:19 +09:00
Guo Xiang Tan fa09a02201 Fix specs. 2017-07-24 15:56:08 +09:00
Guo Xiang Tan 2a17f1ccd7 FIX: Group owners should be able to invite users to their groups.
https://meta.discourse.org/t/group-owner-cannot-send-an-invite-to-a-group/60617/12
2017-07-21 23:48:25 +09:00
Leo McArdle d0b027d88d FEATURE: phase 1 of supporting multiple email addresses 2017-07-20 11:22:27 +09:00
Sam 845170bd6b FEATURE: add support for group visibility level
There are 4 visibility levels

- public (default)
- members only
- staff
- owners

Note, admins and group owners ALWAYS have visibility to groups

Migration treated old "non public" as "members only"
2017-07-03 15:26:57 -04:00
Guo Xiang Tan b5249fb4ca FIX: Send request membership PM to last 5 active group owner. 2017-06-15 11:37:09 +08:00
Guo Xiang Tan a5d3abc9b6 FIX: Create group membership request on behalf of user. 2017-06-13 17:49:21 +09:00
Régis Hanol 038454bde2 FIX: always confirm emails when SSO says so 2017-06-08 01:05:33 +02:00
Guo Xiang Tan 2ee144c27f FEATURE: Add DiscourseEvent trigger when a user logs in.
* Also adds a event trigger when user logs in for the first time.
2017-06-01 17:44:49 +09:00
Neil Lalonde 7821400141 FEATURE: staff can set a timer to remind them about a topic 2017-05-16 14:49:50 -04:00
Guo Xiang Tan 8eecd42856 FIX: Delete topic timer after completion. 2017-05-12 10:28:51 +08:00
Neil Lalonde 55b61e9bea rename topic_status_update to topic_timer 2017-05-11 18:27:53 -04:00
Guo Xiang Tan 55d3547540 Fix random test failure. 2017-05-09 11:27:05 +08:00
Robin Ward fc00032ec8 Check for 404 from spec rather than failure 2017-05-08 16:33:43 -04:00
Robin Ward 777f1f0f47 FIX: Return a 404 if the auth session is not present 2017-05-04 15:35:24 -04:00
Sam 4b6c49b13f correct specs 2017-04-14 15:25:34 -04:00
Guo Xiang Tan c076f7b1aa Fix broken specs. 2017-04-11 10:43:34 +08:00
Guo Xiang Tan f4758a4c4d FEATURE: Allow admins to schedule a topic to be published in the future. 2017-04-04 11:16:05 +08:00
Guo Xiang Tan dc5a6e7cda Remove empty test. 2017-04-03 21:12:20 +08:00
Guo Xiang Tan b6e9871b4b Update `Topic#closed` client side when closing/opening a topic temporarily. 2017-03-31 15:05:00 +08:00
Guo Xiang Tan 34b7bee568 FEATURE: Allow admin to auto reopen at topic.
* This commit also introduces a `TopicStatusUpdate`
  model to support other forms of deferred topic
  status update in the future.
2017-03-31 11:14:18 +08:00
Robin Ward 14410b71fb Convert server side paths to use `/u/` 2017-03-30 10:23:24 -04:00
Guo Xiang Tan 7cf0f39066 Require `Sidekiq::Testing` in rails helper. 2017-03-29 11:10:25 +08:00
Guo Xiang Tan 3ef82bb32c SECURITY: CSRF vulnerabilities in `Admin::BackupsController`. 2017-03-23 10:29:35 +08:00
Guo Xiang Tan 1a7e954e09 FIX: Store custom emojis as uploads.
* Depending on a hardcoded directory was a flawed design
  which made it impossible to debug when custom emojis go
  missing.
2017-03-14 13:07:18 +08:00
Guo Xiang Tan 3d347fb9c4 FIX: Don't mark user as `active` if verified email is different. 2017-03-02 14:24:30 +08:00
Guo Xiang Tan 0e5d490b05 No need for special helper to reset SiteSetting state.
* SiteSetting in tests uses a local provider that resets it.
2017-01-28 10:55:49 +08:00