The auto restart logic was sending a USR2 to the parent process without checking what the parent process actually was. In some situations, it might not be the `bin/unicorn` supervisor.
This commit switches to use a global variable for the supervisor PID. This will be much less prone to unexpected behavior.
* DEV: Allow wildcards in Oneboxer optional domain Site Settings
Allows a wildcard to be used as a subdomain on Oneboxer-related SiteSettings, e.g.:
- `force_get_hosts`
- `cache_onebox_response_body_domains`
- `force_custom_user_agent_hosts`
* DEV: fix typos
* FIX: Try doing a GET after receiving a 500 error from a HEAD
By default we try to do a `HEAD` requests. If this results in a 500 error response, we should try to do a `GET`
* DEV: `force_get_hosts` should be a hidden setting
* DEV: Oneboxer Strategies
Have an alternative oneboxing ‘strategy’ (i.e., set of options) to use when an attempt to generate a Onebox fails. Keep track of any non-default strategies that were used on a particular host, and use that strategy for that host in the future.
Initially, the alternate strategy (`force_get_and_ua`) forces the FinalDestination step of Oneboxing to do a `GET` rather than `HEAD`, and forces a custom user agent.
* DEV: change stubbed return code
The stubbed status code needs to be a value not recognized by FinalDestination
We have a few places in the code where we need to validate various email related settings, and will have another soon with the improved group email settings UI. This PR introduces a class which can validate POP3, IMAP, and SMTP credentials and also provide a friendly error message for issues if they must be presented to an end user.
This PR does not change any existing code to use the new service. I have added a TODO to change POP3 validation and the email test rake task to use the new validator post-release.
* FIX: Ensure the same email cannot be invited twice
When creating a new invite with a duplicated email, the old invite will
be updated and returned. When updating an invite with a duplicated email
address, an error will be returned.
* FIX: not Ember helper does not exist
* FIX: Sync can_invite_to_forum? and can_invite_to?
The two methods should perform the same basic set of checks, such as
check must_approve_users site setting.
Ideally, one of the methods would call the other one or be merged and
that will happen in the future.
* FIX: Show invite to group if user is group owner
This commit adds a listener on (almost) all `.rb` files in the repository. When a change occurs, it checks whether Zeitwerk is responsible for autoloading it. If not, a warning will be printed to the console and the server will be automatically restarted. Optionally, you can pass the `AUTO_RESTART=0` environment variable to prevent auto-restart.
Currently, when the target is not available we're returning the error message "`You are not permitted to view the requested resource`" which is not clear.
The user may have changed their category or tag tracking settings since a topic was tracked/watched based on those settings in the past. In that case we need to alter the reason message we show them otherwise it is very confusing for the end user to be told they are tracking a topic because of a category, when they are no longer tracking that category.
For example: "You will see a count of new replies because you are tracking this category." becomes: "You will see a count of new replies because you were tracking this category in the past."
To do this, it was necessary to add tag and category tracking info to current user serializer. I improved the serializer code so it only does 3 SQL queries instead of 9 to get the tracking information for tags and categories for the current user.
* FEATURE: add support for like webhooks
Add support for like webhooks. Webhook events only send on user membership
in the defined webhook group filters.
This also fixes group webhook events, as before this was never used, and
the logic was not correct.
This commit makes a few changes to improve boot time in development environments. It will have no effect on production boot times.
- Skip the SchemaCache warmup. In development mode, the SchemaCache is refreshed every time there is a code change, so warmup is of limited use.
- Skip warming up PrettyText. This adds ~2s to each web worker's boot time. The vast majority of requests do not use PrettyText, so it is more efficient to defer its warmup until it's needed
- Skip the intentional 1 second pause during Unicorn worker forking. The comment (which also exists in Unicorn's documentation) suggests this works around a Unix signal handling bug, but I haven't been able to locate any more information. Skipping it in dev will significantly speed up boot. If we start to see issues, we can revert this change.
On my machine, this improves `/bin/unicorn` boot time from >10s to ~4s
In development we regularly restart/reload Rails, which wipes out the schema cache. This then has to be regenerated using DDL queries on the database.
Instead, we can make use of the `rake db:schema:cache:dump` command. This will dump the schema cache to a YAML file, and then load it when needed. This is significantly faster than rebuilding the cache from DDL queries every time.
On every request, Rails checks to see whether any ruby code has been changed on the filesystem. The default FileUpdateChecker does this by iterating over every file on the autoload_paths and comparing its modified-time.
In Discourse, our autoload path of `/app` includes the `/app/assets` directory, and therefore thousands of non-ruby files (e.g. node_modules). This makes the `Dir["/app"]` call very slow (>100ms in my case). On my machine, every Rails-handled request spends around 150-200ms in the FileUpdateChecker. This commit introduces a couple of changes to completely eliminate this wasted time:
- The `/app/assets` directory is excluded from the file watchers. For me, this cut the time spent in the file_watcher to around 50-100ms
- Switches our development config to use the `EventedFileUpdateChecker`, which makes use of the `listen` gem to subscribe to os-specific notifications of changes. This completely removes the `FileUpdateChecker` from the critical path
On my machine, topic_list requests now return in around 80ms (previously >200ms). Live code reload still works as it did before
This commit allows site admins to run theme tests in production via a new `/theme-qunit` route. When you visit `/theme-qunit`, you'll see a list of the themes/components installed on your site that have tests, and from there you can select a theme or component that you run its tests.
We also have a new rake task `themes:install_and_test` that can be used to install a list of themes/components on a temporary database and run the tests of the themes/components that are installed. This rake task can be useful when upgrading/deploying a Discourse instance to make sure that the installed themes/components are compatible with the new Discourse version being deployed, and if the tests fail you can abort the build/deploy process so you don't end up with a broken site.
This setting allows admin to de/activate automatic trimming of incoming email.
There are instances where it does wonders in trimming all the garbage content and other
instances where it's so bad that it trims the most important part of the email.
FIX: don't remove hidden content using the style attribute when converting HTML to Markdown.
The regexp used was doing more harm than good. It was way too broad.
FIX: properly elide signatures from emails sent with Front App.
This is fairly safe as Front App nicely identifies signatures in the HTML part.
If the "use_site_small_logo_as_system_avatar" setting is enabled, the site's small logo is displayed as the selected option by the avatar-selector. Choosing a different avatar disables the setting.
When the admin creates a new custom field they can specify if that field should be searchable or not.
That setting is taken into consideration for quick search results.
Adds a webhook to notify when a reviewable score is updated.
This is different from created or status changed as additional flags can
roll in and update the score without updating status. Useful for applications
looking to integrate in with Discourse's scores
This commit allows site admins to run theme tests in production via a new `/theme-qunit` route. When you visit `/theme-qunit`, you'll see a list of the themes/components installed on your site that have tests, and from there you can select a theme or component that you run its tests.
We also have a new rake task `themes:install_and_test` that can be used to install a list of themes/components on a temporary database and run the tests of the themes/components that are installed. This rake task can be useful when upgrading/deploying a Discourse instance to make sure that the installed themes/components are compatible with the new Discourse version being deployed, and if the tests fail you can abort the build/deploy process so you don't end up with a broken site.
This filter hides reviewables with a score lower than the "reviewable_low_priority_threshold" setting. We only use reviewables that already met this threshold to calculate the Medium and High priority filters.
The old share modal used to host both share and invite functionality,
under two tabs. The new "Share Topic" modal can be used only for
sharing, but has a link to the invite modal.
Among the sharing methods, there is also "Notify" which points out
that existing users will simply be notified (this was not clear
before). Staff members can notify as many users as they want, but
regular users are restricted to one at a time, no more than
max_topic_invitations_per_day. The user will not receive another
notification if they have been notified of the same topic in past hour.
The "Create Invite" modal also suffered some changes: the two radio
boxes for selecting the type (invite or email) have been replaced by a
single checkbox (is email?) and then the two labels about emails have
been replaced by a single one, some fields were reordered and the
advanced options toggle was moved to the bottom right of the modal.
Not all videos can be rendered everywhere because some browser may be
missing some codecs. This commit adds a notice on top of video to let
the user know about it.
The "last custom date and time" shortcut for the topic timer and
bookmarks could get into a state where it had an Invalid Date if
the user opened the topic timer modal, clicked Custom Date and then
closed the modal without making changes. This has been fixed, the
last custom date + time will no longer be set in this case and if
somehow the last custom date + time is invalid that option will not
show.
Also improve the wording from just "Last" to "Last custom datetime"
It's important that we don't perform pg_dumps against databases
running behind pgbouncer.
We had an old monkey-patch to prevent this, but following some [recent
internal rails refactoring](5488686851),
the patch no longer works. Instead, we can use the official
`config.active_record.dump_schema_after_migration` option.
Setting this to false in production is recommended by Rails, and is the
default for newly generated Rails applications.
Previously watched words ignored topic titles when applying auto tagging rules.
Also copy has been improved to reflect how the system behaves.
The text hints that we are only watching first post now
* FEATURE: Review every post using the review queue.
If the `review_every_post` setting is enabled, posts created and edited by regular uses are sent to the review queue so staff can review them. We'll skip PMs and posts created or edited by TL4 or staff users.
Staff can choose to:
- Approve the post (nothing happens)
- Approve and restore the post (if deleted)
- Approve and unhide the post (if hidden)
- Reject and delete it
- Reject and keep deleted (if deleted)
- Reject and suspend the user
- Reject and silence the user
* Update config/locales/server.en.yml
Co-authored-by: Robin Ward <robin.ward@gmail.com>
Co-authored-by: Robin Ward <robin.ward@gmail.com>
* soften language around daily limits and add daily
let's try to be as gentle as we can with new users
* better copy suggestion from Blake
* further copyedit improvements
There is a category setting that enforces 1 or more tags must be added to a topic from a specific tag group before creating it. This validation was not being run before the topic was being sent to a review queue for categories that have that setting enabled.
There was an existing validation in `TopicCreator` but it was not correct; it was only validating when the tags did _not_ exist and also only happened on `create`. I now run the validation in `TopicCreator.valid?`
I also improved the error message shown to the user when they have not added the tags required (showing the tag names from the tag group), and changed the composer tag selector to not show "optional" if there are N tags required from a certain group.
The server used to respond with a generic 'error, contact admin' message
which did not offer any hint what the error was. This happened even when
the error could be easily corrected by the user (for example, if they
chose a very common password).
This PR adds a new category setting which is a column in the `categories` table, `allow_unlimited_owner_edits_on_first_post`.
What this does is:
* Inside the `can_edit_post?` method of `PostGuardian`, if the current user editing a post is the owner of the post, it is the first post, and the topic's category has `allow_unlimited_owner_edits_on_first_post`, then we bypass the check for `LimitedEdit#edit_time_limit_expired?` on that post.
* Also, similar to wiki topics, in `PostActionNotifier#after_create_post_revision` we send a notification to all users watching a topic when the OP is edited in a topic with the category setting `allow_unlimited_owner_edits_on_first_post` enabled.
This is useful for forums where there is a Marketplace or similar category, where topics are created and then updated indefinitely by the OP rather than the OP making new topics or additional replies. In a way this acts similar to a wiki that only one person can edit.
* Fixes the z-index of the prompt so it is behind the quick access panels
* Adds a dismiss `X` button (made sure the click target of this was quite big)
* Change structure of HTML to address template lint issues
* Fix aria-hidden not returning true/false
* Reload current page instead of navigating to / when clicking on the prompt message
This commit allows themes and theme components to have QUnit tests. To add tests to your theme/component, create a top-level directory in your theme and name it `test`, and Discourse will save all the files in that directory (and its sub-directories) as "tests files" in the database. While tests files/directories are not required to be organized in a specific way, we recommend that you follow Discourse core's tests [structure](https://github.com/discourse/discourse/tree/master/app/assets/javascripts/discourse/tests).
Writing theme tests should be identical to writing plugins or core tests; all the `import` statements and APIs that you see in core (or plugins) to define/setup tests should just work in themes.
You do need a working Discourse install to run theme tests, and you have 2 ways to run theme tests:
* In the browser at the `/qunit` route. `/qunit` will run tests of all active themes/components as well as core and plugins. The `/qunit` now accepts a `theme_name` or `theme_url` params that you can use to run tests of a specific theme/component like so: `/qunit?theme_name=<your_theme_name>`.
* In the command line using the `themes:qunit` rake task. This take is meant to run tests of a single theme/component so you need to provide it with a theme name or URL like so: `bundle exec rake themes:qunit[name=<theme_name>]` or `bundle exec rake themes:qunit[url=<theme_url>]`.
There are some refactors to how Discourse processes JavaScript that comes with themes/components, and these refactors may break your JS customizations; see https://meta.discourse.org/t/upcoming-core-changes-that-may-break-some-themes-components-april-12/186252?u=osama for details on how you can check if your themes/components are affected and what you need to do to fix them.
This commit also improves theme error handling in Discourse. We will now be able to catch errors that occur when theme initializers are run and prevent them from breaking the site and other themes/components.
We introduced a cap on the number of bookmarks the user can add in be145ccf2f. However this has caused unintended side effects; when the `jobs/scheduled/bookmark_reminder_notifications.rb` runs we get this error for users who already had more bookmarks than the limit:
> Job exception: Validation failed: Sorry, you have too many bookmarks, visit #{url}/my/activity/bookmarks to remove some.
This is because the `clear_reminder!` call was triggering a bookmark validation, which raised an error because the user already had to many, holding up other reminders.
This PR also adds `max_bookmarks_per_user` hidden site setting (default 2000). This replaces the BOOKMARK_LIMIT const so we can raise it for certain sites.
To add an extra layer of security, we sanitize settings before shipping them to the client. We don't sanitize those that have the "html" type.
The CookedPostProcessor already uses Loofah for sanitization, so I chose to also use it for this. I added it to our gemfile since we installed it as a transitive dependency.
This commit allows themes and theme components to have QUnit tests. To add tests to your theme/component, create a top-level directory in your theme and name it `test`, and Discourse will save all the files in that directory (and its sub-directories) as "tests files" in the database. While tests files/directories are not required to be organized in a specific way, we recommend that you follow Discourse core's tests [structure](https://github.com/discourse/discourse/tree/master/app/assets/javascripts/discourse/tests).
Writing theme tests should be identical to writing plugins or core tests; all the `import` statements and APIs that you see in core (or plugins) to define/setup tests should just work in themes.
You do need a working Discourse install to run theme tests, and you have 2 ways to run theme tests:
* In the browser at the `/qunit` route. `/qunit` will run tests of all active themes/components as well as core and plugins. The `/qunit` now accepts a `theme_name` or `theme_url` params that you can use to run tests of a specific theme/component like so: `/qunit?theme_name=<your_theme_name>`.
* In the command line using the `themes:qunit` rake task. This take is meant to run tests of a single theme/component so you need to provide it with a theme name or URL like so: `bundle exec rake themes:qunit[name=<theme_name>]` or `bundle exec rake themes:qunit[url=<theme_url>]`.
There are some refactors to internal code that's responsible for processing themes/components in Discourse, most notably:
* `<script type="text/discourse-plugin">` tags are automatically converted to modules.
* The `theme-settings` service is removed in favor of a simple `lib` file responsible for managing theme settings. This was done to allow us to register/lookup theme settings very early in our Ember app lifecycle and because there was no reason for it to be an Ember service.
These refactors should 100% backward compatible and invisible to theme developers.
This moves the "This site was just updated" modal asking the user if they want to refresh into a subtle prompt that slides down from the header.
Also in this PR I've added a helper to publish message bus messages in JS tests. So instead of this:
```javascript
// Mimic a messagebus message
MessageBus.callbacks
.filterBy("channel", "/global/asset-version")
.map((c) => c.func("somenewversion"));
```
We can have:
```javascript
publishToMessageBus("/global/asset-version", "somenewversion");
```
In Improve invite system, a newly created link only invite cannot
be retrieved via API with the invitee's email once created. A new
route, /invites/retrieve, is introduced to fetch an already
created invite by email address.
This feature used to be controlled by two site settings
enable_personal_email_messages and min_trust_to_send_email_messages.
I removed enable_personal_email_messages and unhide
min_trust_to_send_email_messages to simplify the process of
enabling / disabling this feature.
Changing the invite type from link to email and then copying it was
confusing because it gave user the impression that the invite was
updated and the invite link will reflect the latest changes, but it
did not.
If the user has not been sent any messages, show a message in the quick access menu with an educational message. If the user can send private messages, also show a link to open the "new message" composer:
This also adds a general improvement to the quick-access-panel, to be able to show an `emptyStateWidget` instead of just a message if there is nothing to show in the panel, as well as initial general styles for empty state.
* FEATURE: Cache successful HTTP GET requests during Oneboxing
Some oneboxes may fail if when making excessive and/or odd requests against the target domains. This change provides a simple mechanism to cache the results of succesful GET requests as part of the oneboxing process, with the goal of reducing repeated requests and ultimately improving the rate of successful oneboxing.
To enable:
Set `SiteSetting.cache_onebox_response_body` to `true`
Add the domains you’re interesting in caching to `SiteSetting. cache_onebox_response_body_domains` e.g. `example.com|example.org|example.net`
Optionally set `SiteSetting.cache_onebox_user_agent` to a user agent string of your choice to use when making requests against domains in the above list.
* FIX: Swap order of duration and value in redis call
The correct order for `setex` arguments is `key`, `duration`, and `value`.
Duration and value had been flipped, however the code would not have thrown an error because we were caching the value of `1.day.to_i` for a period of 1 seconds… The intention appears to be to set a value of 1 (purely as a flag) for a period of 1 day.
Fixes `Rack::Lint::LintError: a header value must be a String, but the value of 'Retry-After' is a Integer`. (see: 14a236b4f0/lib/rack/lint.rb (L676))
I found it when I got flooded by those warning a while back in a test-related accident 😉 (ember CLI tests were hitting a local rails server at a fast rate)
browser-update script does not work correctly in some very old browsers
because the contents of <noscript> is not accessible in JavaScript.
For these browsers, the server can display the crawler page and add the
browser update notice.
Simply loading the browser-update script in the crawler view is not a
solution because that means all crawlers will also see it.
Users can now pin bookmarks from their bookmark list. This will anchor the bookmark to the top of the list, and show a pin icon next to it. This also applies in the nav bookmarks panel. If there are multiple pinned bookmarks they sort by last updated order.
The logster initializer tries to adds RailsMultisite::Formatter to the STDOUT logger. In production, the lograge initializer then removes the RailsMultisite:Formatter because the JSON log will include the database.
e10a74694a used `Rails.application.reloader.to_prepare` to defer running the 100-logster initializer, which meant it ran **after** 101-lograge. This meant that we were writing JSON logs with a non-json text prefix.
The `to_prepare` was added because our freedom-patches are now deferred using `to_prepare`, and some initializers were relying on the freedom patches. However, following 1533cbb38b, we decided to load the RailsMultisite freedom patch without `to_prepare`. Therefore, `005-site_settings` and `100-logster` no longer need to use `to_prepare`. Removing it means that these initializers are back to running in sequential order, and the logging issue will be resolved.
The only remaining initializer which depends on freedom patches is `100-i18n`. I've added a comment to explain why.
We previously included this option conditionally when users were replying
or creating a new topic while they had content already in the composer.
This makes the dialog always include three buttons:
- Close and discard
- Close and save draft for later
- Keed editing
This also changes how the backend notifies the frontend when there is
a current draft topic. This is now sent via the `has_topic_draft`
property in the current user serializer.
This PR allows invitations to be used when the DiscourseConnect SSO is enabled for a site (`enable_discourse_connect`) and local logins are disabled. Previously invites could not be accepted with SSO enabled simply because we did not have the code paths to handle that logic.
The invitation methods that are supported include:
* Inviting people to groups via email address
* Inviting people to topics via email address
* Using invitation links generated by the Invite Users UI in the /my/invited/pending route
The flow works like this:
1. User visits an invite URL
2. The normal invitation validations (redemptions/expiry) happen at that point
3. We store the invite key in a secure session
4. The user clicks "Accept Invitation and Continue" (see below)
5. The user is redirected to /session/sso then to the SSO provider URL then back to /session/sso_login
6. We retrieve the invite based on the invite key in secure session. We revalidate the invitation. We show an error to the user if it is not valid. An additional check here for invites with an email specified is to check the SSO email matches the invite email
7. If the invite is OK we create the user via the normal SSO methods
8. We redeem the invite and activate the user. We clear the invite key in secure session.
9. If the invite had a topic we redirect the user there, otherwise we redirect to /
Note that we decided for SSO-based invites the `must_approve_users` site setting is ignored, because the invite is a form of pre-approval, and because regular non-staff users cannot send out email invites or generally invite to the forum in this case.
Also deletes some group invite checks as per https://github.com/discourse/discourse/pull/12353
* FIX: Be able to handle long file extensions
Some applications have really long file extensions, but if we truncate
them weird behavior ensues.
This commit changes the file extension size from 10 characters to 255
characters instead.
See:
https://meta.discourse.org/t/182824
* Keep truncation at 10, but allow uppercase and dashes
Currently the process of adding a custom image to badge is quite clunky; you have to upload your image to a topic, and then copy the image URL and pasting it in a text field. Besides being clucky, if the topic or post that contains the image is deleted, the image will be garbage-collected in a few days and the badge will lose the image because the application is not that the image is referenced by a badge.
This commit improves that by adding a proper image uploader widget for badge images.
Get rid of deprecation related to Zeitwerk autoloader.
Original PR was reverted because of multisite bug #12381 - thank you @davidtaylorhq for fixing it.
I added the last commit to fix that multisite problem.
This commit extends functionality of the expired invites tab, making
it more similar to the pending tab. It also implements a different
layout for mobile.
The cluster name can be configured by setting the `DISCOURSE_CLUSTER_NAME` environment variable. If set, you can then call /srv/status with a `?cluster=` parameter. If the cluster does not match, an error will be returned. This is useful if you need a load balancer to be able to verify the identity, as well as the presence, of an application container.
Staff can send a post to the review queue by clicking the "Flag Post" button next to "Take Action...". Clicking it flags the post using the "Notify moderators" score type and hides it. A custom message will be sent to the user.
This is not recommended. But if you have other protections in place for CSRF mitigation, you may wish to disable Discourse's implementation. This site setting is not visible in the UI, and must be changed via the console.
* FIX: Show resend button only if an email was sent
Otherwise, show the "save and send email" button.
* UX: Copy change
* UX: Show feedback when link was copied
Prior to this change, we had weights for very_high, high, low and
very_low. This means there were 4 weights to tweak and what weights to
use for `very_high/high` and `very_low/low` pair was hard to explain.
This change makes it such that `very_high` search priority will always
ensure that the posts are ranked at the top while `very_low` search
priority will ensure that the posts are ranked at the very bottom.
* FIX: Do not show expired invites under Pending tab
* DEV: Controller action was renamed in previous commit
* FEATURE: Add 'Expired' tab to invites
* FEATURE: Refresh model after removing expired invites
* FEATURE: Do not immediately add invite to the list
Opening the 'create-invite' modal used to automatically generate an
invite to reserve an invite link. If the user did not save it and
closed the modal, the invite would be destroyed. This operations caused
the invite list to change in the background and confuse users.
* FEATURE: Sort redeemed users by creation time
* UX: Improve show / hide advanced options link
* FIX: Show redeemed users even if invites were trashed
* UX: Change modal title when editing invite
* UX: Remove Get Link button
Users can get it from the edit modal
* FEATURE: Add limit for invite links generated by regular users
* FEATURE: Add option to skip email
* UX: Show better error messages
* FIX: Show "Invited by" even if invite was trashed
Follow up to 1fdfa13a099d8e46edd0c481b3aaaafe40455ced.
* FEATURE: Add button to save without sending email
Follow up to c86379a465f28a3cc64a4a8c939cf32cf2931659.
* DEV: Use a buffer to hold all changed data
* FEATURE: Close modal after save
* FEATURE: Rate limit resend invite email
* FEATURE: Make the save buttons smarter
* FEATURE: Do not always send email even for new invites
Mailing list mode can generate significant email volume, especially on sites with a large user base. Disable mailing list mode via site settings by default so sites don't experience an unexpectedly large cost from outgoing email.
I made some adjustments to the invite system copy, to reflect the mockups at https://meta.discourse.org/t/improve-invite-system/173756 - the goal is to try to reduce the number of words and make it as simple as possible, similar to discord.
The user interface has been reorganized to show email and link invites
in the same screen. Staff has more control over creating and updating
invites. Bulk invite has also been improved with better explanations.
On the server side, many code paths for email and link invites have
been merged to avoid duplicated logic. The API returns better responses
with more appropriate HTTP status codes.