Commit Graph

399 Commits

Author SHA1 Message Date
Sam Saffron 62141b6316 FEATURE: enable_performance_http_headers for performance diagnostics
This adds support for DISCOURSE_ENABLE_PERFORMANCE_HTTP_HEADERS
when set to `true` this will turn on performance related headers

```text
X-Redis-Calls: 10     # number of redis calls
X-Redis-Time: 1.02    # redis time in seconds
X-Sql-Commands: 102   # number of SQL commands
X-Sql-Time: 1.02      # duration in SQL in seconds
X-Queue-Time: 1.01    # time the request sat in queue (depends on NGINX)
```

To get queue time NGINX must provide: HTTP_X_REQUEST_START

We do not recommend you enable this without thinking, it exposes information
about what your page is doing, usually you would only enable this if you
intend to strip off the headers further down the stream in a proxy
2019-06-05 16:08:11 +10:00
Sam Saffron b114bcd294 DEV: switch message bus backend to memory for tests
This backend is a bit faster and well tested, this is part of a longer
term plan to have a `backend: :memory, threaded: false` type config for
message bus which we can use in test.

The threading in message bus causes all sorts of surprises in test, it will
be nice not to be beholden to them.
2019-05-29 16:34:55 +10:00
Rafael dos Santos Silva 315a38e0e3 FEATURE: Allow running message_bus in a different redis instance (#7616)
Adds `DISCOURSE_MESSAGE_BUS_REDIS_ENABLED` env var, that when set
to true, will allow Discourse to connect to a different redis
instance for MessageBus needs.

When enabled you can configure the same env vars user for redis,
but prefixed by `MESSAGE_BUS`, eg:

`DISCOURSE_MESSAGE_BUS_REDIS_HOST`
2019-05-28 15:52:43 +10:00
Sam Saffron 30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Sam Saffron 88249932a0 PERF: remove SQL notifiers for every SQL execution
Benchmarking:

```
Benchmark.ips do |b|
  b.report("simple") do
    User.first
  end
end

ActiveSupport::Notifications.notifier.listeners_for("sql.active_record").clear

Benchmark.ips do |b|
  b.report("simple") do
    User.first
  end
end

```

```
sam@arch discourse % RAILS_ENV=production ruby script/micro_bench.rb

Before

Calculating -------------------------------------
              simple      3.289k (± 4.4%) i/s -     16.575k in   5.049771s
After

Calculating -------------------------------------
              simple      3.491k (± 3.6%) i/s -     17.442k in   5.002226s
````
2019-05-09 17:44:46 +10:00
Guo Xiang Tan 152238b4cf DEV: Prefer `public_send` over `send`. 2019-05-07 09:33:21 +08:00
David Taylor 0e303c7f5d
FEATURE: Automatically generate optimized site metadata icons (#7372)
This change automatically resizes icons for various purposes. Admins can now upload `logo` and `logo_small`, and everything else will be auto-generated. Specific icons can still be uploaded separately if required.

## Core

- Adds an SiteIconManager module which manages automatic resizing and fallback

- Icons are looked up in the OptimizedImage table at runtime, and then cached in Redis. If the resized version is missing for some reason, then most icons will fall back to the original files. Some icons (e.g. PWA Manifest) will return `nil` (because an incorrectly sized icon is worse than a missing icon). 

- `SiteSetting.site_large_icon_url` will return the optimized version, including any fallback. `SiteSetting.large_icon` continues to return the upload object. This means that (almost) no changes are required in core/plugins to support this new system.

- Icons are resized whenever a relevant site setting is changed, and during post-deploy migrations

## Wizard

- Allows `requiresRefresh` wizard steps to reload data via AJAX instead of a full page reload

- Add placeholders to the **icons** step of the wizard, which automatically update from the "Square Logo"

- Various copy updates to support the changes

- Remove the "upload-time" resizing for `large_icon`. This is no longer required.

## Site Settings UX

- Move logo/icon settings under a new "Branding" tab

- Various copy changes to support the changes

- Adds placeholder support to the `image-uploader` component

- Automatically reloads site settings after saving. This allows setting placeholders to change based on changes to other settings

- Upload site settings will be assigned a placeholder if SiteIconManager `responds_to?` an icon of the same name

## Dashboard Warnings

- Remove PWA icon and PWA title warnings. Both are now handled automatically.

## Bonus

- Updated the sketch logos to use @awesomerobot's new high-res designs
2019-05-01 14:44:45 +01:00
Sam Saffron 3b95f34e7b DEV: Rails 6 compatability
Without forcing a reload on start internal state in the accelerator can be
off. In Rails 5 not translation is being called so this is not an issue but
in 6 it is called earlier on.
2019-04-29 17:13:30 +10:00
David Taylor 7826acc4a7
DEV: Replace site_setting_saved DiscourseEvent with site_setting_changed (#7401)
* DEV: Replace site_setting_saved DiscourseEvent with site_setting_changed

site_setting_saved is confusing for a few reasons:
- It is attached to the after_save of the ActiveRecord model. This is confusing because it only works 'properly' with the db_provider
- It passes the activerecord model as a parameter, which is confusing because you get access to the 'database' version of the setting, rather than the ruby setting. For example, booleans appear as 'y' or 'n' strings.
- When the event is called, the local process cache has not yet been updated. So if you call SiteSetting.setting_name inside the event handler, you will receive the old site setting value

I have deprecated that event, and added a new site_setting_changed event. It passes three parameters:
- Setting name (symbol)
- Old value (in ruby format)
- New value (in ruby format)

It is triggered after the setting has been persisted, and the local process cache has been updated.

This commit also includes a test case which describes the confusing behavior. This can be removed once site_setting_saved is removed.
2019-04-18 16:48:01 +01:00
David Taylor c687e2b921 FIX: Use `saved_change_to_value?` in site_setting_saved event
Since Rails 5.2, the behavior of `attribute_changed?` inside `after_save` callbacks has changed, so we need to use `saved_change_to_attribute` instead. The site setting local_process_provider in test mode was covering up the issue.
2019-04-17 10:07:00 -04:00
Robin Ward ba6d4b2a8d FIX: Better handling for toggling `must_approve_users`
If you turn it on now, default all users to approved since they were
previously. Also support approving a user that doesn't have a reviewable
record (it will be created first.)

This also includes a refactor to move class method calls to
`DiscourseEvent` into an initializer. Otherwise the load order of
classes makes a difference in the test environment and some settings
might be triggered and others not, randomly.
2019-04-16 15:56:35 -04:00
Robin Ward b58867b6e9 FEATURE: New 'Reviewable' model to make reviewable items generic
Includes support for flags, reviewable users and queued posts, with REST API
backwards compatibility.

Co-Authored-By: romanrizzi <romanalejandro@gmail.com>
Co-Authored-By: jjaffeux <j.jaffeux@gmail.com>
2019-03-28 12:45:10 -04:00
Sam Saffron b418857830 DEV: document max_logster_logs in discourse_defaults.conf
This cleans up logster configuration a bit cause we no longer have to
check if we respond_to anything and keeps the logster limit properly
documented

Followup on da578e92
2019-03-22 14:11:40 +11:00
Osama Sayegh da578e92f2
DEV: configure logster max logs via ENV (#7225) 2019-03-21 16:10:17 +03:00
Osama Sayegh 14c1af0a9e Bump logster version to 2.2.0 (#7200) 2019-03-19 09:39:16 +01:00
Osama Sayegh f86ca5631a
Bump logster to v 2.1.2 (#7052) 2019-02-21 18:59:33 +03:00
Sam 8745f78277 DEV: disable async logging in development on broken Ruby
Ruby 2.5.3 has an upatched issue that crashes unicorn after fork:

https://bugs.ruby-lang.org/issues/14634

This will be patched in 2.5.4 however for now just warn people dev is slower
and disable async logging on the older rubies
2019-02-21 17:20:58 +11:00
Guo Xiang Tan eb9565d8fc DEV: Rescue from readonly error in lograge. 2019-01-21 14:20:19 +08:00
Guo Xiang Tan ee7ab3e2ec Pause MiniScheduler when Sidekiq is paused. 2019-01-18 17:50:24 +08:00
Guo Xiang Tan 234e0f35c5 Fix the build.
6edf285c89
2019-01-17 17:28:09 +08:00
Guo Xiang Tan 6edf285c89 FIX: Avoid rescuing error in login hint initializer.
On the first migration, trying to access the users table will throw an
error in PostgreSQL's log which has been confusing since users will
report it to us when rebuild fails.
2019-01-17 17:22:15 +08:00
Sam a19170a4c2 DEV: avoid require_dependency for some libs
This avoids require dependency on method_profiler and anon cache.

It means that if there is any change to these files the reloader will not pick it up.

Previously the reloader was picking up the anon cache twice causing it to double load on boot.

This caused warnings.

Long term my plan is to give up on require dependency and instead use:

https://github.com/Shopify/autoload_reloader
2018-12-31 10:53:30 +11:00
Guo Xiang Tan e564fe1866 FIX: Sidekiq fails to start if any of the multisite has problems. 2018-12-12 11:30:14 +08:00
Sam a34bc92e1a DEV: update mini profiler
This provides us with instrumentation missing after rails upgrade

Latest version of rails uses exec_params internally which is no longer
routed to intercepted methods in mini profiler 1.0.0
2018-12-10 14:29:20 +11:00
Sam 1a71f98d28 DEV: only publish logs error count to admins 2018-12-05 17:03:37 +11:00
Sam 12f5889c85 DEV: stop logging warning when overriding open scope
ActiveRecord defines automatic scopes for enums, the Poll model defines
an enum for `{open: 1}` this mean Rails wants the scope `Poll.all.open`
to work which in turn means it has to override `open` which is defined
privately.

Rails feature req exists for: https://github.com/rails/rails/issues/34599
which will allow us to define enums without scopes which would resolve this
a lot more cleaner.
2018-12-03 13:52:09 +11:00
Penar Musaraj 03deda2147
Upgrade to FontAwesome 5 (take two) (#6673)
* Add missing icons to set

* Revert FA5 revert

 This reverts commit 42572ff

* use new SVG syntax in locales

* Noscript page changes (remove login button, center "powered by" footer text)

* Cast wider net for SVG icons in settings

- include any _icon setting for SVG registry (offers better support for plugin settings)

- let themes store multiple pipe-delimited icons in a setting

- also replaces broken onebox image icon with SVG reference in cooked post processor

* interpolate icons in locales

* Fix composer whisper icon alignment

* Add support for stacked icons

* SECURITY: enforce hostname to match discourse hostname

This ensures that the hostname rails uses for various helpers always matches
the Discourse hostname

* load SVG sprite with pre-initializers

* FIX: enable caching on SVG sprites

* PERF: use JSONP for SVG sprites so they are served from CDN

This avoids needing to deal with CORS for loading of the SVG

Note, added the svg- prefix to the filename so we can quickly tell in
dev tools what the file is

* Add missing SVG sprite JSONP script to CSP

* Upgrade to FA 5.5.0

* Add support for all FA4.7 icons

- adds complete frontend and backend for renamed FA4.7 icons

- improves performance of SvgSprite.bundle and SvgSprite.all_icons

* Fix group avatar flair preview

- adds an endpoint at /svg-sprites/search/:keyword

- adds frontend ajax call that pulls icon in avatar flair preview even when it is not in subset

* Remove FA 4.7 font files
2018-11-26 16:49:57 -05:00
Sam 42572ff138 Revert font awesome 5 changes
We are still pushing ahead on this 100% just need a bit longer to prepare
all plugins
2018-11-08 16:12:18 +11:00
Penar Musaraj 005e1ecb9b
FEATURE: Update Font Awesome to v5.4.1 and SVGs (#6557)
* First take on subsetting svg icons

* FontAwesome 5 svg subset WIP

* Include icons from plugins/badges into svg sprite subset

* add svg icon support to themes

* Add spec for SvgSprite

* Misc. SVG icon fixes

* Use FA5 svgs in local-dates plugin

* CSS adjustments, fix SVG icons in group flair

* Use SVG icons in poll plugin

* Add SVG icons to /wizard
2018-11-07 13:05:43 -05:00
Kyle Zhao dca830cb73 Revert "FEATURE: [Experimental] Content Security Policy (#6504)"
This reverts commit fb8231077a.
2018-10-19 11:53:29 -04:00
Kyle Zhao fb8231077a
FEATURE: [Experimental] Content Security Policy (#6504) 2018-10-19 10:39:22 -04:00
Davide Porrovecchio 005e1f5373 Add Cache-Control header to CORS (#6490) 2018-10-16 10:46:55 +11:00
Guo Xiang Tan 1c9b5e75e7 DEV: Support post deployment migrations for plugins. 2018-10-09 13:11:45 +08:00
Guo Xiang Tan 40fa96777d
FEATURE: Post deployment migrations. (#6406)
This moves us away from the delayed drops pattern which
was problematic on two counts. First, it uses a hardcoded "delay for"
duration which may be too short for certain deployment strategies.
Second, delayed drop doesn't ensure that it only runs after
the latest application code has been deployed. If the migration runs
and the application code fails to deploy, running the migration after
"delay for" has been met will cause the application to blow up.

The new strategy allows post deployment migrations to be skipped if the
env `SKIP_POST_DEPLOYMENT_MIGRATIONS` is provided.

```
SKIP_POST_DEPLOYMENT_MIGRATIONS=1 rake db:migrate
-> deploy app servers
SKIP_POST_DEPLOYMENT_MIGRATIONS=0 rake db:migrate
```

To aid with the generation of a post deployment migration, a generator
has been added. Simply run `rails generate post_migration`.
2018-10-08 15:47:38 +08:00
Vinoth Kannan 8430ea927e FIX: Generate webhook payloads before destroy events (#6325) 2018-10-05 16:53:59 +08:00
CheshireVillageSoftware f3214889dc FEATURE: Updated CORS config to explicitly specifyhttp methods
See: https://stackoverflow.com/questions/20478312/default-value-for-access-control-allow-methods 

In particular we now explicitly allow DELETE and PUT which is inconsistently allowed depending on browser
2018-09-17 11:01:08 +10:00
Guo Xiang Tan 8dc1463ab3 Enable `Lint/ShadowingOuterLocalVariable` for Rubocop. 2018-09-04 10:16:42 +08:00
Guo Xiang Tan 0a14e0a256 Ensure `params[:files]` responds to `map` in Lograge. 2018-09-04 09:22:54 +08:00
Davide Porrovecchio 1826626272 FEATURE: Add Content-Type header to CORS
- add Content-Type to Access-Control-Allow-Headers
- update test accordingly
2018-08-28 11:19:38 +10:00
Guo Xiang Tan 6d588e43cc FIX: Avoid lograge error when controller doesn't respond to `current_user`.
https://meta.discourse.org/t/broken-third-party-plugins-can-break-logging/94256
2018-08-20 10:59:56 +08:00
Guo Xiang Tan 402e570c77 `Logster::Logger#chained` is now readable in new version of logster. 2018-08-14 09:57:09 +08:00
Sam b504398a91 DEV: keep `bin/unicorn` clean from log noise 2018-08-14 08:49:57 +10:00
Guo Xiang Tan dadbf2edb4 DEV: Log to STDOUT in development like how `rails s` used to. 2018-08-13 17:05:42 +08:00
Sam c1e7d6e39a move patch cause it was causing a side effect 2018-08-07 17:25:28 +10:00
Sam f3549291a3 DEV: use unicorn in development
This commit also cleans up a bunch of pointless noise each time we boot app

- narrative was loading i18n cause redefinition of consts
- discourse.rb was loaded twice as was auth
- bin/unicorn now does all the smart things and boots unicron in dev
- bin/rails s will boot unicorn with no params
- remove bin/puma which only causes confusion
2018-08-07 17:13:47 +10:00
Neil Lalonde 0f6dcb8c40 remove unnecessary require 2018-08-01 10:37:10 -04:00
Neil Lalonde 4ad7ce70ce REFACTOR: extract scheduler to the mini_scheduler gem 2018-07-31 17:12:55 -04:00
Vinoth Kannan af5b88f8e2 Rename approval web hook event type to queued post 2018-07-26 10:29:38 +05:30
Vinoth Kannan 5059dad8f0 FEATURE: Webhook for post approval events 2018-07-25 23:43:19 +05:30
Davide Porrovecchio dd9d815178 FIX: Add User Api Key headers to CORS
- add User-Api-Key and User-Api-Client-Id to Access-Control-Allow-Headers
- update test
2018-07-24 10:28:23 +10:00