Commit Graph

225 Commits

Author SHA1 Message Date
Guo Xiang Tan a1c13eb3c6
DEV: Redis failover should only clear redis recently readonly. 2020-06-09 16:36:31 +08:00
Guo Xiang Tan 8e1681d356 Bump rails_failover. 2020-06-04 15:22:35 +08:00
Guo Xiang Tan 76ce814a62
DEV: Use `safe_each_connection` patch when preloading Rails.
This is to ensure we still boot even if there is a problem with any of
the databases.
2020-06-03 15:36:50 +08:00
Guo Xiang Tan fe9c82994d
DEV: Actually disconnect from Redis connections after fork. 2020-06-02 11:40:16 +08:00
Guo Xiang Tan df62407f35
DEV: Implement multiple keys support for `DiscourseRedis#exists`. 2020-06-01 11:20:26 +08:00
Guo Xiang Tan 3894555b2e
DEV: Prefer `Discourse.redis.reconnct`. 2020-06-01 10:55:53 +08:00
Guo Xiang Tan 3b311f6b15
DEV: Bump `rails_failover`. 2020-05-29 09:02:34 +08:00
Guo Xiang Tan 878f06f1fe DEV: Remove custom connection reaper.
Rails 6 fixed the reaper to use one thread to reap all the connection pools.
2020-05-26 09:09:46 +08:00
Bianca Nenciu 09caf614de DEV: Fix build
Rails.logger is not available when Discourse.deprecate is called from
enabled_site_setting_filter.

Follow up to db46018235.
2020-05-10 15:06:39 +03:00
Bianca Nenciu 8149bfbaf1
FEATURE: Filter settings by plugin (#9692) 2020-05-10 14:07:45 +03:00
Sam Saffron d0d5a138c3
DEV: stop freezing frozen strings
We have the `# frozen_string_literal: true` comment on all our
files. This means all string literals are frozen. There is no need
to call #freeze on any literals.

For files with `# frozen_string_literal: true`

```
puts %w{a b}[0].frozen?
=> true

puts "hi".frozen?
=> true

puts "a #{1} b".frozen?
=> true

puts ("a " + "b").frozen?
=> false

puts (-("a " + "b")).frozen?
=> true
```

For more details see: https://samsaffron.com/archive/2018/02/16/reducing-string-duplication-in-ruby
2020-04-30 16:48:53 +10:00
David Taylor 3d71b68195
DEV: Introduce plugin api for conditionally rendering assets (#9200) 2020-03-13 15:30:31 +00:00
Robin Ward a3f0543f99
Support for transpiling `.js` files (#9160)
* Remove some `.es6` from comments where it does not matter

* Use a post processor for transpilation

This will allow us to eventually use the directory structure to
transpile rather than the extension.

* FIX: Some errors and clean up in confirm-new-email

It would throw an error if the webauthn element wasn't present.
Also I changed things so that no-module is not explicitly
referenced.

* Remove `no-module`

Instead we allow a magic comment: `// discourse-skip-module` to prevent
the asset pipeline from creating a module.

* DEV: Enable babel transpilation based on directory

If it's in `app/assets/javascripts/dicourse` it will be transpiled
even without the `.es6` extension.

* REFACTOR: Remove Tilt/ES6ModuleTranspiler
2020-03-11 09:43:55 -04:00
Daniel Waterworth 83e649d08e FIX: Clobbering plugin files on startup is problematic
On startup, (including when starting a rails console) we manipule a
collection of plugin files. Writing these files is done in multiple
observable steps, which presents opportunities for race conditions and
causes temporary corruption.

This commit uses the write, fsync and rename trick to atomically
overwrite these files instead, but reads them first to avoid unnecessary
writes.

c457d3bf was a previous attempt to fix the same problem.
2020-03-05 11:51:56 +00:00
Sam Saffron 28292d2759
PERF: avoid shelling to get hostname aggressively
Previously we had many places in the app that called `hostname` to get
hostname of a server. This commit replaces the pattern in 2 ways

1. We cache the result in `Discourse.os_hostname` so it is only ever called once

2. We prefer to use Socket.gethostname which avoids making a shell command

This improves performance as we are not spawning hostname processes throughout
the app lifetime
2020-02-18 15:13:19 +11:00
Robin Ward 556d66a97c FIX: Use CDN for the `discourse-internet-explorer`
Previously the CDN was skipped because the assets were not in the proper
place to be uploaded.
2020-01-10 15:06:55 -05:00
Vinoth Kannan 3b7f5db5ba
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547) 2019-12-18 11:21:57 +05:30
Joffrey JAFFEUX 0d3d2c43a0
DEV: s/\$redis/Discourse\.redis (#8431)
This commit also adds a rubocop rule to prevent global variables.
2019-12-03 10:05:53 +01:00
Sam Saffron 0fb497eb23 DEV: use Discourse.cache over Rails.cache
Discourse.cache is a more consistent method to use and offers clean fallback
if you are skipping redis

This is part of a larger change that both optimizes Discoruse.cache and omits
use of setex on $redis in favor of consistently using discourse cache

Bench does reveal that use of Rails.cache and Discourse.cache is 1.25x slower
than redis.setex / get so a re-implementation will follow prior to porting
2019-11-27 12:36:19 +11:00
Martin Brennan 93d7abe372
FIX: Automatically recover from bad sprockets cache in development (#8364)
We were having issues in development mode where the JS code had errors due to a bad cache. When starting a server in development mode in bin/unicorn we now get the git sha of the discourse HEAD and get a git sha of all plugins, and store them in a file. If the sha has changed then we delete tmp/cache to refresh the assets cache.
2019-11-19 09:15:09 +10:00
Daniel Waterworth e79cec9fc7 DEV: Remove handling of category top menu items
Support for these kinds of navigation items was dropped in 88f52514, but
the code for handling these menu items was never removed.
2019-11-11 17:36:29 +00:00
David Taylor 60a235d128
DEV: Allow execute_command to receive a block (#8303)
This makes it easy to run multiple commands with the same keyword arguments. The main use is for using `chdir` across multiple commands. The `Dir.chdir` method is not concurrency safe because it switches the working directory of the entire process.
2019-11-07 15:47:16 +00:00
David Taylor 4312bbe1e7
FIX: Do not load plugin CSS/JS assets when disabled (#8275)
Follow-up to 839916aa49 and 5bd6b70d98
2019-11-01 09:50:31 +00:00
David Taylor 1f88ecf6d8
FIX: Cache `Discourse.system_user` separately for each multisite tenant (#8276)
This was not causing any known issue, because the system user ID is always the same across all sites. However, we should cache this on a per-site basis to be safe.
2019-10-31 15:16:26 +00:00
Dan Ungureanu fdb1d3404c
FEATURE: Add site setting to show more detailed 404 errors. (#8014)
If the setting is turned on, then the user will receive information
about the subject: if it was deleted or requires some special access to
a group (only if the group is public). Otherwise, the user will receive
a generic #404 error message. For now, this change affects only the
topics and categories controller.

This commit also tries to refactor some of the code related to error
handling. To make error pages more consistent (design-wise), the actual
error page will be rendered server-side.
2019-10-08 14:15:08 +03:00
David Taylor d2bceff133
FEATURE: Use full page redirection for all external auth methods (#8092)
Using popups is becoming increasingly rare. Full page redirects are already used on mobile, and for some providers. This commit removes all logic related to popup authentication, leaving only the full page redirect method.

For more info, see https://meta.discourse.org/t/do-we-need-popups-for-login/127988
2019-10-08 12:10:43 +01:00
Sam Saffron 8d5f47dded PREF: optimise preloading application
We preload to ensure as much memory as possible is reused from unicorn master
to various workers using copy-on-write (sidekiq, unicorn)

This migrates the preloading code into the Discourse module for easier
reuse and adds 3 notable preloading changes

1. We attempt to localize a string on each site, ensuring we warmup
the i18n

2. We preload all our templates (compiling .erb to class)

3. We warm-up our search tokenizer which uses cppjieba which is a large
memory consumer, this will only cause a warmup on CJK sites or sites with
the special site setting enabled.
2019-10-07 00:33:37 -04:00
David Taylor 3da9b99dbf FIX: Live reload plugin stylesheets when the color scheme changes 2019-09-17 09:54:55 +01:00
David Taylor be96c4478e
FEATURE: Login with Discord (#8053)
This migrates the functionality of discourse-plugin-discord-auth into core. 

The plugin will automatically disable itself when core is updated: fd0867844d?w=1

For setup instructions, visit https://meta.discourse.org/t/configuring-discord-login-for-discourse/127129
2019-08-30 10:54:19 +01:00
Régis Hanol 00dbc260d3 FIX: ensure avatar sizes are integers 2019-08-27 17:03:20 +02:00
Vinoth Kannan 41f22946c3 FIX: properly load desktop and mobile only plugin css assets.
5bd6b70d98
2019-08-22 08:39:10 +05:30
Vinoth Kannan 5bd6b70d98
DEV: debundle plugin css assets and don't load if disabled (#7646) 2019-08-20 22:09:52 +05:30
Roman Rizzi eb26bee046
DEV: group_list site settings should store IDs instead of group names (#7860)
* DEV: group_list site settings should store IDs instead of group names

* Ship site setting to know when we should migrate group_list settings

* Migrate existing group_list site settings

* Bump migration timestamp and don't set null when migrating is not possible.
2019-07-19 15:17:58 -03:00
Vinoth Kannan 839916aa49
DEV: Debundle plugin javascript assets and don't load if disabled (#7566)
And don't load javascript assets if plugin is disabled.

* precompile auto generated plugin js assets

* SPEC: remove spec test functions

* remove plugin js from test_helper

Co-Authored-By: Régis Hanol <regis@hanol.fr>

* DEV: using equality is slightly easier to read than inequality

Co-Authored-By: Régis Hanol <regis@hanol.fr>

* DEV: use `select` method instead of `find_all` for readability

Co-Authored-By: Régis Hanol <regis@hanol.fr>
2019-07-15 20:22:54 +05:30
David Taylor afb5ec811d FIX: Don't use DistributedCache to store redis readonly state
This can cause unbound CPU usage in some cases, and excessive logging in other cases. This commit moves redis readonly information into the local process, but maintains the DistributedCache for postgres readonly state.
2019-06-25 11:20:34 +08:00
David Taylor ed21128ee6 FIX: Do not change directory when decompressing S3 inventory
In sidekiq, jobs are run in multiple threads within the same process. `cd` affects the entire process, so can cause unexpected issues in other running jobs.
2019-06-13 17:13:50 +01:00
Sam fa2a5f6f56
FEATURE: SKIP_DB_AND_REDIS env var (#7756)
Sometimes we would like to create a base image without any DB access, this
assists in creating custom base images with custom plugins that already
includes `public/assets`

Following this change set you can run:

```
SPROCKETS_CONCURRENT=1 DONT_PRECOMPILE_CSS=1 SKIP_DB_AND_REDIS=1 RAILS_ENV=production bin/rake assets:precompile
```

Then it is straight forward to create a base image without needing a DB or
Redis.
2019-06-13 12:58:27 +10:00
David Taylor 5666316285 FEATURE: Display 'last updated' on dashboard, improve release notes link (#7560) 2019-05-17 15:42:45 +10:00
Sam Saffron 30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Guo Xiang Tan b7b9960558 DEV: Fix missing port in URLs in development mode. 2019-05-06 13:26:57 +08:00
Vinoth Kannan 92cc57b5da FIX: make 'stats' method compatible with multisite.
a8f410a9c5
2019-05-01 23:34:18 +05:30
Vinoth Kannan a8f410a9c5
FEATURE: Create new helper method 'Discourse.stats' (#7388) 2019-04-17 12:45:04 +05:30
Robin Ward ba6d4b2a8d FIX: Better handling for toggling `must_approve_users`
If you turn it on now, default all users to approved since they were
previously. Also support approving a user that doesn't have a reviewable
record (it will be created first.)

This also includes a refactor to move class method calls to
`DiscourseEvent` into an initializer. Otherwise the load order of
classes makes a difference in the test environment and some settings
might be triggered and others not, randomly.
2019-04-16 15:56:35 -04:00
David Taylor 20cda5d689 DEV: Raise exceptions when jobs fail in test mode
Previously jobs would fail silently in test mode. Now they will raise the exception and cause the relevant test to fail. This identified a few broken tests, which I will fix in a followup commit
2019-04-08 15:59:04 +01:00
David Taylor f524f8f811
Remove Yahoo login support from core and deprecate OpenID2.0 (#7310)
- Plugin developers using OpenID2.0 should migrate to OAuth2 or OIDC. OpenID2.0 APIs will be removed in v2.4.0

- For sites requiring Yahoo login, it can be implemented using the OpenID Connect plugin: https://meta.discourse.org/t/103632

For more information, see https://meta.discourse.org/t/113249
2019-04-08 10:38:25 +01:00
Robin Ward b58867b6e9 FEATURE: New 'Reviewable' model to make reviewable items generic
Includes support for flags, reviewable users and queued posts, with REST API
backwards compatibility.

Co-Authored-By: romanrizzi <romanalejandro@gmail.com>
Co-Authored-By: jjaffeux <j.jaffeux@gmail.com>
2019-03-28 12:45:10 -04:00
David Taylor 0d3531c2f1 FEATURE: Allow auth providers to define specific icons
Previously we relied on the provider name matching the name of the icon. Now icon names are explicitly set. Plugin providers which do not define an icon will get the default "sign-in-alt" icon
2019-03-27 13:25:04 +00:00
David Taylor 01dd96d178
FIX: Use default frame size for Instagram login
I don't know why this was set to 1x1 pixels, but it shouldn't have been
2019-03-19 18:57:57 +00:00
Guo Xiang Tan 0e0daa2c91 DEV: Prevent race condition when keeping site in readonly mode. 2019-02-20 10:01:18 +08:00
Guo Xiang Tan c29076152d PERF: Reduce hit to Redis server when keeping a site in reaonly mode. 2019-02-19 10:29:08 +08:00